In this expository talk, I will give an overview of o-minimality as a possible mathematical framework for optimization and deep learning. In particular, I will try to explain why working within a framework such as o-minimality seems essential for proving convergence guarantees for the training process for neural networks.