Suvrit Sra, Massachusetts Institute of Technology

Abstract

Currently, in machine learning there is intense interest in nonconvex optimization. This interest is fueled by the rise of deep neural networks, and also by other more complex tasks in related areas. Although an understanding of why neural networks work so well remains elusive, there has been impressive progress in algorithms, software, and systems for nonconvex optimization. But in today's talk, I want to take a step back from algorithmic advances (fast nonconvex SGD, escaping saddle-point, etc.) --- instead, I want to draw your attention to a new set of tools that expand our repertoire of tractable nonconvex optimization. In particular, I will present a rich subclass of nonconvex problems that can be solved to global optimality (or failing that, solved numerically more efficiently). The geometric concept that I'll discuss is geodesic convexity, which generalizes the usual vector-space (linear) notion of convexity to nonlinear spaces. I will outline how geometric thinking leads to improved models or insights for fundamental tasks in machine learning and statistics, including large-scale PCA, metric learning, and Gaussian mixture models. I will outline both theoretical and practical aspects, including iteration complexity theory, and conclude with some open problems.

Video