Benjamin Haeffele, Johns Hopkins University

Abstract

A wide variety of non-convex problems can be characterized as the composition of a convex function with a convexity destroying transformation. Well known examples include many matrix/tensor factorization and neural network training formulations, where the loss is typically convex but convexity is destroyed by the matrix/tensor product or network mapping, respectively. This talk will describe a general framework that allows one to study a wide variety of non-convex optimization problems using tools from convex analysis. The analysis then provides sufficient conditions to guarantee when local minima are globally optimal as well as when no spurious local minima are present in the loss surface. Applications of the framework in matrix factorization, neural network training, separable-dictionary learning, and dropout regularization will be discussed.