Martin Takáč, Lehigh University

Abstract

In this talk, we address the question if and when do we need 2nd order optimization methods for training deep neural networks and when are the SGD type algorithms sufficient. We will further discuss some challenges when using stochastic and batch Quasi-Newton methods for training DNN. We will conclude the talk with preliminary numerical experiments.