Deep learning as optimal control problems

Elena Celledoni (Norwegian University of Science and Technology)

The motivation of this talk comes from recent work of Haber and Ruthotto, where deep learning neural networks have been interpreted as discretisations of an optimal control problem. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalize.