Seminar: Approximation Properties of Deep Learning

Professor:	JProf. Dr. Diyora Salimova
E-Mail Professor:	diyora.salimova@mathematik.uni-freiburg.de
Lecture:	Monday 12-14 Uhr, online, Zoom link will be provided
Office Hours:	by arrangement, R 208, Hermann-Herder-Str. 10
Assistant:	M.Sc. Ilkhom Mukhammadiev
E-Mail Assistant:	ilkhom.mukhammadiev@mathematik.uni-freiburg.de
Office Hours:	by arrangement, R 211, Hermann-Herder-Str. 10

Content

In recent years, deep learning based techniques have been successfully employed for a multitude of computational problems including object and face recognition, natural language processing, fraud detection, computational advertisement, and numerical approximations of differential equations. Such simulations indicate that neural networks seem to admit the fundamental power to efficiently approximate high-dimensional functions appearing in these applications. The seminar will review some results on approximation properties of deep learning. We will mostly focus on mathematical aspects and techniques to obtain approximation estimates on various classes of data including, in particular, certain types of PDE solutions.

Possible topics (to be extended)

• Physics-informed neural networks (PINNs), Section 2 in Gonon, L., Jentzen, A., Kuckuck, B., Liang, S., Riekert, A., & von Wurstemberger, P. (2024): An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
• Deep Kolmogorov methods for PDEs, Section 3.3 in Gonon, L., Jentzen, A., Kuckuck, B., Liang, S., Riekert, A., & von Wurstemberger, P. (2024): An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
• Operator learning methods, Section 4 in Gonon, L., Jentzen, A., Kuckuck, B., Liang, S., Riekert, A., & von Wurstemberger, P. (2024): An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
• Liu, Liang, & Chen (2024): Characterizing ResNet's Universal Approximation Capability
• Further type of ANNs: CNNS, ResNets, RNNs, Sections 1.4 -1.7 in Jentzen, A., Kuckuck, B., & von Wurstemberger, P.(2023): Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
• Generalization error in Deep Learning, Section 13 in Jentzen, A., Kuckuck, B., & von Wurstemberger, P.(2023): Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
• ResNets calculus, Section 3 in Baggenstos & Salimova (2021): Approximation properties of Residual Neural Networks for Kolmogorov PDEs

Preliminary schedule:

• 28.04 - Shokhrukh Ibragimov: Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality
• 05.05 - Lucas Scherberger: ResNets calculus, Section 3 in Baggenstos & Salimova (2021): Approximation properties of Residual Neural Networks for Kolmogorov PDEs
• 12.05 - Nicolas Maitry: Liu, Liang, & Chen (2024), Characterizing ResNet's Universal Approximation Capability
• 19.05 - Tim Macleod: Physics-informed neural networks (PINNs), Section 2 in Gonon, L., Jentzen, A., Kuckuck, B., Liang, S., Riekert, A., & von Wurstemberger, P. (2024): An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
• 26.05 - Ilkhom Mukhammadiev: to be announced