[IOML] - [2024Winter] - [en] - [Optimization for Machine Learning]


Optimization for Machine Learning [2024 SoSe]
Code
IOML
Name
Optimization for Machine Learning
CP
8
Duration
one semester
Offered
every winter semester
Format
Lecture 4 SWS + Exercise course 2 SWS
Workload
240 h; thereof
60 h lectures
30 h exercises
24 h preparation for exam
126 h self-study and working on assignments/projects (optionally in groups)
Availability
M.Sc. Angewandte Informatik
M.Sc. Data and Computer Science
M.Sc. Mathematik
M.Sc. Scientific Computing
Language
English
Lecturer(s)
Bogdan Savchynskyy
Examination scheme
Learning objectives The students
- can analyze optimization methods for machine learning problems and estimate the area of their potential application
- can competently apply existing algorithms and program packages for inference and learning with graphical models and neural networks
- know typical optimization techniques for inference and learning with graphical models and neural networks
- understand the basics of convex analysis, convex optimization, convex duality theory, (integer) linear programs and their geometry
Learning content The course presents various existing optimization techniques for such important machine learning tasks, as inference and learning for graphical models and neural networks. In particular, it addresses such topics as combinatorial algorithms, integer linear programs, scalable convex and non-convex optimization and convex duality theory. Graphical models and neural networks play a role of working examples along the course. The content of the course includes:
- Convex analysis and optimization: convex sets and functions, polyhedra, (integer) linear programs, basic first-order convex optimization methods and their stochastic variants, LP and Lagrange relaxations
- Graphical Models: dynamic programming, sub-gradient and block-coordinate ascent inference methods, min-cut/max-flow based inference, structured risk minimization for graphical models
- neural networks: architectures, backpropagation algorithm, stochastic gradient descent and its variants for training neural networks.
Requirements for participation recommended are: linear algebra, analysis and any universal programming language (e.g. C/C++/Pascal/python)
Requirements for the assignment of credits and final grade The module is completed with a graded oral examination. The final grade of the module is determined by the grade of the examination. The requirements for the assignment of credits follows the regulations in section modalities for examinations.
Useful literature will beannounced by the lecturer at the beginning of the course