1. Explain advantages of deep learning with respect to the alternative machine learning approaches.
2. Apply techniques for training of deep models.
3. Explain application fields of deep discriminative and generative models.
4. Apply deep learning techniques in understanding of images and text.
5. Distinguish kinds of deep models which are appropriate in supervised, semi-supervised and unsupervised applications.
6. Analyze and evaluate the performance of deep models.
7. Design deep models in a high-level programming language.

The course does not include lectures in English.

Students may receive additional points for presenting a technical seminar.

Two exercises in each half of the semester.

1. Motivation for deep learning; course information; basics of machine learning.
2. Fully connected feed-forward models; loss functions; activation functions; universal approximation theorem.
3. Recovering gradients by backward propagation; learning with gradient descent; differentiable programming.
4. Deep convolutional networks: layers, architectures, visualization, fine tuning, applications, implementation
5. Convolutional models for computer vision; convolutional layers; pooling layers; efficient implementation; classification architectures.
6. Convolutional backprop; recovering gradient in 1D and 2D case; implementation details.
7. Optimization with gradient descent; challenges in learning deep models; learning with momentum.
8. Midterm exam
9. Nesterov accelerated momentum, adaptive momentum; normalization of activations: LRN, batchnorm.
10. Correspondence embeddings; Siamese models; triplet loss.
11. Regularization; penalizing the parameter norm (weight decay); data augmentation and noise injection; early stopping; parameter tying and sharing; bagging and ensembling; excluding activations (dropout).
12. Recurrent models; sequence modeling; applications in natural language understanding; formulation and optimization.
13. RNN cell extensions; exploding gradient; advanced RNN cells; attention.
14. Details of convolutional models; residual and dense connectivity; advanced techniques; model interpretation and prediction explanation; object detection; dense prediction; real time; challenges.
15. Final exam