### Machine Learning

#### Course Description

#### General Competencies

Familiarity with supervised and unsupervised machine learning methods, classification and regression tasks. Understanding of generative, discriminative, parametric, and nonparametric models. Understanding of theoretical foundations of these models, the underlying assumptions, and their advantages and disadvantages. Capability to design and implement a machine learning system for classification, regression, or clustering and to evaluate its performance.

#### Learning Outcomes

- define the basic concepts of machine learning
- distinguish between generative and discriminative, parametric and nonparametric and probabilistic and nonprobabilistic models models
- explain the theoretical assumptions, advantages, and disadvantages of basic machine learning algorithms
- apply model selection and statistical evaluation of the learned model
- apply various classification algorithms, inclusive generative, discriminative, and nonparametric ones
- apply clustering algorithms and cluster validation
- design and implement a machine learning method for classification/clustering and carry out its evalution
- assess the suitability of a machine learning algorithm for a given task

#### Forms of Teaching

**Lectures**Lectures are given for 13 weeks in two two-hour sessions per week.

**Exams**Midterm exam and final exam.

**Exercises**Recitations are given for 13 weeks in one-hour sessions as the need arises.

**Consultations**Weekly office hours.

**Programming Exercises**Programming assignments, demonstrated to the instructor or teaching assistant.

**Other**Weekly homework assignments. Assignments are not reviewed and not graded; solutions are presented and discussed during recitation sessions.

#### Grading Method

Continuous Assessment | Exam | |||||
---|---|---|---|---|---|---|

Type | Threshold | Percent of Grade | Threshold | Percent of Grade | ||

Laboratory Exercises | 30 % | 30 % | 0 % | 30 % | ||

Class participation | 0 % | 5 % | 0 % | 0 % | ||

Mid Term Exam: Written | 0 % | 35 % | 0 % | |||

Final Exam: Written | 0 % | 35 % | ||||

Exam: Written | 0 % | 35 % | ||||

Exam: Oral | 35 % |

##### Comment:

Active participation in class is given 5% bonus points. Exams after the final exam are written and oral, together contributing 70% to the final grade. Students are required to score above 50% on written exam to be admitted to the oral exam.

#### Week by Week Schedule

- Introduction to machine learning and motivation. Machine learning approaches. Machine learning tools. Supervised learning.
- VC-dimension. Inductive bias. Generalization, overfitting and underfitting. Model selection.
- Density estimation. Likelihood function. Estimators. Maximum likelihood estimator. MAP estimator.
- Probabilistic generative models. Naive Bayes classifier. Seminaive Bayes classifer. Smoothing.
- Regression. Least squares method. Generalized linear model of regression. Regularized regression.
- Linear discriminative models. Geometry of a linear model. Perceptron. Multiclass schemes.
- Logistic regression. Regularization. MaxEnt model.
- Midterm exam.
- Support vector machines. Kernel functions. SVM regresija.
- Nonparametric methods. k-nearest neighbors algorithm. Decision trees. Nonparametric regression. Ansambli klasifikatora.
- Classifier evaluation. Evaluation measures. Cross validation. Statistical testing.
- Feature selection. Clustering. k-means algorithm.
- Gaussian mixtures. Expectation maximization algorithm. Density-bsaed clustering. Hierarchical clustering. Cluster validation.
- Dodatna tema / pozvano predavanje. Wrap-up.
- Final exam.

#### Study Programmes

##### University graduate

#### Literature

*Introduction to Machine Learning*, The MIT Press

*Pattern Recognition and Machine Learning*, Springer

*Machine Learning: A Probabilistic Perspective*, The MIT Press

*Strojno učenje -- skripta*,

*The Elements of Statistical Learning: Data Mining, Inference, and Prediction*, Springer

*Info Theory, Inference, and Learning Algorithms;MacKay;2003;Cambridge University Press*,

#### Exercises

#### For students

#### General

**ID**103973

**5**ECTS

**L0**English Level

**L1**e-Learning

**45**Lectures

**15**Exercises

#### Grading System

**89**Excellent

**76**Very Good

**63**Good

**50**Acceptable