Familiarity with supervised and unsupervised machine learning methods, classification and regression tasks. Understanding of generative, discriminative, parametric, and nonparametric models. Understanding of theoretical foundations of these models, the underlying assumptions, and their advantages and disadvantages. Capability to design and implement a machine learning system for classification, regression, or clustering and to evaluate its performance.
- define the basic concepts of machine learning
- distinguish between generative and discriminative, parametric and nonparametric and probabilistic and nonprobabilistic models models
- explain the theoretical assumptions, advantages, and disadvantages of basic machine learning algorithms
- apply model selection and statistical evaluation of the learned model
- apply various classification algorithms, inclusive generative, discriminative, and nonparametric ones
- apply clustering algorithms and cluster validation
- design and implement a machine learning method for classification/clustering and carry out its evalution
- assess the suitability of a machine learning algorithm for a given task
Forms of Teaching
Lectures are given for 13 weeks in two two-hour sessions per week.Exams
Midterm exam and final exam.Exercises
Recitations are given for 13 weeks in one-hour sessions as the need arises.Consultations
Weekly office hours.Programming Exercises
Programming assignments, demonstrated to the instructor or teaching assistant.Other
Weekly homework assignments. Assignments are not reviewed and not graded; solutions are presented and discussed during recitation sessions.
|Type||Threshold||Percent of Grade||Threshold||Percent of Grade|
|Laboratory Exercises||30 %||30 %||0 %||30 %|
|Class participation||0 %||5 %||0 %||0 %|
|Mid Term Exam: Written||0 %||35 %||0 %|
|Final Exam: Written||0 %||35 %|
|Exam: Written||0 %||35 %|
|Exam: Oral||35 %|
Active participation in class is given 5% bonus points. Exams after the final exam are written and oral, together contributing 70% to the final grade. Students are required to score above 50% on written exam to be admitted to the oral exam.
Week by Week Schedule
- Introduction to machine learning and motivation. Machine learning approaches. Machine learning tools. Supervised learning.
- VC-dimension. Inductive bias. Generalization, overfitting and underfitting. Model selection.
- Density estimation. Likelihood function. Estimators. Maximum likelihood estimator. MAP estimator.
- Probabilistic generative models. Naive Bayes classifier. Seminaive Bayes classifer. Smoothing.
- Regression. Least squares method. Generalized linear model of regression. Regularized regression.
- Linear discriminative models. Geometry of a linear model. Perceptron. Multiclass schemes.
- Logistic regression. Regularization. MaxEnt model.
- Midterm exam.
- Support vector machines. Kernel functions. SVM regresija.
- Nonparametric methods. k-nearest neighbors algorithm. Decision trees. Nonparametric regression. Ansambli klasifikatora.
- Classifier evaluation. Evaluation measures. Cross validation. Statistical testing.
- Feature selection. Clustering. k-means algorithm.
- Gaussian mixtures. Expectation maximization algorithm. Density-bsaed clustering. Hierarchical clustering. Cluster validation.
- Dodatna tema / pozvano predavanje. Wrap-up.
- Final exam.