1. Define the basic concepts of machine learning
2. Distinguish between generative and discriminative, parametric and nonparametric and probabilistic and nonprobabilistic models models
3. Explain the theoretical assumptions, advantages, and disadvantages of basic machine learning algorithms
4. Apply model selection and statistical evaluation of the learned model
5. Apply various classification algorithms, inclusive generative, discriminative, and nonparametric ones
6. Apply clustering algorithms and cluster validation
7. Design and implement a machine learning method for classification/clustering and carry out its evalution
8. Assess the suitability of a machine learning algorithm for a given task

Lectures are given for 13 weeks in two two-hour sessions per week.

Recitations are given for 13 weeks in one-hour sessions as the need arises.

Programming assignments, demonstrated to the instructor or teaching assistant.

1. Machine learning tasks and applications, Machine learning approaches and paradigms, Hypothesis, model, parameter space, version space, Inductive learning and inductive bias, language and preference bias, Loss function and error function, Overfitting and model selection, empirical and structural risk minimization
2. Least-squares regression, maximum likelihood estimation for linear regression, Ridge regression, Maximum entropy model, Feature mapping functions
3. Hypothesis, model, parameter space, version space, Perceptron (learning paradigms, Hebbian learning, competitive learning, Boltzmann learning)
4. Logistic regression, Generalized linear models (exponential family, ML and MAP estimation)
5. Support vector machine for classification.
6. Lazy classification (k-NN), Kernel functions (RBF, graph kernels, Mercer kernels, linear kernels), Kernel trick
7. Bagging and boosting, Stacked generalization
8. Midterm exam
9. Maximum likelihood estimator, Maximum aposteriori estimator, Laplace estimator, Beta-binomial model, Dirichlet-multinomial model
10. Bayes decision rule for classification, Naïve Bayes classifier, Multivariate Gaussian Bayes model
11. Bayesian networks, Exact inference (belief propagation, variable elimination, clique trees), Monte Carlo inference (rejection sampling, importance sampling)
12. Gaussian mixture model for clustering, K-means algorithm, K-medoids algorithm
13. Confusion matrix-based performance measures (accuracy, precision, recall, sensitivity, F-score), Multiclass performance measures, Receiver operating characteristic (ROC) analysis, Resampling error estimation (cross-validation, nested cross-validation, leave-one-out), Statistical testing of classifier performance, Debugging classifier algorithms
14. Feature selection (filter methods, subset selection, wrapper method)
15. Final exam