Learning Automata

Course Description

Fundamentals and definitions of machine learning. State and action probabilities, and standard patterns of machine learning behaviour. Mathematical models of neuron and neural networks, diversity and properties of learning alorithms. Claster analisys, knowledge representation, compression and representation from uncomplete data. Basics of associative memories. Recursive networks and processing of time dependent patterns. Learning algorithms: Back-propagation with extensions, Kohonen and Cascade correlation algorithm. WTA (Winner-Take-All). Feature extraction by competitive learning - interactive or on-line learning, Sarsa and Q-learning algorithm, Markov models. Intelligent agents and selfadaptive control of communication services.

General Competencies

After completing this course students will be able to analyse and design learning automata based solutions, and to select suitable encodings and fitness functions for specific information processing and communication tasks. Students will be able to design neural network and develop reinforcement learning techniques by using simulation tools and Java programming.

Learning Outcomes

  1. to define concept, methods and architectures typical for machine learnings
  2. to explain how machine learning operate and basic purpose
  3. to apply knowledge about machine learning for telecommunication services
  4. to analyze functions of machine learned components, as well as their interactions in order to find appropriate solution
  5. to analyze organization of machine learned model
  6. to define basic components for realisation of needed function by machine learning
  7. to create machine learned models including various types of recognition and self-adaptation
  8. to evaluate and assess solutions based on different methods of machine learning

Forms of Teaching


Lectures, with lecture notes and presentations available in advance on the web.


Midterm exam and final exam.

Laboratory Work

Complex laboratory assignments that include building machine learning models, defining learning and testing parameters, learning and testing procedures, and explanations of the results.


Regular consultations hours with lecturer, four tems every week.

Acquisition of Skills

Literature search on machine learning models and methods. Building software environment for machine learning design and analysis.

Programming Exercises

Modelling and testing using software tools for machine learning.

Other Forms of Group and Self Study

Home works related to case studies for machine learning models.

Grading Method

Continuous Assessment Exam
Type Threshold Percent of Grade Threshold Percent of Grade
Laboratory Exercises 50 % 20 % 10 % 20 %
Homeworks 50 % 10 % 5 % 10 %
Class participation 0 % 5 % 0 % 0 %
Attendance 0 % 5 % 0 % 0 %
Mid Term Exam: Written 50 % 20 % 0 %
Final Exam: Written 50 % 20 %
Final Exam: Oral 20 %
Exam: Written 50 % 40 %
Exam: Oral 30 %

All laboratory assignements should be completed succefully.

Week by Week Schedule

  1. Definitions of knowledge and intelligence. Application area and fundamental properties of machine learning. Markov chains and learning automata. Deterministic and stochastic models. State and action probabilities and standard pattern of learning model behavior. Learning models and methods classification. Knowledge based models.
  2. Natural and artificial model of neural network. Neuron model and network architectures, one layer and multilayer networks. Feed-forward and feedback networks. Random connectivity.
  3. Supervised learning. Gradient descent methods, problem of internal representation - analyse and network topology adaptation.
  4. Clusters and cluster analysis. Metrics and distances, examples. Learning of one layer networks. Hebb and Widrow-hoff learning algorithms.
  5. Learning of multilayer networks. Learning internal representations by error backpropagation – BP algorithm. Selection of learning parameters, generalization capability. Case studies based on telecommunication traffic control.
  6. Variation of BP algorithm. Learning with momentum and learning in time. Time delay neural network and “Backpropagation through time” algorithm.
  7. Achiving of optimal topology structure - criteria, methods, algorithms. Activation function selection based on cluster structure. Cascade correlation algorithm.
  8. Midterm exam.
  9. Associativity, memory, compression and reading from data with gaps. Learning algorithms, topologies and capacities.
  10. Williams and Zipser algorithm for real time learning of recurrent neural networks. Elman and Jordan networks.
  11. Pre-processing data for neural networks. Selection of learning, testing and validation data sets. Overfiting prevention, and methods for improving generalization. Colored Petri nets, model components and properties.
  12. Unsupervised learning, goals and applications. Cluster, dimensionality reduction, topografic mapping, hidden data. Self-organizing maps. Variants of Kohonens neural networks and learning algorithms. Winner take all method. Case studies from telecommunication services area. Coloured Petri nets - multisets and occurence graph.
  13. Fundamentals of reinforcement learning. Bellman equations. The model of intelligent agent, agent environment, determination of goals and rewards. Markov property and Markov decision processes. Protocol and communication process modelling by coloured petri nets.
  14. Knowledge discovery in databases. Overview of decision trees and decision tree construction. Decision tree rules and prunning. ID3, C4.5 and C5 algorithms. Problem task.
  15. Final exam.

Study Programmes

University graduate
Telecommunication and Informatics (profile)
Specialization Course (1. semester) (3. semester)


(.), Reinforcement Learning:An Introduction Richard S. Sutton, Andrew G. Barto MIT Press, Cambridge 1998,
(.), Neurodynamic programming Dimitri P. Bertsekas, John Tsitsiklis Athena Scientific, MIT 1996,
(.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition David E. Rumelhart MIT Press 1987,
(.), Digitalni automati Mladen Tkalić Liber 1991,
(.), Learning automata - An Introduction K.S. Narendra, M.A.L. Thathachar Prentice Hall 1989,

Associate Lecturers

Laboratory exercises


ID 86446
  Winter semester
L1 English Level
L1 e-Learning
30 Lectures
0 Exercises
15 Laboratory exercises
0 Project laboratory

Grading System

85 Excellent
70 Very Good
60 Good
50 Acceptable