Natural Language Processing

Data is displayed for academic year: 2023./2024.

Lecturers

Assoc. Prof.
Marina Bagić Babac
Prof.
Gordan Gledec

Course Description

Theoretical fundamentals of natural language processing. Language collections: dictionaries and corpora, markup systems. Learning from the corpus: learning new words, solving ambiguity problems, language models. Grammar: Hidden Markov models (HMM), context-independent grammar (CFG) and others. Application of grammatical models in corpus markup and parsing. Linguistic pre-processing and post-processing in text synthesis. The impact of natural language processing applications on social development and language change. Methods and tools for machine translation.

Study Programmes

University graduate
[FER3-HR] Audio Technologies and Electroacoustics - profile
Elective Courses (1. semester) (3. semester)
[FER3-HR] Communication and Space Technologies - profile
Elective Courses (3. semester)
[FER3-HR] Computational Modelling in Engineering - profile
Elective Courses (3. semester)
[FER3-HR] Computer Engineering - profile
Elective Courses (3. semester)
[FER3-HR] Computer Science - profile
Elective Courses (3. semester)
[FER3-HR] Control Systems and Robotics - profile
Elective Courses (3. semester)
[FER3-HR] Data Science - profile
Elective Courses (3. semester)
[FER3-HR] Electrical Power Engineering - profile
Elective Courses (3. semester)
[FER3-HR] Electric Machines, Drives and Automation - profile
Elective Courses (3. semester)
[FER3-HR] Electronic and Computer Engineering - profile
Elective Courses (3. semester)
[FER3-HR] Electronics - profile
Elective Courses (3. semester)
[FER3-HR] Information and Communication Engineering - profile
Elective Courses (3. semester)
[FER3-HR] Network Science - profile
Elective Courses (3. semester)
[FER3-HR] Software Engineering and Information Systems - profile
Elective Course of the profile (3. semester)
Elective Courses (3. semester)
[FER2-HR] Computer Engineering - profile
Recommended elective courses (3. semester)
[FER2-HR] Computer Science - profile
Recommended elective courses (3. semester)
[FER2-HR] Software Engineering and Information Systems - profile
Recommended elective courses (3. semester)
[FER2-HR] Telecommunication and Informatics - profile
Recommended elective courses (3. semester)

Learning Outcomes

  1. identify computational complexity of NLP problems
  2. evaluate open source NLP tools
  3. manipulate text and speech corpora
  4. participate in speech synthesis projects
  5. participate in speech recognition projects
  6. participate in machine translation projects

Forms of Teaching

Lectures

Alive lectures

Laboratory

Research Project Tasks

Grading Method

Continuous Assessment Exam
Type Threshold Percent of Grade Threshold Percent of Grade
Seminar/Project 0 % 40 % 0 % 40 %
Mid Term Exam: Written 0 % 30 % 0 %
Final Exam: Written 0 % 30 %
Exam: Written 50 % 60 %

Week by Week Schedule

  1. Language Models, Corpus methods, n-grams, colocations
  2. Computational morphology, computational semantics (formal semantics, semantic role labeling)
  3. Part of speech tagging, Hidden Markov Models
  4. Deterministic and stochastic grammars, constituency and dependency grammars (CFG, PCFG)
  5. Parsing algorithms (CYK, Chart), lexicalized parsing, dependency parsing
  6. Distributional semantic models and word embeddings, understanding and applying text embeddings
  7. Log-linear models, Parameter Estimation in Log-Linear Models
  8. Language models, smoothing, evaluation
  9. Midterm exam
  10. Neural Language Models, Large Language Models, Open Source Models
  11. Prompt Engineering
  12. Finetuning Large Language Models
  13. Agents, chained calls, and memories to expand use of LLMs. LLMs with Semantic Search. Retrieval Augmented generation (RAG).
  14. Evaluating and Debugging Generative AI Models, Quality and Safety for LLM Applications
  15. Final exam

Literature

(.), Daniel Jurafsky, James H. Martin (2019.), Speech and Language Processing (3nd edition), Prentice Hall,
(.), Christopher D. Manning, Hinrich Schütze (1999.), Foundations of Statistical Natural Language Processing, MIT Press,
(.), Ruslan Mitkov (ed.) (2005.), The Oxford Handbook of Computational Linguistics, Oxford University Press, USA,
(.), Steven Bird, Ewan Klein, Edward Loper (2009.), Natural Language Processing with Python, O'Reilly Media, Inc.,

For students

General

ID 222553
  Winter semester
5 ECTS
L0 English Level
L1 e-Learning
30 Lectures
0 Seminar
6 Exercises
15 Laboratory exercises
0 Project laboratory
0 Physical education excercises

Grading System

90 Excellent
80 Very Good
70 Good
50 Sufficient

Similar Courses