Cloud Data Engineering Technologies

Course Description

Students will learn the basic processes of data engineering projects, such as data entry, storage, transfer, transformation, cleaning, validation and transfer. Also source code management procedures, serverless programming environments and continuous code integration and delivery procedures which are essential on today’s cloud data engineering projects will be addressed. Through assignments students will gain practical skills in working with the GO programming language, Docker containers and the Kubernetes cluster, MongoDB NoSQL database, GIT Jankins and Terraform.

Forms of Teaching

Lectures

Lectures will be held in 7 cycles lasting 2 hours per week in laboratory classrooms at the faculty.

Laboratory

Laboratory exercises will be held in 7 cycles of 2 hours per week in the laboratory classrooms at the Faculty.

Grading Method

Continuous Assessment Exam
Type Threshold Percent of Grade Threshold Percent of Grade
Laboratory Exercises 50 % 40 % 50 % 40 %
Seminar/Project 50 % 40 % 50 % 40 %
Attendance 50 % 20 % 50 % 20 %

Week by Week Schedule

  1. What is a data platform, cloud platforms and building data pipelines
  2. Data sources, transfer of local data to the cloud, retrieval from relational databases, program interfaces and public sources, data ingestion
  3. Using Pub / sub agent for message forwarding and Auro message format, metadata and data origin
  4. Data consolidation and transformation, message schema control, schema catalog construction and how to solve the problem of message schema evolution
  5. Using software containers, managing the Kubernetes cluster, and using the MongoDB NoSQL database
  6. Development of programming interfaces in GO programming language (REST API, Swagger, oAuth authentication), serving through message forwarding agents, data democratization
  7. Code development applications aligned with the DevOps concept, using GIT, Jankins, Terraform, establishing development, test, presentation and production environments in the cloud
  8. Final project

Study Programmes

University undergraduate
Skills (6. semester)
Skills (6. semester)
[FER2-HR] Computer Engineering - module
Skills (6. semester)
[FER2-HR] Computer Science - module
Skills (6. semester)
[FER2-HR] Control Engineering and Automation - module
Skills (6. semester)
[FER2-HR] Electrical Power Engineering - module
Skills (6. semester)
[FER2-HR] Electronic and Computer Engineering - module
Skills (6. semester)
[FER2-HR] Electronics - module
Skills (6. semester)
[FER2-HR] Information Processing - module
Skills (6. semester)
[FER2-HR] Software Engineering and Information Systems - module
Skills (6. semester)
[FER2-HR] Telecommunication and Informatics - module
Skills (6. semester)
[FER2-HR] Wireless Technologies - module
Skills (6. semester)
University graduate
[FER3-HR] Computing - study
Skills (2. semester)
[FER3-HR] Electrical Engineering and Information Technology - study
Skills (2. semester)
[FER3-HR] Information and Communication Technology - study
Skills (2. semester)
[FER2-HR] Computer Engineering - profile
Skills (2. semester)
[FER2-HR] Computer Science - profile
Skills (2. semester)
[FER2-HR] Control Engineering and Automation - profile
Skills (2. semester)
[FER2-HR] Electrical Engineering Systems and Technologies - profile
Skills (2. semester)
[FER2-HR] Electrical Power Engineering - profile
Skills (2. semester)
[FER2-HR] Electronic and Computer Engineering - profile
Skills (2. semester)
[FER2-HR] Electronics - profile
Skills (2. semester)
[FER2-HR] Information Processing - profile
Skills (2. semester)
[FER2-HR] Software Engineering and Information Systems - profile
Skills (2. semester)
[FER2-HR] Telecommunication and Informatics - profile
Skills (2. semester)
[FER2-HR] Wireless Technologies - profile
Skills (2. semester)

Literature

Martin Kleppmann (2017.), Designing Data-intensive Applications, Oreilly & Associates Incorporated
John Arundel, Justin Domingus (2019.), Cloud Native DevOps with Kubernetes, O'Reilly Media
Manoj Kukreja, Danil Zburivsky (2021.), Data Engineering with Apache Spark, Delta Lake, and Lakehouse, Packt Publishing Ltd
Alan A. A. Donovan, Brian W. Kernighan (2015.), The Go Programming Language, Addison-Wesley Professional

For students

General

ID 234065
  Summer semester
3 ECTS
L2 English Level
L1 e-Learning
15 Lectures
0 Seminar
0 Exercises
15 Laboratory exercises
0 Project laboratory

Grading System

90 Excellent
80 Very Good
65 Good
50 Acceptable