Interactive Simulation Systems

Data is displayed for academic year: 2023./2024.

Lecturers

Assoc. Prof.
Siniša Popović

Course Description

The course introduces students to the field of interactive simulation in real time using the virtual and augmented reality technologies. Students are acquainted with examples of interactive simulation systems in real time, such as different types of training simulators, and specific requirements on architecture and performance of these systems are emphasized. The course underlines the importance of parallelization of complex simulation problems through decomposition and allocation of tasks, as well as the importance of system performance optimization due to interaction with the user in real time. Methods and tools for interactive simulation based on virtual and augmented reality are analyzed, with an emphasis on particularities of graphical modeling and rendering of the virtual and augmented world. Diverse multimodal input/output interfaces are presented, which enable complete and effective real-time interaction with the user. The course also presents various mathematical models to simulate the kinematics and dynamics of moving objects in real time within the virtual and augmented world. Particular emphasis is placed on the importance of assessing the human factor impact in interactive simulation systems for training related to various professional stressful occupations, like special forces, intervention police, first responders. An overview of methods and tools is given for modeling different multimodal emotional stressors, including visual and auditory stimuli generators, computerized cognitive tasks and serious games, with simultaneous analyses of relevant psychophysiological features extracted from heart rate, skin conductance, respiration, voice, dynamics of the oculomotor system, activity of the prefrontal cortex and related regions etc.

Study Programmes

University undergraduate
[FER3-EN] Computing - study
Elective Courses (5. semester)
[FER3-EN] Electrical Engineering and Information Technology - study
Elective Courses (5. semester)

Learning Outcomes

  1. Explain main conceptual elements of interactive simulation systems
  2. Explain the concept of real time and recognize latency sources and problems in the context of interactive simulation systems
  3. Define various types of input-output devices in interactive simulation systems based on virtual reality
  4. Describe most common representations of terrains and 3D objects, as well as major visualization stages in interactive stimulation systems
  5. Explain the importance of mathematical modeling and the application of numerical methods in interactive simulation systems
  6. Describe the importance of interactive simulation systems in training for stressful tasks
  7. Outline examples of methods and tools for modeling different multimodal emotional stressors
  8. Explain the assessment of human factor impact in interactive simulation systems for training related to various professional stressful occupations

Forms of Teaching

Lectures

There are 15 weeks of classes. First cycle consists of 7 weeks of lectures, followed by midterm, while second cycle consists of 6 weeks of lectures, followed by final exam.

Independent assignments

Laboratory exercises and students' independent work in consultation with the course instructors includes development of limited functionalities of interactive simulation systems or serious games, as well as measurement and data analysis related to human factor in interactive simulations. Writing up and presenting the report on the conducted work is also expected.

Laboratory

Laboratory exercises and students' independent work in consultation with the course instructors includes development of limited functionalities of interactive simulation systems or serious games, as well as measurement and data analysis related to human factor in interactive simulations. Writing up and presenting the report on the conducted work is also expected.

Other

Possible demonstrations of the developed interactive simulation systems, serious games and systems for generating laboratory stress-inducing stimuli and analyses of multimodal psychophysiological reactions. Possible organization of visits to companies working in the field of interactive simulation systems.

Grading Method

Continuous Assessment Exam
Type Threshold Percent of Grade Threshold Percent of Grade
Class participation 0 % 10 % 0 % 10 %
Seminar/Project 0 % 50 % 0 % 50 %
Mid Term Exam: Written 0 % 20 % 0 %
Final Exam: Written 0 % 20 %
Exam: Written 0 % 40 %

Week by Week Schedule

  1. Cost-effectiveness and safety of human training, Driving simulators; Simulators of civilian and military airplanes; Missile system simulators
  2. Driving simulators; Simulators of civilian and military airplanes; Missile system simulators, Main elements and system architecture
  3. Main elements and system architecture, Simulation loop, real-time constraints and concurrency, Human factor issues associated with latency in interactive simulation systems
  4. Educational use-cases for demonstration of interactive simulation concepts
  5. Head orientation and position tracking (acoustic; Magnetic; Mechanical; Optical; Inertial), Tracking other body parts (eye tracking; Hand tracking; Full-body tracking)
  6. Tracking other body parts (eye tracking; Hand tracking; Full-body tracking), Miscellaneous input interfaces (physiological signals; Speech; Facial expressions; Gestures; Locomotion), Simulator-specific input interfaces, Visual channel (head-mounted display; CAVE; Dome; Multi-projector screens)
  7. Visual channel (head-mounted display; CAVE; Dome; Multi-projector screens), Auditory, motion and tactile channels, Simulator-specific output interfaces
  8. Midterm exam
  9. Basic concepts in 3D computer graphics modeling (points; Polygons; Normals; Coordinate systems; Transformations; Color; Material; Light; Texture), 3D model representations for interactive simulation systems (polygon mesh; Scene graph)
  10. 3D model representations for interactive simulation systems (polygon mesh; Scene graph), Terrain representation as digital elevation model and triangular irregular network
  11. Basic overview of graphics rendering, Basic optimizations of graphics rendering in interactive simulation systems (levels of detail; Culling), 3D modeling and graphics rendering software tools for interactive simulation systems
  12. Mathematical modeling using ordinary differential equations (ODE), Analytical versus numerical solving of ODE model, Basic numerical methods for ODE solving (forward Euler; Midpoint method)
  13. Elicitation of stress and emotions, Measurement and estimation of stress and emotions from physiological features, Measurement and estimation of stress and emotions from vocal features
  14. Measurement and estimation of stress and emotions from facial features, Interactive computerized cognitive testing, Biofeedback; Serious games with multimodal feedback for therapy and rehabilitation
  15. Final exam

Literature

(.), Nastavni materijali s predavanja, koji će biti dostupni na internetskoj stranici predmeta,
Igor S. Pandžić (2004.), Virtualna okruženja : računalna grafika u stvarnom vremenu i njene primjene, Element
Grigore Burdea, Philippe Coiffet (2003.), Virtual Reality Technology, John Wiley & Sons Inc.
Nathaniel I. Durlach, Anne S. Mavor (1995.), Virtual Reality: Scientific and Technological Challenges, National Academies Press
James A. Coan, John J. B. Allen (2007.), The Handbook of Emotion Elicitation and Assessment, Oxford University Press
Stuart J. Russell, Peter Norvig (2002.), Artificial Intelligence: A Modern Approach (2nd edition), Prentice Hall

For students

General

ID 223351
  Winter semester
4 ECTS
L1 English Level
L1 e-Learning
30 Lectures
0 Seminar
0 Exercises
6 Laboratory exercises
0 Project laboratory
0 Physical education excercises

Grading System

90 Excellent
75 Very Good
60 Good
50 Sufficient

Similar Courses