Stefano Calzolari

Stefano Calzolari's picture

Ph.D. candidate in Ingegneria Informatica E Dei Sistemi , 39th cycle (2023-2026)
Department of Control and Computer Engineering (DAUIN)

Docente esterno e/o collaboratore didattico
Department of Control and Computer Engineering (DAUIN)

Profile

PhD

Research topic

Emotions aware Embodied Conversational Agents (E2CA)

Tutors

Research interests

Computer graphics and Multimedia
Data science, Computer vision and AI
Software engineering and Mobile computing

Biography

I obtained a MSc degree in Cinema and Media Engineering (LM-32) in April 2023 and I started my PhD career in Computer and Control Engineering in November of the same year. My research group is the Computer Graphics & Vision Group (CG&VG) which mainly operates in the fields of Augmented, Mixed and Virtual Reality, Human-Computer Interaction (HMI), User eXperience (UX), Computer Vision, Machine Learning and Artificial Intelligence.

My research topic specifically focuses on the Embodied Conversational Agents (ECAs) are virtual characters capable of simulating a human-like conversation using natural language processing and multimodal interaction. ECAs can be used in a plethora of applications from education to training, healthcare, virtual assistants and virtual companions. In many of these fields, improving ECAs effectiveness and realism requires to make them capable of expressing believable emotions and leverage the analysis of human affects to create a strong empathic bond with the end-users.

The aim of my research is twofold.
The first objective is to develop a framework for prototyping Embodied Conversational Agents (ECAs) for Extended Reality (AR/VR) applications, enabling the integration of the key components necessary for human-like simulation:
  • Animation: leveraging Motion Matching and Inverse Kinematics (IK) techniques
  • Emotions: utilizing models such as Pleasure-Arousal-Dominance (PAD) or the Circumplex Model
  • Speech: employing Speech-To-Text (STT), Text-To-Speech (TTS), and lip-sync technologies
  • Behavior: decomposing the agent's behavior into atomic and composable Actions using classical systems like Finite-State-Machine (FSM), Behavioral Trees (BT), or Goal Oriented Action Planning (GOAP), and incorporating Large Language Models (LLMs) for reasoning and Action planning through modern Artificial Intelligence (AI) techniques.
The second objective is to evaluate how these systems can enhance the “Believability” of the agents as perceived by users during interaction. The main tools used for this evaluation include UX questionnaires and user-agent interaction data such as interaction frequency, user trajectory data, and eye-tracking data.

Teaching

Teachings

Master of Science

Bachelor of Science

  • Informatica. A.A. 2024/25, INGEGNERIA AEROSPAZIALE. Collaboratore del corso

Research

Research groups

Publications

Latest publications View all publications in Porto@Iris