TIVOM

The project aims to develop a conductor support system that enables the inclusion of blind and visually impaired musicians within orchestras.

Building on a motion-tracking system that captures the conductor's movements, the project will:

• develop a wearable vibrotactile device that conveys the conductor's gestures to musicians through tactile feedback;
• integrate facial expression tracking to capture the conductor's expressive cues;
• develop a machine learning algorithm optimised for low-cost hardware platforms, such as Raspberry Pi, capable of recognising conducting gestures and facial expressions in real time;
• translate these recognised patterns into vibrotactile signals that enable musicians to receive the conductor's instructions with greater accuracy.

The proposed solution will be validated through experimental testing involving musicians with visual impairments, to demonstrate its effectiveness in real-world orchestral settings.

The project will make a significant contribution to research in the fields of multimodal human–computer interaction and machine learning for real-time applications on low-cost hardware platforms. It will also strengthen the connection between science and music by fostering interdisciplinary collaboration among electronics and telecommunications engineering, human–computer interaction, and ethics—fields that have had limited opportunities to work together to date.

In addition, the project will raise awareness among artists, educators, and the wider public of the artistic and social implications of Internet of Musical Things (IoMusT) technologies, with particular attention to the needs of blind and visually impaired musicians. The proposed technology will promote inclusion and equal access to musical activities while supporting collaborative music-making and music education for people of all ages.

Finally, the project offers strong potential for technology transfer through the combination of a low-cost wearable prototype and machine learning algorithms optimised for embedded platforms such as Raspberry Pi.

Project’s partner

• Conservatorio Guido Cantelli di Novara
• Conservatorio Giuseppe Verdi di Como
• Coro del Politecnico di Torino
• Coro Jubilate di Legnano

Project Team

• Cristina Rottondi (DET), PA
• Antonio Servetti (DAUIN), Senior researcher
• Matteo Sacchetto (DET), Research fellow