Research database

This project sets out with the goal of creating an inclusive Extended Reality (XR) platform for networked musical interactions in the Metaverse, to achieve novel forms of musical expression and foster music composition, performance, and education. The Metaverse relates to a vision of a virtual, digital world which is parallel to the real, physical world, where each user owns and interacts through his/her own avatar. The "Musical Metaverse" (MM), the Metaverse part dedicated to musical activities, is currently in its infancy. As of today an investigation about the technical, artistic and inclusivity-related opportunities and challenges posed by the MM has not been conducted yet. Moreover, current XR platforms are unable to effectively support real-time networked musical interactions between geographically displaced musicians due to unsolved technical issues, such as the guarantee of communication latencies of less than 30 ms. Furthermore, only a few studies have investigated collaborative music making in XR, and little is known about the influence of diversity factors in MM interactions, such as gender or disability. Investigations on all these topics are crucial for enabling MM ecosystems that musical stakeholders can utilize in efficient, creative and responsible ways. Given the timeliness and expected importance of the MM, such investigations are useful to provide developers with guidelines for creating better musical XR environments and interactions between stakeholders. The project aims to achieve three interdependent objectives: 1) To gain a rich understanding of musicians’ needs for different musical activities in the MM, and design with them methods to satisfy their needs; 2) To develop new fundamental methods for the synchronous interconnection of musicians within collaborative XR environments; 3) To explore and empirically study new services for musicians exploiting the developed methods. The project adopts a methodology consisting of an iterative paradigm based on 3 partially overlapping phases, Design-Develop-Evaluate. The phases are iterated twice, aiming to progressively increase the technology readiness level and achieve a tangible proof-of-concept of the envisioned technology. The focus will be on classical, pop, rock, and experimental music, which are genres selected for covering a wide and heterogeneous scope in terms of music creation and consumption. Moreover, three musical activities will be investigated, namely composition, performance and education. The project brings together 2 technical universities and 2 musical institutions with complementary skills in different fields: music, human-computer interaction, XR, networking, machine learning, sociology, ethics, and innovation. Such interdisciplinary approach is necessary to address the project vision and respond holistically and critically to its significant challenges. The project is a high-impact endeavour from which basic science, art, industry and society can benefit.