Research database

NEURAL pioneers the integration of on-device AI technologies with human-centric inputs such as electroencephalogram (EEG) signals, voice, and silent speech (e.g., electromyography) to personalize SLMs. A central innovation of NEURAL lies in its ability to learn and adapt continuously. Traditional local assistants offer limited capabilities and are unable to evolve dynamically with user interactions. NEURAL overcomes these constraints by implementing novel continual learning methodologies. NEURAL stores conversations with the assistant, augmented by occasional remote queries to a cloud-based expert large language model (LLM), which will be used as ground-truth high-quality data for model adaptation. To process these data, advanced strategies will be used to prevent catastrophic forgetting while integrating new user-specific insights. NEURAL will be built as a distributed collaborative system consisting of three levels: i) on “far-edge” devices such as wearables, always-on triggers will be used to process high-frequency speech and silent speech signals efficiently to wake-up the NEURAL assistant; ii) the personalized SLM-based assistant will execute on a higher power edge device (smartphone or equivalent). Further, this platform will execute two additional software blocks: a) signal processing algorithms to extract meaningful latent representations of EEG/speech inputs; b) out-of-distribution and privacy-preserving detection module to generate the reply locally or ask for a remote LLM one; iii) on the cloud, an omniscient LLM that can process the query, while storing it and golden LLM reply for model adaptation over time. This architecture ensures all sensitive data are processed locally, calling the LLM only for non-sensitive data or if the user explicitly requests it. Scientifically, NEURAL significantly advances multimodal interaction, bio-signal analysis, edge AI, and continual learning, establishing a framework for developing bio-signal-powered personalized AI assistants. Even in isolation, all the contributions provide breakthroughs in their respective field. Further, the project's methodologies for deploying computationally intensive models onto resource-constrained hardware provide broadly applicable insights relevant to expanding the boundaries of edge AI. The societal implications of NEURAL are profound, facilitating broader accessibility and inclusion. By incorporating silent speech modality, the assistant accommodates users with speech impairments or those who do not want to speak in public, dramatically enhancing usability across diverse populations. Moreover, by fostering secure, personalized interactions, NEURAL aligns closely with European data protection standards, setting new benchmarks for ethical AI deployment. Economically, NEURAL’s innovations could stimulate significant growth in the IoT and wearable tech sectors, catalysing a shift toward decentralized, energy-efficient AI solutions.