Improving FND-CLIP for Multimodal Italian Fake News Detection

With the proliferation of multimodal data sources, detecting fake news content has become more and more challenging as fake content can be hidden in either visual or textual news elements. Our work addresses the automatic detection of Italian fake news in a multimodal setting, where both the textual and the visual components potentially contribute as sources of fake content. We propose FND-CLIP-IT, an extension of the state-of-the-art FND-CLIP multimodal architecture, exploring the integration of several additional components, such as a sentiment-based text encoder, a data augmentation stage based on back-translation, an image transformation module based on Discrete Fourier Transform, as well as different approaches to combine and weigh the input embeddings. Thanks to its effectiveness in combining visual and textual content, our solution contributes to fighting the spread of disinformation in the Italian news flow.

Presenter: Lorenzo Vaiani

Biography

Lorenzo Vaiani is a Ph.D. student in Computer Engineering at Politecnico, and he got a Master’s Degree in Computer Engineering from the Politecnico in 2021. His research focuses on multimodal learning, especially in the vision-language domain. He is currently working on applying deep learning models to social-media multimodal data to address problems such as hate speech and fake news detection.

Event organised by SmartData@PoliTO center.