3D Scene Understanding: from 3D point-based to graph-based representations of the world

A new Ellis Turin Talk will be held on Thursday, 7 December 2023, titled "3D Scene Understanding: from 3D point-based
to graph-based representations of the world"

Autonomous systems must understand the 3D spatial layout world they navigate and interact with. To fully operate in the wild, a fundamental step is to build representations of the 3D world that are reliable and human-interpretable. In this lecture, I will provide a walkthrough on recent advancements in generating 3D models of the world that are semantically meaningful and that can be used to solve high-level tasks. We will first provide fundamentals on 3D geometry and how it is possible to localize objects in multi-view by using structure from motion and multi-view geometry principles. Such geometrical reasoning can then be used to generate a real-world representation using 3D scene graphs computed with Graph Neural Networks that encode both geometric structure and visual appearance of the objects present in the scene. Finally, we will demonstrate how these models can be effective for several tasks such as camera re-localisation, SLAM, active visual search and Visual Question and Answering.

Speaker: Alessio Del Bue - Researcher leading the PAVIS (Pattern Analysis and computer VISion) research line of the Italian Institute of Technology in Genova.