These CIFRE: Augmented spatio-temporal perception of complex environments for autonomous robotics

Environment

This PhD is a collaboration between Inria’s ACENTAURI research team and NXP Semiconductors’ Vision Technology Engineering Center (VTEC).

ACENTAURI focuses on intelligent, autonomous, and mobile robotics, with expertise spanning perception, decision‑making, and multi‑robot collaboration. The team develops hybrid AI approaches that combine model‑based and data‑driven methods, validated on real robotic platforms such as autonomous cars, AGVs, and drones. Their work targets smart territories, smart cities, and smart factories, emphasizing robust multi‑sensor cooperation and strong industrial transfer.

NXP Semiconductors designs the processors that power next‑generation embedded intelligent systems, ensuring they are safe, secure, fast, and reliable. Future autonomous vehicles, robots, drones, and mobile devices will rely on NXP neural processing units (such as the eIQ Neutron NPU) to achieve high‑performance inference. The VTEC team in Sophia Antipolis develops the software ecosystem that enables efficient vision pipelines on NXP hardware. To push technological limits, NXP designs optimized AI architectures tailored to customer needs and NXP processors.

This PhD sits at the intersection of advanced robotics, multi‑sensor perception, and efficient AI architectures, contributing jointly to scientific research and industrial innovation.

Motivation and Objectives

Robotic systems are becoming increasingly complex, involving multiple cooperating robots and heterogeneous sensors operating across large, dynamic environments. Achieving optimal task execution requires maintaining a global, time‑evolving representation of the environment and extracting from it a compact, task‑specific representation usable for real‑time decision‑making.

The objective of this PhD is to design and implement a multi‑layer, large‑scale environment representation for robotics, integrating geometry, appearance, and semantic information from stereo vision and LiDAR sensors. Existing approaches typically address only small‑scale areas and rely solely on vision; this research aims to extend them to large dynamic environments.

Building on recent advances such as 3D scene graphs (e.g., MIT’s Kimera/Hydra), the work will:

• Define the layers and structure of the multi‑sensor, multi‑level representation.
• Develop efficient tools to build, maintain, and query these representations.
• Leverage graph‑based methods for tasks like SLAM, exploration, navigation, and place recognition.
• Explore hybrid AI approaches that bridge rule‑based methods with data‑driven neural architectures.
• Validate the system through real‑world experiments using ACENTAURI’s instrumented robots.

The final framework will be implemented in C/C++ under ROS2 and evaluated through both simulation and real datasets.

More information about NXP in France...

#LI-8e4d