EAM-ECDG: Enhancing Autonomous Mobility through Edge Case Adaptation ans Domain Generalization

The EAM-ECDG project epitomizes an innovative AI approach to significantly advance autonomous mobility by addressing the critical challenge of unpredictable edge cases. By tightly integrating Large Language Models (LLMs) with advanced decision-making and reinforcement learning (RL) principles, the project aims to develop a robust agent capable of human-like reasoning in rare and complex traffic situations. The project has three primary objectives:

• Develop an advanced autonomous driving agent: This model combines LLMs, RL paradigms, and robust sensor fusion into a unified agent. It reasons about current and future environmental states, as well as the intentions of other traffic participants, ensuring safe and efficient decision-making even in highly atypical corner cases.
• Introduce a novel state prediction module: Leveraging the theoretical equivalence between autoregressive attention mechanisms and non-linear state estimators, this module accurately forecasts future scenarios. It addresses challenges such as LLM hallucinations by deeply grounding predictions in multimodal observation and learning through human interaction.
• Implement a prioritized communication framework: Deployed on 5G edge servers, this system allows the autonomous agent to seamlessly exchange critical messages and prioritize dynamic traffic data (such as compressed delta-frames) with infrastructure and other vehicles, ensuring ultra-reliable Vehicle-to-Infrastructure (V2I) communication.

Funded by the European Union – NextGenerationEU through the Recovery and Resilience Plan for Slovakia under the project No. 09I03-03-V04-00395 (Zlepšenie autonómnej mobility prostredníctvom adaptácie okrajových prípadov a zovšeobecnenia domén).