Model-Based Testing meets Large Language Models: Automating Intelligent Scenario Generation and Safety Assessment with ScenicNL

ABOUT THIS PROJECT

At a glance

Scenic is a probabilistic programming language for modeling intelligent autonomous systems and the worlds in which they operate. Scenic and its associated open-source toolset can be used for a variety of use cases, including testing/verification, safety/risk assessment, synthetic data generation, etc. The goal of the proposed project is to reliably generate Scenic code from natural language descriptions, including real-world crash/incident reports. This would enable much easier applications of Scenic including automated testing of AVs in diverse safety-critical scenarios, systematic data generation, and design space exploration. The proposed Scenic Natural Language AI System (ScenicNL) will leverage a variety of LLMs, compositional prompting strategies, and feedback from existing tools to generate Scenic programs that represent natural language descriptions obtained from real-world sources.