Bridging Simulation and Real Data with Dynamic SCENIC Programs


At a glance

Sensor realism is one of the most fundamental challenges for the practical use of simulation in autonomous vehicle development and testing. There is a pressing need for techniques to efficiently validate simulation data and results against real-world datasets and testing. In this proposed research, we will develop a novel data-driven approach to automatically validate simulation results against real time-series data. We plan to develop an approach to query a set of labelled, real-world videos of any sensor type (e.g. 2D RGB, 3D LiDAR point cloud, and depth-camera) with a program modeling an autonomous driving scenario in the SCENIC programming language. The query outputs a subset of the videos that are members of the scenario represented by that SCENIC program. The system under evaluation can then be tested on queried real-world videos for validation. If the query returns none or too small subset to properly validate, then we have preventatively identified a failure scenario that is barely encountered on roads so far, providing directed guidance for road testing. Through this approach, we plan to bridge the gap between simulated, synthetic, corner-case data and real-world time-series datasets.

principal investigatorsresearchersthemes

Sanjit Seshia

Alberto Sangiovanni-Vincentelli


Perception, Simulation, Verification, Scenario Specification, Datasets, Querying, Synthetic and Real Data

This project is a continuation of: "Bridging Simulation and Real-World Data with SCENIC Queries".