Bridging Simulation and Real-World Data with SCENIC Queries


At a glance


Sanjit A. Seshia

Alberto Sangiovanni-Vincentelli


Perception, Simulation, Verification, Scenario Specification, Datasets, Querying

We propose a method for developers to automatically validate failure scenarios, which are identified in simulation, in reality. We will initially focus on testing and validating a given perception module on visual/LiDAR images. Suppose a developer found a failure scenario for a given perception module in simulation. This scenario is characterized using synthetic image scenes from the simulator. To check whether the perception module also fails in reality, we propose to develop a technique to query a large labeled, real-world dataset of images, which companies mostly possess nowadays, and find a subset of the dataset which belongs to the failure scenario. Then, we test the perception module on this subset to validate whether failures in simulation in a certain scenario holds true in reality. Through this approach, we plan to bridge the gap between simulated, synthetic, corner-case data and real-world datasets.