Pedestrian-Vehicle Interaction – Data Sets and Analysis

The research work described herein, titled ‘Pedestrian-Vehicle Interaction – Data Sets and Analysis,’ is conducted as a core research activity under the Infrastructure project.

Predicting pedestrian intents and understanding the interactions between pedestrians and vehicles are critical for autonomous vehicles (AV) to act in a safe, efficient, and smart manner. This project is aimed at collecting a meaningful video data set that capture pedestrian and vehicle movements and preforming analysis to gain insights for modeling the interaction behaviors and enabling implementation of intelligent AV driving policy.

In this study, we explore the use of three different data sets. The BDD (NEXAR) video data set, which is captured by an app running dashboard-mounted phones, will also be explored. These videos and images provide a perspective of pedestrian behaviors, from the inside of a vehicle. One other data set is from unmarked mid-block pedestrian crossings. It is captured near the north gate of UC Berkeley campus where pedestrians often spontaneously walk across. Another data set is acquired from the City of Ningbo in China, where data from selective signalized intersections have been obtained. Even though the crossing is under signal intervention, many mixed-traffic interactive behaviors can be observed among pedestrians, motorcycles and vehicles. The diversity of viewing angles and traffic environment from the three aforementioned data sets offers this study a rich source for in-depth investigation.

Using dense optical flow and pixel intensity information, algorithms are trained to segment each input video frame at the pixel level three object classes including pedestrians, background (road, building) and foreground (tree branches, leaves), and vehicles. The analysis of these data sets can provide a foundation for the modeling of pedestrian-vehicle interactive behaviors. Moreover, since the videos contain data from different settings and environments, they will allow us to evaluate the cultural and regional impacts on pedestrian-vehicle interactions and their implications.