Interaction and Communication Between Pedestrians and Autonomous Vehicles

ABOUT THE PROJECT

At a glance

Motivation and Research Objectives
Human Factors (HF) challenges need to be resolved prior to the deployment of Automated Vehicles (AVs) on public roads. HF research plays an important role in the process of the technological development. This significant area of research helps define the roles of human driver/passenger and the machine/automated vehicle. One typical research question is about how and when to transfer control between human and machines for semi-autonomous vehicles, as well as whether human drivers should be out of the loop for highly and fully autonomous vehicles1. Another type of research question is related to the communication between AVs, other vehicles and vulnerable road users, such as pedestrians and cyclists. This line of research agenda is aimed to (1) determine how other road users will be interacting with AVs, (2) ensure public acceptance and trust in AVs, and (3) confirm that pedestrians and other road users are aware of the automated systems’ capabilities and limitations. AVs will only be acceptable and deployable if they are proven to be communicative and safe for all the other road users.
In this project, we propose to explore the interaction between AVs and pedestrians in an urban driving environment. The objectives of this project are partitioned into three phases. They are depicted in Figure 1.

Figure 1. Research Objectives of This Project

• The first objective of this project is to identify and abstract the scenarios when vehicles and pedestrians interact with each other. Interaction patterns between human-driven vehicles and pedestrians may vary due to many factors, such as: (a) regulation difference (e.g., rights of way at both signalized and un-signalized intersections); (b) regional differences resulting from local culture (e.g., high density of population) and (c) situations with pedestrians violating traffic regulations because of improper infrastructure design or road construction. This phase will be conducted in conjunction with an existing BDD project: Pedestrian-Vehicle Interaction – Data Sets and Analysis, https://deepdrive.berkeley.edu/project/pedestrian-vehicle-interaction-%E2%80%93-data-sets-and-analysis
• The second objective of this project is to design different modalities of interaction interfaces with which AVs can effectively communicate with pedestrians in the vicinity. Exemplar considerations are: (a) how to initiate the communication; (b) how to handle and disambiguate error situations; and (c) what types of feedback from pedestrians are critical for AV’s perception and trajectory planning in specific scenarios.
• The third objective of this project is to conduct testing and compare different interaction and interface designs and to evaluate pedestrians’ vision, perception, and comprehension regarding different designs. The goal is to produce simple and effective interface design concepts.

 Technical Approaches
A schematic diagram for our proposed approach is shown in Figure 2.
In the 1st phase of the study, we will identify the critical scenarios, in which vehicles and pedestrians interact with each other. Through combined efforts with another BDD project, video, and image analysis will be conducted to extract both pedestrian behavior and vehicle trajectory. A unified time axis with a local geographic coordinate system is set up for each data set. All the data that belong to same data set will be mapped to this time-geography space. Pedestrian behavior will be defined as a series of basic action elements, including walk forward, step back, stop, speed up, slow down, turn left, turn right, head turn left, head turn right, wave hand and so on. These actions are recognized directly or deduced from trajectories.

Figure 2. Schematic Diagram of This Project

In the 2nd phase of the study, interaction flows and multi-modal interfaces will be designed for communication between AVs and pedestrians, using symbols, texts, headlights, or audio messages. Detailed graphical and auditory presentation will be designed to convey the messages in those various scenarios identified in the 1st phase.
In the 3rd phase of the study, we will conduct field testing in the Richmond Field Station and evaluate the interaction flow design as well as the interface design. The Wizard of OZ method will be used to collect data in the field, in which an experimenter will be sitting on the back seat of the car, recognize the intention of the pedestrian, present the pre-defined messages through the AV display. Auditory messages will be prompted through external speakers. Text and symbol messages will be displayed through external LED displays. At the same time, pedestrians will be asked to wear an eye tracking device in order to obtain their eye movements and gaze behavior. Pedestrians head and body movement as well as behavior data while interacting with the car will also be recorded by video cameras. The BDD experimental vehicles will be used.
Expected Outcome
The project is expected to generate the following outcome to add to the BDD depository: (1) A set of safety critical scenarios with vehicle and road users’ interactions; (2) A set of field testing data capturing vision and behaviors of pedestrians while interacting with AVs; and (3) Recommendation and design concepts for interaction and interface design for communication between AVs and pedestrians.
 

This project laid the groundwork for the current project:  Implicit Communication of Automated Vehicles: Using Vehicle Movement Cues for Coordination with Pedestrians at Intersections