Human Action and Intention Prediction in Urban Environments

ABOUT THE PROJECT

At a glance

The deployment of Automated Vehicle (AV) to transportation is motivated by their predicted beneficial outcomes such as safety and efficiency improvements. In the foreseeable future, drivers will still be engaged in driving tasks but with increasing levels of automation added. Therefore, in these partially automated vehicles, driver’s physical and emotional states should be monitored by the AV to better understand driver intention and to be able to respond appropriately. This is especially critical in automation of Level 3, per SAE standard J3016 , which needs the driver to take control back in certain situations, while she or he may be carrying out secondary tasks and attention has drifted away from the primary driving tasks. Driver physical and emotional cues may present a rich source of information and feedback for L3 automation. Understanding what is going on with driver inside of a vehicle and providing feedback to the driver by the automated system accordingly is essential in roadway safety. 

Although significant work exists in the space of human-machine interaction for automated vehicles, there has been limited study of challenging traffic situations such as urban environments; for example, vehicles negotiating with traffic, pedestrians, bicycles and etc. In this project, we propose to explore driver emotion, action and intention while they use L3 automation in the challenging traffic situation of urban environment. Additionally, we will explore the effectiveness of different modalities of vehicle feedback on the driver. 

The research plan of this project is partitioned into two phases as follows: 
Phase 1: The first task of this project is to design different modalities of feedback that AV provides for driver in a challenging urban environment situation. One of the challenging situations is when AV is negotiating and communicating with pedestrians at intersection. This situation will be stressful for driver if he/she has been kept out of communication loop. Exemplar considerations to design different feedback modalities for such situation are: (a) how to initiate and maintain the communication; (b) how to handle and disambiguate error situations; and (c) what critical information should be conveyed to driver. In this phase, we will leverage the findings of the previous BDD project , which was about the information and design of interaction and communication between AV and pedestrians. 

Phase 2: The second task of this project is to simulate L3 automation and monitor driver emotion, and action to predict driver intention. We will use various methods including eye movement, bio-signal (skin conductance, heart rate and etc.), body movement and subjective data for understanding and predicting driver intention . The goal is to identify physical and emotional cues to predict driver intention in the negotiation situation between AV and pedestrian.