Behavior Forecasting in Structured Environments

People are everywhere! There are roughly 7.4 billion people in the world, and by nature, human beings are social, interacting all the time. Providing machines with the ability to understand and reason about human behavior is arguably one of the most important, intriguing, and potentially useful challenges of modern engineering. Applications are ubiquitous, ranging from automating surveillance, to caring for the elderly or disabled, to providing coaching, to self-driving cars.

In the proposed work, we are interested in forecasting human behavior. Solving the problem of anticipating future events requires recognition of present events, as well as an additional understanding of event sequences, consequences, and predictive cues. Humans are all the time anticipating how others will react and respond to their actions, and these anticipations help shape daily interactions. Machines designed to interact dynamically with humans will require a similar skill. Even machines that merely observe human interactions, such as surveillance systems, can benefit from such foresight, as it could help recognize anomalous behavior indicative of a safety threat. Furthermore, learning to predict behaviors could subsequently help improve learning to recognize behaviors, which would be useful for, e.g., content-based retrieval.

During this project our research will involve predicting lane changing behavior of other drivers on the road. In particular, we propose to predict when another driver will change lanes, and into which lane he or she will move. Much like in the sports setting, on the road, drivers are actively trying to anticipate the movement of other drivers. Such foresight can help avoid collisions. Much of the work in this domain has focused on predicting lane changes from trajectory data, whereas we propose to predict lane changing behavior from visual input, building on methods developed in our work on sports analysis. Predicting lane changes from visual input could be useful as it could eventually provide drivers with such real-time information as "car coming up on the right might enter your lane".