Combining human and artificial intelligence in autonomous vehicles could push driverless cars more quickly toward wide-scale adoption, according to researchers at the University of Michigan Transportation Research Institute (UMTRI).

That’s the goal of a new project that relies on a technique called instantaneous crowdsourcing to provide a cost-effective, real-time remote backup for onboard autonomous systems without the need for a human to be physically in the driver’s seat.

Today’s autonomous vehicles can drive relatively well in typical settings, but they fail in exceptional situations—and it’s those situations that are the most dangerous. Designing autonomous systems that can handle those exceptional situations could take decades, and in the meantime, we’re going to need something to fill the gap.

—Walter Lasecki, an assistant professor of computer science and engineering and a leader of the project

The need for human safety drivers in vehicles such as Waymo’s recently introduced autonomous taxis undermines their cost advantage compared to traditional ride sharing services, the researchers say. It also keeps the era of cars as autonomous rolling living rooms out of reach. Most researchers agree that machines won’t be able to completely take over driving duties for years or even decades.

Instantaneous crowdsourcing differs from earlier efforts at remote human backup in that it can provide human responses in just a few milliseconds—potentially fast enough to help dodge a swerving vehicle or maneuver around a piece of roadway debris. It utilizes connected vehicle technology and a remotely located control center.

Here’s how it would work—all within the space of five seconds or less:

1. Software in the vehicle would analyze real-time vehicle data and electronically guesses 10-30 seconds into the future to estimate the likelihood of a “disengagement”—a situation where the car’s automated systems could need human help.

2. If the likelihood exceeds a pre-set threshold, the system contacts a remotely located control center and sends data from the car.

3. The control center’s system analyzes the car’s data, generates several possible scenarios and shows them to several human supervisors, who are situated in driving simulators.

4. The humans respond to the simulations and their responses are sent back to the vehicle.

5. The vehicle now has a library of human-generated responses that it can choose from instantaneously, based on information from on-board sensors.

Such a system might sound expensive and cumbersome, but Robert Hampshire, a research professor at UMTRI and U-M’s Ford School of Public Policy, says it would be far less expensive than having a human driver in every vehicle. This could make it particularly valuable to ride-sharing and fleet operators. And the huge volume of miles driven combined with the fact that autonomous vehicles only rarely need human assistance could drive economies of scale that would bring down the cost per vehicle.

There were 3.2 trillion miles driven in the US last year, and the best autonomous vehicles averaged one disengagement every 5,000 miles. We estimate that if all those miles were automated, you’d need around 50,000 to 100,000 employees, distributed city by city. A network like that could operate as a subscription service, or it could be a government entity, similar to today’s air traffic control system.

—Robert Hampshire

The algorithm-based screening at the beginning of the process makes it more useful than earlier attempts at remote human assistance, which required the vehicle to stop, contact a remote call center and get instructions before proceeding.

Another key to making the system work on the ground will be designing it in a way that’s workable for the large number of employees, says Hampshire.

Like the job of air traffic controllers, this work could be stressful and cognitively complex. So we’ll be looking at ways to make it less intense and mentally fatiguing.

—Robert Hampshire

The developers are currently working on the software platform. They hope to have humans testing the system by the end of the project’s first year, with the system capturing data from actual vehicles by the end of the second year.

The basic premise behind instantaneous crowdsourcing was validated in a paper titled “Bolt: Instantaneous Crowdsourcing via Just-in-Time Training,” which was presented at the ACM CHI 2018 conference.

We introduce the look-ahead approach, a hybrid intelligence workflow that enables instantaneous crowdsourcing systems (i.e., those that can return crowd responses within mere milliseconds). The look-ahead approach works by exploring possible future states that may be encountered within a short time horizon (e.g., a few seconds into the future) and prefetching crowd worker responses to these states. … Through a series of crowd worker experiments, we demonstrate that the look-ahead approach can outperform the fastest individual worker by approximately two orders of magnitude. Our work opens new avenues for hybrid intelligence systems that are as smart as people, but also far faster than humanly possible.

“Bolt: Instantaneous Crowdsourcing via Just-in-Time Training”

The USDOT project aims to adapt it for use in autonomous vehicles. In addition to USDOT, this project is funded by the Center for Connected and Automated Transportation at UMTRI, Mcity and U-M’s Mcubed.

Resources

• “Bolt: Instantaneous Crowdsourcing via Just-in-Time Training”, CHI 2018