Consumer Reports ranks Tesla Autopilot second among self-driving systems

In the latest ranking of automaker self-driving systems, Tesla's popular Autopilot system ranked second to Cadillac's Super Cruise system.

The ratings come from Consumer Reports, which tested the cars on its test track and on local highways, in both lead and following positions.

The organization rated the systems on their capability and ease of use; how they ensured that drivers stay engaged and don't become complacent; and to what degree the systems allow drivers to engage them in circumstances where they're not designed to be used.

Cadillac CT6 Supercruise

Consumer Reports noted that none of the systems qualify as self driving, even though some automakers market them as such. (It even contacted Volvo to suggest it remove its Pilot Assist system from the Autonomous Driving section of its website, which the automaker did.)

The organization rated Tesla's Autopilot system the most capable and as having the best performance among the four it tested, but it scored the worst at keeping the driver engaged, which dropped it below Cadillac's Super Cruise system in the overall rankings.

DON'T MISS: IIHS: Self-driving systems aren't that, and aren't ready for prime time

In addition to a camera monitoring whether the driver is paying attention to the road and has his or her hands on the steering wheel, Cadillac only allows Super Cruise to be engaged on 130,000 miles of divided highways in North America with no cross traffic, says spokesman Danny Nordlicht.

These modern self-driving systems are becoming more common even in affordable cars as they begin to trickle down from high-tech luxury machines and have drawn attention from the public—and from authorities—after a series of accidents involving Tesla's Autopilot as well as self-driving systems from Uber and Waymo.

Tesla Model 3 dashboard in Autopilot testing with IIHS [CREDIT: IIHS]

"The problem is that these drivers are assuming that their cars are in control of the situation—but they’re not,” said Kelly Funkhouser, program manager for vehicle usability and automation at Consumer Reports. The problem comes more from human nature than technical limitations. “When humans are asked to monitor automation, they lose interest," she says. "That leads to slower reaction times, and can cause crashes.”

Research from the University of Utah, (conducted by Funkhouser before she came to CR) found that driver reaction times lengthened by almost 50 percent when such partial automation systems were used. The longer the features were turned on, the worse the driver's reaction time. Drivers who used a phone while driving were worse off with the automation systems turned on than with them off.

All the systems that CR tested—from Cadillac, Tesla, Nissan, and Volvo—had safeguards to warn drivers to pay attention if they took their hands off the wheel for too long. Super Cruise warns drivers if they divert their attention for four seconds. Autopilot took 24 seconds before warning drivers. A 2009 study by Virginia Tech and NTHSA showed that the chance of crashing or having a near miss more than doubles when drivers have their eyes off the road for even two seconds.

After two visual warnings and an audible warning, all the systems shut down. Autopilot and Super Cruise brought the car to a gradual stop in its lane and turned on the hazard lights. Nissan's system applied the brakes abruptly to get the driver's attention, and Volvo's just shut off and left the car driving at speed.

Both Nissan and Volvo engineers told CR editors that their systems intentionally don't keep the car centered in its lane in order to keep the driver engaged.

CHECK OUT: Tesla Model 3 earns Consumer Reports recommendation after software brake upgrade

Nissan's Pro-Pilot Assist system ranked third in the test, and Volvo's Pilot Assist last.

In similar tests of self-driving systems by the IIHS released this summer, IIHS found that none of the systems it tested (including Autopilot and Volvo's Pilot Assist) were ready for prime time or qualified as "self-driving." Although Autopilot was among the best at keeping the car centered in its lane, Tesla's automatic emergency braking system, which intervenes to slow the car before an accident if Autopilot is disengaged, was the only such system to allow the car to crash into a stationary balloon.

Consumer Reports did not evaluate the cars' automatic emergency braking systems.

Tesla releases own safety ratings

On the same day that Consumer Reports released its findings, Tesla produced its own report regarding its vehicles' crash frequency. The company reported that its vehicles experience an accident or near miss once in every 1.92 million miles driven. Tesla says that number is reduced to one in every 3.34 million miles when Autopilot is engaged.

NHTSA does not have statistics on near misses, but it's data shows that the average car is involved in an accident every 492,000 miles.

Tesla says it plans to continue to publish accident and near-miss statistics for its cars quarterly.