2016 State of the Art Autonomous Cars

In 2015, NASA researcher Ken Goodrich and I collaborated to catalogue the state of the art of human-automation interfaces, culminating in an paper presented at AIAA and published in AVIATION conference proceedings. Ken Goodrich and I are both small aircraft pilots, though Ken is considerably more experienced; we both have studied the interfaces of large commercial transport aircraft in safety analyses and fall into that club of airplane safety nuts who are able to spend hours discussing any commercial aircraft accident over the last 40 years. Ken has more interest in the study of human factors, while I am a car autonomy fan. Writing the 2015 article gave us an opportunity to analyze and discuss air accidents, the 2015 Amtrak crash outside of Philadelphia, and car automation.

At the time I thought an annual update to the State of Automation paper would be valuable, so with this blog I’m updating the vehicle section. To update that work, I refer to the grading scales used previously. (The original paper can be read at GrowKudos, https://goo.gl/6kNUHs ) In just one year, there has been a lot of progress in commercially available semi-automated cars; the higher end cars are filling out and advancing capabilities, and established capabilities are more widely available in less expensive cars.

One scale rates how far the automation assists the human. In order for the car to take the place of the human in the various tasks of driving, the car needs to be able to perceive the road and/or the environment (such as when the car senses a highway lane using visualization), be able to reference its actions against what it should do, in the case of when the car senses a car in front of it and slows the auto-cruise speed to avoid a collision. In different functions, sometimes the automation warns the driver (eg a lane departure warning); sometimes the vehicle acts without the driver, e.g., to keep the car within the lane in autocruise. In some automation, the car is empowered to prevent certain behaviors – anti-lock brakes are one example. In others, the car can integrate functions. The best examples of integrated functions right now are airplane autopilot, which can conduct the entire en route portion of flight, navigating a flight plan, which includes vertical and lateral flying, turning the aircraft to follow navigation signals and maintaining altitude, even down to approach. Though so far most autopilots do not also have a conflict avoidance tie-in, making them short of autonomous.

Perceive (P) The ability to sense and process information into a form that supports understanding of the immediate situation.
Analyze and decide (D) The integration of perceived and stored information to understand the current situation; identify and project future options; and rank these options to support selection of preferred actions.
Warn/inform (W) Rather than acting independently, the automation informs or warns a human operator of its recommendations or analysis. An example would be a lane-departure warning system for cars. Rather than steer the car away from the lane boundary, such a system warns the operator of the need to act and possibly indicates in which direction.
Act (A) Sustained control activity to manage a vehicle’s trajectory (such as autopilot systems) or other systems (such as engine controllers) according to a specified plan or to achieve and maintain a desired condition. The desired plan or outcome may be specified by another automation agent (such as a flight management system) or the human operator.
Limit (L) A restricted form of act in which the automation has control authority to prevent or limit specified conditions or outcomes. This limiting involves modifying, reducing, or even overriding inputs from the human operator that exceed design limits or may result in an unsafe outcome such as a collision. Examples include envelope protection systems on aircraft, antilock-brakes on cars, and limiting speed or lane position. Limiting can be implemented such that it cannot be nominally overruled by the operator (hard limiting) or such that it can be (soft limiting).

Integrate (I) Higher-level coordination between multiple automated elements or functions. For example, changing lanes in heavy traffic may involve close coordination of both speed and steering to smoothly merge into an opening between neighboring cars traveling at a different speed. At the highest level (i.e. level 3 below), integrate implies a level of understanding of the environment akin to that of a human, though it need not mimic the human. An example of sophisticated integration would be the classic example of anticipating that a ball rolling into the street between parked cars might be followed by a running child and posturing other system elements accordingly. The importance of this sort of meta-understanding to safety is difficult, if not impossible, to quantify at present, yet it is likely to be a factor in safe, efficient operations.

The National Highway Transportation Safety Administration publishes an automation scale, similar to one by the International Association of Public Transport. We used a very similar, but simple, four-level numerical scale to indicate how comprehensive the implementation of that function of driving is for a specific vehicle. The four levels are:

0) None: At level 0, the automation has no capability.
1) Limited: At level 1, the element is partially supported by automation, but significant portions of normal operation are not.
2) Nominal: At level 2, the element is fully performed by the automation for normal or routine operations but there are known, rare-normal and non-normal situations that are not supported.
3) Comprehensive: At level 3, the automation performs appropriately in all normal and non-normal situations not considered to be extremely improbable. Its aggregate performance may exceed human levels, in response time, response reliability, adding up to improved safety.

Driving can be separated into functions, which makes it easier to rate the vehicle functions, instead of facing questions about averaging the capability of the car across disparate categories. For example, auto-cruise is highly automated, while re-fueling is not. Borrowing from safety literature, a 2012 paper by Hemm, Horio, DeCicco and Lee divides safe pilotage into functions. This allows rating the functions by different levels of automation. As with the 2015 paper, this rating is designed to show how close we are getting (or not) to a safe, self-driving car.
All the cars in this list now have automatic cruise with automatic steering, so all qualify as semi-autonomous. Self-parking is becoming more common, too. A few have become famous for use or mis-use of those tools, such as the Q50 driving video when the driver climbs into the back seat, and of course the fraudulent claims of a few Tesla owners.

 Cadillac CTSInfiniti Q502017 Mercedes Benz E-classBMW X5Tesla S
Safe separation from other vehiclesP1 D0 A0 I0 W2 L0
Warns driver of failing separation; will be automated to follow lane in 2017 model year. (I1)Does not change lanes.
P2 D1 A1 I1 W2 L1
Active Lane Control. Assistive separation, maintains in-trail interval. Does not change lanes. Prevents lane changing under conflict.
P2 D1 A2 I1 W2 L1
Follows lane markings, straight and mild curves. Changes lanes when indicated by driver, with automated collision avoidance. with safe separation. Actively avoids oncoming collisions; change lanes, braking.
P2 D1 A2 I1 W1 L1
Assists driver; audible alert, flashing light and vibration. Maintains in-trail separation. Traffic jam auto-drives by following other cars.
P2 D1 A2 I1 W2 L1Auto cruise and auto-steer with driver feedback at intervals required. Changes lanes when indicated by driver, with automated collision avoidance. Actively avoids side, frontal impacts.
Safe separation from ground and obstaclesP0 D0 A0 I0 W0 L0
Not available
P0 D0 A0 I0 W0 L0
Not available
P2 D1 A1 I1 W1 L1
Brakes for pedestrians and cyclists in front of the car.
P1 D0 A1 I1 W1 L1
Car overrides driver to avoid non-moving collisions; self park is automated
P2 D1 A1 I1 W1 L1
Brakes for pedestrians and cyclists in front of the car. Self-park in simple parking.
Hazards and pace (eg sense hazards and react)P0 D0 A0 I0 W0 L0
Not available
P0 D0 A0 I0 W0 L0
Direct Adaptive steering resists drift from wind, bumps, inattention. Traction power redistribution.
P1 D0 A0 I0 W0 L0
Vehicle-to-X communication technology warns driver of slippery roads, emergency vehicles, congestion, construction. Appears to be short range GHz signaling.
P1 D1 A1 I1 W1 L1
Senses loss of traction and informs other systems. Moving map for driver (perceive) of route.
P3 D0 A2 I2 W0 L0
Not exactly what is intended here but - Does not lose traction. Floats in floods and wheels provide paddle action. Moving map for driver (perceive) of route.
Maintain ride for passenger safety; eg smoothed braking, smoothed speedP0 D0 A0 I0 W0 L0
Not available
P0 D0 A0 I0 W0 L0
Direct Adaptive steering damps road feedback, for straighter, smoother driving.
P0 D0 A0 I0 W0 L0
Not available. Airbags move occupants away from impact point when sensing impact
P0 D0 A0 I0 W0 L0
Not available
P2 D2 A0 I0 W0 L3
Autonomously slows on curves in auto cruise, with or without autosteer. Limits speed in autocruise; limits autosteer on bad roads.
Maintain fuelP2 D0 A0 I0 W1 L0
Fuel gauge
P2 D0 A0 I0 W1 L0
Fuel gauge
P2 D0 A0 I0 W1 L0
Fuel gauge
P2 D0 A0 I0 W1 L0
Fuel gauge
P2 D1 A1 I1 W1 L0
Nav unit prompts driver to supercharger if car is low on range. Range mode fits car performance into available battery life.
After-market inductive charger available for electric cars.
Maintain vehicle performanceP0 D0 A0 I0 W1 L0
Fault annunciation not available
P1 D0 A1 I0 W1 L0
Fault annunciation including steer by wire error reporting. Reverts to mechanical steering.
P1 D1 A3 I2 W2 L2
When sensing a condition that threatens the motor the car will not leave Park, but does not warn driver.
UnknownP1 D1 A1 I1 W1 L3
Car warns of low tire pressure. Car can be “bricked” remotely by owner if stolen.
Maintain navigation (auto navigate)P1 D0 A1 I0 W1 L1
Warns driver of lane position error
P1 D0 A1 I0 W1 L0
Assistive lane control. Inferred override if driver does not signal lane change.[1]
P2 D1 A2 I1 W1 L0
Assisted drive or automated mode; hands free
P1 D0 A0 I0 W1 L0
Lane assist; car provides alerts if car is drifting
P2 D1 A2 I2 W2 L0
Autocruise and autosteer. Self park and summon to/from parking.

Changes from 2015

Tesla was not on the in 2015, but now is upon it, with a lot of semi-autonomous features, which were rolled out to consumers in summer of 2015. Some fundamental changes are needed to a car to enable it to self-drive, and designing the car from scratch enabled Tesla to jump out ahead, though those features and being out in front go hand-in-hand with a certain cost. In 2015, the Cadillac was among the leaders with an advanced suite of forward-looking sensors, but the integration of lane-following with auto-cruise that was promised for the 2017 model year has been delayed, so there are actually no changes for 2016 to this table for Cadillac. Not pictured is Audi, because its AI-driven autonomous car, a version of the A7, is not available commercially yet. While most car manufacturers automated auto cruise and auto steer for highway driving, Audi looked at the driving experience and decided that being stuck in traffic was the task that most drivers would rather be rid of, and automated that, a feature that the commercially available 2017 BMW X5 now offers as well.

This table in 2015 featured the Mercedes Benz S-class, but for 2016, the E-class, a more affordable model, where Mercedes Benz has pretty much matched the Tesla in self-driving capabilities. In one functional category Mercedes Benz has moved ahead, as in its adoption of Vehicle-to-X traffic warning. Vehicle–to-X seems limited now, as it depends on equipped-vehicle-to-equipped-vehicle signals, and this Bosch capability seems limited to Mercedes Benz, but with increasing adoption this will become a dominant reason to go to cars with Vehicle-to-X. In the congested NorthEast US there are enough Mercedes Benz around that equipped owners will get plenty of signal. Mercedes Benz has a lengthy background in autonomous driving, explored some in the 2015 paper.


Hemm, R.V., B. Horio, A. DeCicco and D.A. Lee, “Assessment of System Safety Risks for NextGen Concepts and Technologies,” AIAA Aviation Technology Integration and Operations (ATIO) Conference, September 2012.