AI traffic jam pilot: topology of the actuators. Click to enlarge.

The driver activates the traffic jam pilot with the AI button on the center console. On freeways and highways where a physical barrier separates the two carriageways, the system takes over the driving task if the car is traveling at less than 60 km/h (37.3 mph) in nose-to-tail traffic. The traffic jam pilot manages starting, accelerating, steering and braking. It can also handle critical situations such as vehicles cutting in closely in front.

The signals that the traffic jam pilot needs for highly automated driving are supplied by the central driver assistance controller (zFAS). With the traffic jam pilot active, the driver can relax. Drivers can take their hands off the steering wheel permanently and, depending on national laws, focus on a different activity that is supported by the car—such as watching a TV program on the 10.1-inch display in the center console. In this instance the Audi virtual cockpit supplies abstract representations that symbolize the motion and surroundings of the new A8.

During highly automated travel a small camera in the driving area detects if the driver tires or falls sleep. If that happens, a multi-stage warning is given. As soon as the speed rises above 60 km/h (37.3 mph) or the line of vehicles breaks up, the traffic jam pilot informs the driver that they need to take charge of driving once again. If they ignore this prompt and the subsequent warnings, the new A8 is braked to a standstill.

The introduction of the Audi AI traffic jam pilot requires the statutory framework to be clarified in each individual market, along with the country-specific definition of the application and testing of the system. In addition, a range of approval procedures and their corresponding timescales will need to be observed worldwide. Audi will therefore be adopting a step-by-step approach to the introduction of the traffic jam pilot in production models.

Audi AI remote parking pilot and garage pilot. With the Audi AI remote parking pilot and the Audi AI remote garage pilot, the A8 drives independently and without a driver into a parallel or right-angle parking space in several maneuvers, or forward into a garage. Both systems access the steering, accelerator, brakes, tiptronic and also the optional dynamic all-wheel steering.

What is new here is the driver is outside the car throughout the entire parking maneuver. The driver starts the maneuver from a smartphone by pressing the Audi AI button in the myAudi app. To monitor it, the driver holds it pressed and watches a live display from the car’s 360˚ cameras on the smartphone. Once the parking maneuver is over, the system automatically engages the tiptronic P position and switches off engine and ignition.

When ready to drive off, the driver can start the A8 by smartphone and drive it out of the parking space or garage again. If the driver is still behind the wheel, he or she also has the option of activating the parking pilot using the AI button in the center console. Like the AI button in the myAudi app, it needs to be help pressed throughout the entire process.

The sedan can also be steered accurately into a garage if it needs to describe an arc. Inside the garage, the A8 can approach very close to the walls and obstructions such as bikes—or it will interrupt entering if there is not enough space for the car. This special function benefits especially from the innovative laser scanner and is not available in any other competitor model.

Sensors. The new A8 features a new, complete set of sensors. The full set comprises:

• twelve ultrasonic sensors on the front, sides and rear;
• four 360˚ cameras on the front, rear and exterior mirrors;
• one front camera on the top edge of the windshield;
• four mid-range radars at the vehicle’s corners;
• one long-range radar on the front;
• one infrared camera (night vision assist) on the front; and
• one laser scanner on the front.

Sensors and cameras, with zFAS. Click to enlarge.

zFAS and sensor fusion. Until now, driver assistance systems were managed by spatially isolated control units. Audi will be the first automobile manufacturer to bundle these in a central domain architecture. To this extent, the function portfolio, the required sensors, electronic hardware and the software architecture have been combined into a single central system. Right from the outset, full attention was paid to this, and especially to the safety concept.

As a result of the vast sensor information bundled in the zFAS, it computes an entire model of the vehicle surroundings at lightning speed and provides this information to all assistance systems. It is thus also the central interface for all functions of piloted driving.

Audi developed the zFAS with an international leading team of technology partners. It integrates high-performance chips—the Tegra K1 from NVIDIA, the Aurix from Infineon, and the Cyclon V from Altera—which are supplemented by the EyeQ3 processor from Mobileye, the world leader in image processing algorithms for the automobile industry. Its modular concept makes the zFAS flexibly scalable and thus future-proof.

Artificial intelligence and machine learning. Artificial intelligence will soon make it possible for piloted vehicles to react appropriately in highly complex situations, similar to the way in which a human driver would, or perhaps even better. As a sub-branch of information technology, artificial intelligence looks at equipping machines with similar capabilities to those of human beings. This might be achievable, for example, using machine learning.

Machine learning is therefore a pre-requisite for artificial intelligence. The basis for this comes from mathematics and statistics. In the most complex of situations, algorithms will independently find patterns and rules – and will make decisions based on these. In the not- too-distant past, research in the field of artificial neural networks (i.e. the imitation of signal connections within the human brain) made major progress. Deep learning emulates networks of the brain on a computer. This requires enormous computing power and a broad base of data.

In intelligent and piloted vehicles, there will be numerous use cases for machine learning in the future. Thus Audi is evaluating different methods—for example supervised learning or deep reinforcement learning—with the aim of finding the optimal approach for each of these use cases. To this end, Audi is working closely with top businesses from the software field, as well as with leading universities.

One of the most important fields of application of machine learning is currently object and environment recognition. In the Audi A4, A5, Q5 and Q7 models, object recognition has already been implemented in series production with the help of supervised learning. For this purpose, a trained system is used: the learning process is thus complete before the car goes into production.

Even in the new Audi A8, supervised learning is used for object recognition. Image processing developed by technology partner Mobileye is based, among other things, on the deep learning method. This involves deep neural networks being trained using various data sets. In this way, the neural network learns to classify a diverse range of objects—as cars, as cyclists, as pedestrians. The data retrieved as part of this process is then made available to the final version of the driver assistance system software as well as to that of piloted driving.

Through this process, the new Audi A8 also detects free spaces, i.e. spaces in which it can drive. This is a major requirement for the new Audi AI traffic jam pilot.

The laser scanner. Audi’s laser scanner is mounted in the front bumper. This component, roughly the size of a fist, emits light pulses on several vertical planes. A mirror distributes them fanned out over a field about 80 meters (262.5 ft) deep and through a beam range of 145 degrees. The wavelengths of the flashes are in the near infrared range, making them invisible and harmless to the human eye.

They are reflected by the objects in front of the car and are returned to the laser scanner in far less than a microsecond. There, they are detected by photodiodes. The time this process takes is used to compile a detailed, deeply contoured image of the surroundings. Together, the long-range rada, front camera, and laser scanner form a trio of sensors, the various strengths of which complement each other.

The laser detects all kinds of objects—including non-metallic ones—with precision and has a wide beam angle. When it comes to those criteria it beats radar, which has a range of up to 250 meters (820.2 ft), a greater height range and persistently delivers readings even in rain and fog.

Laser scanners and radar also work in the dark if the front camera comes up against its limits despite the bright headlights of the new A8. In good visibility conditions it produces high-resolution images of the vehicle’s near and mid-range surroundings. Thanks to the image database it is capable of classifying many individual objects, such as automobiles, trucks, cyclists and pedestrians. Deep learning methods are used for the first time by the image processing system. It uses neural networks as part of its self-learning approach when determining which characteristics are appropriate and relevant for identifying the various objects. This satisfies another key condition for progress towards autonomous driving.

With the highly connected sensor technology, the assistance systems in the new A8 also respond even sooner to objects, for instance when they detect the end of a tailback and initiate a brake application—an advance on the previous model. Even the navigation profits from sensor data fusion and enables the sedan to identify its position with absolute precision in many situations.

41 driver assistance systems. The fourth-generation A8 offers 41 driver assistance systems. Audi groups the assistance systems in three packages: Park, City and Tour, with the Tour package standard in Germany. Customers who opt for the Audi AI assist package plus obtain all three packages and also the Audi AI remote garage pilot.

The Audi AI Park assist package comprises the Audi AI parking pilot and the wide-angle 360 degree cameras that make maneuvering safer. The driver can have various views of the car’s immediate surroundings displayed on the on-board monitor, to some extent with superimposed guide lines. Maneuvering assist provides steering movements and independently applies the brakes to avert the risk of bumping into static and moving objects. To protect the alloy wheels, the Park package also includes a curb warning.

The newly developed crossing assist is part of the City assist package. If the mid-range radars detect critical crossing traffic in front of the car, the system warns the driver and makes a brake application if need be. The new A8 also alerts the driver to risks when changing lanes. The exit warning system indicates vehicles or cyclists approaching from the rear when opening the door. Light guides in the doors indicate the danger optically. Cross traffic assist kicks in if the new A8 is reversing out of a right-angle parking space.

Another feature of the City package is the pre sense 360° safety system. It detects collision hazards all around the car and initiates targeted preventive measures—whether a full brake application, adjusting the seats or tightening the belts. The Audi AI active suspension additionally raises the body in the event of a side-on collision, deliberately directing the external forces towards crash-active structures. This new function is called Audi pre sense side and is part of the pre sense 360˚ system. It, too, uses the sensors integrated into the central driver assistance controller (zFAS), including the mid-range radars at the corners of the vehicle with a measuring range of about 75 meters (246.1 ft).

The central system in the Tour assist package is the adaptive driving assistant (ADA). It is a progression from the adaptive cruise control (ACC) of the previous model and also integrates a lane assist function and traffic jam assist. This means the driver has support for longitudinal and lateral control over the entire speed range from 0 to 250 km/h (155.3 mph). Bottle neck assist guides the new A8 through construction sites and similar zones. The system in addition promotes an efficient driving style by incorporating navigation data and car-to-X content such as traffic sign information.

The A8 is the only model in its competitive field to accelerate or slow down in response to speed limits, bends, junctions and roundabouts without the need for driver intervention, while automatically drawing on route information from the navigation system. To complement the predictive advice in the Audi virtual cockpit, the driving pedal in the new A8 emits pulses.

The Audi pre sense front safety system, also part of the Tour package, can prevent rear-end collisions or reduce their severity. It incorporates a warning and braking function for vehicles, pedestrians and cyclists. The system is active over the car’s entire speed range up to 250 km/h (155.3 mph).

The Tour package also includes turn assist and collision avoidance assist. Turn assist monitors the lane of oncoming traffic when turning across it, thus helping to avoid a collision with an oncoming vehicle. It also monitors oncoming vehicles turning across the lane of the A8. It is the only model in its field of competitors to feature collision avoidance assist that also offers individual wheel braking as well as steering assistance, for more precise avoidance of obstructions. Camera-based traffic sign recognition and emergency assist, which brings the automobile to a standstill if the driver is no longer able to do so, complete the range.

Park assist and night vision assist are available as standalone options. The latter uses an infrared camera to detect pedestrians and larger wild animals at long distances in the dark and warns the driver of their presence.