Our system leverages expert human guidance to improve the efficiency and reliability of the Aurora Driver.
Self-driving vehicles will save lives, improve access to transportation, materially change the economics of goods delivery, and allow cities to reclaim the space currently occupied by parking lots. We’ve spoken at length about our development approach and the myriad of ways we’re designing our Driver with safety and efficiency in mind. But to be truly impactful, our self-driving fleet must also operate reliably at scale.
Safety and reliability are distinct objectives. Safety sets a hard prerequisite for deployment: we will keep trained personnel behind the wheel until we are confident that our Driver can operate more safely than the average human. Reliability affects the quality of our service after we make that determination and deploy self-driving vehicles without safety drivers.
We will not deploy our vehicles in a commercial setting until we are confident that the Aurora Driver meets our safety standards. To facilitate deployments in new operational domains while maintaining high reliability, Aurora is investing in teleassist. This set of technologies allows trained specialists to monitor our vehicles and provide high-level guidance when needed.
Read on to learn more about teleassist at Aurora, how it’s different from remote control, and what this investment means for our business moving forward.
Why teleassist?
Teleassist will maximize the availability of our services.
Availability is the amount of time that vehicles powered by the Aurora Driver are capable of fulfilling their mission, whether that’s taking people to work or hauling goods across the country. When partners request service, we must ensure that one of our vehicles is always able to respond and can do so without undue delays. Greater availability means more rides, faster delivery, more satisfied passengers, and a better return on investment.
The challenge here is that the world is predictably unpredictable; we’ll never be able to anticipate all of the unique (and sometimes bizarre) events that can occur on the road. While we’re designing the Aurora Driver to respond appropriately in all circumstances, there is still the possibility that this caution could cause it to get “stuck” in uncommon or complex scenarios. For example, say it perceives a traffic guard waving vehicles toward an obscure or unexpected detour. A human driver would see the gesture, watch other vehicles move toward the detour, and intuitively understand what they should do. In its early days, our Driver may lack this type of contextual understanding and would therefore choose to come to a safe stop to await further instruction.
At best, these types of situations are frustrating–no one wants to be late because their vehicle freezes at a detour. At worst, trips could be canceled entirely. The quality of the services we provide depends heavily on our ability to identify hiccups and get our Driver back on track as quickly as possible. Humans can do this fairly easily, but it’s not practical to permanently retain a full-time supervisor in each vehicle. We need a way to leverage human intuition while preserving the many benefits of a largely independent self-driving fleet.
Enter teleassist, a system specifically designed to do just that.
Remote specialists are not safety drivers
You may be wondering why we’d develop teleassist when we already have humans that sit in the front of our vehicles. Are teleassist specialists meant to replace safety drivers?
For us, the answer is no. Safety drivers are present in our vehicles for as long as operating safely requires that they be ready to assume control. In contrast, teleassist specialists are available for our vehicles when operating reliably requires that they be ready to provide guidance.
Safety drivers ensure that our vehicles operate safely before we have determined that the Aurora Driver can independently operate more safely than the average human, while teleassist specialists ensure high availability after safety drivers have been removed. That means safety drivers will be present until we’re sure the Aurora Driver meets our safety standards in new operational domains, but teleassist specialists will aid our vehicles long thereafter.
Teleassist, not remote control
To maximize availability without compromising safety, teleassist at Aurora will look much different than the remote control system you may have been imagining. Our teleassist specialists won’t be sitting in a distant room turning a wheel, pressing a pedal, and otherwise directly engaging the controls of a remote vehicle. We’re taking a different approach because we’re not comfortable with the risks posed by remote control driving.
First, network latency is a challenge. Latency describes lag, or the time it takes for data like camera feeds to travel from our vehicles to remote operation stations, and vice versa. We would need to keep latency low to enable good remote control driving, but this is very difficult to achieve with today’s technology. In addition, limitations in wireless network coverage and periods of high usage can slow down and even interrupt transmission between vehicles and remote operators. Achieving a connection with low latency is already hard; making that fast connection consistently available is even harder.
Second, even if we could achieve consistently low latency, humans are at a disadvantage when they drive remotely, largely due to a lack of situational awareness. If the video feed suffers from blind spots, poor resolution, and/or optical distortion, remote control operators will have limited visibility. This is dangerous because it’s difficult to react to things you can’t see. It’s also difficult to know exactly how much to press the gas or turn the wheel when you can’t feel the vehicle’s movement.
Third, statistics show that remaining attentive and alert is difficult for human drivers, especially over long distances. We’re building the Aurora Driver to alleviate this issue, so it doesn’t make sense to reintroduce risk by placing this burden on our teleassist specialists.
We’re reducing sensitivity to all of these problems by building a teleassist system that complements rather than replaces the Aurora Driver. We’re not giving teleassist specialists direct access to the vehicle’s steering, brake, or throttle. Instead, we’re building an API layer that allows them to collaborate with — rather than replace — the Aurora Driver.
Rather than allowing direct control of our vehicles, we will build a teleassist API that can pass information between teleassist specialists and our self-driving software system.
For example, let’s return to our traffic guard scenario. The Aurora Driver stops at the junction and waits….and waits. After a moment of uncertainty, the system notifies teleassist that a member of our fleet has stopped in the middle of its route. A teleassist specialist reviews the situation, and sees the traffic guard signaling traffic down an alternative street. Armed with this context, the teleassist specialist tells our Driver that it should proceed on the alternative route, and the Driver executes this maneuver when it’s appropriate to do so. The vehicle continues along without losing too much time.
Teleassist specialists will have access to contextual information like live feeds of the camera and autonomy views when responding to support requests.
Notice that the Aurora Driver is still doing the actual driving, and the teleassist specialist just helps point it in the right direction. Teamwork at its finest.
Designing for humans in the loop
Before we can build a teleassist application, we need to extend our self-driving software so that it can collaborate effectively with humans. These additions present interesting engineering challenges that we’re excited to tackle in the coming months.
First, we’ll build an API to translate specific data from our software system into something human-readable and vice versa. Our software modules communicate in a series of numbers. To make this intelligible to humans, we’ll tag those numbers with text that makes sense on a teleassist specialist’s screen (e.g. “Should I go around this vehicle?”). We’ll also translate the guidance provided by our teleassist specialists back into the appropriate format for our software system.
In addition to this communications layer, we’ll modify our software stack so that it can reason about guidance from the teleassist team. For example, our motion planner is built to think about high-level potential actions (e.g., “Should I stop and wait, or should I move around this car?”). When the teleassist team provides feedback, the motion planner will determine whether it can safely execute that guidance and make adjustments as necessary.
Safely, quickly, and broadly
As the network of vehicles powered by the Aurora Driver grows, our team of engineers will continue working behind the scenes to augment its capabilities and expand the road networks, traffic scenarios, and weather conditions it can handle. This continued need for support and close collaboration between Aurora, the vehicles we power, and the networks on which we operate is why we call our product a service and our “customers” our partners. Moving forward, teleassist will play a big role in these partnerships and our deployment, helping the Aurora Driver make a difference quickly, broadly, and at scale.
