How do we know if a self-driving vehicle is safe enough to drive on public roads? It’s a question that has been asked since society first started talking about the potential for self-driving cars to grace our roads. At Aurora, we’re using a safety case-based approach, a defined way to evaluate when our vehicles are safe enough to operate on public roads and to assess that they are not creating an unreasonable risk to motor vehicle safety.
Today we are sharing our initial version of the Aurora Safety Case Framework—the first self-driving Safety Case Framework that applies to both autonomous trucks and passenger vehicles. We believe a Safety Case Framework is the most effective and efficient path to safely pulling the safety driver and it’s an imperative component for any company looking to operate without a safety driver and safely deliver commercial-ready self-driving vehicles at scale. The Aurora Safety Case Framework assesses the entire development lifecycle of our vehicles, allowing us to accelerate our path to deployment and determine when self-driving vehicles are acceptably safe for public roads. We are the only AV company currently operating in our industry to publicly share its Safety Case Framework.
To drive the adoption of self-driving technology, transparency and collaboration are critical. Sharing our work and progress allows the industry to work together to push forward the standards of safety, and it ensures everyone, from regulators and partners to the general public, understands our safety approach. With this commitment to knowledge sharing, we recently offered our perspective on standing up a Safety Management System (SMS), and today, along with sharing the Aurora Safety Case Framework, we’re chatting with Aurora VP of Safety Nat Beuse to dive into why this approach is crucial for our industry and how it shapes our work at Aurora.
Take a look through the Aurora Safety Case Framework and learn more about the principles and applications of our framework.
Today Aurora unveiled its Safety Case Framework. At a high level, what does the framework do, and how does it ensure Aurora can measure its safety work?
Nat: Aurora’s Safety Case Framework captures different elements that are critical for evaluating the safe development, testing, and operation of a self-driving vehicle on public roads. While a safety case-based approach is frequently used in a variety of other safety-centric industries—including aviation, nuclear, medical, automotive, and oil and gas exploration and extraction—it is not yet the norm in the self-driving industry. We think that this is something other self-driving companies should strongly consider using as they move toward commercialization.
At Aurora, we approach safety as a continuous process, not a static checklist of to-do items, and our evidence-based approach is critical internally and externally. Within our company, our Safety Case Framework is how we continuously review evidence and evaluate the Aurora Driver’s performance and development against internal standards to ensure we are confident putting self-driving vehicles on the road both with and without a vehicle operator. Externally, it enables us to effectively share our approach and progress with partners, customers, regulators, and the general public. This transparency is critical and helps us build trust, which is important when deploying any new technology.
This first version of the Aurora Safety Case Framework we are publishing today includes the top four levels of our claims. Its further development will follow an iterative process, so we fully expect that it will evolve over time—in both the short and long run—as we learn more, experience more, and expand our testing operations to new environments and vehicle platforms. This framework is comprehensive, meaning it’s designed to cover testing with vehicle operators, as well as without. At the same time, it’s built to be adaptable, so we can tailor it to different scenarios and environments.
What makes Aurora’s framework an industry first?
Nat: For one, no other company currently operating in our industry has shared a Safety Case Framework. We’re sharing our framework now because we think it's crucial to show our roadmap for safe development and deployment, not just within our industry, but to the public. Some other key differentiators include:
We’re addressing trucking and passenger mobility, all in one: From day one, we’ve developed the Aurora Driver to learn from its experience across multiple use cases – large trucks moving goods on highways, passenger vehicles moving people on highways, or those vehicles moving goods and people on suburban and urban roads. Consistent with this common architecture of Aurora Driver’s hardware and software, we’re proud to be the first company to publish the method for a company to determine that its self-driving system—whether integrated into an autonomous truck or a passenger car—is safe enough for public roads. While we tailor the framework for the specific vehicle platform and operating domain, this approach enables our team to work from one set of transparent guidelines, ensuring Aurora can develop quickly and operate safely.
We’re evaluating the entire development lifecycle, not just deployment: Instead of just focusing on safety for a final commercially deployed product, our Safety Case Framework is adaptable, with claims that support different aspects throughout the lifecycle of our self-driving vehicle development, from testing to deployment. This means that we will be able to adapt the safety case claims we have published today to different vehicle platforms, vehicles with an operator behind the wheel, and vehicles on testing tracks as well as public roads.
We’re focused on the entire enterprise, not just the vehicle: Rather than being limited to just the vehicle, our Safety Case Framework lays out the safety story for Aurora as a self-driving enterprise: the vehicle, people, processes, culture, and supporting programs and systems of our organization.
Why is this Framework important and how does it ensure you’re building a safe Aurora Driver?
Nat: Ultimately, Aurora’s Safety Case Framework helps to assess the design and development of the Aurora Driver and is aligned with our product development roadmap. For each major product milestone, we will examine which claims are relevant and develop the corresponding evidence. A claim is an assertion that we are making, such as “G3.1 Safety performance indicators are measured, analyzed, and used to monitor safety.” The appropriate evidence, which we are actively working internally to develop, will be tailored to substantiate each individual claim, and so may be composed of, for example, test results, peer reviews, audits, or assessments.
Our Safety Case Framework has a top-level claim that Aurora’s self-driving vehicle is acceptably safe to operate on public roads. Naturally, one would ask, what do you mean by that? How do you know it’s acceptably safe? Well, we break that claim down into five principles or subclaims—Proficient, Fail-Safe, Resilient, Continuously Improving, and Trustworthy. So then it’s natural to ask, what does proficient mean? Click through the visual below to see how we break down this subclaim even further.
Why not just amass a massive fleet to drive more and more miles on the road as a way to validate safety?
Nat: While that option might sound attractive, and while we’ve seen competitors talk about safety progress in terms of the number of on-road miles they’ve driven, it’s not the right approach to build a commercially viable product. Given the complexity of driving and the infinite number of scenarios one might encounter in a given operational domain, this brute force method does not allow a company to adequately amass enough miles to understand the safety performance of a commercial product. As RAND noted, “Autonomous vehicles would have to be driven hundreds of millions of miles and sometimes hundreds of billions of miles to demonstrate their reliability in terms of fatalities and injuries. Under even aggressive testing assumptions, existing fleets would take tens and sometimes hundreds of years to drive these miles—an impossible proposition if the aim is to demonstrate their performance prior to releasing them on the roads for consumer use.” Further, even if this approach did work, once a software or hardware change is made to the technology, there’s no way to confirm safety performance other than to drive those miles all over again.
In contrast, we’re taking a more deliberate and comprehensive safety argument approach with a Safety Case Framework. This approach allows us to understand the magnitude of engineering efforts that are necessary to prove why we are acceptably safe on public roads for commercial operations. With that insight, before driving in a new environment or expanding to a new vehicle platform, we can understand the breadth and depth of evidence we will need to develop in order to remove the Vehicle Operator as we deploy commercially. We believe the only viable way to validate that a self-driving vehicle is safe enough to drive on the road is to develop a framework of claims and the evidence to support those claims. Building a safety case framework allows us to demonstrate exactly how we are approaching safety and the many factors we are taking into consideration—a stark comparison to simply reporting on miles driven or disengagements, which do not necessarily provide support for the fact that a vehicle is safe for any specific context or environment. This structured approach is the only way Aurora believes we can safely commercialize our self-driving vehicles.
What’s next for Aurora when it comes to safety? Can we expect you to be sharing more from your Safety Case Framework over time?
Nat: It’s easy for companies in our industry to proclaim their technology is safe, but we're proud to share the work we’re doing to prove it. We’re publishing our Safety Case Framework as part of our commitment to transparency and collaboration, and because we believe that all AV companies need to work together to make our roads safer. We hope this encourages other companies to put together their own safety cases, which we believe are crucial to safely deploying self-driving vehicles on public roads. Ultimately, a strong safety case-based approach should signal — to employees, partners, regulators, and the general public—that the product being put on the road is acceptably safe.
We will continue to partner with internal and external teams to review and refine our safety work, adapt and apply this framework to our changing operations, and share more in the coming months!
