HeyJaredAutonomous System Safety by Phil Koopman
Are autonomous systems such as robotaxis safe? And what does "safe" even mean? Plus other posts on Computer System Safety, Embedded Systems, Software. 100% Human-Written.
- Indexed issues, last 90 days
- 6
- Latest publication
- Sep 27, 2026
- Audience
- Checking…
- Earliest in this view
- Jul 11, 2026
Latest issues
What’s the Deal with Careful & Competent Computer Drivers? (opens the original)
Read excerpt
This week, Junko and I took on the topic of Careful & Competent driving as it applies to autonomous vehicles. The AV industry desperately wants safety to be exactly equal to a low crash per mile metric. But as we see robotaxis blow by school buses, drive the wrong way in traffic, ignore construction lane closures, and so on, it is becoming obvious that we aren’t going to get what ordinary folks think is safe driving without looking at bad driving behavior — even if it doesn’t happen to result in
Theoretical AI Doom Stories Are Distracting Us from Real AI Harms (opens the original)
Read excerpt
It’s not hard to find discussions about existential risks to humanity being presented by AI. One flavor of the story goes that someday soon we’ll have created a real-life version of Skynet that will take over the world, killing mankind. I can’t say that is impossible, but it’s not at the top of my list of things to worry about.I have a somewhat different take on the AI safety problem. I worry less about Skynet waking up sometime soon. That does not mean real harm can't come from AI, but I concen
Inside Analysis Video Podcast on AV Safety (opens the original)
Read excerpt
I was delighted to be a guest on a discussion on the Inside Analysis & DM Radio show with host Eric Kavanagh and also technical expert (and CMU alumnus) Mark Madsen.Quoted Show summary:Promises, promises! We’ve been told time and again that self-driving cars will save lives. But there’s more to safety than crash data. If an autonomous vehicle blocks a fire truck, for example, that’s a severe edge case. And there are many others like it. Most people tend to focus more on stories than stats, which
Getting Better at Telling the AV Safety Story (opens the original)
Read excerpt
I’m delighted to share the discussion I had with Stéphane about autonomous vehicle safety from a different point of view: the technical safety arguments only work if the story is compelling, so let’s talk about the story.Key insights:The safety story for AVs has been rewritten twice already, and needs to be rewritten againSafety is decided by rare events, and those rare events typically present unfavorable optics for the industryMaybe what people want is not statistical crash “safety”, but rathe
I Talk Edge Cases on the Center for Auto Safety Podcast (opens the original)
Read excerpt
I was delighted to be the guest on The Center for Auto Safety’s podcast There Auto Be A Law (S4 Ep48). The discussion top was Edge Cases, remote assistants, robotaxi fixes that didn’t actually work, and how NHTSA has declared that safety includes things other than crash rates.Episode Description: “Phil Koopman is back. Federal regulators warned autonomous vehicle operators to fix dangerous interactions with first responders and promised meetings and recall pressure, sparking debate over NHTSA’s
Publishing over time
Last 90 days. Choose a month to open its work.
Recurring subjects
Named in the text we hold. One piece can cover several.
Audience
No verified audience measurement yet.
About this data
Counts cover the work we have indexed. Tone needs enough text and a confident classification. Excerpts and episode notes are not full articles or transcripts.
Identity or attribution wrong? Suggest a correction.