> I like the auto-auto term. Trying to use it now.

Cheers!

> But that said, aren't you holding automation to a different standard than humans?

Yes and no.

Humans drive preternaturally well en mass. So much so that when I realized it I began to suspect that "Guardian Angels" might be real. Then I learned more about human unconscious motor neurology. Driving performance is one of the few places where the unconscious mind is blatant. Nevertheless we have set up a system that amounts to a "mayhem lottery". When I say "they can't come soon enough" I really mean it.

Now eventually auto-autos will be super-human in capabilities and performance. They don't have blind-spots, they can use radar and lidar and mass sensors, etc. They will also have the benefit of perfect hindsight across the whole fleet (if we set it up sensibly!) Further, traffic infrastructure will evolve to facilitate automated traffic, with sensors and comms embedded in signals and even the road. They'll be able to share knowledge in real-time of local road conditions and keep track of people and (non-self-driving) vehicles around them cooperatively. Sooner or later, it won't be possible for someone to surprise one of these things.

And all this will happen at an ever-accelerating rate. (Which means that if you just wait, making them gets exponentially easier!)

In the meantime, it's not okay to use public space to test crude early prototypes. People have already died.

I was truly shaken and upset when I read that the Uber car that killed that woman crossing the road with her bike had determined 1.3 seconds before it struck and killed her that it needed to initiate emergency braking, yet the emergency braking system had been disabled because it didn't work well enough. The machine knew it had to stop, it had the capability to stop, but it was prevented from stopping by bad decision-making driven by the desire to put unsafe machinery on public roads.

That's tragic and truly pathetic, and I would say the makers committed murder. Certainly, if you or I built a robot and sent it out into the world and it killed a person we would have to defend our actions with something better than, "Well it's not perfect. Give it some time."

We could make a e.g. self-driving golf cart that could safely take my mother to her doctor's appointments and back. She's got dementia so we don't want her to take the bus even though she's physically able. It would have a top speed of 3~5 mph, and obviously it would have to travel restricted routes where it wouldn't interfere with traffic, and a there are a host of other concerns. But it would work, we could build it today, there's a market for it, and it would have a really hard time killing anyone in a collision. It would be lightweight as well as slow-moving and covered in airbags inside and out with pre-impact detection and triggering.

Once you start to think outside the box (sorry) there's all kinds of scope for incrementally automating traffic.

Trying to go straight to imitating full-on automobiles with human drivers on public roads is insane, it's hubris. And again, people have already died. So this is not a theoretical concern. There's been a funeral man.

> Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now!

It's not okay for robots to kill people just because people do, no.

(I think a sane transportation network would have at least four interlocking but separate systems: pedestrian, bike, car/truck, & rail. Cf. "Pattern Language" C. Alexander. But I'm getting off on a tangent.)

> I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a bicyclist when I was only going about 15mph, because she basically appeared out of nowhere before I (or she) could stop.

That sucks! Sorry dude.

But yeah, what I'm saying is a robot moving in public should have a hard limit on its kinetic energy (half mass times velocity squared, I just looked it up.)

A robot moving in public with enough energy to injure, maim, or kill is an extraordinary hazard at our current state of development.

If you want to make a safe one, you can keep it lightweight, or you can slow it down.