Teaching a Self-Driving A.I. To Make Human-Like Decisions
By: Oliver Cameron
Tags: Self Driving Cars, Autonomous Cars, Technology, Machine Learning, Startup

In this situation, both the optimization-based and machine-learned models within our behavior planner had determined that the right course of action in this instance was to yield to the static pedestrian in the roundabout. This was a safe action, but something a rider in our robotaxi would not be pleased to wait for. A human driver would undertake them on the right albeit slower than usual.

In High-Quality Decision Making, we teach our self-driving A.I. to navigate this situation—and importantly, others that resemble it—by adding “rich” data to our dataset. This rich data has two components:

The recorded driving data of the event. Explicit instructions on what our self-driving A.I. did well (an affirmation) or where it needed to improve (a correction) in that event.

In the above instruction, a member of our data team took a look at the recorded driving data and codified—explicitly—the actions they would have taken if they were in the driver’s seat. In this case, the author says we should have navigated the vehicles and the pedestrian on the right, and that we should have done so slower than usual.

What’s special here—and what makes this data rich—is that other machine learning approaches (e.g., end-to-end) may feed recorded driving events with just subtle signals (i.e., driver take-over) into their model, without also feeding explicit detail on was right or wrong in the event. It is then up to the machine-learned model itself to infer exactly what the self-driving A.I. did right or wrong, with the theory being that with enough data, it will figure this out by itself. Our approach explicitly feeds our machine-learned model detailed instructions—interpretable by both machine and human—on exactly how a human would have handled the situation. With this smaller, richer data—and by bounding what the machine-learned model is tasked to do—we have achieved great results.

After training the machine-learned model with our newly added rich data, our self-driving A.I. can now navigate the scenario. The yellow ghost car that gets stuck is utilizing the behavior planner before re-training.

After training, our robotaxi navigates the situation with ease.

It’s important to note that this situation being handled correctly after adding the explicit instructions is not what’s exciting (it is a form of overfitting, after all), but rather how we are seeing the machine-learned model generalize from a relatively small dataset to other situations.

With now thousands of pieces of rich data—similar to the above example—fueling the machine-learned model in High-Quality Decision Making, we are now regularly observing our self-driving A.I. making human-like decisions. These decisions are often not wow-worthy, but it’s the sum of many subtle driving interactions—like nudging for a pedestrian or slowing down appropriately—that makes for a human-like ride.

Here are a few examples of those subtle decisions made by High-Quality Decision Making.

Our self-driving A.I. slows down before nudging right to give the pedestrian more room.

Our self-driving A.I. yields for the crossing pedestrian, but doesn’t wait for them to exit the road before making the unprotected turn.

Our self-driving A.I. slows down to understand if the overtake can be made without interfering with the pedestrians.

Our self-driving A.I. makes a cautious overtake with multiple pedestrians around us.

Our self-driving A.I. overtakes pedestrians cautiously on a curve.