The Socially Optimal Future of Self-Driving Cars
By: Eric Li
Tags: Policy, Safety, Self Driving Cars

Photo by Andrea Cau on Unsplash

Americans spend more than 70 billion hours a year driving over 3.2 trillion miles (Bureau of Transportation Statistics; Johnson, 2019). From daily commutes to cross-country road trips, cars are the most popular mode of transport in the US and across the globe. In the Digital Age, increased technological capabilities have resulted in incredible innovations in the automotive experience; modern electric vehicles have allowed travel with no carbon emissions, and lane-departure warnings have lowered the injury rate of single-vehicle, sideswipe, and head-on crashes by 21% (Insurance Institute for Highway Safety, 2017). But what will be most revolutionary is what removes the person from the process: self-driving cars.

For vehicles, there are six levels of automation from no automation (level 0) where the driver is in full control to conditional autonomation (level 3) where the driver must be ready to take control at all times to full autonomation (level 5) where the vehicle drives completely on its own (National Highway Traffic Safety Administration). This paper will only consider the fourth and fifth levels of autonomation in which the driver is not required to play a constant role in the driving process as the vehicle deals with those details. In other words, the person enters the car, and the car does the rest.

The arrival of autonomous vehicles (AVs) such as these will have huge implications for safety. An analysis by the National Highway Traffic Safety Administration (2018) found that 94% of car crashes are attributable primarily to driver error, such as distracted driving, lack of sleep, or speeding. With algorithms that don’t face the same sort of pressures from fatigue and cannot get drunk nor distracted by texts, self-driving cars can reduce traffic fatalities by up to 90% and save 30,000 lives in the US alone and avoid those tragic stories about a text that resulted in a crash and a death (LaFrance, 2015). But to portray AVs as a cure-all for driving accidents would be to oversell its capabilities. After all, the artificial intelligence that powers these technologies is only as good as the inputs that it is provided with. For example, using just a few pieces of black and white tape, computer science researchers were able to trick stop sign identifier algorithms into mischaracterizing stop signs as other objects such as speed limits (Eykholt, 2018). Moreover, sometimes, there are no errors, but rather the algorithm is placed in a difficult decision. Consider a situation in five people are jaywalking and the car can either hit the jaywalkers or swerve onto the sidewalk and hit one other person — a modern trolley problem. These situations, no matter how unlikely, need to be accounted for within the coding. With humans behind the wheel, this sort of situation never faces intense scrutiny, because humans are understood to be flawed actors that are unable to process information instantly. The fact that these types now can be accounted for means that we have to hard code some abstract aspect of morality that isn’t well-defined.

However, since the operators of self-driving cars are not humans, but rather algorithms, what about accountability? What happens when graffiti artists cause stop signs to be misinterpreted as speed limits? What happens when the algorithm decides to swerve into a lone bystander to avoid a couple walking across the street? To develop effective proposals to tackle these two issues of algorithmic error and the trolley problem, an analysis of the current and potential frameworks for liability is needed.

CURRENT LIABILITY FRAMEWORKS

Negligence

Negligence is the framework that almost all current car accident litigation uses. It essentially operates off the principle that the defendant was not reasonably careful in some activity (in this case, driving) which caused some sort of harm. More technically, there are a few requirements to hold someone negligible (Goguen).

Duty of reasonable care: law required the defendant to be careful

Breach of duty of reasonable care: defendant did not act as a reasonable person should act in that situation

Defendant caused plaintiff’s injuries

Plaintiff suffered some loss

However, it’s impossible to apply this same framework of negligence to an autonomous vehicle (AV). If the defendant isn’t driving the car, then the defendant can never really breach the duty of care as they were not even in control of the car. As such, perhaps it is not the defendant that should be held liable but rather the self-driving algorithm as it is the algorithm that is in control of the car.

Product Liability

Product liability cases are cases in which the plaintiff is harmed by a product defect, or in this case, an error in the algorithm of the car. There are similar requirements to hold a company liable under the product liability framework (Legal Information Institute).

Defendant sells the product being used

Defendant is a commercial seller of the product

Item defective when the product was sold

Plaintiff suffers an injury

Defect was the cause of the injury

This sort of framework works well for algorithmic errors as the litigation process would reveal such errors; a stop sign misidentification error would satisfy the requirement that the item was defective when the product was sold. Additionally, since the litigation and settlement process would incur costs, this framework provides a direct incentive for manufacturers to create the safest algorithm — the safer the algorithm, the less expensive the litigations, and the less additional costs.

However, this framework fails when it comes to tackling cases involving the trolley problem, because the issue wasn’t a defect but rather an issue with the genuine design. A subset of product liability, design defect deals specifically with defects that have been built into the product. To prove design defect, though, the plaintiff must show that the product is unreasonably dangerous due to a design flaw. But what is unreasonably dangerous? In the classic trolley problem — where the trolley is headed towards five people and can be diverted to hit one instead — is it unreasonably dangerous for the single person? The vagueness of the wording provides discretion for the judge, but the trolley problem has plagued philosophers for decades without a rigorous answer, and it wouldn’t make sense to ask a judge to answer that question for a court case.

No-fault insurance

Another potential avenue of dealing with accountability is no-fault insurance, under which, the insured are not held liable for the crash regardless of fault in the crash (US Chamber Institute for Legal Reform, 2018). For this, states usually institute a no-fault insurance policy where each insurance company compensates their client for the cost of their injuries (Insurance Information Institute, 2018). This avoids the issue of accountability for the trolley problem — as it recognizes that there will be victims just as a result of the algorithm and those victims deserve compensation — but also avoids the issue of accountability for algorithmic errors. If there were a systematic issue in the self-driving algorithm that resulted in consistent deaths, the manufacturer would never be held responsible. For example, if there were a stop sign misidentification error that only occurred in heavy rain, this error may never be discovered as the issue. Manufacturers could investigate the issue on their own after a crash, but beyond consumer perception, there would be no concrete incentive to build the safest algorithm — unlike the product liability framework’s incentive of avoiding litigation costs.

Victim Compensation Fund

Governments could also establish a victim compensation fund to deal with accountability. The National Childhood Vaccine Injury Fund was established to increase vaccinations without having to establish fault when somebody was injured (US Chamber Institute for Legal Reform, 2018). Under a similar program, governments could establish such a fund to promote the public good with payment from manufacturers (perhaps via a tax on each self-driving car sold). Similar to no-fault insurance, this could avoid the issue of liability for the trolley problem but do the same for algorithmic errors; there would be no incentive for car manufacturers to make their algorithms as safe as possible, because the compensation fund would cover the cost of any crashes. However, compared to no-fault insurance, a victim compensation fund would be desirable, because it’s not entirely the fault of the purchasers of the car (who are effectively at fault as they pay the insurance that pays their costs) but rather the fault of the algorithm.

The Socially Optimal Liability Framework

The product liability, no-fault insurance, and victim compensation fund frameworks all have their benefits. Product liability deals well with algorithmic errors but not so well with the trolley problem. On the other hand, the no-fault insurance and victim compensation fund frameworks are the opposite; they deal well with the trolley problem by offsetting responsibility but don’t ensure that manufacturers create algorithms without the lowest possible algorithmic error rates. Considering the much higher potential occurrence of algorithmic errors as compared to trolley problem situations — the trolley problem has never happened in the history of conventional driving — it would be socially optimal to adopt a product liability framework and add the victim compensation fund framework to deal with issues of the trolley problem. However, because accountability for the trolley problem would essentially be avoided, the socially optimal algorithm to deal with the trolley problem should also be considered. To design such an algorithm, the desires of the stakeholders (the consumers, government, and manufacturers) must be considered.

PERSPECTIVES ON THE TROLLEY PROBLEM ALGORITHM

Consumers

Assuming that incorporation of AVs is a positive thing (due to increased safety) and society should move towards that direction, it would be important to ensure that the morality of the self-driving cars is one in which the consumers can agree with; otherwise, these cars would have no demand market and would never be adopted as they would go unsold. That being said, there are moral inconsistencies within what consumers want. In a Science Magazine research paper, Bonnefon, Shariff, & Rahwan (2016) found that people overwhelmingly viewed a utilitarian self-driving car (that would save the maximum number of people in any crash) as more moral but personally would prefer to buy a car that protects the passenger at all costs, even if that means killing 10 or 20 pedestrians. The paper describes how the likelihood of purchasing an AV drops by about half when it is utilitarian (19 out of 100) as compared to passenger-protecting (50 out of 100). In that case, perhaps it would be best to create cars that are passenger-protecting to facilitate the ubiquitous incorporation of AVs, because there should be a relatively minimal number of lives lost due to trolley problems as compared to the much larger number of lives saved via the increased safety of self-driving algorithms.

Government

The government primarily wants to promote the general welfare (perhaps via enforcement of a certain algorithm or subsidies for manufacturers that use such an algorithm) but have multiple axes to consider: the public opinion, economy, and safety.

Regarding public opinion, the government could be interested in pursuing a policy that aligns with the citizens’ desires — one in which the algorithm prioritizes the passengers. On the other hand, the government could align with public opinion by instituting the algorithms that the general public desires to live in. The MIT Media Lab conducted a specific analysis of moral desires in multiple trolley problem cases in their article published in Nature, one of the world’s top academic journals. They discovered that the strongest global preferences are to save more lives over fewer lives and younger lives over older lives (Awad et al., 2018). The government could use these opinions in their consideration of what sort of algorithm to promote. This complicates the situation further, though, as the principle of preferring younger lives to older lives seems to conflict with the Fourteenth Amendment’s right to equal protection which states that all citizens are entitled to the same protection of their rights. Thus, the government may want to just promote a purely utilitarian algorithm that just values lives — ignoring the more discriminatory aspects of these moral preferences.

Regarding the economy, the government could view improving the economy as improving the general welfare of the populace. In doing so, they would want to promote the development and innovation of self-driving cars. This could result in the government being laxer about governmental regulation to encourage self-driving car industry development. This is already an issue with interstate economic competition; Ron DeSantis, the governor of Florida, signed a bill allowing AVs on the roads without requiring a human at the wheel and encouraged AV companies to relocate to Florida to “bring high-paying jobs to the state while making [Florida] roads safer” (Farrington, 2019). Similar intercountry economic competition could occur, especially between the US and China. The Chinese market for AVs is estimated to be the largest global market for AVs, in the trillions of dollars. The US government would thus have an incentive to have lax regulations (or even subsidies to directly encourage development) so that they could tap domestic and international markets, increasing US economic growth. Thus, the government would want to promote an algorithm that would be desirable for consumers — one that prioritizes passenger safety.

Regarding safety, the government could want to promote an algorithm that makes people as safe as possible — namely, a utilitarian algorithm. This would also align with the public desire to live in a world with utilitarian cars. On the other hand, the government could also try to maximize the adoption of AVs to promote general safety by promoting an algorithm that prioritizes the passenger.

Considering these three axes, the government seems to most desire promoting an algorithm that maximizes passenger safety. This would be in line with public opinion on the individual desire to ride in a passenger-prioritizing car. It would also promote economic development in the AV industry as well as increase general safety by increasing the rate of adoption of self-driving cars.

Manufacturers

The Business Roundtable, an association of the CEOs of major US companies, redefined the purpose of a corporation this year by expanding it beyond just the interests of the shareholders. They describe the purpose of a corporation as delivering value to customers, investing in employees, dealing fairly and ethically with suppliers, supporting the communities in which a corporation works, and generating long-term value for shareholders (Business Roundtable). However, regarding deciding an algorithm to deal with the trolley problem, only delivering value to customers and generating long-term value for shareholders are relevant. Investing in employees, dealing with suppliers, and supporting the communities are not directly related to deciding what algorithm to use.

These two ideas of customer value and shareholder profit converge in maximizing profits with an emphasis on what consumers want; thus, manufacturers should consider how to maximize consumer demand. Manufacturers would want to create an algorithm that prioritizes passenger safety to increase demand for the vehicle.

The Socially Optimal Trolley Problem Algorithm

With the interests of the consumers, government, and manufacturers considered, the socially optimal algorithm seems to be one that prioritizes passenger safety. This would allow for the greatest consumer demand for self-driving cars, increasing profits for manufacturers and increasing adoption of these objectively safer vehicles, which would also support the governmental goal of promoting welfare in both the economic and safety sense.

POLICY PROPOSAL

The government should institute a liability framework of product liability for algorithmic errors and a victim compensation fund for trolley problem issues. Moreover, regarding the trolley problem, the government should promote a passenger-prioritizing algorithm via a subsidy to manufacturers that produce cars that use such an algorithm. It should not mandate such an algorithm, though, because if there were cars with utilitarian algorithms that were popular with consumers, that would increase the number of saved lives. The only benefit of passenger-prioritizing algorithms is that they would be more widely adopted. However, such utilitarian cars would likely fall out of production as consumer demand would be much lower.

CONCLUSION

Criticisms

Though instituting an algorithm that prioritizes passenger safety seems to be the most conducive to safer roads as the likelihood of such a trolley problem situation being extremely low, there is likely to be public outcry at the idea of promoting an algorithm that inherently values the lives of the passenger over the lives of a pedestrian as utilitarian AVs are viewed as more moral. While this public pushback is an important consideration, there is no definitive evidence that, given the reasoning behind such a decision, there would still be substantial pushback. However, even assuming that there is pushback, manufacturers still have an incentive to produce passenger-prioritizing vehicles due to their increased consumer demand. Perhaps there would be such strong negative sentiment against the creation of self-driving cars with such an algorithm but given the substantially higher desire to purchase a passenger-protecting vehicle as compared to a utilitarian one, this seems unlikely.

Unaddressed Concerns

Though the proposed liability framework is socially optimal, it is not perfect. Consider the issue of adversarial AI, a technique designed to fool artificial intelligence algorithms with specially designed input. As previously mentioned in the context of stop signs, AI algorithms can be fooled with a decently minimal amount of effort. Keen Security Lab (2019) of the Chinese technological company Tencent was able to fool the Tesla autopilot system to change lanes into oncoming traffic with just three stickers placed on the ground to suggest that the lane was curving left. In instances like these, the defectiveness of the product is unclear. Does it qualify as defective because such a simple difference resulted in a monumental change or is liability only on the person with malintent who designed the input? Clearly, the person with malintent ought to be held accountable but the algorithm also needs some sort of improvement. These sorts of issues can be almost impossible to predict, and perhaps the best option is to just rely on dealing with these issues on a case by case basis. The presented socially optimal framework would need to be expanded upon to deal with issues such as these.

Additionally, consider the assumption of certainty underlying the analysis of the trolley problem algorithm. In this paper’s analysis as well as the analysis by Bonnefon et al. (2016), the survival and deaths are considered to be guaranteed, unlike the real world where everything is probabilistic. What should be done when the likelihood of saving the passenger isn’t 100%? What if there is a 0.01% chance that the passenger of a car could be saved if the car hits the pedestrians (killing the pedestrians with 100% certainty)? Presumably, consumers would be less averse to purchasing a car that decides to save the five pedestrians in this case as they (the passenger) would likely still die. Future studies about uncertainty and the marketability of trolley problem algorithms need to be conducted.

Implications

These socially optimal accountability frameworks and trolley problem algorithms have direct implications on the future actions of the government and manufacturers regarding self-driving cars but also for all AI that has to determine tradeoffs between life and death.

The increasing incorporation of AI into warfare is one other area that could benefit greatly from a similar system. For instance, if an algorithm were to misidentify a civilian as a soldier and kill this civilian, this type of error would fall under product liability and the developer of the algorithm held accountable. On the other hand, when determining the tradeoff between achieving a military objective (such as killing a terrorist leader) when it could hurt innocent civilians, there could be a similar victim compensation fund for the civilians harmed. The specific details of the fund as well as the tradeoff between military objective and civilian lives — can’t just disvalue the civilians because achieving a military objective could result in a much larger number of deaths than a car crash — would have to be worked out, but a similar framework is applicable.

Conclusion

Self-driving cars are increasingly becoming more common. However, the issue of accountability still clearly plagues the industry; how can the ubiquitous incorporation of such a vehicle be allowed when there’s no effective accountability mechanism? Through analysis of current frameworks for car crash liability, a product liability and victim compensation fund seem to be the socially optimal liability framework. The product liability framework would deal well with issues of algorithmic error by creating an incentive for manufacturers to create the safest algorithms and allow for the improvement of defects discovered via litigation. The victim compensation fund recognizes the impossibility of accountability in the trolley problem while still providing support to victims. Moreover, considerations of the consumer, government, and manufacturer reveal that the socially optimal algorithm for the trolley problem would be one that prioritizes the passenger. Though such a system could seem unethical, it would maximize the adoption of self-driving cars, thus decreasing traffic fatalities, because the likelihood of a trolley problem situation happening is minuscule.

References

Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., …Rahwan, I. (2018). The Moral Machine Experiment. Nature, 563(7729). https://doi.org/10.1038/s41586-018-0637-6.

Bonnefon, J., Shariff, A., & Rahwan, I. (2016). The Social Dilemma of Autonomous Vehicles. Science, 35. https://doi.org/10.1126/science.aaf2654.

Bureau of Transportation Statistics. (n.d.). U.S. Vehicle-Miles. Retrieved from https://www.bts.gov/content/us-vehicle-miles.

Business Roundtable. (n.d.). Our Commitment. Retrieved December 17, 2019, from https://opportunity.businessroundtable.org/ourcommitment/.

Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., …Song, D. (2018). Robust Physical-World Attacks on Deep Learning Visual Classification. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2018.00175.

Farrington, B. (2019). Florida Law to Allow Autonomous Cars When They’re Ready. Retrieved from https://apnews.com/c0c1b57dd4c74f7c861cf9622e1cc2f6.

Goguen, D. (n.d.). Car Accidents Caused by Negligence. Retrieved December 9, 2019, from https://www.nolo.com/legal-encyclopedia/car-accidents-caused-by-negligence-29537.html.

Insurance Information Institute. (2018). Background on: No-fault Auto Insurance. Retrieved from https://www.iii.org/article/background-on-no-fault-auto-insurance.

Insurance Institute for Highway Safety. (2017). Lane Departure Warning, Blind Spot Detection Help Drivers Avoid Trouble. Retrieved from https://www.iihs.org/news/detail/stay-within-the-lines-lane-departure-warning-blind-spot-detection-help-drivers-avoid-trouble.

Johnson, T. (2019). Think You’re In Your Car More? You’re Right. Americans Spend 70 Billion Hours Behind the Wheel. Retrieved from https://newsroom.aaa.com/2019/02/think-youre-in-your-car-more-youre-right-americans-spend-70-billion-hours-behind-the-wheel/.

LaFrance, A. (2015). Self-Driving Cars Could Save 300,000 Lives Per Decade in America. Retrieved from https://www.theatlantic.com/technology/archive/2015/09/self-driving-cars-could-save-300000-lives-per-decade-in-america/407956/.

Legal Information Institute. (n.d.). Products Liability. Retrieved December 9, 2019, from https://www.law.cornell.edu/wex/products_liability.

National Highway Traffic Safety Administration. (n.d.). Automated Vehicles for Safety. Retrieved December 10, 2019, from https://www.nhtsa.gov/technology-innovation/automated-vehicles.

National Highway Traffic Safety Administration. (2018). Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey (Publication DOT HS 812 506). Retrieved from https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812506.

Tencent Keen Security Lab. (2019). Experimental Security Research of Tesla Autopilot.

U.S. Chamber Institute for Legal Reform. (2018). Torts of the Future: Autonomous Vehicles. Washington, DC: Silverman, C., Goldberg, P., Wilson, J., & Goggans, S.