When Technology Fails, Who’s Responsible?
By: Amber Case
Tags: Ideas, Cloud Computing, Design, Tech, Accountability

We see less extreme examples of this every day: Products and services with failure scenarios that simply haven’t been considered in their overall design, even when a breakdown can cause serious distress or worse. Typically, the failure is due to a variable that wasn’t seriously considered or even recognized early on in the planning process — or if it was, it was overridden by management as a marginal problem. And somehow, we seem to have resigned ourselves to this as an inevitability of high tech’s ubiquitous “beta phase.”

For instance, consider a popular “smart” pet feeder, designed to feed your animal through a smartphone app. In 2016, the company’s Twitter account was bombarded with panicked users, many of them on vacation, far from home, who suddenly realized their feeder was offline, leaving them unable to feed their cherished pets.

Why? The company had designed the feeder to work with a third-party server from Google and hadn’t bothered to add a failsafe mode in the event the server ever went offline. Somewhere in its planning process, the company decided not to acknowledge that Google could ever suffer server downtime. They did, however, build the possibility of failure into their terms of service, with a clause saying that the company was not accountable for service outages. Such clauses are common for tech companies, a uniquely legalistic way of evading responsibility.

The assumption that cloud services are 100% reliable still persists to this day, especially in the design of internet of things devices, which often depend on them. But they’re not foolproof: According to Network World, during a recent 17-month period, Amazon Web Services, Google Cloud Platform, and Microsoft Azure experienced 338, 361, and 1,934 hours of downtime, respectively. During each of those hours, it’s possible that tens of millions of consumers were using cloud-dependent services that failed, causing anything from mild inconvenience to serious danger.

As is often the case, this is a new tech problem that was anticipated by earlier tech wisdom that’s been buried beneath our current era of “move fast and break things.” In the mid-1990s, Mark Weiser, principal scientist at Xerox PARC, spelled it out in his seminal essay, “The Technologist’s Responsibilities and Social Change.”

Weiser’s Principles of Inventing Socially Dangerous Technology:

1. Build it as safe as you can, and build into it all the safeguards to personal values that you can imagine.

2. Tell the world at large that you are doing something dangerous.

“Most engineers,” as Weiser explained, “will defend as strongly as possible the value of their work and leave it to others to find fault. But that is not enough if one is doing something that one knows has possibly dangerous consequences.”

It would be wonderful if tech companies would willingly embrace Weiser’s principles, but for the corporate culture of many companies, it may be too late. We are seeing what they’ve wrought: massive companies built on technology in which little thought has been put into fundamental safeguards and little transparency into the dangers the technology poses.

The tech press should not consider themselves exempt from these principles. All too often they do. In their product reviews, for example, a potential failure point (“this service pauses if wifi connectivity is lost”) is usually shrugged away in passing as a mild inconvenience, instead of being red-flagged for what it really is: a potential deal-breaker for the product as a whole. Where are the warning labels that read, “these products could result in the death of your pet if there is a wifi outage while you’re away”?

A new institution might be needed. In 1894, when electricity was still very much an unpredictable, dangerous technology, a company called Underwriters Laboratories introduced safety and testing standards. They were quickly embraced by the burgeoning industry and are still in use today. Perhaps it’s time for Silicon Valley to create an Underwriter Laboratories of its own, placing its many products and services under serious, rigorous scrutiny.