London DevOps #74.1 - Unknown unknowns and how to know them - Dylan Ratcliffe
Channel: London DevOps
Date: 2023-06-16

It feels like outages always happen in the worst possible place, and as soon as we understand them, they happen somewhere else. Turns out it’s true. So how can we deal with something that seems to be actively working against us? By changing the way we build mental models. The STELLA Report (Woods DD, Ohio State University) explains how our mental models of how a system works are constantly challenged by outages, and why it’s always the unknown unknowns that cause the biggest problems. We can’t possibly know about these things in advance (otherwise they’d be known unknowns), so instead we need to be able to develop mental models more quickly; in response to a specific outage, or as preparation for a given change. We’ll talk about why this is so important, and why current tools really aren’t suited to job.

Dylan is founder of Overmind and ex Puppet Professional Services Engineer. He's worked with some of the most complex ops environments in the world and is convinced that fear is one of the biggest problems facing devops teams.