Engineering Principles for Unruly Artists
Tags: leadership, healthydebate, productivity, management
Published: 2020-12-30 12:32:13

Leading a team of intelligent and passionate engineers can feel too much like herding cats at times. Problem solving of any kind is a deeply creative activity, and with creativity comes mess. Lots of mess. This is particularly the case when there isn't just one artist painting on canvas, but many, with their respective paint brushes fighting to manifest their unique ideas.

We all come from different backgrounds, have differing educations, and unique life experiences. These individual perspectives are built up over a lifetime and they can and should lead to healthy (and most importantly, productive) technical discussions.

However, that's not always the case! I've seen teams where the conflicting viewpoints constantly trigger meaningless debates. Debates that divert the team's finite energy and brainpower from the key tasks at hand.

I've seen teams where specific members have such entrenched views they can't begin to see a different way of working. They often end up going off on their own and writing code only they can work with effectively.

What's the result? Paint everywhere.

This crazy artistry is why we see the most effective teams being those that are well disciplined, working in lockstep. Each member playing their own unique role but in concert with everybody else.

It's tempting to look at that discipline and determine that it must come from an autocratic command and control approach, with rules and orders flowing down from the top to the implementers below.

This is "easy" for leadership to do, it's also incredibly lazy. Nobody has to think about integrating conflicting ideas, even if that means good ideas, better ideas, are lost. Because the "best" way, the only way, has already been dictated.

The problem is, it would be impossible to write up a complete list of rules for how to engineer software. The list would be insanely long, and many of the rules would conflict with each other. Not to mention you'd need to know the context of each situation to judge whether the rules had been followed appropriately.

This kind of dictatorial rule-setting ends up being counter-productive as it requires either dumb obedience or cognitive overload. Neither are desirable when cognitive energy is at such a staggering premium. Hiring more people and more expensive skill sets simply isn't the smartest move.

So what's a better way to manage creative individuals?

# Practical Principles
A good analogy for principles are the guide rails bowling alleys provide for children and the perennially inaccurate. They don't guarantee you're going to hit a strike on your first bowl but they do ensure the resulting output is within an acceptable range.

Successful entrepreneur Elon Musk and others talk about reasoning from first principles in order to solve problems more effectively. Why should software engineering be any different?

Instilling a set of principles in a team allows for much faster implementation and iteration. There's just no need to think about _how_ to architect solutions or to confirm every decision with a superior, because an acceptable range of "hows" has already been defined.

Anything that reduces cognitive load and opens up more mental capacity for solving problems is a winner in my book. The trick is for the principles to leave sufficient scope for innovation, personal flair, and team input, without being so loose as to be useless.

The principles I personally like to follow err towards maintainability and practicality. I have a bias for getting stuff done rather than indulging in an academic sense of correctness, because for me, it's more important to have a mostly correct but useful output, than a perfectly correct pipedream.

That probably comes from a background in innovation rather than engineering. A different perspective for sure. Now of course this won't be appropriate for all situations so please modify as appropriate!

# My Principles
This list is my current best effort as of December 2020 and I fully expect it to change over time as I learn new things. The principles below are in ascending order and build upon each other.

### 1. Simplicity
Generally said, it's better to favour simple solutions over complex ones. Sometimes the overarching solution may be inescapably complex, but the constituent parts of that solution can still be kept simple themselves.

Simplicity is good for many reasons, not least because we're only human and we will all make mistakes. In short, _simple_ means easy to understand... easy to understand means safe to change... safe to change means a low risk for introducing bugs... and so on.

Typically I'd consider a concept to be simple if a developer can hold the entire working knowledge of it in their head at one time. Obviously this doesn't apply to entire software systems, but rather to each individual part.

### 2. Consistency
It's no good just choosing simplicity over complexity. The way code is written and the way problems are solved should also be broadly consistent across a codebase. Having 500 unique approaches might yield marginal innovation in some areas, but it's painful, time consuming, and costly to maintain such unlimited variety.

Consistency also applies to conforming to the generally accepted way of doing things both inside and outside your current team and organisation. That's because developers coming from elsewhere will probably expect things to work a certain way.

We have international standards for a reason, the same reason that we tell developers "don't repeat yourself" and consider "code reusability". Doing things differently and reinventing the wheel for no good reason introduces cognitive overhead. And bugs sure love to take advantage of overloaded developers.

### 3. Legibility
Simple and consistent code lends itself nicely to legibility - the quality of being clear to read and easy to digest. Maybe you can impress yourself with the shortest possible piece of code. Great. Pat on the back. Now give that to someone else after you're long gone, or even your future self next year, and hope they can figure it out in a sensible amount of time.

Whether it's showing off or laziness, short code is a pain to unpick. It's a pain to review. It's a pain to modify. A bit more verbosity to make your intentions clear goes a long way to explaining your thought process to other developers.

Engineering is a team sport that requires constant communication through code, and as with any form of communication you can't expect others to magically devine what's going on inside your head. Make it explicit. As in life it's better to over-communicate than to under-communicate.

### 4. Responsibility
In all likelihood, code that is simple, consistent, and legible, probably only does one thing. That's because code that only has one responsibility is much easier to reason about. It's easier to understand how the internals work and how it fits into the larger picture as a whole.

Singular responsibility is a sturdy design principle that applies at all levels. From the application as a whole, to each microservice, class, function, variable, all the way down to individual lines of code. Everything benefits from having a clear, singular purpose.

It also makes it infinitely easier to reuse code in multiple places when that code only does one thing, but does it well. It becomes possible to stitch together (compose) units of code in multiple different ways to produce multiple different outcomes. A core tenet of functional programming.

### 5. Predictability
Code that is simple, consistent, legible, and has a singular responsibility, provides an amazing foundation for predictability. Now we're really getting to the crux of the matter, because for me, predictability is the holy grail of building reliable, bug-free software.

Often our software can seem like a wild beast. We release it into the wilderness* (*also called production) with what we think is a good understanding of how it will behave, only to find that when our wild beast meets reality its behaviour is anything but rational.

Of course we use automated testing, code quality checks, partial roll-outs, automated rollbacks, code reviews, and a whole host of other techniques to assess the behaviour of our code. And these things are absolutely necessary in an imperfect world.

But we could have made our lives a whole lot easier if only we had started with code that's predictable in the first place. That means following all of the above principles as well as more concrete techniques such as minimising side effects, avoiding shared state, and so on.

If we can predict how our code is going to behave before we release it into the wild we'll be able to pre-empt and correct much more of its undesired behaviour before it impacts users.

### 6. Reliability
Hopefully by now you get how this works. Code that is simple, consistent, legible, with a singular responsibility, and is predictable, has the potential to be considerably more reliable.

Reliability refers to two things here; firstly that the software conforms to whatever specifications have been set out, whether that comes from internal stakeholders or from the customer(s) themselves.

This is important because we need to know what the software _should_ be doing even if that's at a high level (we don't want to be doing waterfall here). As discussed above we can use automated testing and associated techniques to assess this.

Secondly, reliability refers to the software behaving as expected when end-users get their hands on it out in the wild (AKA production). Now I'm not saying that it should be perfectly bug free. A known bug that reliably results in the same behaviour every time it occurs may be a nuisance but it's a manageable nuisance.

Reliable software is software that consistently behaves in the same way every time (that also includes being "up" and available when a user needs to access it, in case you're asking). The key here is consistent behaviour, users will be able to work around limitations and bugs, and as a team you'll be able to fix them that much easier.

# Conclusion
So there you have it. For software to be **reliable** it must first be **predictable**, which requires that each part has a single **responsibility**, that the code is **legible**, and that problems are solved in a **consistent** way, recognising that **simple** solutions are usually better than complex ones.

These are the six principles by which I guide my teams to design and build high quality software. It may be an ever evolving set of principles but it's a set that's carried me a fair distance so far, and I hope with a bit more tweaking, much further into the future.

Comments, war stories, and differing opinions are welcomed! :)