Not Quite Broken: How Business-as-usual Software Development Produces Barely Designed Software Systems
Tags: healthydebate
Published: 2020-09-26 21:59:44

It is something of an open secret among long-time developers that practically every software system you work on as a for-hire developer in the private sector is, more or less, a huge mess;  your job is merely to try to slow the inevitable code rot, that, like entropy, threatens to one day plunge the system into such a disorganized, overly complicated, chaotic state of "spaghetti code" that the only feasible solution is to completely rewrite it.

There is, by way of shared humor, of course, the inevitable half-joking references to Dilbert style bosses to explain why this is -- running the gamut from ineffective to possessing a perverse pleasure at orchestrating such an outcome -- but rarely, if ever, is it acknowledged that much of the blame lies with the core processes of for-profit software development.

The above assertions may seem overly harsh, but they directly follow from the processes that for-profit software development teams are constrained to operate within. For, despite the supposed "self-organizing" nature of Agile software development teams, the reality is:

* Every task a developer is working on was assigned to them by the business. Their "leader" or "manager" or whatever title their boss has, is, primarily, the friendly face of management to the workers and vice-versa. Rarely, if ever, does the boss have any real significant power to advance any desired goals of the team or their own goals, unless they already coincide with the goals of the stakeholders. (They have other powers, but they're primarily concerned with "managing" the workforce in terms of interviewing potential new hires, approving PTO requests, etc.) They, like the rank-and-file developers, are employees able to operate only within the narrowly permitted freedoms that *their* bosses afford them.

* Developers are discouraged, implicitly or explicitly, from devoting too much time to any one task, and are especially discouraged from devoting "sprint time" (which is the entire workweek in most companies) to any non-assigned tasks

* Those in upper levels of management who actually write up the ubiquitous "tickets" that form the currency of software development to the rest of the business often have only a vague idea of how the system should actually work (not being technical experts), but a much better idea of how it should *not* work

The consequences of the above 3 realities lead directly to the common state found in almost every software development team: you rarely, if ever, get to work on any "feature requests" because almost ALL your time is spent fixing "bugs" (which can range from actual clearly incorrect behavior of the system to the changing whims of whoever originated the "bug" ticket). 

Even more rare is an opportunity to "refactor" your system, write additional unit tests, or perform any work that wasn't directly assigned. Any thought of discussing high-level architectural concerns (such as S.O.L.I.D principles, etc.) with the team is laughable. Merely bringing up the word "refactor" during any of the team's meetings is a sure-fire way to have QA start to panic as they consider the thought of everything breaking before the next production push (an understandable concern given how little time they have to actually focus on any one ticket).

Now, a certain percentage of "bug" tickets is simply a consequence of software development in general: developers are humans and thus no software system will be perfect. But if almost *all* of the tickets being worked on *every* sprint are bug tickets it directly follows that the system that is demonstrating these defects has a highly defective design, by definition. 

This leads back to my initial assertion: these "business-as-usual" practices and principles lead to software systems that are, just barely, but not quite broken. In other words, they are barely designed. They have gone through the theatre of "planning", "development", and "verification" (usually by a separate under-staffed, overworked "QA" department), but none of those practices have been done in any way that truly fits the meaning of those words.

Note to the reader, especially the non-developer reader (what few there may be in this community): the following can, but shouldn't, be understood as a concise insider's description of software development; it omits some very important steps, however, and is primarily intended to provide a brief idea of what "planning", "development", and "verification" typically boil down to in common practice.

* During "planning", the whole team is presented with an often just-now-seen "ticket" they are expected to read, evaluate, and estimate in the few minutes provided during, for instance, "planning poker". 

Given the complex nature of the software system and the ambiguous nature of the ticket, this is little more than a hopeful guess. The business barely cares if the estimate is a 5 or 8 (if using the common unit of story points) since all they *really* care about is how long it will take before the ticket is moved to "done". Very little discussion occurs in most cases. There are far too many tickets that need to be "estimated" to allow any significant time to discuss an approach, let alone the merits of alternative approaches, to any one of them anyway.

* During "development", the developer *finally* has the opportunity to actually consider the business's ask (usually a "bug"), as recorded in the ticket. 

From this point on, the developer will mostly work solo, with the main touch-point with the team on the actual implementation to be the later "code review" whereby another developer looks over the code and, if they don't believe there's anything obviously wrong with it, approves it. 

Typically, the first step will be to try to reproduce the bug in a separate local development environment. This is often not straightforward at all since there might be little information on what even *caused* the bug.

Once a developer has the ability to reproduce the bug, they then will attempt to use their diagnostic and analytical skills to determine a likely cause.

Finally, they will try to identify the quickest, least invasive way to adjust the system as little as possible in order to correct the defect in a way that will require the least amount of "verification" (i.e. re-testing by QA).

Already, it's clear that the above process isn't geared towards an "optimal" solution, but merely a "workable" one.

Ideally, as part of this process, the developer would want to communicate with the person who requested the ticket to clarify any unstated assumptions, but, since that would take too much time, they instead end up implementing a minimal literal "fix" that, unless they have a *very* thorough understanding of the *entire* mechanisms of the system exhibiting the "bug" (a practical impossibility), probably ends up fixing that particular defect, but introduces several more in the process, thus setting the stage for the future bug tickets that will come as a direct consequence.

During "verification" a, usually separate, "QA" department (often just a single person or a handful; almost always drastically understaffed and overworked) does their best to repeat the steps that caused the bug and, provided that this brief walkthrough doesn't exhibit the bug, marks it as "done", freeing up the developer to pick up another of their pre-assigned tickets to work on.

Now, none of this is a reflection on the fine developers who work on such systems, who are fighting a Herculean uphill battle where every day a ticket is not "done" is one day too many from the business's perspective of constantly at-risk deadlines, but it is a complete farce to say the system is a "designed" system in any fair use of that word. A designed system is one that has been planned out, thought out, its pieces created to fit together into a cohesive whole. Houses are designed. Almost all modern physical products are designed. Software is not. Software is cobbled together by overworked, underpaid, highly talented, creative, and persevering individuals who have very little actual say in how anything is done. 

So the next time you are faced with a website, app, or another piece of software and you ask yourself "who the hell would design it this way?" You now have your answer: it wasn't. It was barely designed. And the developers who are responsible for it can hardly be considered the "authors". Until the state of "business-as-usual" software development changes to actually give freedom to the developers to implement quality software we will continue to have barely designed, not quite broken systems, and you have the profit-is-everything, time-is-money mindset of the corporate world to thank for it. 

The opinions expressed within this post are entirely my own. They are not those of dev.to, the unnamed company I work for, or the Internet at large. If you agree, and especially if you disagree, with any of the above, I *highly* encourage stating so in the comments below (and, note to moderators, why I tagged this with "healthydebate"). All I ask is any comments be logically sound, meaningful contributions, and constructive.