Accelerate (the book): Scrum and DevOps, Better than PB&J?
By: Grant Gadomski
Tags: Book Recommendations, DevOps, It Management, Scrum, Agile

If any two-bit snake oil salesman tries to convince you that Scrum will solve all your development woes, kindly (or forcefully) show them the door. Scrum is a process methodology, it has nothing to do with development practices. Contrary to popular belief, there’s nothing in Scrum about Test Driven Development (TDD), Continuous Integration/Deployment (CI/CD), or proper Code Reviews. Luckily, there are a number of development-centric solutions that can significantly improve a development team’s ability to deliver well-built features quickly, Scrum or no Scrum. Lean and CI/CD are specifically addressed in the book Accelerate, by Nicole Forsgren, PhD, Jez Humble, and Gene Kim.

Continuous Delivery

Continuous Delivery has five key principles, listed out in the book as (pg. 42):

Build Quality In Work in Small Batches Computers Perform Repetative Tasks, People Solve Problems Relentlessly Persue Continuious Improvement Everyone is Responsable

The Definition of Done(DoD) is the team’s best friend, even if it doesn’t feel like it when they’re scrambling to finish stories one day before the sprint’s end. A genuinely mature Scrum team will notice the positive aspects of the DoD, and understand the headache it saves them down the line. A good Definition of Done bakes the requirement for quality directly into each and every story delivered by the team. It usually indicates that high-quality tests must be written and passing, any needed documentation has been created, and a code review process has been applied. Scrum (especially with a team new to it) is all about going slow to go fast. These requirements of quality for every story at implementation-time may seem like a ball-and-chain at first, but they can dramatically reduce the amount of bug fixing and maintenance work down the road.

One of the most important aspects of Scrum (and I cannot stress this enough) is that stories must be done for them to count towards the team’s velocity. This simple but radical idea not only means that stakeholders see the tangible results of their investments sooner (meaning your project’s more likely to stay funded), but the team usually has to re-think the way they define and deliver software. For one, stories must be small enough to fit into one sprint. This may seem annoying during the planning stage, but smaller stories directly correlate with both easier integrations, and a tighter feedback loop with the product owner, meaning that most required rework will be known about in two weeks instead of two months. The team may also need to rethink the way they deliver features. Traditionally all features would be written and tested before a deployment is even mentioned. Once everything’s finished, a large, painful, merge conflict-ridden integration process would occur, with a difficult, time consuming, pounding-Red-Bulls-at-3am style deployment process as their only reward. Since Scrum requires that stories are done, in a literal sense this means that stories are integrated and deployed, at least to a test environment for stakeholders to try them out. Establishing a robust CI/CD process takes some time, money, and elbow grease (both from a technical and an organizational level), but a well-built pipeline means that deploys are a button push away, and teams can deliver software as soon as they’ve built and tested it.

Lean

Although not as popular as Agile in the software world, Lean has become a common buzzword used by CTOs to mean “the department works faster with fewer people”. In all seriousness, Lean is a simple management concept encompassing four key components (pg. 76):

Limiting Work in Progress Visual Management Feedback from Production Lightweight Change Approvals

Only one of the four is specifically mentioned in Scrum, but all are incredibly worthwhile additions to any Scrum implementation. Limiting the current work in progress is an idea addressed by Kanban (Scrum’s slightly less popular sibling), which insists that only a few items exist in each development status at any one time. When tasks are small but often not delivered by sprint’s end, the cause may be each person having too many simultaneous work items. A good Scrum Master will suggest column limits on the team’s Scrum board in an attempt to improve task completion rates.

Visual management is the one Lean management aspect that boilerplate Scrum incorporates. By utilizing a Scrum board in a location that everyone can see, Scrum provides constant insight into the project’s current status. This means the team can frequently adapt to the Sprint’s current status, and work with the product owner to ensure that each sprint can successfully deliver value.

For the vast majority of software projects, the number one measure of success is how well they delight, serve, and improve the lives of their end users. Since no human being can read minds (and if you can, please get out of my head), a system’s ability to serve is based on (at best) direct needs from end users, and (at worst) guesses based on vague requirements. Even the most well researched, well thought out requirements based on users having hands-on time with prototypes can quickly become outdated. By the time you build the software, users’ wants have already moved on. Scrum allows a team to react quickly to feedback from production, through its short sprints and flexible backlog. A perfect agile project has new, fully usable functionality deploying to production (at least) every sprint, and the product owner can constantly update the product backlog to incorporate feedback from users. This feedback can be reflected in the end product in as little as one sprint.

For decades, a common thorn in the side of enterprise developers and architects everywhere has been change approvals. While the intentions behind most change approval processes are good, they can often devolve into bureaucratic, paperwork-heavy, time-consuming headaches that suck time away from serving end users and providing business value. Instead of relying on a large quality review process at the end of the project, Scrum significantly decreases the need for heavy change oversight in two ways:

Quality is built into each sprint: Although code reviews and pair programming aren’t explicitly mentioned in textbook Scrum, Scrum does require that all work marked as done is fully completed, testing and all. It’s up to the team to define their quality control procedures, but for any features to be sent out the door, they must fully pass all quality assurance checks agreed upon by the team (usually codified in the Definition of Done). Deployments are smaller, and happen more often: Since the “perfect” Scrum team is deploying every sprint, deployments aren’t large, all-nighter ordeals. Heavyweight deployment processes simply do not work with Scrum’s frequent delivery of functionality. Instead of 7 months worth of work being moved to production at once, only two weeks to a months worth is. This usually means easier integrations, and fewer headaches per deploy.

Conclusion

One of the most refreshing things about this book was its academic-like rigor and emphasis on valid, statistically significant data when making any points. Software engineering & the study of how organizations build software are still very young fields of study (compared to mechanical engineering and management-oriented organizational behavior), and don’t yet have the academic weight they deserve. Unlike other knowledge-based fields, software developers rarely read academic papers, and the number of software-focused PhD candidates remains low compared to other disciplines. Instead much of the information distilled throughout the software community comes from personal experiences, intuition, and inductive reasoning. By utilizing the scientific method and testing hypotheses with good hard data, Accelerate’s conclusions can be formally defended, leaving the reader confident in the validity of these findings.

Although I focused on the defined facets of DevOps and how they cooperate with Scrum’s artifacts and values, Accelerate contains a wealth of great information on how DevOps indirectly shapes an organization’s IT Architecture and Culture. The ideas and practices in a high-performing DevOps organization are quite similar to those of a true Agile organization, and these two distinct but complimentary methodologies have been proven to impact many companies’ bottom lines. Overall I would highly recommend that you give this book a read.