Closed-loop feedback control with Scrum
By: Gunther Verheyen
Tags: Agile, Serious Scrum, Empiricism, Scrum

Scrum has its roots in ‘new product development’ [1] and empirical process control, but is mostly known for its impact on the world of software (product) development. Under ‘new product development’ we understand all work to create, deliver and sustain products.

As Scrum enters many other domains of society, it is most helpful to keep in mind that Scrum, in a broader sense, is a simple framework that supports people in addressing complex challenges (in often complex circumstances). New (software) product development surely falls into this category of work, but there are many more complex endeavors for which people are looking at Scrum nowadays.

It is quite essential to understand what the empirical approach is that Scrum intends to implement through its rules and roles. It is quite essential to understand how Scrum is designed to tackle complexity, in order to grasp the depth of the rules and roles and achieve more with Scrum, more successfully tackle your challenges.

Some characteristics of the process of new product development will helps us understand what complexity entangles. These characteristics can be transposed to many other problems, helping us to acknowledge their complexity and unpredictability.

Complexity

There are many variables that have an impact on product development work; features and consumer expectations, competition and market conditions, skills and experience, people and teams, technology and techniques, interfaces and integrations, company strategies and governance, budgets and timelines, regulations and legislation, to name just a few variables. Not all variables are known. Not all known variables behave in a predictable way. Even if a variable is known, its behavior -now or in the future- may be unknown, or different from what is anticipated. Not all known variables are controlled by the people performing the actual work, although they have to incorporate the impact of these variables on their work.

The outcomes are hard to describe in an exact and detailed way; before, at the beginning, or even as the work advances. Every product is unique and the functionality that is appreciated can only be established when released. The product development activities to be undertaken aren’t predictable with any degree of high precision. They have not been performed before, in the same way, under the same conditions, several times. Every product is unique. Techniques and technologies evolve constantly. The environment evolves constantly. Users, markets and stakeholders act unpredictably.

At the heart of such complex challenges are people with cognitive capacities and distinct personality. People are not robots, cogs or replaceable pieces of machinery. People are not predictable ‘resources’, which brilliantly adds to the complexity already described.

Ultimately, more is unknown than is known. There is a high degree of unpredictability.

The degree of dynamism of a challenge requires the right process to be in place:

Open-loop systems

An open-loop system is designed for execution of a series of preempted steps to result in a defined outcome in a single run. Such a system assumes a near-perfect predictability of the variables that influence the process as well as of the process activities themselves.

For larger problems, typically a chain of open-loop subsystems is created, which in an end-to-end view is still an open-loop system. The output of each subsystem is the input to the next subsystem. Although theoretically risks should be confined to the smaller subsystems, in practice the whole system’s vulnerability increases exponentially. In situations of -even moderate- turbulence and change, deviations and variances accumulate across the various subsystems, e.g. in timing and quality. It is not uncommon for the accumulated problems to surface only at the end of the final subsystem, undermining the total results and outcomes.

Predictive plans and hand-overs of work between separate functional teams/departments are implementations of open-loop thinking. Predictive plans can only include known variables, their known details and their anticipated behavior, upon the illusion that no unknowns exist. Open-loop thinking invites lengthy upfront consideration of all elements of the plan, and ultimately attempt to foresee the unforeseeable.

An open-loop system is -by design- unfit for the amount of disruptions and unknowns typical for complex challenges.

Closed-loop systems

Closed-loop systems implement frequent opportunities for inspection so adaptations can be implemented. The actual outcome of the system is compared in a timely fashion against a desired outcome. Desires may change. Variances or undesired results are eliminated or corrected in the next or in future runs. Not all variables and parameters need to be known precisely and in detail, as the process is self-correcting.

Such closed-loop system requires and creates transparency. Reality is inspected, and exposed, so that appropriate adaptations are undertaken. The people performing the inspections have clear and agreed standards in place to inspect-and-adapt against. Inspection for reporting and status purposes is pointless. Inspection without adaptation is pointless.

Closed-loop feedback control with Scrum

Scrum implements empirical process control (aka empiricism) to support people in addressing complexity through closed-loop thinking. Scrum thereby replaces the open loops of traditional, phase-gate, staged or similar processes. Scrum enables players to halt the rat race of traditional processes.

Scrum defines regular opportunities to inspect and adapt. Players are empowered (and expected) to regularly stop, reflect, learn from inspections, gather feedback over the output and change course, re-organize, update priorities, improve, adapt. Scrum brings reality back in the game. Scrum brings transparency. Transparency is pleasant when all is progressing well. Transparency is even more crucial, although less appreciated, when all is not progressing as hoped for. It is critical however for re-positioning and corrections.

In Scrum all work is organised in Sprints, the smallest phase or cycle possible. Sprints can combine into larger efforts. Sprints are time-boxed to assure timely inspection and adaptation. A Sprint forms an ‘inspect and adapt’ cycle of no more than a few weeks that wraps the 24-hours ‘inspect and adapt’ cycle of the Daily Scrum:

At the Daily Scrum the people doing the work inspect their progress, and identify their most important work to do next within the container of the Sprint. A team uses Sprint Backlog, the Sprint Goal and a progress visualization to self-organize within the Sprint. The daily cadence assures they never get out of sync for more than 24 hours.

A Sprint is a cycle that starts with identifying and interiorizing the most important ideas (e.g. product slices) instantiated on Product Backlog. Sprints end with an inspection of the product Increment that was actually released or could be released, as well as how it was built, the process, the interactions, the technology at play.

All events of Scrum set a frequency for the inspection and adaptation process, where the artifacts contain the information to be inspected and adapted. Scrum describes the accountabilities needed to perform the inspections and adaptations. Within a Sprint, additional feedback loops are created, e.g. through agreed work and development standards.

Organizing work in Sprints allows people to take a breath, to break with the traditional rat races, and work at a sustainable pace. Explicit reflection moments are introduced that are crucial for humans to thrive in cognitive, creative work of high complexity.

[1] Takeuchi, H., Nonaka, I. (January-February 1986). The New New Product Development Game. Harvard Business Review.

Do you want to write for Serious Scrum or seriously discuss Scrum?