Agile Project Planning (APP) — Part 2: Misunderstandings around RE Process, product backlog and User Stories
By: Hood Group
Tags: User Stories, Agile Methodology, Re, Requirements, Product Backlog

In the part 1 of this blog series, we have discussed, that the question of plan- and document- based requirements engineering (RE) or agile RE can be wrong. In this part, I will focus on the misunderstandings around the RE process, product backlog and user stories.

There is no one-size-fit-all RE process.

Requirements Engineering is basically a process-independent discipline. You should adapt this discipline to the requirements and context of your project. This primarily concerns the type of collaboration and the chronological sequence of activities. However, proven concepts and artifacts of requirements engineering are still necessary and useful, no matter which process model is used. Requirements always exist on different abstraction levels (see information model below). In order to be able to work value-oriented, the separation of problem and solution is especially necessary.

Can you imagine why the solution level should always be separated into functional (what) and technical (how) solution?

Correct! The functional level tends to be more stable than the technical level. The separation also creates the possibility to consider alternative technical solutions for a desired functionality consciously.

Each of these levels can contain several refinement levels, which allow you to go from coarse to fine. The connection of requirements within these levels, as well as to “outside”, such as the test cases, allows us to inspect the important questions of traceability and test coverage.

All these information elements on the various levels must be recorded and managed, regardless of the process model, so that they could always be available to all participants. What can be information elements? These include visions and goals, the definition of the context, the system scope and interfaces to other systems, detailed functional requirements, quality requirements and boundary conditions, as well as a glossary — the definition of relevant technical terms, and / or domain model.

To manage these information elements, we need a suitable tool. Appropriate tool could be in one case a lightweight Wiki while in another case a heavyweight requirements management tool, which supports the efficient management and linking of thousands of requirements. In an agile environment the Product Backlog is often used for this purpose.

Requirements and Product Backlog

The management of requirements is a task of requirements engineering. NOT the task of Product Backlog Management. The Product Backlog is a planning instrument. According to the SCRUM Guide,

“The Product Backlog is an ordered list of everything that is known to be needed in the product. It is the single source of requirements for any changes to be made to the product. The Product Backlog lists all features, functions, requirements, enhancements, and fixes that constitute the changes to be made to the product in future releases. Product Backlog items have the attributes of a description, order, estimate, and value.”

One of the misunderstandings is that the Product Backlog is intended to manage requirements. Collecting and managing the requirements (the content) and controlling the implementation of this content are different tasks. Therefore, they require different views and techniques.

Why is the separation between the view on the content and the view of control so important? — So that the different aspects, which are also partly connected with different responsibilities, can be handled efficiently.

The Product Backlog is an ordered list of elements with the properties description order, estimate and value. To manage the content, other properties are needed, such as structured representation through requirements hierarchies. In addition, the relationships between the requirements (and also to other artifacts) must be managed in a clear way.

Our experience at HOOD shows over and over again that mixing these two aspects (content and implementation) in the same tool unfortunately causes the product backlog to become a garbage dump after a short time. The Product Owner can then no longer use it in a meaningful and efficient manner.

Requirements and User Stories

Another problem and thus another misunderstanding when dealing with requirements in an agile environment is the focus on user stories.

User Stories are placeholders for requirements. They are the promise to talk about requirements if the user story is important enough to be planned and implemented in the next or after the next sprint. User Stories are therefore primarily a communication tool. They should help to implement the principles of the Manifesto of Agile Software Development. Especially important in this respect are principle 4 — “Business experts and developers must work together daily throughout the project.” and principle 6 –“The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” When we write them on cards and move them across walls, they are excellent planning artifacts to deliver values to our customers sprint by sprint. But they are not useful documentation artifacts for requirements. A larger number of user stories quickly leads to what James Shore calls “Story Card Hell”, where we cannot notice the context (user’s start-to-end-usage scenarios) for the user stories, just like one cannot see the wood for the trees.

Creating a common understanding

How can we avoid misunderstandings and create a common understanding of the requirements? How can we create a common picture of the requirements in the minds of the participants on all levels of abstraction, from the vision via the big picture to the detailed requirements on a functional and technical level? Direct communication is always the best way to create common images.

To convert these common images into a deliverable product increment, we also need documentation. We have to describe the problems (why-requirements), the functional (what-requirements) and technical (how- requirements) solutions, in natural language, with diagrams, with examples, etc., in order to be able to work on them together. In the pre-agile time, we often documented too much and talked to each other too little. In my experience, the second motto of the manifesto — “Working software over comprehensive documentation” is sometimes misunderstood, so that it leads to the fact, that we still talk too little to each other and now also document too little. This is however not a progress.

How can we get over the usual pitfalls and misunderstandings of requirements engineering in an agile environment? Is there a procedure that integrates seamlessly into the SCRUM framework? In the third part of this blog series, I will introduce the Agile Project Planning (APP) framework in detail.

Would you like to learn more about agile product planning right now? Then I recommend my multi-day curriculum of Agile Project Planning (APP) for you, or your team.

Uwe Valentini,

Coach, Consultant, Trainer at HOOD Group