Working with technical debt (a PM perspective)
By: Andrei Chirila
Tags: Technical Debt, Software Development, Project Management, Risk Management

What’s happening?

Things haven’t been going so well lately.

It’s been a couple of months since you and your team were able to complete a sprint successfully, without leaving any tickets behind. Simple functionalities or minor changes seem to be taking way longer than they should and everyone seems to be underestimating how long it will take them to do things. The teams velocity keeps going down even though everyone seems to be working extra time to meet the sprint goal. Besides that, bugs keep showing up out of nowhere and the more you fix, the more pop up out of nowhere. Making any kind of schedule seems impossible and if it keeps going like this you can definitely say that you’re not going to hit your end of project deadline.

There might be a lot of reasons for projects not meeting their deadlines but if the situation sounds a little bit like the one above, you might be facing a case of : high technical debt.

Why is technical debt so scary?

Well, it’s in the name: it’s debt and if it accumulates you might be spending all the resources that you have into paying that debt rather than generating business value.

I’ll spare you the technicalities. What’s important to know is that technical debt is normal on any project and it’s even useful. Whenever you need fast results and you “take a shortcut”, you create a little bit of debt. Sometimes, you will want to get fast results to meet a deadline, get a functionality out so it can be tested by users or just to show the stakeholders that you get things done and build some trust. That’s not a problem as long as you pay that debt (rewrite the code) sooner rather than later. Fast results now means more work later so be mindful about when you choose to go fast, and when you take time to fix things up.

But how do you know that you have a problem with technical debt and that you should start worrying about it? Well, I’ve already given a hint on how such a project looks like in my first paragraph, let me just put some bullet points in front so it’s easier to follow:

functionalities or changes that seem easy on a first glance end up taking a lot more time than expected ;

; velocity is smaller than usual or is all over the place (some sprints it’s low, other sprints it’s back to normal) although not much changes from sprint to sprint;

is smaller than usual or is all over the place (some sprints it’s low, other sprints it’s back to normal) although not much changes from sprint to sprint; releases are getting buggier and the more time you are spending on bugs (anything over 20% should be a cause for concern) the more bugs seem to appear;

and the more time you are spending on bugs (anything over 20% should be a cause for concern) the more bugs seem to appear; your tester seems to be overwhelmed at times and suddenly everyone starts doing testing to make sure all bugs are “caught” before the release;

to make sure all bugs are “caught” before the release; sometimes, it seems that you are spending more time fixing stuff, rather than developing .

. estimates (even simple ones) are hard to get right even when you add a buffer and take into consideration the worst case scenario;

(even simple ones) are hard to get right even when you add a buffer and take into consideration the worst case scenario; even though people seem to be working more than usual deadlines either aren’t met or, when they are, it is because of some superhuman effort;

If most of these apply, then you probably have a problem that you need to fix. If only some of these apply or they apply to a lesser extent (for example not all releases are buggy but one in 2 or 3 are) then maybe you are somewhere on the edge and you need to make a decision — pay now or pay more later.

So, you’re working with technical debt

It doesn’t even matter how you got to this point. It just happened and now, here you are: your schedule is a mess, releases become stressful and team morale is low.

First of all let’s see how this looks like from the perspective of your colleagues.

Technical debt makes coding a lot slower as there are areas that you need to handle carefully in order not to cause any more problems. Usually it means that the code is written poorly and, if we’re talking about a big project, it might be the case that nobody in the team even knows what exactly that piece of code does. Changes in some areas need to be done in 10 different places and whenever a bug shows up, you’re not sure what caused it and need to spend a lot of time investigating. Usually developers know where these high technical debt areas are in the project and tend to avoid them. Working on a task that requires them to make changes to that code causes frustration (“oh, it’s gonna take forever”) and anxiety (“I have to be extra careful who knows what I’m going to break”). It’s the equivalent of playing Jenga but if the tower falls then you not only have to clean everything up and put the pieces back together but you’re doing it while the pieces are on fire.

In order to make sure that the Jenga tower doesn’t fall, whenever a change is made extra testing is done by everyone in the team. Everyone, especially the tester on the team, is feeling the pressure of not letting any bugs go into production. This just makes tasks take even longer than normal and developers become “duck tape programmers” — trying to ship on time but desperately holding everything together behind the scenes.

How do I manage this?

Fist of all, I have to tell you that I lied. Remember a couple of paragraphs ago when I said that it doesn’t matter how you got here? Well, I lied, it does, because how you got here tells you how you’re going to get out.

High technical debt can have many causes :

Sometimes it’s not your fault — this is an old project and you and your team “inherited” it this way or the stakeholders simply took the decision of going into debt knowing the risks because they needed fast results.

Other times, the fault is somewhere in the middle — constantly changing specs and priorities meant that you had to be flexible and adapt still, you never tried to confront the stakeholders and explain that all these fast paced changes will impact the quality of the project on the long run.

And sometimes it’s completely your fault — you’ve underestimated the work that needed to be done, the team had a lot of junior programmers and, in trying to keep up with your initial estimations, you’ve opted a little bit to often for fast results rather than good quality. Or maybe you just didn’t put that much emphasis on good engineering methodologies like code reviews.

Let’s start off with the team.

I think the best way to manage this is to be honest. Just put it out there: “Guys, this is what we’re facing … I can see that you are frustrated, believe me so am I … Let’s see what we need to do in order to change the situation.”

Have a meeting dedicated to technical debt with your team and brainstorm ideas on how to fix the situation . Try to understand how you got there and identify things that can be changed. It is also very important not to blame anyone. Start the discussion from the mindset that everyone did the best they could given the knowledge and information that they had at that specific moment.

. Try to understand how you got there and identify things that can be changed. It is also very important not to blame anyone. Start the discussion from the mindset that everyone did the best they could given the knowledge and information that they had at that specific moment. Adopt good work methodologies and stick to them (unit testing, code reviews even peer programming can work). Remember what John Wooden said: “If you don’t have time to do it right, when will you have time to do it again?”. So start working on doing things right from the start (or at least from now on).

(unit testing, code reviews even peer programming can work). Remember what John Wooden said: “If you don’t have time to do it right, when will you have time to do it again?”. So start working on doing things right from the start (or at least from now on). Make a wall of “Technical Debt” and put post-it notes with areas or functionalities that you know need refactoring. Start including these in your sprints one by one while still delivering business value to your stakeholders. This will also add a new sense of ownership as everyone will want to get rid of the post-it notes on that wall.

What about the stakeholders?

Well, like any other problem it won’t get solved unless you talk to them. Be honest about the state of the project and explain to them the concept of technical debt. If stakeholders aren’t aware of the concept of technical debt, they will award fast shipping rather than quality. Once they become aware that fast comes with a cost, they might start thinking differently.

You will most probably be facing three kinds of stakeholders and the result will depend on your relationship with them:

Nr.1> The ones that knowingly took decisions that increased technical debt (“We need this big functionality by next month”). You will have to prove to them that although they seem to be getting results fast, this will hit them hard on the long run and that changing specs in the middle of the sprint isn’t useful either. Practically, you have to convince them that you need to go slower or everything will fall apart. You might find that they don’t trust you (“It worked fine until now. It’s not going to crash if we do this little thing also. Don’t worry and just focus on delivering.”) so you will need data to prove your point.

The ones that knowingly took decisions that increased technical debt (“We need this big functionality by next month”). You will have to prove to them that although they seem to be getting results fast, this will hit them hard on the long run and that changing specs in the middle of the sprint isn’t useful either. Practically, you have to convince them that you need to go slower or everything will fall apart. You might find that they don’t trust you (“It worked fine until now. It’s not going to crash if we do this little thing also. Don’t worry and just focus on delivering.”) so you will need data to prove your point. Nr. 2> The ones that took these decisions (“Priorities changed, we need this by next week”) without knowing the risks. This happens especially when, trying to please them, you started working ASAP without addressing potential downfalls. Maybe if you told them or made sure they understand the risk, they would have realised it’s not that important or urgent and things would have been different. Depending on how bad the situation is and on how much you’ve tried (or didn’t try) to warn them, you might be facing a lack of trust (“It’s your fault, you never told us this can happen.”) or you might gain some trust (“Thanks for making us aware of this. Well, what do you suggest we do?”).

The ones that took these decisions (“Priorities changed, we need this by next week”) without knowing the risks. This happens especially when, trying to please them, you started working ASAP without addressing potential downfalls. Maybe if you told them or made sure they understand the risk, they would have realised it’s not that important or urgent and things would have been different. Depending on how bad the situation is and on how much you’ve tried (or didn’t try) to warn them, you might be facing a (“It’s your fault, you never told us this can happen.”) or (“Thanks for making us aware of this. Well, what do you suggest we do?”). Nr.3> The ones that had no clue that you are struggling to meet the deadlines and that quality is suffering because of that. They made no changes to the initial plan but you underestimated the work that needed to be done and you ended up building the debt yourself. It probably the hardest situation to be in because you are trying not to loose the trust of the stakeholders while explaining why things are going slow.

Whatever the case is, you need to have a conversation and present the situation. Depending on the situation and what kind of stakeholders you have (Nr.1, Nr.2 or Nr.3) you will have to prepare differently for the conversation. Here are some things that can help:

A ratio of time spent developing to time spend fixing bugs (if you think you have a problem with technical debt but you haven’t been tracking this yet, you should). Developing brings value, bug fixing doesn’t. If your team is wasting 30% of it’s time looking for and fixing bugs then you aren’t delivering value 30% of the time (translate this into money for a bigger impact). (good for Nr.1)

to (if you think you have a problem with technical debt but you haven’t been tracking this yet, you should). Developing brings value, bug fixing doesn’t. If your team is wasting 30% of it’s time looking for and fixing bugs then you aren’t delivering value 30% of the time (translate this into money for a bigger impact). (good for Nr.1) Burndown and velocity charts showing the amount of work being done with the current state of the project. (good for Nr.1)

showing the amount of work being done with the current state of the project. (good for Nr.1) A short description of the state of the team . The goal here is to make them understand that if the team is frustrated or in burnout, then you won’t be able to deliver neither quality or quantity. (good for Nr.1 and Nr.2)

. The goal here is to make them understand that if the team is frustrated or in burnout, then you won’t be able to deliver neither quality or quantity. (good for Nr.1 and Nr.2) A list of areas that need refactoring and potential risks to the business if they are not refactored. The idea is to make the stakeholders aware that if something goes wrong in those areas, their whole business might suffer (sales going down, bad reviews etc.). (good for Nr.1 and Nr.2)

and if they are not refactored. The idea is to make the stakeholders aware that if something goes wrong in those areas, their whole business might suffer (sales going down, bad reviews etc.). (good for Nr.1 and Nr.2) Examples of bugs that almost made it live and could have had a big impact on the business. Giving a real example (“Last month we had a bug that made you unable to introduce your credit card detail if you had a discount on your items”) will make them pay more attention. (good for Nr.1 and Nr.2 and even for Nr.3 if you are showing that you are aware of the issues and are working on them)

that almost made it live and could have had a big impact on the business. Giving a real example (“Last month we had a bug that made you unable to introduce your credit card detail if you had a discount on your items”) will make them pay more attention. (good for Nr.1 and Nr.2 and even for Nr.3 if you are showing that you are aware of the issues and are working on them) A plan regarding changes that can be made in order to improve quality (for example how you could include one area that needs refactoring in each sprint for the next 6 sprints while still delivering business value). (good for Nr.2 and Nr.3)

regarding changes that can be made in order to improve quality (for example how you could include one area that needs refactoring in each sprint for the next 6 sprints while still delivering business value). (good for Nr.2 and Nr.3) An updated schedule showing how milestones need to change in order to develop new features and work on the technical debt at the same time. (good for Nr.2 and Nr.3)

showing how milestones need to change in order to develop new features and work on the technical debt at the same time. (good for Nr.2 and Nr.3) Details on other solutions found inside the team and how this will help reduce bugs and increase quality (for example having a technical two hour meeting with the developers each sprint to discuss new functionalities and how to approach them before starting to code). (good for Nr.3)

and how this will help reduce bugs and increase quality (for example having a technical two hour meeting with the developers each sprint to discuss new functionalities and how to approach them before starting to code). (good for Nr.3) A list of what you need from them. It might be anything from putting a stop to new functionalities until you pay some of the debt, to move some of the deadlines a little bit further away or to add another developer to the team. Be sure that you are mindful of their needs as well as yours and that you find the solution that fits both. (necessary for all of them)

And now you are done with technical debt!

I wish it would be that easy. What you should be doing now is asking yourself — “What is the next step?”. Do you have a problem and need to deal with it right now, or is this just something you should be thinking about, making sure it doesn’t happen in the future?

Before leaving I’m going to remind you just one thing: technical debt isn’t always bad, you just have to keep it under control.

Oh, and here is a quote I think sums up this idea pretty well: