Optimizing Bugs Fix Policy
By: David Cohen
Tags: Policies And Procedures, Process, Engineering, Management, Quality Assurance

On the two last engineering organizations, I led as a VP, when I joined, we faced that kind of dilemma daily. When we analyzed it, we figured out that there are just too many open bugs all across. In this reality while the support is trying to call what is the right SLA, the developers are overwhelmed with the number of open bugs without a clear idea on how to prioritize between them, therefor calling loose deadlines, and the client-facing teams are also trying to figure out how to move in this mess and lack of SLA communication, while not counting on the R&D to solve soon enough (or even to solve at all), therefore expediting those bugs, more than needed.

The solution, was a combination of “Zero Bugs Policy”, formulating a “fix policy”, and enhancing communication interfaces. This was achieved in some easy-to-implement / straightforward steps.

Reducing the number of open bugs.

Go over the list of all your open bugs and decide for each one of those if it is a real bug, which requires a code change and a new version deployed, or is it some logical issue (e.g statistical AI-related) and then move it into a task (if required), or is this some configuration issue, for which, you should open a support ticket and let your technical support personnel solve it soon enough. Even if this is a real bug, decide if it is something you need to fix, if it’s minor and already alive for a few months, it probably not worth a fix, either-wise if it is on some component you are going to replace soon, don’t fix it. If it still “survives” as a bug — You should solve it fast enough (not longer than 30 days), which means bugs are not aging in the system.

Issues workflow

This step resulted in reducing the number of open bugs in the system from high 100’s into low 10’s. Now it’s clearer and easier to communicate over a shorter list of bugs. On average every team has not more than 5 open bugs to solve in any given moment.

Formulate the fixing policy and correlate between it and contractual SLAs

The next step was to make sure everyone is aligned on what are the fix policies required for different bugs. Adding custom fields into JIRA, using some of the existing JIRA fields for bugs, and making sure that all the fields that are needed for this formula are mandatory when ticketing a new bug — We, automatically (JIRA script based) formulated a “bug score”, which is advised to be revisited in some months frequency.

Such fields can be the Number of effected clients and end-users (we are a B2B, SAAS product). The effected component. Is there any workaround?. Which client is it? (prioritizing clients with some VIP score which the client-facing organization can reset every month). Is this a mission-critical function?

Once we had this score we tied it with the “fix policy”. We decided that we want to support the following SLAs: Incident (ASAP), 48 hours, 1 week, 30 days.

Excel based (before we automated it) calculator to figure out bug’s fix policy

This step result was that every bug in the JIRA has a clear fix policy. Statistically, we found out the 95% of the fix policies that were the automation result, were all agreed by the client-facing team, technical support team, and R&D organization. We also add a JIRA process in which one can argue the automatically calculated score.

Monitor Bugs Fixing progress

The last step was to create visualization and a push/pull call for action items over our fixing policies.

JIRA dashboards and data studio reports that present all the needed information, including aging bugs, policy meeting grade (on a team level), and bugs amount (created and solved).

Zero Bugs Policy — Compliance

A list of bugs that didn’t meet the fix policy

Push notifications, Slack channel, and Email based were sent to the bug’s reporter and assignee whenever a bug wasn’t progressing at the right pace. We allowed a maximum of 3 days in which the bug’s assignee could have “argued” the “fix policy” and/or confirm and verify the bug is on him. We alerted “close to aging” bugs and we nudged when a bug was crossing its fix policy.