Agile, hackathons, and the danger of local maxima
By: Nicholas Heal
Tags: Agile, Product Management, Leadership, UX, Development

I love agile. For the first few years of my career I worked in a chaotic waterfall environment. My first experience of switching to agile was a breath of fresh air.

Agile is a rational, considered, and sensible approach to apply in development, and I can imagine a lot of other settings too. There are drawbacks though, and there’s one I want to explore a bit here in the hope of starting a discussion.

One of the best features of agile is the retrospective. It forces your team to get together, and analyse your performance in the previous sprint. Pick out highs, lows, and generally any thoughts. The idea being that you refine, and edge closer to peak performance.

Sometimes I see teams cycle through issues, because they only track their retrospective results back one or two sprints. They end up fixing one problem by creating another problem, and then switch back a few sprints down the line. Ping-ponging between issues like that isn’t ideal, but sometimes you have to make trade-offs when there is no perfect solution, and change can be beneficial in itself.

The question I want to raise here ties in to machine learning though. When I was at university I remember being fascinated by the courses I had relating to the use of evolution in machine learning. The essence of it is that you seed your algorithm with some random values, and they each give you a good or bad result. You then discard the bad ones, and mash the good ones together in a variety of ways to see which ones produce better, and which ones worse. Each generation you run through the strongest survive, and the weakest don’t, hence the reference to evolution.

It all sounds good so far, but you can hit a problem with local maxima. On the graph below imagine the flat axes that run from 0 to 50 represent two parameters you put in to your algorithm, and the vertical height of the chart represents how good a pairing those parameters are.

The issue with evolution is that each generation must be better than the last. That can get you stuck if you start in the wrong place. Imagine you start in the corner closest, and then evolve up to that first peak marked ‘Local Maximum’. Once you’re there you’re stuck. You’re not going anywhere, because the only way to get to the true ‘Global Maximum’ is to go down hill first. In other words, things are going to get worse before they get better.

In machine learning the technique often used to avoid local minima, and maxima is the stochastic gradient descent. This doesn’t guarantee escaping the localised peaks, but it means the evolution has an element of extra randomness, which makes it more likely that your optimisation will jump out of a localised peak, and in to the global one.

Tying this back to agile, I wonder sometimes if teams end up in this same position. Optimising their approach every sprint, but never able to jump across to a greater mountain. And if so how do you combat it?

From machine learning it would seem that adding a bit more randomness to our sprint retrospectives might help. Agile often evangelises keeping consistent teams, but this would suggest that it might be more effective to routinely move people across agile teams to inject some variation. I’d also love to try swapping retrospective notes between teams at random as well. Where I work we list some ‘things we should do better next sprint’, and ‘things we should stop doing next sprint’. It would be great to swap those between teams to see if there’s consensus.

Hackathons come in here as well. Teams have to strive for consistency 99% of the time to help achieve business goals efficiently, which means they naturally gravitate to local maxima. Randomness on the way to the top is frowned upon. There has to be a channel where people can demonstrate new ideas that are completely ‘outside-the-box’ in a rapid prototyping environment, and hackathons are the epitome of that. They are the opportunity to jump to another mountain to see if it’s taller than the one you’re currently standing on.

The other thing I’d like to flag as well — thanks to my love of metrics — is that sprint retrospectives are opinion-based in my experience. We decide what we think went well, and what we think we should change, but we don’t ties those things in to metrics that show if our opinions match our outcomes. In the end, if you produce no work for two weeks, and then say at the end that there was nothing you could have done better, that’s not the perfect sprint. Our conscience’s prevent us from doing that, but I still have some concern that our team thinking we’ve made out sprint process better may not correlate with producing a better outcome for our customers.

As always please leave any comments down below. I’d be fascinated to know if anyone has already tackled these issues. Like I say, I am a huge fan of agile, and it’s the fact that we can question and improve the very process itself that makes it so effective.

All the best,

Nick