Hackathon Sprint — Approach Projects Differently
By: Zdenek Letko
Tags: Teamwork, Creative Process, Agile, Software Engineering

Recently, we have had the opportunity to approach a new project from a totally different angle. And you know what? We enjoyed the ride and also fulfilled expectations by delivered more than anticipated. This post describes how we took a step away from our standard processes, which are there to guide and protect us, and tried something different.

A Different Project

Our product owner often has a clear vision of what should be implemented within an iteration that might span one or more sprints. In this case, the vision was unclear but the internal deadline was firm. We were expected to deliver a reporting over data which actually didn’t exist in the database yet. The database should have been filled by another team and even the database schema was not yet agreed.

Our discussions revealed that the firm deadline was motivated mainly by an internal need for reports which could be demoed to a validation group to get customers/prospects on board early and to get their feedback as soon as possible. Bingo. This is an ideal situation for getting wild and prototype something. But, how should one quickly prototype something while keeping high quality of our codebase with stress on pair reviews, test coverage and all the standard processes we have?

Hackathon Rules

We defined some rules in the team to define what is acceptable and encouraged:

We have one sprint (2 weeks) full team capacity (backend, frontend, operations, data science, quality assurance) to deliver a solution. The team is therefore capable of touching any part of our product from infrastructure to frontend.

No overtimes, everything should be done within normal working hours. This is not a hackathon with sleeping bags and pizza.

Minimal cross team dependencies. Therefore, nobody blocks us and we do not consume any capacity of other teams.

The solution must follow the latest product requirements — mainly UI design.

We can build reporting over generated data that should model expected reality before the data are going to be available.

We can use whatever tooling available to us to deliver the solution.

Internal standards (such as unit/integration test coverage, high coding standard, etc.) must be challenged. The team have to agree what still makes sense in this new situation.

Pair review stays as we want to learn from each other.

Hacks and shortcuts must be documented on the project wiki page so the future us know where a technical debt is buried.

We must follow deployment and security rules — our solution must be runnable in our existing infrastructure and must not represent any security risk.

Only one team member will not participate in the hackathon. The rotating position of so-called sprint hero is usually responsible for shielding the team during the sprint from ad-hoc inputs and requests from other teams, watching development as well as production environments, and solving any issues that might disrupt the team. We need this position to do the job for us even during this hackathon sprint.

Preparation

We had a week or two prior to the sprint to actually finish our previous tasks and to get ready. One of us started to look into possible technologies we can leverage. They studied different libraries, consulted some of the infrastructure solutions with infrastructure and operation teams. The final proposal was to use a combination of Flyway for loading a generated data set into a separate database (not interfering with our production data), Speedment for quick generation of Java ORM and service endpoints, and our internal API-gateway to control whether a standard production reporting engine or our hackathon result is used as source for the reports. The frontend part was therefore meant to be production ready from the beginning.

In the meantime, our product, UX team and the team responsible for delivering the real data finalised UI expectations and database schema. Moreover, additional requirements on the generated data were defined such that reports look natural and API was proposed to conform with our existing reporting infrastructure. We were ready, heading for the weekend.

The Sprint

Due to our extensive preparation we were productive from day 1. Such that after we kicked off the hackathon, people were already building the solution. Someone started to build a new micro-service and wire it into our infrastructure, another deep dived into the backend logic, frontend guys started development against the API mocks we had, operations engineer jumped on creation of a new database for us and our data scientist started to work on a reusable script to produce a reasonable dataset for the reports. I have never seen before so close and enthusiastic cooperation of the whole team discussing topics, helping each other, completely focusing on a single goal and having fun in the same time.

The first challenge came the second day. We have discovered that almost everyone is waiting on an input from someone else. The stand up ended with a battle plan shown on the picture:

The vertices on the graph represent people and edges some blocking dependencies. The person in the middle is our operations engineer. There were also discovered dependencies on external teams (under the dashed red line). At the end, it was nothing critical — just some required infrastructure code reviews and some parallel work being done on infrastructure to synchronise with. The visualisation of dependencies helped us to understand the situation and to define priorities for particular team members as well as to adjust expectations.

The work progressed smoothly since then. All issues were being addressed as they were discovered. Interaction among team members was quite frequent and the contours of the solution started to appear. I really liked our stand-ups which changed from a status update oriented to problems oriented. Normally only a limited number of people actually cooperate within our normal sprint due to multiple work streams we usually handle. This time, everyone was fully involved and each team member was coming with information to share or problem that needs an attention.

Finally, the integration of data from our data scientist with the backend and frontend also wasn’t fluent as it discovered some missed corner cases. But again, all those findings by developers or our quality engineer just lead to a quick regeneration of data and their deployment to the development environments together with some minor fixes here and there. All was amazingly quick and smooth.

At the end of the sprint, we did deliver a new micro-service incorporated in our production infrastructure that provided realistic reporting over generated data with very easy possibility to switch to real data when ready. The solution was based on few technologies that we have introduced into Wandera and all was working. The result was accepted by the product owner which did a demo even to our CEO and the team received positive feedback very soon. Well done team!

Lessons Learned

There were multiple findings and great impact of the experiment on the team and the team members. I’ll try to summarise at least some of them.

Pawns or players? That is a popular question to get insight into a team health in our company. After this experiment, all team members felt like players as they had opportunity to contribute and were involved in many decisions on the way.

As already said, the interaction and cooperation of the team members was intense and outstanding. In a sense, this was a sprint long team building.

The hackathon was a unique opportunity to have focussed time to innovate in our own way and more naturally than in our day to day. We were even able to crack a long term issue within the code base.

Quick feedback from the product and management is important. Mainly, when everyone in the team feels that some good piece of work has been delivered.

Of course, there are things to question and improve as well.

We have created a technical debt and we knew about it. Some, such as covering the solution with our standard level of automated tests, have been addressed already in a few sprints after the hackathon. Some, such as use of a non-standard database, are still on-hold and needs to be addressed in the future.

Given all the effort invested before the hackathon sprint as well as after it, makes the feature overall price quite similar to what we would have invested if we had approach the project the standard way. The difference can be seen only in the time of delivery, much shorter feedback loop and fun.

The team agreed that the hackathon sprint was a welcome break in the routine but we do not suggest to have every sprint/feature delivered in this style. For us, it was definitely a refreshing experiment and we look forward to try it again when a suitable project knocks on our door. But, we still trust in our standard battle-tested and always evolving processes which are there to protect and guide us.

Also, we do not want to endup maintaining and supporting a codebase built on prototyping. The zoo of languages/technologies would be immense.

Overall, I also appreciate that we were trusted to do stuff differently despite the challenging internal deadline. This is part of a safe environment we have in Wandera from my point of view.

And what about you? Do you see any project around where you could torpedo the standard delivery process and take your team for an enjoyable bumpy ride?