Adoption of engineering standards at Typeform
By: Cesar Lugo
Tags: Organizational Change, Github, Data, Quality Assurance, Software Development

Photo by Scott Webb on Unsplash

How we’ve influenced organizational behavior with discussion, alignment, and tooling.

Authors:

Members of the Engineering Intelligence Team: César Lugo, Kao Felix, Javier Gómez, Andrei Danilov.

The story is old: An engineer sees an opportunity for improvement. Maybe it’s the logs that look wildly different across every service they maintain, making everything harder to debug. Or it could be the error handling that is inconsistent: sensible in one service, confusing in another, and missing altogether in yet another. Turns out people are tired of repeating the same comments in code reviews spending time arguing about the form instead of the feature. “We should make it consistent” and “this should be a standard” are common reactions in this scenario.

In small teams with a few dozen repositories, adopting standards might require the work of just a couple of motivated engineers. However, in the context of a growing organization with hundreds of engineers split between dozens of teams, working across hundreds of repositories — with conflicting priorities and communication overheads — motivation alone won’t do the trick.

At Typeform, we have been tuning up for more than a year an inclusive framework for technical organization change. This framework involves the whole lifecycle: proposal, discussion, approval, tracking, and accountability.

The flow of a standard, from proposal to reporting on adoption.

Starts with the standard proposal

Anyone can submit a standard proposal. Having an open process encourages people with different roles and perspectives to participate. A proposal could come from platform engineers, product engineer leads, infrastructure, and even managers. This open process also helps to find possible drawbacks, blockers, and improvements. Comments could come both from the engineers implementing the changes, and the stakeholders of any potential improvements in quality, velocity, reliability, and so on.

Lifecycle of a proposal:

Draft: Collaborative stage where authors work together to bring a proposal to life. They follow a template that guides them in creating a proposal that:

Is well reasoned.

Is supported by data.

Has considered and compared alternatives.

Has a plan for implementation.

Has a strategy for measuring adoption.

2. Feedback: The proposal document is ready for async review by domain experts, and to be commented on by anyone that is interested. Observations, support, improvements, and objections are raised at this stage, shaping up the final version of the proposal. When enough feedback has been received, and all points have been addressed, the proposal moves to the next stage.

3. Final Decision: This stage is a meeting with the final decision maker (by default, our Director of Platform), the author(s) of the proposal, and any specialist in the area when needed. Pros and cons are weighted, feedback is reviewed, and a decision is made: Moving forward or Not for Now.

4. Moving Forward:

The decision is communicated during the Engineering All Hands meeting.

The documentation describing how to adopt the standard is rolled out.

Plans and objectives for adoption are set.

The tracking of the adoption is implemented and displayed in an open dashboard.

A screenshot of our standard proposals board lifecycle, with some examples.

Set your adoption metrics

The evaluation of a new standard is defined and implemented in code.

A dashboard with the visualization of the adoption rate of all standards over time is open to all in the organization.

Company-wide and team metrics are available.

The adoption rate is communicated frequently in all hands, and in weekly scheduled notifications.

Our Engineering Intelligence team has put in place a system to evaluate, visualize and communicate the state of standards adoption over time. This avoids approved ideas from falling under the rug of day-to-day work and important new features. The dashboard also gives us a way to quantify our accumulated technical debt and maintainability costs, which leads to more data-informed decision-making.

A screenshot of one of the tiles of our dashboard. It shows the progression of the adoption of several standards over time.

A key part of the success of this evaluation system comes from the fact that if a team owns a domain, this team also has the ownership of the code that evaluates a standard in that domain.

For example, one of our Platform Teams proposed and implemented a standard to migrate our repositories from Travis CI to Github Actions. The Intelligence team provides the infrastructure and the system for the evaluation to take place but, the Platform team owns the specific code to measure the adoption of this migration, and they also provide support to other teams to comply with this standard.

A code extract of our Travis CI -> Github Actions migration evaluation.

Report what matters

Adoption rate objectives are set by management

Capacity and roadmap planning is done by teams

Progress is updated in recurring ceremonies

A screenshot of one of the tiles of our dashboard. It shows the overall adoption of all standards.

Rolling out a new objective needs to be done with enough time and visibility so teams have a good head start to comply with it. Initially, standards were required as soon as they were approved, but we received push-back from teams with lots of new features and priorities on their plate. So we reached a sweet spot: if a standard is approved in 1 quarter, it will be required for all teams to comply with it by the end of next quarter. This has worked well so far. We have a stream of new standards becoming mandatory each quarter, and teams are fully prepared to work on them.

Without the technical leadership behind this framework, no amount of effort would move the needle. Our leadership team has bet on reserving capacity, aligning stakeholders, giving clear communications, and setting the stage for teams to report and track their improvement. This allows us to impact the behavior of the engineering organization in a consistent, coherent, and measurable way.