Evolution of architecture in startups
By: Roshan Alexander
Tags: Programming, Microservices, Startup Life, System Architecture, Startup Lessons

This blog is purely based on my personal experiences with architecting systems in multiple startups at different stages.

Sharing my observations on the journey the startups I was a part of, went through, moving from ideation phase to scaling for tens of millions of users or transactions.

Architecting systems in a hyper growth startup comes with its own unique set of challenges. You will be juggling too many pieces in parallel. Tradeoffs and context play key roles in deciding the architecture design solutions which best fits the situation.

I have often seen engineers being unnecessarily ashamed of the solutions that they have built in the early stages; which seems obsolete now and banging their heads on why they could not have envisioned or built a better architecture at the initial stage itself.

In my experience the architecture evolution that startups go through while on their path from ideation to scaling to tens of millions of users, can be broadly classified into 3 stages.

Phase 1 : Making things work

This is the phase where the startup is still figuring out the product market fit, and the road map looks a lot vague for the organisation. There are multiple experimentations around the initial ideas and hypothesis. And proving or disproving them, is the only way in which the product evolves. Since this is a make or break phase for the company, speed of execution is the key here. So the entire emphasis at this stage, should be on solutions that could be implemented with a shorter time to market duration to fasten the feedback loop.

Want to read this story later? Save it in Journal.

Since the road map is quite volatile at this stage, spending efforts on coming up with the best possible and scalable architecture or solutions, would actually be counterproductive.This might mean that we end up over engineering the solutions; investing a lot of time, which could possibly get scrapped even in its entirety or worse, the startup runs out of money before achieving product market fit.

The solutions that are implemented at this phase are largely driven by the skill set of the team. The team itself would be rather small, and the solutions for most problems, would be biased, retrofitted back and limited by their current skill set and experience . This reflects in most of the early architectural and tech decisions, whether it is about choosing a language, framework, tech stack, or how the system is designed. This is acceptable, as long as we are making things work in terms of proving or disproving our core hypothesis.

Phase 2 : Making working things survive

This is the phase where the startup gains some traction and starts growing. At this stage, the road map starts to look a bit more clearer than the initial stage.

This is a tricky stage, since you also need to add scalability to the list of things being juggled, and at the same time you cant take your foot off the speed of building and iteration.

This means all the previous systems that were built, have to be made scalable in one form or another to survive for the immediate future, but with minimal efforts. This could be by some short term solutions, like vertically scaling the systems, adding a caching layer, or a queue in between for asynchronous processing, or throwing in some replicas or shards or probably rebuilding some smaller systems that are at the end of their life cycle.

This is to make sure that we are providing our existing users with the best experience possible, while also maintaining the growth and product velocity, through continued building, experimenting and gaining insights.

Phase 3 : Building a stable and sustainable architecture

This is the phase were the startup starts to see sustainable growth and is projecting a huge scale in the near future. The startup would have mostly figured out its USP, target audience and its core use cases. The organization would also be at a stage were it could predict the growth for the immediate future, with a decent level of accuracy.

The solution approaches selected at this stage should be having scalability as one of their main deciding factors, and not as an afterthought. In addition to this, all the previous systems that were built, have to be reviewed, and made sure that they could survive the test of time for the predicted growth.

Cost of not acknowledging this phase

Many startups that worked very well in the initial two phases make the mistake of not acknowledging this phase. This comes at a big cost, since we are loaning time from our future, to build the present.

There are multiple reasons that can make even the best startups overlook this phase, or delay acknowledging it.

Non-realization that the phase has arrived

This is very common and genuine reason, since the phases overlap quite a bit, and the line is pretty blur, on when the transition towards the next phase should start happening.

This is very common and genuine reason, since the phases overlap quite a bit, and the line is pretty blur, on when the transition towards the next phase should start happening. Optimizing for the short term

The strategy of building things the same way, without worrying too much about the scalability or state of the system, appears beneficial in the short term. Since there is still growth in numbers and a decent product velocity, and most systems would probably still survive, at-least in short term. This often means chasing short term efficiency over long term effectiveness.

The strategy of building things the same way, without worrying too much about the scalability or state of the system, appears beneficial in the short term. Since there is still growth in numbers and a decent product velocity, and most systems would probably still survive, at-least in short term. This often means chasing short term efficiency over long term effectiveness. Skill Gap in Team

If the team itself is unaware of better system designs, then the solutions proposed by the team, would be just marginal improvements over the existing systems.

Building a system for 100K users is usually a little bit different from building it for 1K or 10K users. For a million users, we might need to go back to the drawing board and rebuild some things from scratch, with a drastically different approach. Building for a 10 million might call for an entirely different tech stack or even up-skilling the team.

When the systems start breaking, we might already be out of time for implementing a sustainable solution. Post this, it becomes a vicious cycle of adding hacks, and then more hacks on top of them to make the previous hacks sustain, thereby my making the system a spaghetti of unsustainable hacks. This also increases the chances for a sort of tech Domino effect, where changing one part of the system, takes down multiple other parts. This causes the team to lose confidence and hesitant in touching some core parts of the system.

As a combined result of all this, the velocity at which the product evolution happens, would start to become painfully slow, asymptotically approaching zero.

At this point, the organization would be forced to de-prioritize things in the product roadmap in favour of architecture backlogs, rebuilding things for scalability and stability. This would become necessary, because retaining the current users with a seamless experience, is as important, if not more, as acquiring and growing the users or use cases.

These are the things I have seen work well in this phase :