Career Growth Frameworks in Software Engineering: A Review
By: Simon Gerber
Tags: Engineering Mangement, Management, Programming, Software Development, Careers

Background and Context

There is an apocryphal story of a company that hired a consultant to help identify and improve their culture. The consultant asked, “what do you have to do to get promoted around here?” Upon hearing and recording all the responses, the consultant handed that back to the company and said, “There’s your culture.”

A career framework is inextricable from your company’s culture and should be put in place very carefully. It can take a substantial amount of time to ‘get it right’ and it adds a large amount of planning and process. It’s tempting to simply copy what other people have done, or what you have seen done at previous employers. However, we must be aware that there is a vast amount of social and historical context bound up in the Way Things Are.

Career Ladders — Historical Context

Prior to the industrial revolution, middle-management wasn’t really a thing. Usually no one other than the owner of an enterprise had much to do with running it. There were not many large organisations outside of the Church and Military. Those that did exist were sadly mostly based on slave labor.

Post industrial revolution, the concepts of division of labor and management came into their own as ways of optimising mass-production and exploitation of resources. Workers were seen as cogs in the industrial wheel and control structures were put in place to optimise production.

A well known example is Taylorism, also known as ‘scientific management’, named after Frederick Winslow Taylor (1856–1915). Taylor sought to break down the manufacturing process into repeatable, well-understood tasks that could be measured and optimised¹. In his vision, engineers would identify and break down the processes which the workers would then perform.

While Taylor had hoped his methodologies would reduce friction between workers and management, his methods focused on the separation of planning from execution and direct from indirect work. This had the effect of turning workers into mindless, replaceable cogs in the machine.

Following Taylorism was Fordism, pioneered by Henry Ford in his manufacturing plants. In contrast to Taylorism, which sought to optimise workers, tasks, and machines, Ford’s contribution was to introduce the assembly line to bring the work to the workers. It also included a focus on minimising costs rather than optimising profits.

Worker turnover was high. Supposedly, he had to hire 52,000 people in 1913 to fill a workforce of 14,000, but Ford sought to offset that by increasing pay. While heavily criticised, the efficiency gains that Taylorism and Fordism brought to American manufacturing arguably contributed to America winning the second world war and allowed them to dominate the industrial world for some time after.

In post-WWII America, rigid corporate structures were dominant. Technological and social progress was slow (as compared to today), leading to a general expectation of stability and an unwritten promise that employees would be loyal and employers would provide lifelong employment. Although, if you were a worker in a Ford plant, that meant subjecting your household to regular inspections to make sure you were behaving in the American way².

The corporate structures that were emerging at this time grew out of the success of ‘scientific management’ and were by and large a transcription of this process. The management layer were Taylor’s engineers, defining and breaking down the process, which was then doled out to the unskilled workers for execution. In this context, a successful career in the corporate world would be marked by a rise from unskilled worker into skilled management.

Following on from this, in the late 1970s, economic boom-bust cycles started to shatter the illusion of stable, permanent employment, just before globalisation destroyed it entirely. The workforce was now multi-generational, shifting and unstable. With manufacturing being moved off-shore, there was also an employment shift towards knowledge workers. Innovation and creativity gained greater emphasis.

Software engineering as a profession was coming into its own around this time and was shoehorned into the traditional corporate model even as that model was losing relevance. Not only was the traditional corporate model losing relevance, but the role of the programmer within it drastically changed over a short period of time.

Computer was, after all, originally a job title. It referred to a person who computed and tabulated information such as navigational data or interest rate tables. This usage of computer dates back to the 17th century and was not regarded as a creative profession. As mechanical and electronic computers started to take over the actual work of computation ,the role of the programmer emerged, but still in the shadow of a world-view that treated computing as part of the production line.

As an example, see the seminal text “The Mythical Man Month,” written by Frederick Brooks in 1975, which provides a fascinating glimpse into the dawn of software development as a profession. Much of the book is still relevant, but some was outdated within years of publication, such as his description of the surgical team, in which one “rock star” programmer (the surgeon) writes all the code while a team around him take on the mundane work of desk-checking the code and punching it in.

The rapid evolution of computing and of programmers into product developers or software engineers, against the backdrop of a corporate world rooted in Taylorism, is clearly going to be a fertile ground for misalignment and misunderstandings.

Problems with the traditional career ladder

Perhaps the most well-known criticism of career ladders is the ‘Peter Principle,’ which dates back to the late 1960’s. If an employee does well they are promoted. Promotions cease when they are promoted to a level at which they do not do well. As a result, over time, every position tends to be occupied by an employee who is incompetent to carry it out.

In contrast to the Peter principle, which assumed promotions were granted out of a genuine desire to recognise and reward performance, we have the Dilbert Principle. Coined in the mid 90’s, it supposes that promoting the incompetent into management is the most effective way to get them out of the way of the people who do the ‘real work’. I.e — the developers and software engineers.

In software engineering, and other knowledge-based disciplines, it is a common complaint that there needs to be a way to reward and compensate skilled employees that does not require a move into management. The most typical reaction to this complaint has been to introduce a dual track career ladder. The idea is to provide an alternative set of titles which pay as much as (or more than!) management while allowing an employee to remain an individual contributor.

An imaginary, but typical, “dual track” career ladder

This is fine in theory. However outside of a few outliers like Google (where ‘Google Fellow’ is a technical role equivalent to a VP) it is often implemented inside the framework of a traditional, hierarchical corporation with all the Taylorist baggage that comes with it. The result is that technical roles are still seen as workers, and hence inferior to management roles, or that the hierarchy and bureaucracy that exist in spite of the dual-track path are not conducive to an environment where engineers feel valued.

So what can we do to make engineers feel valued?

Intrinsic vs. Extrinsic rewards

Intuitively we understand that we want a satisfying career and enough money to support our desired lifestyle and, if applicable, dependents. As an employer, we may also wonder if part of the deal is to motivate employees to work harder and do better, if there is a reward to be gained.

This was certainly a key facet of Taylorism: The idea of the ‘carrot and the stick.’ The theory that workers can be motivated by a combination of rewards and punishments. Whether this can work for repetitive factory work is debatable. But even if it does, software developers are not factory workers. Once the basic requirements of adequacy and fairness have been met, external rewards cease to be a strong motivating factor. In fact, in some cases they can even diminish motivation.

Daniel Pink speaks of this in his book, “Drive,” which is often named by technical leaders as one of their most influential reads. It can be intrinsically rewarding to perform a task you enjoy, like crafting well-architected micro-services or finding a particularly elegant solution to a design problem. But once the external reward becomes the reason to perform the task, it can start to take over and kill the intrinsic enjoyment.

Another problem with purely external motivation is that it can cap performance if implemented poorly. For example, if you will receive a bonus for achieving a 10% improvement in some business metric, why would you aim for 15%? If you’ll get a bonus for being ‘on time’, according to some arbitrarily set date, why would you try and get it done any faster? There can also be temptation to game the system to achieve the rewards, encouraging short-term thinking, loss of creativity and, in the extreme, cheating and unethical behaviour.

The purported solution is to ensure intrinsic motivational factors are acknowledged and included. A growth framework cannot be solely about deciding how much someone is paid and what their title is. It must also expose and elevate internal motivational factors.

The three prime factors, as described by Pink, are Mastery, Autonomy and Purpose, which are relatively self-describing.

A career ladder — or other framework — can help us assign compensation, which takes care of extrinsic motivation. But can they help us with intrinsic motivation?

The Debate

Traditional career ladders, and the top-down hierarchy they imply, are seen as anathema by many who share the progressive ideals of Kaizen [continuous improvement], collaboration, autonomy, and organisational agility. See, for example, this post by Graham Lea, who argues among other points that assigning skill levels is a regression problem, not classification, so titles and levels don’t make sense. Netflix has a similar structure (or lack thereof).

Others argue that ladders or grades are necessary to provide a framework for encouraging growth, motivating staff and providing an objective framework for assessing performance. The criticisms levelled at hierarchies are somewhat mitigated by proposing alternative terms like ‘career paths’ and ‘career lattices’.

The problem is compounded when we expand the scope of the debate to specialism vs generalism. There is a growing resistance from some quarters to the idea of having a ‘architect’ or ‘tech lead’ dictating technical choices from the top-down. This contradicts autonomy, which reduces intrinsic motivation. Rather, with autonomous teams and micro-services, we can try to create an environment where good architecture and decision making is everyone’s responsibility.

But if something is everyone’s responsibility, then it often becomes no-one’s responsibility. Pat Kua argued that even an autonomous, self-sufficient team needs a tech lead. The anti-title camp provided a counter-argument, which was then rebutted by Pat Kua. After the dust had settled others weighed in with their thoughts on the debate — making it clear that the issue is complicated and divisive.

The idea of having any sort of ladder or lattice at all also raises another question. How are we going to place engineers onto our ladder? On which rung do they stand? Perhaps we need to measure their performance?

Measuring developer performance

Tell me how you measure me, and I will tell you how I will behave. — Eliyahu M. Goldratt

You can’t. It doesn’t work. We know that, now.

The assumption that developer performance can be measured comes from a flawed analogy with the manufacturing world. But personally, I can relate more closely to a construction metaphor. The speed at which a brick wall can be constructed is directly correlated with the speed at which you can lay bricks. So the speed at which a program can be built correlates with the speed at which you can add lines of code, right?

History has taught us that we categorically have to say no, this is wrong. Like most incorrect but popular theories, there is a naive oversimplification followed by an extrapolation which provides a compelling grain of irresistible logic but ignores other pertinent facts. Nobody ever just builds a wall for the sheer pleasure of it. The construction serves a purpose. The wall would be part of a bigger construction project with a noble goal like housing the downtrodden or reducing congestion on our roads.

Construction projects rarely run on time. Often the larger the project the bigger the overruns and the longer everything takes. But why? Surely it’s a simple calculation based on the productivity of your brick layers?

Oh… did we forget the planning committee and the government paperwork and the architects and the site surveyors and the community consultation and the blind spiders and the ground works and the brick layers and the tilers and the waterproofing and wet weather and … well, I can probably stop now.

In our construction analogy, what role do software engineers play? The brick-layers or the architects? The ground-works team or the surveyors? The water-proofers or the community consultation committee?

The answer is most of them. Especially with the recent push towards cross-functional teams and T-shaped employees. A software engineer’s goal is not to write code, it’s to build product.

This is the fundamental reason why all attempts to objectively measure individual developer productivity ultimately fail. Intuitively we understand that the role of a UX designer is not to draw wireframes and mock-ups, but to design experiences. Wireframes are a means to an end, but we would never think to count ‘number of wireframes’ as a performance metric for a designer. Yet — with software engineering , the myth persists — as evidenced by the number of metrics that have been explored over the years, such as:

Working hours

Source lines of code

Defect rate

Function points

Story points

Code “impact”

Many of the metrics above actually can be useful if applied at the team or project level. At a minimum they help build a picture of business as usual, so you can understand if changes you are making are having an impact. Such data driven decision making is in vogue at the moment, but is outside the scope of this essay. However, as a tool for assigning a software engineer a rank within a career framework, we must conclude that defining and measuring metrics is not what we are looking for.