Pitfalls of full-stack engineering
Tags: team, management, fullstack
Published: 2020-11-07 00:11:06

In this day and age `full-stack` job listings come left and right. Companies have very limited headcount or a breakneck growth and they want to make the most out of the people they hire.

However, regardless what we wish there are no superheroes out there (or they cost an astronomical price and probably have very bad work-life balance) and it is time to discuss the where the so-called **full-stack engineering** can and cannot perform. I intend to give you all the viewpoints necessary to draw your own conclusions.

*Disclaimer:* Currently I work kinda full-stackish (sometimes I write a new SQL query or help in refactoring a `react-native` app). Usually I do mostly frontend development, but I was working on the backend for a couple of years and even was doing automated end-to-end testing in C#. But I would not call myself a `full-stack engineer` as there are way too many layers to be familiar with; I find the idea dangerous with a lot of bad consequences - hence this article.

To see when `full-stack engineering` _works_ out and when _does not_ we need to understand what an engineer really _knows_ and to see how big the actual stack needs to be covered by one person.

## The anatomy of an engineer's knowledge

It is important to know that actual software development is a lot more than understanding the _syntax_ of a programming language or the ability to traverse a _binary tree_.

> As there is **reusable** and **not-reusable code** there is **reusable** and **not-reusable engineering knowledge**. 

### Knowing a programming language `[not reusable]`
The _quirks_ and the _syntax_ of a language cannot be really reused, unless the other language is almost the same - you need to relearn keywords, rules, gotchas and indentation. The less similar two languages the more time you will need, e.g. jumping from `JavaScript` to `Rust` is a bigger one than from `C#` to `Java`.

For instance the idea that `this` is a bindable context of a `function` and not a pointer to the object containing said `function` in JavaScript itself cannot be really reused for another language. That construct (I think) only exists in one particular language and that knowledge is lost when the language is abandoned.

### The ecosystem of a language  `[not reusable]`
This means the modules or packages available (like `react` or `express.js` or `SQLAlchemy` for `Python`) at your disposal usually through a package management system  like `npm`, `pip` or `pip3` for Python, `NuGet` for `.NET`... 

This encompasses the knowledge of which of these libraries is great and which has issues, what tool is the most productive for a given task and so on.

This type of information is slowly accumulated and **not-reusable**.

### Platform specific gotchas `[not reusable]`

**Languages and ecosystem do not exist in a vacuum**. There is the operating system or a sandbox around them (say `MacOS`, `Firefox` or a particular `Android` version). The medium our code runs on interacts and sometimes dominates how we write things. Couple of examples:

- knowing that older browsers won't be able to process certain `UTF` characters therefore completely dropping the rendering of a page
- when you see timeout fails you check if the clock of a `virtual machine` you use is in _sync_ with the real world or not (developing the very idea _took me 2 days of bug hunting_)
- running two tasks with `&&` won't work on `Windows` (hence the existence of [`npm-run-all`](https://github.com/mysticatea/npm-run-all)
- or you can be an excellent `Java` programmer who knows nothing about how the `Android` system handles permissions...

These can only be earned by paying the **“iron price”**: struggling through mysteries, scrolling through **GitHub** issues, asking for help on **stackoverflow** - you can be stuck debugging these for days time to time! 

Therefore this is a very important part of engineering!
**Platform specific knowledge can make or break releases!**

> Having a **team member** that scores high on `platform specific experience` is able to defuse future landmines people would not even start searching for. 

### Abstract engineering `[reusable]`
This is what you learn when you learn **clean coding**, the `SOLID` principles, **design patterns** and **anti-patterns**, **test triangle** and the art of **getting rid of unnecessary layers**.

This is knowing what to _reuse_ and what _not to reuse_. This is your ability to [smell the code](https://martinfowler.com/bliki/CodeSmell.html). Usually sharpened by getting familiar with more than one language and by using a lot of well and not-so-well designed libraries, languages and tech. 

To give you another personal example, when I had to fix a data engineer's python code (_lacking any training beforehand_) I immediately searched if there was a better way to replace variables in a string than positional `%s` signs:

```python
sql = 'SELECT %s FROM %s WHERE id = %s' % ('my_table', 13, 'name')

# versus
sql = 'SELECT {column_name} FROM {table_name} WHERE id = {id}'
.format(id=13, table_name='my_table', column_name='name')

# Did you notice that in the first version the order 
# of the arguments passed is wrong? Not likely to happen in
# the second version - it would look odd immediately
```

As it turned out there was already a syntax for `named placeholders` - which is a superior way since there is a clear indication of intent and also changing the order of parameters has no consequence.

**This is the most reusable knowledge.**

It also helps you with the adoption of a new language: you know what to put into the search bar to get what you look for.

Tip of the day: [Head first design patterns](https://www.amazon.com/Head-First-Design-Patterns-Brain-Friendly/dp/0596007124) is a book seemingly about `Java` but in reality it gives you the _whys_ behind every pattern. Not just for object oriented programmers.

### Tools of productivity `[reusable]`
This ranges from getting the most out of your `IDE` with the right setup to **how to organize your day** to get stuff done. It is also knowing when to stay to finish a problem and when to go home. Or when to decline the wrong meetings.

## How _full_ is your stack?

Now let's switch the perspective to the problem an _engineering team_ would need to solve.

In an average enterprise application or software service the following layers will be there between the **end-user** and the **machines** serving them:

- [frontend] **UI**: layout masonry (CSS or Views/Screens in mobile development), UX principles, accessibility, design, site-building, animations, typography (i.e. do you know the [rules](https://every-layout.dev/) how to make a page look great in 20 different resolutions?) and also the _size of the stylesheet_ served
- [frontend] **dealing with the API and application state**: it means making calls to a server, handling authorization tokens, cookies, bundling code to be consumed by a browser or a phone, event handling, UI testing etc. or the _"JavaScript part"_ (in a web app)
- [backend] **accepting and serving data** from the frontend: input validation, API security basics, integration testing, performance and memory limits...
- [service-to-service communication]: creating an effective **micro-service architecture**, message queues, circuit-breakers, contract testing...
- [database] **performant and safe DB queries, safe DB updates**: from writing actually human-readable SQL code to hinting the query optimizer; but also backups, replicas, distribution, updating schemas to a new version (surprisingly common task)...
- [testing] **testing strategies**, end-to-end testing, unit testing, mocking, test coverage, etc.
- [devops] **building** a simple-to-use and resilient **CI/CD pipeline**, knowledge of Docker and of `containerisation`, service monitoring, remote access and debugging. Setting up environments like `staging` and `production` and also knowing the operating systems that serve these efforts, the ability to resurrect dead virtual machines...

### The burden of knowing it all
To be a `full-stack` dev you need to know all these _on paper at least_ to be true to your title. Maybe not `devops` some would say, or maybe `site-building` is for a design team.

There is also the sad fact that you can't finish learning a stack - _you have to maintain your knowledge constantly_. **New versions** are coming out (like switch `Python 2.x` to `Python 3.x`), **security vulnerabilities** are discovered and we have to adjust how we write code (check the rationale behind [Rust](https://www.rust-lang.org/)).

Or then there is **scaling** an application: your company had 1000 users and now you have to serve 10 million; working at a new scale requires new methods to tackle it.

Another example from the **relational database** world. At the university I have learnt how insanely useful are `foreign keys` in maintaining data consistency - and then I was learning at work how impossible is to use them when you have to maintain a performant, huge database.

> In short, every new language you learn, every new platform you add to your CV there will be a _"maintenance cost"_.

That is time dedicated to: 
* read tutorials
* follow discussions and blogs
* to experiment with what you know
* and to toy with new members of the ecosystem
* .... 

Just as any new feature you add to your codebase would require work just to make sure it still works years later.

> Code quality will eventually suffer when stuff is solely written by `full-stack` engineers 👩‍💻 👨‍💻 if the team was not balanced by expertise, or there are no specialists to review, mentor and refactor.

## When can you be an effective `full-stack` engineer?
Only if there is a lot to share between the stacks. E.g. if the language of the backend and the frontend is the same, like for `React` frontend and `node.js` backend this is a moderately easy task.

Or having a `Java` backend serving a `Java` mobile (`Android`) codebase.

The same is true when companies move people across teams; if team `A` is working on `Android` with plain `Java` and team `B` is working on the same platform with `Kotlin` the time spent on getting familiar with the new tech will be dramatically different than making `C# .NET` monolith developers adopt a `Node.js` microservice architecture (happened to me - most stressful year of engineering).

There is a time factor here: for a seasoned developer _a year_ is required before they can _think_ in a new language (that is _not forcing the logic_ of their _previous one_ onto the new one), and maybe another before they can _excel_ in it.

## Good ideas
Now that I laid out a little framework it will be easy to discuss why certain ideas work and why others don't.

### Map your team's expertise
Instead of thinking in `frontend` - `backend` - `full-stack`, try this: list all the technologies and responsibilities you have in your team, then assign people and their level of expertise next to it:

| Tech      |     Devs       |  Skill levels   |
|-----------|----------------|-----------------|
| node.js   |  Erica, Mike   | Expert, Junior  |
| Postge/SQL|  Mike          | Junior          |
| CSS/SASS  |  Rubio         | Mid             |
| Angular   |  Erica, Rubio  | Junior, Expert  |
| Kubernetes|  -             | -               |

And so on...

### Look for variety
This little exercise above will tell you in which area you need to invest in training your team or _who should you hire_. Maybe the person you will end up with is a `Python` **DevOps** folk who is also adept at `SQL` and can help the team out.

Or you can also realize that maybe you employ a `kubernetes`-free solution since no one has the expertise for it.

This will also mean that you will spend a bit more time reading CVs... Will this person open up new possibilities maybe?

### `Full-stack` is still great for startups
When working at scale is not a concern (yet) it makes sense to go with generalists who can ship features across a lot of technologies. It is great when you have to deliver prototypes and test with real users or stakeholders. 

## Bad ideas

### The `full-stack` feature team obsession
If you run the company in an [OKR](https://en.wikipedia.org/wiki/OKR) fashion - that is most teams have to have a customer-facing measurable outcome - there will be a temptation to go with the _"safest choice"_: having **generalists** aka `full-stack` devs. The idea here is wherever the direction will be (more frontend or more backend tasks) you will have the devs. 

Who will also produce `spaghetti code` when they don't work with their strongest skills.

> Therefore a long-term engineering vision can only be successful having specialists assisting the efforts.

**Rigid team structures** coming from the _obsession of measuring team performance_ will get in the way of **productivity**.

Think about it, why is such a **taboo** to have some specialists dynamically be assigned to teams temporarily needing them?

### Going `full-stack` being a necessity for promotion
Just as I wrote above there is almost no chance you can _"finish"_ learning frontend or backend or database. 

> Therefore going `full-stack` is not adding new knowledge to an engineer; it is merely _splitting_ their attention.

In reality it is not much of a _step-up_ than a _side-step_. And if you remember the table above, even a senior dev can have junior knowledge in a new stack, so in fact they will be `senior juniors` (senior _salaries_ with junior _output_).

(_Fun fact of the day_: the word `medior` is wrong in both Latin and English; you can't be _more medium_).

### Company stack is across too many platforms
Even this article is on the skeptical side it _is_ possible to execute `full-stack` development quite effectively, however the prerequisite is to have as much overlap as possible between said stacks, to make sure that the **ecosystem** and the **platform specific gotchas** are largely the same.

For example `Android` devs would find it easier to work on a `Java` backend.

If you expect `JavaScript` / `CSS` frontend developers to magically learn working with `Java` microservices in their spare time - it won't happen.

### Wonder weapons 🤯
There is an expectation that certain classes of problems can be dealt with a _wonder weapon_ therefore eliminating the skill requirement from an engineering team. I.e. _less devs to feed, more profit_.

This is a very popular thing in site-building: putting together a `bootstrap` kind-of design system and hoping that engineers don't need to learn CSS. But then **user and design requirements start to pour in** and people start using **negative margins** 😱, scroll-bars mysteriously appear in certain browsers 🥶 but not in others and so on...

The same is true for infrastructure - many promise that you can fire your `DevOps` team, but when something _really breaks_ you will need to pull in the expertise who knows how to make a linux pod dance what has happened to it.

### Promising people they can be `full-stack` in 3 months 😳
I am just in shock to see that code-camps can promise you can be a `full-stack` developer in 3 months. So please no, teach them the language, teach them how to search for answers. 

### Getting false confidence by copy-pasting from `stackoverflow`
With a bit of experience, the right IDE and search skills you can fix any minor bugs or get started in basically any language: 

"Oh, I just have to run `kubectl get namespace` and I’ll get all namespaces!" 

It feels like you have just unlocked a new platform, but in reality this is no better than "wingardium leviosa". You don’t know if either `wingardium` or `leviosa` has any additional parameters, you just parrot back these words that do something. As I stated before have the humility to not claim a stack until you have spent a year in that ecosystem.

## Wrapping it all up
I know this was a long article - but to have a meaningful discussion about the `full-stack` trend a lot of pieces had to fall in place. I hope I have given you enough tools to think in the right granularity.

> Thanks for reading it all and let me know your opinion (or corrections) in the comments!