WTF even is Software Engineering?
Tags: sdlc, kiss
Published: 2024-04-24 09:10:31

While figuring out [why coding is so overcomplicated](https://dev.to/matthijsschuurman/why-is-coding-so-ridiculously-overcomplicated-489h) it occurred to me that the majority of it isn't due to the coding itself. But rather the whole [SDLC](https://en.wikipedia.org/wiki/Systems_development_life_cycle) is to blame. But what actually is that? And what should we be doing? And how do we keep it as simple as possible?


## Waterfall
Back in the day we used to do [waterfall](https://en.wikipedia.org/wiki/Waterfall_model), which has its pro's and cons... The projects we used to run were quite ok, at least we managed to meet the deadline with all required features and the quality was good enough. Now I'm not here to promote waterfall (although [Iguazu](https://en.wikipedia.org/wiki/Iguazu_Falls) is absolutely stunning) but it does nicely illustrate what is needed for software engineering:

![Waterfall timeline](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntmlvnjh3aisspvazthk.png) Why is this relevant? Whatever your methodology, spend the fucking time! Don't skimp on analysis, design, dev, QA or ops... it will bite you in the arse!


## Agile
Now fortunately we moved to a more iterative approach where we can evaluate what we've built more frequently. As said, you still need to spend the time but the big difference is that it's a lot more intertwined. Especially QA, deploy and run have become an integral part of the cycle... and honestly it's fucking awesome! 

![Agile timeline](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yib6aad2an5ch9hatvok.png) This is an indication of how it could look... it kinda implies [Sprints](https://en.wikipedia.org/wiki/Scrum_(software_development)#Sprint) but a more [Kanban](https://en.wikipedia.org/wiki/Kanban) style approach would work equally well. Most importantly ensure everyone is involved on a continuous basis...

### Team work
For me the best part about agile (aside from it being called [ASD](https://en.wikipedia.org/wiki/Agile_software_development)) is that it forces you to work as a team. Everybody has to be communicating with each other otherwise it's not gonna fly. I can't stress this enough: 
- **PO, Architecture, Dev, UX, QA & DevOps all need to be fully aligned!**

Any meeting you have on a feature, from conception to deployment, make sure everybody is in it. And for the love of god, take the time to discus... assumption *is* the mother of all fuck-ups. Even if you think everything is clear, it isn't!


## Requirements
I like [requirements analysis](https://en.wikipedia.org/wiki/Requirements_analysis) as a means to determine **what** is needed as it ignores the question of **how** something gets implemented. Don't let a PO come up with a solution, that's what UX, Software & DevOps engineers are for. Sorry PO's, no it's not your job... by all means raise suggestions and examples of how things can be solved but leave it to the team to come up with the **how** and jointly validate it meets the requirements.

Also please ignore the customers' cries that "it has to work like this otherwise it's useless" because that ruins the product... think about all the other customers and really distill what the underlying problem is... **what** is required? 


## Reporting
And the first requirement to consider is:
- As a **Product Owner**, I **must** receive **usage information** for feature X, so that I can validate  **it's useful** to our customers.

Normally we leave this till the very last moment and consequently don't implement it... but please please please do this first, you want to prove something is useful or not... cuz if it isn't... BIN IT! There's nothing worse than useless crap in your product... not only is it cumbersome for your users but it also has to be [maintained](https://dev.to/matthijsschuurman/why-is-coding-so-ridiculously-overcomplicated-489h#fluidity).


## Architecture
This may very well be the most important piece to allow the whole thing to work... but it deserves a whole separate post... or even a series... or rather go get a proper architect and consider what is needed for your organisation, cuz there is no one-size-fits-all.

That said, there are some pointers:
- Like software your **architecture should be flexible**... it's a balancing act between solid guidelines and change to facilitate the ever changing requirements
- **Reducing dependencies** and **compartmentalization** helps to create a more sustainable environment
- **Hire an architect**, someone who knows its shit and actually has time to solve things in the long term... when you build a house you don't just throw builders at it, there is a structure

## DevOps
You'll eventually need to be able to enable a feature in production, but there are a whole host of things that need to be sorted before that:
- Logging for **monitoring** & **incident management**
  - Like [reporting](#reporting) this is always considered last, don't make that mistake!
- Automatic **provisioning** of your infrastructure 
  - Doesn't matter whether you do cloud or on premise, automate it
- **DX**... Developer eXperience is a thing
  - Enable your developers to focus on their core task
- Automatic **validation** of code
  - Code scans, QA, linting, the whole nine yards
- Automatic **deployment** to various environments
  - If you need to manually login, you're doing it wrong
- Feature flags for manual **enabling of features**
  - This is your actual go-live, not the deployment to production

In case this isn't clear yet, DevOps is the fundamental backbone of everything the teams does, not just ops. The more you can automate the better... There's a reason we're moving to [Platform engineering](https://platformengineering.org/blog/what-is-platform-engineering).


## Design
[Less is more](https://en.wikipedia.org/wiki/Less_is_more), [KISS](https://en.wikipedia.org/wiki/KISS_principle)... we as users already got too much shit being thrown at us, we don't need more. So keep the UX as simple as possible, distill till you nailed it. 

Also from a technical/architectural perspective try to keep it as simple as possible. Design it so it's easy to implement and have as few dependencies as possible. This does however imply an [architecture](#architecture) that facilitates compartmentalization so features don't bleed over into one another. 

There's a lot more to be said about design but that's defo a topic for another day!


## Coding & Validation
These 2 things have become so intertwined (at least in my twisted mind) that I think we shouldn't be considering them as separate disciplines. When you've built a brick wall, you don't just walk away... you check whether it's according to spec. Do the same with the code.

I'm still not sure what the best approach is when it comes to validation but a substantial part involves the dev... or inversely the QA engineer is also to a large extent a dev. That person may not per se write the unit tests but it makes sense to have tests written in the same way so they can be created, used and improved together. Keep in mind that tests are a ridiculously valuable part of development, it can make your life so much easier!

**But** don't make the mistake of assuming Dev & QA are the same person! They have a distinctly different focus and you don't want to have devs grade their own homework. 


## So how do we KISS?
You will need to do all this, however begin as simple as possible and build from there:
- Define & implement a **base architecture**
   - Spend time on this please and ask for help
- Define how you **analyse & design feature with the team**
   - Tickets, whiteboards, design tools, tech specs, whatever floats your boat as long as it's clear to the whole team
- Define your **reporting requirements** and implement basics
- Setup basic **automations for coding & validation**
   - Simple things like checking in your `.vscode` directory is already super useful
- Setup basic automatic **infrastructure provisioning** & **deployment scripts**

Stick to the basics, don't make it too big... just make sure you at least start.

### Rinse and repeat
Use frequent [retrospectives](https://en.wikipedia.org/wiki/Retrospective#Software_development) to adjust course and do bigger sessions every couple months to look at the fundamentals.
- **Continuously evaluate** how well you're doing 
- Find **standardisation** opportunities in everything you do (design, coding, QA, DevOps, architecture, everything!)
- **Automate** the fuck out of your SDLC