How to Squeeze Test Driven Development on Legacy Systems
Tags: oop, tutorial, tdd, legacy
Published: 2020-11-26 03:12:32

*We all love T.D.D. We know its benefits, we have read a thousand tutorials on how to build a system using this technique. But this not feasible for currently legacy systems.*

# What is TDD? 

[Test-driven development](https://en.wikipedia.org/wiki/Test-driven_development) (TDD) is a software **DEVELOPMENT** process that relies on the repetition of a *very short* development cycle.

We turn requirements into very specific test cases.

We improve software so all tests pass. 

This is opposed to incorporating functionality that has not been proven to comply with requirements. 

Created in 2003 by Kent Beck (also the [xUnit testing Framework](https://en.wikipedia.org/wiki/XUnit) testing system author).

## The Cycle

1) Add a test (*it must fail*)

Solely based on behavior. Forgetting everything about accidental implementation.

2) Run all tests.
The new test must fail. All the rest should pass.

3) Write the simplest possible solution to make the test pass.
The programmer must not write code that is beyond the functionality the test checks. ([K.I.S.S.](https://es.wikipedia.org/wiki/Keep_It_Simple) and [Y.A.G.N.I](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it) design principles)

If the all test passes restart the process or ...

4) (optionally) make a refactor (when code stinks).

**NEVER DO BOTH 1 and 4 Together.**

## Design Benefits 

* Testability and better class interfaces

* Simpler designs (KISS, YAGNI, Gold Plating avoidance, [Fake it till you make it](https://en.wikipedia.org/wiki/Fake_it_till_you_make_it), [Fail Fast](https://maximilianocontieri.com/fail-fast))

* Isolation on failures (less debugger or logging uses).

* Design by contracts. 

* Modularization 

* Bottom up building.

* Normal Use cases and Exceptions (Alternate cases) separation.

* Full branches coverage (we cannot add code without a test covering it). 

* Instant feedback / psychological rewards.

* Small steps incremental approach.
 
* Based on Wittgenstein learning ideas by incremental examples and Cognitive Behavioral Therapy.

* Defer implementation issues and Premature optimization.

{% post https://dev.to/mcsee/code-smell-20-premature-optimization-4moe %}

## Requirements

Test must be in *full environmental control*.

No Globals, No [Singletons](https://dev.to/mcsee/singleton-the-root-of-all-evil-50bh), [No Settings](https://dev.to/mcsee/code-smell-29-settings-configs-3eh3), No [Database](https://dev.to/mcsee/code-smell-31-accidental-methods-on-business-objects-gpm), No Caches, No External API Calls and no side effects at all.

TDD can detect [coupling](https://maximilianocontieri.com/coupling-the-one-and-only-software-design-problem) problems. 

Solving them leads to cleaner code focused on business logic alone and *encapsulating [implementation decisions](https://maximilianocontieri.com/no-silver-bullet)*. 

We must deal with coupling problems using *test doubles*: [mocks](https://dev.to/mcsee/code-smell-30-mocking-business-3glm), stubs, fake objects, spy, proxies, dummy objects, etc.

{% post https://dev.to/mcsee/code-smell-30-mocking-business-3glm %}

# Working on existing systems

According to the popular myth, we can't use TDD on existing systems. This is not true. Let's show [an example](https://maximilianocontieri.com/how-to-decouple-a-legacy-system).

## The real world example

* We have a ticketing system needing to showcase several artists performing live-streaming during COVID-19 pandemic.

* Users can search for artists based on a type-ahead selector on a React application.

* System performs database queries on a heavy concurrent back-end system.

* We need to remove *redundant* SQL queries matching part of artists names. 

* [Like SQL Operator](https://www.w3schools.com/sql/sql_like.asp) is very expensive on relational systems, and we are not allowed to change back-end architecture.

# The Problem

We need to simplify redundant searches. That's all.

```
SELECT * FROM ARTISTS
WHERE ((artist.fullname LIKE '%Arcade Fire%')
OR (artist.fullname LIKE '%Radiohead.%') 
OR (artist.fullname LIKE '%Radiohead%') 
OR (artist.fullname LIKE '%Sigur Ros%') 
OR (artist.fullname LIKE '%Sigur%')))
…
``` 
System will execute just:
``` 
SELECT * FROM ARTISTS
WHERE ((artist.fullname LIKE '%Arcade Fire%')
OR (artist.fullname LIKE '%Radiohead%') 
OR (artist.fullname LIKE '%Sigur%')))
…
``` 

Since this part is redundant and expensive to the database.
``` 
OR (artist.fullname LIKE '%Radiohead.%') 
OR (artist.fullname LIKE '%Sigur Ros%') 
``` 
   
## Let's get to work

> Always start with the simplest problem 
 
1) Add a test (empty case) 

{% gist https://gist.github.com/mcsee/9c07677d781d362b513265371ee425ae %}

Notice:
* Class **LikePatternSimplifier** is not created yet.
* No function **simplify()** is defined.
* We number tests according to definition order.
* First test is the easiest one and also the *Zero Case* of Zombies methodology.

{% post https://dev.to/mcsee/how-i-survived-the-zombie-apocalypse-59gj %}

> Test fails (as expected). Let’s create the class and the function.

3) Write the simplest possible solution to make the test pass.

{% gist https://gist.github.com/mcsee/32aa44cff1af0494bed7666bbf9ec68c %}

Notice:
* First solution is always hard-coded.

* * *

> Continue with another trivial case

1) Add a test (simple expression) 

{% gist https://gist.github.com/mcsee/a4076fb03f7af62a82d88ee7d803ae3f %}

3) And the simplest solution for both cases.

{% gist https://gist.github.com/mcsee/548c0df24149b087b6e5bd30e3e866c0 %}

Works like a charm in both cases.

We are taking **baby steps**, **slicing** the problem and following **divide and conquer** principle.

* * *

> Continue with another (not so simple) case:

1) Add a test (two independent expressions).

{% gist https://gist.github.com/mcsee/63f2766fcbb24aeb2fe03c2b4be15166 %}

Code works correctly without changes. Is this a good test?

We will discuss it on a more advanced article.

Let's move on.

* * *

> Continue with a desired and juicy business case.

1) Add a test (one expression containing the other).

{% gist https://gist.github.com/mcsee/9719eeea751c2eef90efb6717bf8503b %}

> Let’s make it work.

3) Add the simplest solution for all the already written cases.

This is an ugly algorithmic solution, but we will improve it with a refactoring once we become more confident.

We cannot *fake it* anymore. We need to *make it*.

{% gist https://gist.github.com/mcsee/ff2a499a73034fa621ae684742de6d0a %}

Ugly, not performant, undeclarative and complex.

We don't care. We need to gain confidence and learn on the domain.

Luckily, we will soon have time for better solutions.

* * *

> Continue with another case.

1) Add a test (left expression containing the right one) 

{% gist https://gist.github.com/mcsee/851bbc77560f890f6676031d2c1cb033 %}

… and we are Green, so we are covering the business rule stating that terms order is not relevant. (Commutative Property).

We make it explicit so no smart refactor can ever break it!

* * *

> Move on with another (not so simple) case.

1) Add a test (Capitalization is not relevant to MYSQL engine but our users might not be aware of that). 

{% gist https://gist.github.com/mcsee/5e195837ad99e937c9951d4cea7b5036 %}

2) We run the tests and **the new one** is broken. Let’s fix it!

3) The simplest solution for all the already written cases (with the new case).

{% gist https://gist.github.com/mcsee/e0db739062069689de3efcd018192e39 %}

… and tests are all green again with the ugly improved solution.

* * *

Code smells and we have several test cases. We need a better solution.

4) Let’s refactor the solution with a more efficient and readable one.

{% gist https://gist.github.com/mcsee/0beea3108976e868fdc78292b24ffbf4 %}

* * *

> Let’s test first production scenario requested by customers.

1) Add two unrelated redundant prefixes

{% gist https://gist.github.com/mcsee/2bd6fdca179095400a4f5f6f9e52a696 %}

And it works !

* * *

> We inject it on our legacy code:

Before

{% gist https://gist.github.com/mcsee/25cb8c5f34c73399e24d9b338941f687 %}

Let's inject it.

{% gist https://gist.github.com/mcsee/820585e06dad19fde4cddd17d357a473 %}

* * *

![Old car](https://dev-to-uploads.s3.amazonaws.com/i/62o6p7enwl9umtdepo10.jpeg)

# What really happened

Up to here, we worked in isolation scenario. 

Software development is a group activity. 

The quality assurance engineers found additional possible benefits.

Pattern could be in the middle of the string. 

Customer agreed to add this functionality.

> Lets consider those cases.

{% gist https://gist.github.com/mcsee/ab38f62dcecbe3f1a421f4eddca93b9b %}
  
* * *

New cases are broken since they were not represented by a previous one. We keep fixing them.

4) Let’s change the solution to cover all previous cases and the new ones.

{% gist https://gist.github.com/mcsee/c1a978fb7b7ca24fed559ab83844887c %}

## The end is the beginning

- TDD works in all stages.

- Using CI/CD codefix went into production.

- Happy ending.

![Terminator](https://dev-to-uploads.s3.amazonaws.com/i/4gx0ntjehh6996zt26gp.png)

Once we submitted the *intelligent* SQL simplifier something bad happened.

This was actual SQL after terms of bad handling:


```

SELECT * FROM artists WHERE (())  

``` 

This SQL generation mistaken as an empty condition.

* * *

> So we will fix it TDD Way.
We isolate the defect and add it as a broken TDD Case

{% gist https://gist.github.com/mcsee/399681e65513e52b989477831d2c1f69 %}

Of Course, it fails since previous implementation brought an empty solution (and a customer complaint).

We can fix it by doing a duplicate's remover case-insensitive pre-processor at the beginning of simplify function:

{% gist https://gist.github.com/mcsee/0dae05cfead4089b3da9c93669e193f1 %}

To see if we must test private methods please visit [shoulditestprivatemethods.com](shoulditestprivatemethods.com) 

* * *

Tests are green again. 

Not dealing with case-sensitive duplicate's algorithm worked again.

> Lets consider a different order.

{% gist https://gist.github.com/mcsee/b82191e3251fff7d23fb97bc640f7dae %}

> Against our intuition we see it fails.

This is because the unit is bringing a ‘Yes’ instead of a ‘yes’.

The solution depends on the product owner. We can:

1) normalize all outputs.

2) change the test based on the property that our *SQL Engine* is case-insensitive on text fields.

We choose 2) 

* * *

Tests are green again

> We add more tests considering mixed cases.

{% gist https://gist.github.com/mcsee/04555137bdad3fd71b3845e21c0e8585 %}

* * *

## Missing Opportunities

Tests worked with the new Solution and given expected SQL.

Case went to peer review.

One of the reviewers asked about *not like* comparison finding an improvement opportunity.

We asked our customer-on-site for agreement.

If user chooses not to see *‘head*’ => it is choosing not to see *‘Radiohead‘* and *Talking heads*.

In SQL: NOT LIKE ‘%head%’ implies NOT LIKE ‘%Radiohead%’ which is redundant in an AND condition.

Our simplifier was already aware of that, so we injected in a second place being confident tests were already covering that scenario.

* * *

# Conclusions

- Quality Assurance engineering should add a broken integration test before correcting the implementation that should comply with it.

- Implementing on a big system requires special techniques to gradually remove coupling.

- TDD influenced all the written code.
We have all the new code covered. (17 unit tests and 3 SQL Generation tests).

- We gained confidence on every new case add ensuring they didn’t break previous ones.

- We should [only test private methods](http://shoulditestprivatemethods.com/) using method objects/FunctionAsObject or reflection.

- We used TDD on Development, Code Review, QA fixing and Production Defects Life-cycle.

- We developed a parallel customer system (the tests). They will always be our first omnipresent user.

- Solutions were as simple as possible.

- It’s __very possible__ to make TDD on existing projects with lots of code.

- TDD does not replace or overlap QA Process and tasks. 

- Multiple roles were involved and added value: Developers, QA engineers, Customers and Code Reviewers.

- We faked it until we made it.

- TDD does **not guarantee** a good design.

- We should never **change or optimize** not covered code.

- IF you like this article please [let me know](https://twitter.com/mcsee1), so I can write more on TDD on Legacy systems.

* * *

> Legacy Code is all the code without tests.

[_Michael Feathers_](https://maximilianocontieri.com/software-engineering-great-quotes)

#Credits

<span>Photo by <a href="https://unsplash.com/@polarmermaid">Anne Nygård</a> on <a href="https://unsplash.com/s/photos/old-junk-car">Unsplash</a>