An iterative validation of a new implementation
Tags: crystal, testing, refactor
Published: 2020-12-21 22:46:13

---
title: An iterative validation of a new implementation
published: true
description: A process to replace or refactor a function with no unit test in a program. The goal is to improve our confidence in the end result.
tags: crystal, testing, refactor
---

Sometimes I need to replace or refactor a function in a program. In this post I want to share a process I’ve used to make such change. The goal is to improve our confidence in the end result.

Suppose you have a function `f` we need to migrate. For the sake of simplicity, we will assume `f` will never raise an exception.

```crystal
def f(input : String)
  # ... stripped ...
end
```

There are no unit tests, but said function is widely used through the whole suite of tests.

**First step**, leave the original function and define a wrapper.

```crystal
def original_f(input : String)
  # ... stripped ...
end

def f(input : String)
  original_f(input)
end
```

**Second step**, define a new implementation:

```crystal
def new_f(input : String)
  # something new we hope will work
end
```

**Third step**, run both implementations and detect invalid behaviors of `new_f` with respect to  `original_f`. These invalid behaviors can be either crashes or unexpected results.

To detect them we are going to change the wrapper with a bit of code that you will probably not like at all. Relax, it will be gone at the end.

```crystal
def f(input : String)
  original_result = original_f(input)

  begin
    new_result = new_f(input)
  rescue e
    e.inspect_with_backtrace(STDOUT)
    File.write("bug.txt", input)
    exit(1)
  end

  if original_result != new_result
    File.write("bug.txt", input)
    puts "\n\nUnexpected result: #{input.inspect}.\n  Expected: #{original_result.inspect}\n       Got: #{new_result.inspect}\n\n"
    exit(1)
  end

  original_result
end
```

If there is an unexpected result the program will stop. Immediately. This can be done differently but, for the sake of simplicity, we are stopping at the first invalid behavior.

We run the whole suite of tests and, if there is a crash, a `bug.txt` file will be created with the `input` that caused the invalid behavior.

We can use a smaller program to work on the input that causes the crash.

```crystal
def t(input)
  original_result = original_f(input)
  new_result = new_f(input)

  if original_result != new_result
    pp! input, original_result, new_result
  end
end

# Some trivial cases, maybe
t("")
t("abc")
t("   abc    ")
t("   ")

# The input that caused the crash
t(File.read("bug.txt"))

# You could also use a test framework, of course.
```

This way we can work on `new_f` until the identified case is fixed.

**Iterate the process** of running the whole suite of tests until there is no crash.

Now you have a `new_f` that works as `original_f`. At least, _to the extent the suite of tests needs_. Note that we are not talking about only unit tests of `f`.

Drop the "ugly" code to compare both implementations. Drop the `original_f`. We don't need them anymore. Leave only the `new_f` as the implementation of `f`.

```crystal
def f(input : String)
  # something new we hope will work
end
```

You are done! 🎉

This process was explained with [Crystal](http://crystal-lang.org/) but it can be adapted easily to other languages.

Programming is a tool. Yet, as a tool, it can be used as a process. We made temporal additions to our codebase as part of the process. There is no need for the code you write to always be permanent in the code base.

There are more advanced processes and tools in software verification. This is a small example that might motivate you to look into them.