Resilience to errors in front-end frameworks
Tags: vue, react, svelte, elm
Published: 2020-06-14 10:33:58

(In a hurry? [Jump to the conclusion](#conclusion)!)

This is an attempt to measure how good are front-end frameworks in dealing with simple errors.

I took the implementations of [TodoMVC](http://todomvc.com/) made in 5 different frameworks: [VanillaJS](http://vanilla-js.com/), [React](https://reactjs.org/), [Vue](https://vuejs.org/), [Svelte](https://svelte.dev/) and [Elm](https://elm-lang.org/).

Then I injected 7 different errors and I analyzed the behavior.

After injecting error in the 5 implementations, I measured the outcome using this scale (ordered from best to worse):

* A. Impossible to introduce such error
* B. Compiler error
* C. 
    * **Nothing render** on the screen and there is an **error** in the console **at start**
    * The app **render** and there are no bugs
* D. The app **render** on the screen, it has a **bug** and an **error** in console **at start**
* E. The app **render** on the screen, it has a **bug** and an **error** in console **during** execution
* F. The app **render** on the screen but it has a **bug**, end there are **no errors** in the console

They are ordered in a way that I find more desirable. For example, having the app not showing at all (rank C) is better than having it appear on the screen but with bugs (ranks D, E, F) because in the first case it's easier to note that there is something wrong.

The best case is when errors are not even possible to be added at all (rank A) or are intercepted by the compiler (rank B). The compiler-error case is only possible for Elm, Svelte and React JSX. Everything else is not compiled.

The worst case is when the app renders on the screen and there are no errors in the console whatsoever (case F). It means that if we don't test our application carefully we are likely to deploy a bug in production.

I also rated the accuracy of the errors:

* **Very accurate**, errors that possibly suggest the solution to the problem (add a `+` to the score).
* **Regular**, errors that give a good explanation and position of the error (no changes to the score).
* **Unclear**, error with bad explanation or wrong indication about the error position, (add a `-` to the score).












# Disclaimers <a name="disclaimers"></a>

* My bias: I work primarily with Elm and I am an advocate for functional programming.
* I use the word "front-end frameworks" in a very loose way here, so that it includes all of them because some of them are languages with a compiler, others are just Javascript libraries. 
* I changed the code with a plain text editor. No linters or any other plugin.
* I didn't run any automatic tests. A good development environment should be able to detect all these issues through the assistance of the IDE or the test suite. Here I am judging the naked framework.
* I didn't use Typescript or Babel to transpile the code. I only used the Svelte and Elm compiler as these are integrated parts of the language.
* Using the code of the [TodoMVC](http://todomvc.com/) application. The result may be affected by the quality of that particular implementation. Different implementations written using the same framework could get different scores.
* Sometimes it is not simple to introduce exactly the same error in different languages/implementations so sometimes I had to modify the code slightly.
* All the code modifications are available at [github.com/lucamug/framework-resilience](https://github.com/lucamug/framework-resilience). Each of the seven errors has its own branch so you can verify how these errors were implemented.

Let's get started!









<hr>
<a name="error-1"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/6ovkzvfi2qs7dj5f0dve.png)
# 1. Misspelling one HTML element 

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/20046064724cbc210bca3afcfeae20148c01710e?diff=split)

I changed the opening of a `<span>` element to `<spam>`, leaving the closure intact, in the footer of the application: `<spam>...</span>`

## Outcome

### Vanillajs

The application renders but the footer is not there. There is an error in the console: `Cannot set property 'className' of null`.

 This is a typical example of an unhelpful error, the text does not contain any hint on what could be the cause and the file where the error happens (`view.js`) is different from where the error is (`index.html`).

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0yvueska0cvvi0gp7xsf.png)

Rank: **D-**

### React

The application doesn't render and shows an error in the console at start:

`Uncaught Error: Parse Error: Line 33: Expected corresponding JSX closing tag for spam`
    
This would be intercepted by the JSX precompiler.

Rank: **B**

### Vue

The application renders, but the footer has a wrong layout.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/619uc96kyotx7t1wog5i.png)

There is a clear error in the console:

`vue.js:525 [Vue warn]: Unknown custom element: <spam> - did you register the component correctly?`

Rank: **D**

### Svelte

Nice compiler error

`ParseError: </span> attempted to close an element that was not open`

Rank: **B+**


### Elm

Changing only one element is not possible in Elm. A span element, for example, is created with `span [] [ text "content" ]`.

Rank: **A**








<hr>
<a name="error-2"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qa35wfdhzzkma7u2bkbn.png)
# 2. Misspelling two HTML elements

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/6cfcf33c0ff29a29b85f945002e4a1e74a342d6b?diff=split)

This time we use `spam` both in the opening and in the closing element. This is a subtle mistake because we could have wanted to use a custom HTML type. But let's assume that is a mistake and see which framework detects it.

[Details of code changes](https://github.com/lucamug/framework-resilience/commit/c31db55970355b195489565f14e522c3a6c43eb0?diff=split)

## Outcome

Elm in the only framework that has a compiler error:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8crh1qpqh6myr9hfkwxr.png)

In Elm, to create a `spam` element you need to write

`node "spam" [] [ text "content" ]`

instead of

`spam [] [ text "content" ]`

Other frameworks either just work or they have a warning:
 
This is the case of Vue that has this in the console:

`vue.js:525 [Vue warn]: Unknown custom element: <spam> - did you register the component correctly? For recursive components, make sure to provide the "name" option.`


* Rank Elm: **B+**
* Rank Vue: **C+**
* Everything else: **C**








<hr>
<a name="error-3"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/p8e7dgob6jk1huun0c6x.png)
# 3. Forgetting a space between `a` and `href`

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/1d0c7859a0ad5b9faa048c2eaafb6957a576e6b0?diff=split)

Just replace `<a href="...">` with `<ahref="...">`. I did this in the footer button "Active" that displays the active todos.

## Outcome

### VanillaJS

No errors at all for VanillaJS, but the button "Active" doesn't work.

Rank: **F**

### React

React return the error `Unexpected token <ahref="#/"` and it doesn't render the application. Another error that will be detected by the JSX compiler.

Rank: **B**


### Vue
No errors at all, the same as VanillaJS, and the footer is also broken:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tbmg6u53q1aycsg13sz0.png)

Rank: **F**


### Svelte

Compiler error `ParseError: Expected >`

Rank: **B**


### Elm

Is difficult to add this error as `a` and `href` are separated by a `[` in Elm:
    
`a [ href = "url" ] [ text = "label" ]`
    
so I added the error this way
    
`a [ ahref = "url" ] [ text = "label" ]`
    
That generate this compiler error `I cannot find a 'ahref' variable: These names seem close though: href`

Rank: **B+**








<hr>
<a name="error-4"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8nklybknb93w5dykdd1h.png)
# 4. Misspelling the content of a `href`

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/289278078377787e77174ec9f396a31976b5796f?diff=split)

Remove the content of href in the "Active" button. From `href="#/active"` to `href="#/activa"`

## Outcome

Changing the content of a string, as expected, goes undetected by all frameworks except VanillaJS and makes the link "Active" not working anymore.

VanillaJS give an error at runtime:

`controller.js:254 Uncaught TypeError: this[("show" + activeRoute)] is not a function`.

Strangely enough, the Elm application kept working also with this modification!

I investigated the reason and I found out that

1. The Elm application apply changes "onClick" instead of waiting for the url changing
2. The Elm application store the entire model in the local storage while all other applications only store the list of todo items

As I consider this some kind of "cheating", I downgraded Elm to the same rank of all other applications.

This error is telling us that having strings in the code is usually a bad idea. This is why in Elm, and probably in other frameworks too, we usually write the routing differently. 

Specifically

1. We create a custom type containing all possible routes
2. We create a function `routeToString` that convert such type into a string containing the url of the route

So, using the function `routeToString` as href when we create links, assure that this type of errors cannot happen. It also has the nice side effect of making the code more maintainable if, in the future, we decide to change the url format or name.

Rank: VanillaJS get an **E-**, all other applications get an **F**.









<hr>
<a name="error-5"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1pf5d7ju0h4lu1jbn3je.png)
# 5. Corrupting the initial state

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/e14aa535b0d3494b7b88bbdaf06c0a8c1acf9438?diff=split)

We change the code where the state is initialized, changing the name of an object key or a variable.

## Outcome

### Vanillajs

There is an error in the console:

`store.js:21 Uncaught ReferenceError: todos is not defined`

and the app is not working. The error happens only if the local storage is missing.

Rank: **D**


### React
 
Unclear error in the console: 

`app.jsx:96 Uncaught TypeError: Cannot read property 'filter' of undefined`

but the file with the error is todoModel.js:18

The app doesn't render.    

Rank: **C-**

### Vue

Unclear error in the console: 

`vue.js:525 [Vue warn]: Property or method "todos" is not defined on the instance but referenced during render. Make sure to declare reactive data properties in the data option.`

Rank: **C-**


### Svelte

The compiler gives a warning 

`Plugin svelte: 'items' is not defined`

but not an error, so it is still building the application.

The application doesn't render and in the console there is the same error `Plugin svelte: 'items' is not defined`

Rank: **Between B- and C-**. I will go with **B-** as maybe is possible to configure the compiler so that it will stop the compilation, instead of just giving a warning.

### Elm

In Elm we change a key of a record from `entries` to `entriez`. This is the compiler error:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ute20yrbvfpjgh14kkh6.png)

Rank: **B+**









<hr>
<a name="error-6"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wu506s7fpcefa8bxw81o.png)
# 6. Corrupting the function that add a new todo item

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/5dd4029df4fa3504ef4dc1c249440597cfebf6cd?diff=split)

Here I mostly change the variable name containing the title of the new todo.

The interesting part of this error is that it reveals itself only when users are adding a new item, so it is harder to detect compared to the others.

## Outcome

### Vanillajs

Error when adding a new todo:

`controller.js:17 Uncaught ReferenceError: title is not defined`

Rank: **E**


### React

Error when adding a new todo:

`todoModel.js:36 Uncaught ReferenceError: title is not defined`

Rank: **E**


### Vue

Changed "value" to "valua". Error when adding a new todo:

`app.js:72 Uncaught ReferenceError: value is not defined`

Rank: **E**

### Svelte
 
Changed "description" to "descriptiom". Error when adding a new todo:

`bundle.js:1031 Uncaught ReferenceError: description is not defined`

Rank: **E-**


### Elm
 
Changed "desc" to "descr". Compiler error:
    
`I cannot find a 'descr' variable. These names seem close though: 'desc'`

Rank **B+**







<hr>
<a name="error-7"></a>
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ydltjxh3o6glho14yox2.png)
# 7. Forgetting to add the method `.length` to calculate the active todos

* [Code diff.](https://github.com/lucamug/framework-resilience/commit/2b48b2d9d54ac16193532efb3590b36c4e5475d3?diff=split)

Usually the total quantity of active todo is calculated by filtering all todos and counting the remaining items with `length`.

I remove `,length` where possible. If the implementation was using a different system for counting active todos, I refacto that a bit so that I could introduce the same mistake.

## Outcome

### Vanillajs

The app output `[object Object]` instead of the number of completed todos:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lx5d5wwl5h52xll52twg.png)

Rank: **F**



### React 

The app output the entire list of items instead of the number of completed todos:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/84uzz715gpbq7kb5osli.png)

Rank: **F**


### Vue
 
The app output the entire list of items instead of the number of completed todos:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/617565ajmtu63o5rb37f.png)

Rank: **F**


### Svelte

The app output `[object Object]` instead of the number of completed todos:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/v3273py0gx01qs6kx4nn.png)

Rank: **F**


### Elm

Changing from

`entriesCompleted = List.length (List.filter .completed entries)`

to

`entriesCompleted = List.filter .completed entries`

Generate this compiler error:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/auabowceb8l2qfuuzkmr.png)
    
Rank: **B**











# Conclusion<a name="conclusion"></a>

This is the summary

```
  Error n.   1   2   3   4   5   6   7
--------------------------------------
VanillaJS    D-  C   F   E-  D   E   F
    React    B   C   B   F   C-  E   F
      Vue    D   C+  F   F   C-  E   F
   Svelte    B+  C   B   F   B-  E-  F
      Elm    A   B+  B+  F   B+  B+  B
```

Let's try to quantify the result giving some number:

`A: 10, B:  8, C:  6, D:  4, E:  2, F:  0`

For the errors we add 1 when there is a `+` and subtract 1 when there is a `-`.

```
  Error n.   1   2   3   4   5   6   7
---------------------------------------------
      Elm   10   8+  8+  0   8+  8+  8 = 50++++ = 54
   Svelte    8+  6   8   0   8-  2-  0 = 32+--  = 31
    React    8   6   8   0   6-  2   0 = 30-    = 29
      Vue    4   6+  0   0   6-  2   0 = 18+-   = 18
VanillaJS    4-  6   0   2-  4   2   0 = 18--   = 16
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1c51utv1vlz286r8rs1l.png)

It seems evident from the result that a compiler helps to detect these type of errors (Elm, Svelte and React).

Elm leads the rank mostly because it is strictly typed and the fact that everything in Elm is code, including HTML.

And also thanks to the [high quality of its error messages](https://twitter.com/ID_AA_Carmack/status/735197548034412546).

## Further developments

* Is it possible to have a more objective analysis?
* Was it fair to mix framework with compilers and framework without?
* How different would the result be with Babel and/or Typescript?

Thank you for reading and let me know what you think in the comments below.

---

*Illustrations by Katerina Limpitsouni, from https://undraw.co/*