Encoding mess with Javascript
Tags: javascript, encoding, base64, utf8
Published: 2020-07-14 16:14:11

---
title: Encoding mess with Javascript
published: true
description: Transfer data by URL by encoding them is harder than it looks and you need to be careful about evil characters.
tags: [javascript, encoding, base64, utf8]
//cover_image: https://direct_url_to_image.jpg
---

Today, I launched a new feature on a product I'm working on, letting you deploy a cloud function quickly. (Not the subject but if you're interested it's [here]( https://liteflow.com/functions)).
Like any launch, there are always some last minutes issues, and I discovered something interesting.

I was trying to transfer data from one page (the landing page) to another one (the console). These data contain the content of the function you want to deploy so, a lot of characters like `"`, `{`, `(`... that's where it gets funny.

The encoding was done with the following flow:
- The landing page stringifies the object with [`JSON.stringify`](https://www.w3schools.com/js/js_json_stringify.asp)
- Convert in base64 of that with [`btoa`](https://www.w3schools.com/jsref/met_win_btoa.asp)
- Encode for URL with [`encodeURIComponent`](https://www.w3schools.com/jsref/jsref_encodeURIComponent.asp)
- Redirect to the other website with result in the query of the URL
- The other website decode [`decodeURIComponent`](https://www.w3schools.com/jsref/jsref_decodeURIComponent.asp)
- Decode again with [`atob`](https://www.w3schools.com/jsref/met_win_atob.asp)
- Extract the object with [`JSON.parse`](https://www.w3schools.com/Js/js_json_parse.asp)

So far so good. We should have:
```js
const data = {...}
const encoded = encodeURIComponent(btoa(JSON.parse(data)))
// other website
const decoded = JSON.parse(atob(decodeURIComponent(encoded)))
data === decoded
```

But no...
Some characters like `>` (used to define an arrow function) are messing up with the encoding, and it is impossible to recover the data (more details [here]( https://stackoverflow.com/questions/30106476/using-javascripts-atob-to-decode-base64-doesnt-properly-decode-utf-8-strings)).

So if you already had this issue here is a sweet fix:
You can add another `encode/decodeURIComponent` before the `atob`/`btoa` call. 

So now your flow would be:
- The landing page stringifies the object with [`JSON.stringify`](https://www.w3schools.com/js/js_json_stringify.asp)
- _Encode the string with [`encodeURIComponent`](https://www.w3schools.com/jsref/jsref_encodeURIComponent.asp)_
- Convert in base64 of that with [`btoa`](https://www.w3schools.com/jsref/met_win_btoa.asp)
- Encode for URL with [`encodeURIComponent`](https://www.w3schools.com/jsref/jsref_encodeURIComponent.asp) (not entirely necessary as base64 is already URI compatible but good to have anyway)
- Redirect to the other website with result in the query of the URL
- The other website decode [`decodeURIComponent`](https://www.w3schools.com/jsref/jsref_decodeURIComponent.asp)
- Decode again with [`atob`](https://www.w3schools.com/jsref/met_win_atob.asp)
- Decode again with [`decodeURIComponent`](https://www.w3schools.com/jsref/jsref_decodeURIComponent.asp)
- Finally, extract the object with [`JSON.parse`](https://www.w3schools.com/Js/js_json_parse.asp)

So that would be equivalent to:
```js
const data = {...}
const encoded = encodeURIComponent(btoa(encodeURIComponent(JSON.parse(data))))
// other website
const decoded = JSON.parse(decodeURIComponent(atob(decodeURIComponent(encoded))))
data === decoded
```
Now the data can be decoded with the right content :tada:

I struggled a bit today about that, so I thought I would share it. If you have any ways to do that properly, I would love to hear your feedback, and if you want to try to break it again, you can do it here https://liteflow.com/functions.