A micro API in Firebase and Cloudflare Workers
Tags: firebase, cloudflare, typescript, serverless
Published: 2020-07-27 03:06:41

## Intro 

This project started as a late-night inspiration after reading a Hackernews [article](https://rushter.com/blog/github-profile-markdown/) -- I set out to build a tiny REST(ish) API using Firebase!

To put it succinctly, Github added a feature which allows you to embed Markdown on your profile page. Obviously a core feature of Markdown is the ability to embed images, and although Github throws images behind their own CDN, critically, they obey `Cache-Control` headers from the source.

![Demo](https://dev-to-uploads.s3.amazonaws.com/i/8oeo5jjhyr6piq4dtu52.png)
<figcaption>My Github with profile Markdown</figcaption>

This let's us do fun stuff like count visitors or deliver randomized images a-la "hit counters" from websites of the early 2000s!
![hit counter](https://dev-to-uploads.s3.amazonaws.com/i/3j6ftpyfdij72u6khll0.jpg)
That is, of course, only if we include `Cache-Control: max-age=0, no-cache, no-store, must-revalidate` in our response image headers.

-------------

## Building with Firebase

Wanting to get my feet wet with Firebase, I explored building a serverless API with ~~3~~ 4 endpoints (plus who wants to spend money on EC2 or App Engine):

- `setImage` (POST)
  - The user should pass a valid image URL that they'd like to embed -- in response they will receive a random ID token
- `getImage` (GET)
  - This is the image link you would embed in Markdown, with your ID token as a query parameter. The image gets fetched and streamed back as a response -- while also keeping track of hit counts
- `getStats` (GET)
  - Requires the same ID token as a query parameter and returns the current visitor count as JSON.
- `randomEmoji` (GET)
   - Delivers a random emoji from Twitter's FOSS [emoji set](https://github.com/twitter/twemoji)

I really wanted to write this in TypeScript and Firebase provides fantastic [documentation](https://firebase.google.com/docs/functions/typescript) on how to setup serverless TS functions.

The gist of it is that you handle each function is just like a Express.js endpoint with a request and response.
```javascript
export const randomEmoji = functions.https.onRequest(async (req, res) => {
    res.set('Cache-Control', 'max-age=0, no-cache, no-store, must-revalidate');
    const target = emoji[Math.floor(Math.random() * emoji.length)];
    const url = `https://raw.githubusercontent.com/twitter/twemoji/master/assets/svg/${target}.svg`
    const response = await fetch(url);
    if (response.ok) {
        res.setHeader('content-type', response.headers.get('content-type') || '');
        return streamPipeline(response.body, res);
    }
    res.status(404).end()
});
```
I wanted to delve briefly into how the serverless functions work -- but it's not that complicated and can be found in this [repo](https://github.com/braeden/braeden/).

Additionally I needed a quick persistent database for this API -- [Firebase Real-time Database](https://firebase.google.com/docs/database) worked great for this. It again has fantastic documentation and since the serverless functions already have "Admin" access to the DB, it required minimal configuration.

![Firebase DB Setup](https://dev-to-uploads.s3.amazonaws.com/i/4c7yhiefvcsaxepyulck.png) 
<figcaption>Firebase K/V Setup</figcaption>


**Here's the core of `setimage`:**
```javascript
const db = admin.database();
const id = [...Array(5)].map(() => Math.random().toString(36)[2]).join('')

await db.ref(`${id}`).set({
    url: req?.body?.url || '',
    count: 0
})
```

**Here's the core of `getImage`:**
```javascript
const db = admin.database();

const d = await db.ref(`/${req.query.id}`).once('value');
const url = d?.val()?.url || '';

const response = await fetch(url);
if (response.ok) {
    await db.ref(`/${req.query.id}/count`).transaction(c => c + 1)
    // We use transactions here to avoid DB issues w/ concurrent requests
    return streamPipeline(response.body, res);
}
```
As you can see: it's dead-easy to interface with the DB and setup these serverless functions (Firebase provides incredible velocity to create new projects and APIs).


Feel free to hit the API yourself! Docs are in the postman collection in the above repo. Here's your random [emoji](
https://us-central1-gh-img.cloudfunctions.net/randomEmoji).

https://us-central1-gh-img.cloudfunctions.net/

{% github braeden/braeden %}


-------
## Problems with Firebase (with 'serverless' in general)

Serverless is dead cheap and dead easy -- you don't have to manage containers, servers, Kubernetes, or fancy CI/CD -- you just write your code and go!

The problem arises with the "Scale to zero" mentality which affects AWS Lamdas, GCP Functions (Firebase) and Azure Functions. It keeps users paying per invocation/seconds of execution, but it means there's a cold-start time payoff.

Especially in a time critical event, like fetching images in one-off events, having 3000ms delay for the first invocation response absolutely kills UX.

Check out this awesome article comparing cold-start times between cloud providers: https://mikhail.io/2018/08/serverless-cold-start-war/

-----
## The Fix?

![Cloudflare Workers](https://dev-to-uploads.s3.amazonaws.com/i/c0b68cifxi0szbddvr95.jpeg)

Cloudflare is best known for their dominance over the CDN market, but they have a really neat product I stumbled upon a few days ago! 
#### Cloudflare Workers
It's serverless compute but with these enticing promises: 
> - Cold starts up to 50× faster than other platforms
> - Your code runs within milliseconds from your users worldwide

> Service Workers are background scripts that run in your browser, alongside your application. Cloudflare Workers are the same concept, but super-powered: your Worker scripts run on Cloudflare’s edge network, in-between your application and the client’s browser.

Building `randomEmoji` in Cloudflare was fairly easy -- although there is definitely room for improvement in the local testing and automatic deployment.

The [example 'template' workers](https://developers.cloudflare.com/workers/templates/) that litter Cloudflare's website are a godsend to get started with any use-case!

```javascript
addEventListener('fetch', event => {
    event.respondWith(
        (async () => {
            const target = emoji[Math.floor(Math.random() * emoji.length)];
            const url = `https://raw.githubusercontent.com/twitter/twemoji/master/assets/svg/${target}.svg`
            const image = await fetch(url)
            const {
                readable,
                writable
            } = new TransformStream()
            // I <3 object destructuring
            image.body.pipeTo(writable)
            const r = new Response(readable, image)
            r.headers.set('Cache-Control', 'max-age=0, no-cache, no-store, must-revalidate')
            return r
        })()
    );
});
``` 
We just need to provide a IIFE inside an event listener -- this wipes the floor with GCP's serverless speed.

The code content is very similar, some documentation searching is required but only for the response creation and piping.

Hitting the `randomEmoji` [endpoint](https://emoji.braedensmith.workers.dev/) speed comparison:
- Cold-start
 - GCP: 3500ms
 - Cloudflare: 400ms
- Warm-start
 - GCP: 330ms
 - Cloudflare: 160ms

**An attentive reader would notice we skipped migrating 3/4 of our 4 HTTP endpoints**

Here's the caveat, if you need some sort of Key-Value store (which we do), Cloudflare offers a tempting solution, but you'll have to pay $5/mo for the pleasure.

A few more [details](https://developers.cloudflare.com/workers/reference/storage): 
> Workers KV is a global, low-latency, key-value data store. It supports exceptionally high read volumes with low-latency, making it possible to build highly dynamic APIs and websites which respond as quickly as a cached static file would.
> Workers KV is generally good for use-cases where you need to write relatively infrequently, but read quickly and frequently.

This sounds exactly like our use-case! (Obviously it's a scale overkill, but we're exploring cool tech.)

**Checking out Cloudflare KV to migrate completely away from Firebase**
- We want to store a unique ID as key, and have `url` and `count` as properties on that object.

We must first create a KV Namespace either using `wrangler` or an online interface -> we can share this between different workers if we wanted. (It will be called 'gh' in the examples below.) This namespace needs to bound to our worker again via web interface or `wrangler.toml via wrangler kv:...`.

*As far as I can tell, URL params are not supposed to be used with Workers -- but they seem to work if you manually parse them.* 

I found a super helpful [community post](https://community.cloudflare.com/t/parse-url-query-strings-with-cloudflare-workers/90286/2) that included a snippet on parsing out query strings.

For simplicity's sake we'll move the (image/visitor count) API under one worker: `getStats will just be a POST with a getStats: true`

**Handling Routes**
```javascript
addEventListener('fetch', event => {
  const { request } = event
  if (request.method === 'POST') {
    return event.respondWith(postResponse(request))
  } else if (request.method === 'GET') {
    return event.respondWith(getImage(request))
  }
});
```

**postReponse() (getStats and setImage)**
```javascript
async function postResponse(request) {
  const body = await request.json()
  if (body.url) {
    // setImage
    const id = [...Array(5)].map(() => Math.random().toString(36)[2]).join('')
    await GH.put(id, JSON.stringify({
      url: body.url || '',
      count: 0
    }));
    return new Response(JSON.stringify({
      id
    }))
  } else if (body.id) {
    // getImage
    const kvObject = JSON.parse(await GH.get(body.id));
    return new Response(JSON.stringify({ count: kvObject.count }));
  }
  return new Response('Not a valid POST request')
}
```
**getImage()**
```javascript
async function getImage(request) {
  const params = {}
  const url = new URL(request.url)
  const queryString = url.search.slice(1).split('&')
  queryString.forEach(item => {
    const [key, value] = item.split('=');
    if (key) params[key] = value || true;
  })
  const kvEntry = await GH.get(params.id)
  if (kvEntry) {
    try {
      const kvObject = JSON.parse(kvEntry);
      const image = await fetch(kvObject.url);
      const {
        readable,
        writable
      } = new TransformStream();
      image.body.pipeTo(writable);
      const r = new Response(readable, image)
      r.headers.set('Cache-Control', 'max-age=0, no-cache, no-store, must-revalidate')
      kvObject.count++;
      await GH.put(params.id, JSON.stringify(kvObject));
      return r;
    } catch (e) {
      console.error(e);
      return new Response('URL was invalid');
    }
  }
  return new Response('ID not found');
}
```

As you can see this move from Firebase to Cloudflare Workers was fairly easy and only required a minimal rewrite. There's still tons of room for improvement -- not returning just plaintext, better error handling, TypeScript rewrite and more. Also not having to stringify and parse JSON in the KV store, an necessity based on the value type restriction to: string, ReadableStream, ArrayBuffer.

The performance benefits of Cloudflare work fantastically for porting this particular micro API!

----
## Thanks!

Thanks for reading and I hoped you learned something about some of the serverless market!

All of the code mentioned in this article, and the API details if you would like to use this in your own README can be found here:

{% github braeden/braeden %}