Persist data in Vercel(NextJS) serverless backends
Tags: javascript, programming, tutorial, serverless
Published: 2024-06-28 08:28:38

_Photo by [Aron Visuals](https://www.pexels.com/photo/cropland-duiring-night-time-1743165) on Pexels, chosen for its similarity to Vercel's logo_
<br/>
###TL:DR;
+ This is not an introduction however, here is a good intro to serverless backends on Vercel by [Vishal Yadav](https://dev.to/vyan/how-to-deploy-your-backend-on-vercel-using-verceljson-a-step-by-step-guide-5a78)
+ Persisting data by storage in global objects doesn't work in Vercel functions because they are ephemeral.
+ Writing files to the directory of the project throws a server crashing error while reading files doesn't
+ The workaround used involves writing and reading files with unique filenames to a temporary directory; `/tmp`, allowed by Vercel but absent in the docs

---

I recently migrated my web app from Railway to Vercel only to encounter bugs in my Node.js server due to the short-lived nature of Vercel functions.
I had used an object to store intermediate data for the `url` parameter of each `GET` request. The said object reverts back to `{}` everytime the serverless function is hit.

The following comprises the steps I took in working up a solution as well a link to the web app to see a working version of the code and logic below.<br/>


## Figuring out likely problems and their solutions
### Unique filenames, unique data
Filenames for stored data should be unique to avoid mixing them for every request.
Say the exported function in `api/root.js` is reached by requests to `https://<unique-url-to-app>/api/root`. It may at first suffice to generate a unique string from the parameters of the `GET` or `POST` requests however, serializing request URLs into unique filenames leaves room for duplication not to mention, frustrations with invalid strings for filenames.<br/>

One solution to generating UUIDs is `require('crypto').randomUUID()` if it is supported by the current node version you choose to work with.
Otherwise, here is a useful function for generating UUIDs gotten from the [1loc repo](https://github.com/phuocng/1loc/blob/master/contents/generate-a-random-uuid.mdx)

```js
const uuid = (a) =>a
  ? (a ^ ((Math.random() * 16) >> (a / 4))).toString(16)
  : ([1e7] + -1e3 + -4e3 + -8e3 + -1e11).replace(/[018]/g, uuid);
```

### Making reading/writing data similar to using objects: {"data":"data"}
The code design is done in a way that is asynchronous yet ergonomic in the way getting and setting properties on objects are.<br/>

+ **Writing** data is done after which a callback is supplied the generated UUID wherein it may be sent to the client as part of the response

```js
/* api/store.js */
let fs   = require('fs'),
    path = require('path'),
    dir  = '/tmp/',
    uuid = a =>a
      ? (a ^ ((Math.random() * 16) >> (a / 4))).toString(16)
      : ([1e7] + -1e3 + -4e3 + -8e3 + -1e11).replace(/[018]/g, uuid);

module.exports = {
  write: (data, cb, id)=>fs.writeFile(data, path.join(dir, id=uuid()),
    function(err) {
      if(err) throw err;
      else cb(id)
  })
}
```

Using it is as follows; reading the data comes after writing it

```js
let {write} = require('./api/store'),
    data;

write(data, function(uuid){
  /*the uuid is sent to the client which then uses it as a sort of unique string to retrieve its associated data by sending it along with its requests at a later time to be read by store.read */
  response.send(JSON.stringify({uuid})/*may be sent as either JSON or text*/)
});
```


+ **Read**ing of stored files is wrapped with a `Promise` to await the result of its callback as follows:
```js
/* api/store.js */
let fs   = require('fs'),
    path = require('path'),
    dir  = '/tmp/';

module.exports = {
  read: uuid=>new Promise((resolve, reject)=>{
    fs.readFile(path.join(dir, uuid), (err, buffer)=>{
      if(err) reject(err);
       else resolve(buffer.toString('utf-8'))
    })
  }),
  rm: id=>fs.unlinkSync(path.join(dir, id))
}
```

Which can then be used as follows

```js
let {read, rm} = require('./api/store'),
    data;

if(data = await store.read(/*uuid*/)) {
  
  response.send(data);
  rm(/*uuid*/) //may be removed or left alone
}
```

---


## Final and working draft of the code in `api/store.js`

```js
/* store.js */
let fs     = require('fs'),
    uuid   = (a) => (a ? (a ^ ((Math.random() * 16) >> (a / 4))).toString(16) : ([1e7] + -1e3 + -4e3 + -8e3 + -1e11).replace(/[018]/g, uuid)),
    /* may be made asynchronous */
    rm     = id=>fs.unlinkSync(path.join(dir, id)),
    dir    = '/tmp/',
    path   = require('path');

/*writes/reads data as text/plain, open to modifications*/
module.exports = {
  read: id =>new Promise((resolve, reject)=>fs.readFile(path.join(dir, id), (err, buffer)=>{
      if(err) reject(err);
      else resolve(buffer.toString())
    })
  ),
  write: function(data, cb, id) {
    fs.writeFile(path.join(dir, (id=uuid())), data, _=>cb(id))
  },
  rm 
}
```

---

## Using it in the wild

Here is how I used it to store and retrieve results generated for each valid request in this [webapp](https://rmrf-css.vercel.app)

```js
let store  = require('./store');
module.exports = async function(request, response, uuid) {
  let {url, id} = request.query;
  if(uuid = await store.read(id)) {
    /* exists */
    response.send(/* data */),
    /* you may remove the file or leave it since the contents of the /tmp/ directory are likely temporary */
    store.rm(uuid)
  } else {
    store.write('data', function(id) {
       /* send the unique id for the stored data in some way as JSON or text */
       response.send(id)
    })
  }
}
```

---

## Conclusion
The `/tmp/` directory, from its name, may be susceptible to changes from other programs on the server and therefore is not advisable to persist data in it in the manner you would a database.

Vercel provides [object blobs](https://vercel.com/docs/storage/vercel-blob) for persistent storage but it is not used in this code base since its need for storage is temporary and only throughout the short runtime of the serverless function which is expected to send a reponse within a maximum allowable time of 25s if its runtime is _edge._

This maximum duration is [configurable](https://vercel.com/docs/functions/configuring-functions#maximum-duration) if the runtime of the function is set to something aside _edge_ like _nodejs_ as follows
```js
export const runtime = 'nodejs';
export const maxDuration = 15;
/*rest of the code in the file goes here*/
```

Serveless functions are a great way to start building the backend for a website or webapp with less worrying over edge networks, scaling, server distribution, IP-based content delivery, etcetera