Immutable File Storage
Tags: database, programming, sql, node
Published: 2022-04-07 19:07:13

**Sensitive files should be stored immutably.** This seems like a no brainer, but why isn't it being implemented in modern applications?

I think that many developers are turning to Blockchain technology to solve security problems. Momentum for projects is then soon slowed by the steep learning curve and cost of operating blockchain transactions.

By improving the security of traditional technologies, such as databases, developers can continue using a familiar stack. Immutable databases are great for storing sensitive files. Using the [bSQL](https://bsql.org/docs/overview) language you can easily store immutable files. 

> Immutable data is stored in "pen" not "pencil" and you wouldn't sign a contract in pencil.

## Demonstration

This post will demonstrate how to set up immutable file storage in bSQL using `Node`. The full code can be found on [GitHub](https://github.com/blockpointSystems/bsql-file-storage). 

Let’s define the basic workflow of our example application.
- Store a file with a description in the database.
- Confirm the file hasn't been illicitly modified.
- Export the file from the DB and read the file and its corresponding description.

_Disclamer:_ I am relatively new to `Node` feel free to make comments and suggestions to help me improve the code.

## Setting Up the Database
In order to store files immutably, you'll need an immutable database. The bSQL community slack channel gives you direct access to engineers to help you get set up, by joining the [slack](https://join.slack.com/t/bsqlcommunity/shared_invite/zt-169oaigpu-R3W1D5_vc6OyNnKy4thNoA) you'll receive an access code for a free bSQL instance.

## Establishing the connection
We will be working with the database directly. In order to do so we will need the [node connector](https://github.com/blockpointSystems/mdb-odbc-nodejs). The current version is `3.20.0`, but you should install the latest. The package requires protobuf, so if you are installing manually you will need to run `npm install google-protobuf grpc` **AND** `npm install mdb-server`.

Now you can import the package.

```js
import * as mdb from "mdb-server";
```
And establish the connection:

```js
let conn = await mdb.CreateConnection(
    {
        username: "your_username",
        password: "your_password",
        serverAddress: "0.0.0.0",
        serverPort: 5461,
        databaseName: "master",
        parameters: new Map([["interpolateParams", true]])
    }
)

// Connect to the database
await conn.connect()
```

## Building the Containers
In order to store a description and its corresponding file, we [create a new database](https://bsql.org/docs/database/create-database) and blockchain to link our descriptions to our files.
```js
await conn.exec(`CREATE DATABASE ${dbName}`)
```
In bSQL blockchains are like sql tables. There are many different blockchain types, we will be using a `HISTORICAL PLUS`. The blockchain created by the [CREATE BLOCKHAIN](https://bsql.org/docs/management/create-blockchain) command will have the following columns:
- **id** is a primary key that is incremented after each insertion.
- **file_id** references the corresponding file stored in the file store.
- **description** is a packed string that describes the file.

```js
 await conn.exec(`CREATE BLOCKCHAIN ${dbName}.${blockchainName} HISTORICAL PLUS (
        id UINT64 AUTO INCREMENT PRIMARY,
        file_id UINT64 FOREIGN KEY [document_store.sys_file, sys_file_id],
        description STRING PACKED)`)
```

## Store the local file in the database
bSQL has built in file storage. This makes it very easy to store files. There is no way to delete or update an existing file once it's stored. This ensures that centralized authorities can't illicitly modify sensitive documents.

This command stores the file `blockpoint.png` located in the `app` directory. It uses the response to store the `file_id` and a `despcription` in the reference table.
```js
let resp = await conn.storeFile(imageName, "app/blockpoint.png", "png")

await conn.exec(`INSERT ${dbName}.${blockchainName} (file_id, description) VALUES 
    (?, ?)`,
            [resp.getLastInsertId(), description])
```

## Check the validity of the database
Data in bSQL is hashed and linked together. We can check the recompute hashes and compare them to the hashes stored on insertion by running [CHECK VALIDITY](https://bsql.org/docs/database/check-validity).

```js
resp = await conn.exec("CHECK VALIDITY")
```

## Export the file and save it to a local directory
Exporting the file is a two-step process. The file will be saved to `app/exports/out_image.png` when `resp.saveFile()` is called.

```js
resp = await conn.exportFile(imageName, "app/exports/out_image.png", "png")
    await resp.saveFile()
```

## File Versioning
If there was an amendment to a document, how could we extend this application to update the file?

Use cases:
- Building Codes
- KYC documents
- Legal Contracts

You could simply store the new file, run an [AMEND](https://bsql.org/docs/mutations/amend) statement of the reference blockchain. This would give you access to the new version and the previous version as well.

How would I access the previous version?
We can set a database session to work with a previous version of the database. I recommend checking out [SET TRANSACTION QUERY TIME](https://dev.to/cassidymountjoy/hindsight-20-20-querying-any-point-in-time-2noi).