First Steps With TinyBase
Tags: webdev, javascript, opensource, tooling
Published: 2023-02-16 15:28:12

So [TinyBase](https://tinybase.org/) was one of those thing that came across my feed, and I wondered what the point was. It's a reactive client-side data store, which takes a little parsing to understand.

What exactly is a data store? Simply, it is a structure we use to store data. An array, an object, a Map, a Set... these are data stores.

What exactly do we mean by a "client-side data store"? It's a library that lets us build objects or arrays in the browser. Which, on the face of it, seems silly. We can always just create an array or object, right?

The key lies in the "reactive" bit. Not only does this allow us to construct a data store, but it allows us to respond to changes in that data store. We can *observe* those changes, and respond appropriately.

## Reactive Data Store

What do we mean by respond appropriately? Any sort of action we might like. Think of the event listeners in the DOM, only with *data*. 

* We might want to update the DOM, say a grid of employee `<card>` elements, when one is added or removed. Or we might need to update the contents of a card if one is edited.
* We might want to store data locally when a change happens (`localStorage`, `sessionStorage` or IndexedDb). Tinybase provides for that.
* We might want to handle some remote call, say passing a message to a Socket.
* We might not want to store data locally - when something changes, we might want to handle that change via a *debouncer*, so when the user stops editing for a given period, we handle a remote save request. While Tinybase provides for the remote storage, we would write our own debouncer.

Note that last, it is one of the strengths of this library. It doesn't dictate *how* we might use it or encapsulate it or consume it, it simply does this one thing, and does it quite well.

Note that TinyBase actually provides two storage mechanisms: we can store values by key, or we can store tabular data. That is, we can do either or both of these things:

```js
// we can set key/value pairs, just as in localStorage:
const store = createStore()
  .setValue('currentProject', projectId)
  .setValue('colorMode', 'dark');

console.log(store.getValues()); 
// {
//   currentProject: '36b8f84dd-df4e-4d49-b662-bcde71a8764f',
//   colorMode: 'dark'
// }

// or we can store tabular data:
store.setTable('todos', {
  crypto.randomUUID(), {
    title: 'Look into TinyBase',
    description: 'A pretty darn okay reactive local data store.',
    priority: 'normal',
    done: false
  }
});
```

So with the first part of that, we defined two keys: `currentProject` and `colorMode`. We defined values for them, and we're able to use either `store.getValue(key)` or `store.getValues()` to view those things.

In the second, we are defining a row of data in the `todos` table in our `store`. Note that we didn't specify any particular ordering or columns to that table, we simply went ahead and created the object. We *could* (and in future parts we *will*) define schemas or relations between tables - this project will let us do this, and more.

It's a data store powerhouse, but it is intentionally limited to that domain. It does one thing well, without needing to do all things.

## A Quick Idea...

To begin, let's consider how we might create a `Todos` data model, first with plain javascript and then by leveraging TinyBase.

```js
// Todos-service.js
let todos = [];

const isAMatch = (id) => (obj) => Object.keys(obj)[0] === id;

export const add = (todoObject) => {
  const id = crypto?.randomUUID();
  todos = [...todos, {[id]: todoObject}];
  return {[id]: todoObject};
}

export const update = (todoId, todoObject) => {
  todos = todos.map( todo =>
    isAMatch( todoId)(todo) ?
      {[todoId]: {...Object.values(todo)[0], ...todoObject} :
      todo
    )
}
export const remove = (todoId) => {
  todos = todos.filter( todo => !isAMatch(todoId)(todo));
}
export const findById = (todoId) => todos.find( isAMatch(todoId) )
export const findAll = () => [...todos];
```

All our basic functionality for `Todo` things to be collected, in a nice tidy package. Later, when we want to consume it, we simply

```js
import * as Todos from './services/Todos-service';

Todos.add({
  title: 'Learn about TinyBase', 
  descripton: 'Client-side reactive data stores!',
  due: '2023-02-17',
  priority: 'normal',
  done: false
});
console.log(Todos.findAll() );
```

And that's great. Let's see the same basic functionality with TinyBase:

```js
import { createStore } from 'tinybase';
const store = createStore();

export const add = (todoObject) => {
  const id = crypto?.randomUUID();
  store.setRow('todos', id, todoObject);
  
  return {[id]: store.getRow('todos', id) };
}

export const update = (todoId, todoObject)=>
  store
    .setPartialRow('todos', todoId, todoObject)
    .getRow('todos', todoId);

export const remove = (todoId) =>
  store.delRow('todos', todoId);

export const findById = (todoId) =>
  store.getRow('todos', todoId);

export const findAll = () => 
  store.getTable('todos');
    
```

This is giving us all the same functionality as an array, but it is *abstracting* that internal array into an external data store. And if this was all we were doing with it, we really haven't gained anything.

But as a first step, we can see that, basically, the use is very similar - only rather than using array methods, we're using TinyBase `store` methods.

## Getting Started

### 1. Setting up shop

To build this one, we'll use a bundler and a few packages. Of late, my preferred bundler is Vite - it is quick, clean, and minimal. So we'll open the console and:

```sh
yarn create vite vanilla-todo-app --template vanilla
```

That will create a directory, `vanilla-todo-app`, and set up the `package.json` and dependencies for us. Then we will `cd vanilla-todo-app` to get in there, and

```sh
yarn add tinybase
```

And that will both install the `package.json`, as well as adding the TinyBase package for us. If we have other dependencies we might like, we can add them - but for what we're about to do, that's everything we'll need.

At this point, we can open this in our editor. I'll be using VS Code, simply because it's fairly universal:

```sh
code .
```

This opens the editor with the current directory as the workspace. We will be able to clean up the template quite a bit, we don't need any of the content in the `main.js`, or the files to which it refers - so we can delete `javascript.svg` and `counter.js`, and remove everything but the `import './style.css'` from the `main.js`. At that point, we're ready to start!

### 2. Creating a Store

Now we need to create a data store. And we'll want to place it into its own file, importable by others - we might want to allow for multiple data sets (for example, we might want a `"todos"` and a `"projects"`). Let's start there.

1. Create a `src` directory to keep the root tidy, we'll work in there for the most part. Within there, we'll create a `services` directory.
2. Inside that `services` directory, we'll create a `store.js` file.

```js
// /src/services/store.js
import { createStore } from 'tinybase';

const store = createStore();

export store;
```

And there we go. We have our datastore defined! At this point, that's all we'll need in the `store.js`, though later we'll add a few other useful things to this file.

While we could interact directly with the `store` wherever we might need, a better approach might be to define an *interface* that can consume a particular service. With that, if we choose to swap out our data provider later, it would only require editing one place rather than scattered throughout our code.

### 3. Defining an Abstract Model

The interface methods for each are fairly standard: `add`, `update`, `remove`, `byId`, and `all` will do. We'll start by defining a generic `Model.js`:

```js
// src/models/Model.js
import { store } from '../services/store';


const Model = (table) => {
  const add = (object) => {
    const id = crypto.randomUUID();
    store.setRow(table, id, object);
    return {[id]: object };
  }
  const update = (id, object) =>
    store
      .setPartialRow(table, id, object)
      .getRow(table, id);  
  const remove = (id) => store.delRow(table, id);
  const byId = (id) => store.getRow(table, id);
  const all = () => store.getTable(table);

  return {
    add,
    update,
    remove,
    byId,
    all
  }
}

export default Model;

```
We've defined this as a factory function. To consume it, we could simply call `const Todos = Model('todo')`, providing it with the name of the data table we wish to use.

Now, when we are adding an object, we are getting a `uuid` for the project, and we are using the TinyBase `setRow` method to create a row in the given table.

> Side note, when I refer to the `projects` table, it may be easier to think of the store as an object, and the `projects` as a key on that object that contains an array of `{[id]: object}` things.

When we update a project, TinyBase provides the `setPartialRow` method. With that, we provide a table id, a row id, and an updater object. That updater object doesn't need to redefine all the properties in our original object, only the ones we might want to update. And the `setPartialRow` method returns the TinyBase store instance, so we can chain that and call `getRow()` to get and return the updated value of our object. 

To delete a row, we simply call `delRow` with the table and row ids.

Now, if we look at that code, there is nothing there that is *unique* to the `Project` model. In fact, it's fairly abstract - the only thing identifying it is the `const table = 'projects'` line. And that is deliberate.

To create the `Todos.model.js`, we can use the same code. Simply by changing that one line to `const table = 'todos'`, we will be performing those same CRUD operations elsewhere in our store.

And this is a pretty good abstraction, to my mind. We can reuse this as a template for each of our data models. 

### 4. Creating Instance Models

```js
// src/models/Todos.model.js
import Model from './Model'; 

const Todos = Model('todos');

export default Todos;

```

And that's all we need at this point. We can do the same for projects:

```js
// src/models/Projects.model.js
import Model from './Model';

const Projects= Model('projects');

export default Projects;
```

At that point, we have two functional tables of data we can stuff into our `store` service.

## Let's Consider Structure

Up to this point, we haven't really considered *how* we should structure things. We set up a data store, we defined an interface to consume that data store, and we created a couple of basic data models.  But we would do well to step back and think about how our data should be structured.

Let's examine the `Todos` model first, in isolation. 

```js
// Todo:
{
  [id]: {
    title: 'Default Todo Title',
    description: 'Default description',
    priority: 'normal', // ['low','normal','high','critical']
    created: '2023-02-02',
    due: '2023-02-09',
    done: false
  }
}
```

So note that we have that `priority` key, which can be one of four values. Now, if we wanted to get all the high priority todos, we could simply get them all and use `.filter`, but TinyBase gives us an option. This would be a good candidate for an *index*.

With an index, we can select all rows from a table that match a given index value. When we query the index, we get back an array of keys, all of which meet that index condition. And that array we get back is *reactive* - so as we add/edit/remove rows, the indexes are dynamically updating.

So we have a basic structure - the `priority` key will be indexed, and we want to be able to get two key pieces of information back from our `Todos`: all currently-used indexes, and sets of ids that meet a given index. So we'll be adding two methods to the `Todos` object: `priorities` and `byPriority`. The first will get an array of `priority` keys, while the second will get a complete list of the Todos with a given priority value.

But we're using the `Model` to generate the `Todo.model` - can we somehow add to or *compose* that?

## We Have the Power!

We can, actually. We want to first add an export to the store service, allowing for indexing:

```js
// src/services/store.js
import { createStore, createIndexes } from "tinybase/store";

export const store = createStore();
export const indexes = createIndexes(store);
```

That will let us set up an index in the `Todos.model.js`:

```js
// src/models/Todos.model.js
import Model from './Model';
import { store, indexes } from '../services/store';

indexes.setIndexDefinition(
  'byPriority',
  'todos',
  'priority'
);

const Todos = Model('todos');

export default Todos;
```

At that point, we have defined our index column. `setIndexDefinition` is a method on the `indexes` instance, and we tell it to create an index with the id of `byPriority` (so we can retrieve it later), that is working on the `todos` table, and in particular is indexing the `priority` field in that table.

## Extending the Data Models

In the above `Todo.model.js`, we now have a good index, but we aren't actually *using* it yet. And what we'd like to do, if possible. But what that means is, we want to take in the basic `Model` functionality, and add to that.

```js
// src/models/Todos.model.js
import Model from './Model';
import { store, indexes } from '../services/store';

indexes.setIndexDefinition(
  'byPriority',
  'todos',
  'priority'
);

const Todos =(()=>{
  // our base functionality...
  const baseTodos = Model('todos');

  return {
    ...baseTodos
  }
})();

export default Todos;
```

So we have changed `Todos` from simply being an instance of our `Model` factory to being an IIFE that is returning all the methods of that `baseTodos`, which is *still* the instance. We're *composing* the functionailty from `Model` with some custom methods.

Within that `Todos` IIFE, let's add this:

```js
  const baseTodos = Model('todos');

  // get all the todo ids associated with this project
  const priorities = () =>
    indexes.getSliceIds('byPriority');
  const idsByPriority = (priority) =>
    indexes.getSliceRowIds('byPriority', priority);
  const byPriority = (priority) =>
    idsByPriority(priority).map( baseTodos.byId )

  return {
    ...baseTodos,
    priorities,
    byPriority
  }
```

So we've added two methods to the `Todos` model: `priorities` and `byPriority`. The first gets all the currently-used `priority` values, while the second gets the `todos` themselves with a given  `priority`. 

To get the array of `priority` values, we use the `Index` module's `getSliceIds` method. That gets us all possible key values for the indexed cell (all the possible values for `priority` currently used in our data store).

The `Indexes` module also gives us the `getSliceRowIds` method, which simply gets the id for each row that meets its condition. In our case, the condition is a matching `priority`.

And we can leverage that in the `byPriority` function - we get the ids for each row, and then use those to get each individual row for the project.

Finally, we *spread* the `...baseTodo` object, exposing its methods on the `Todos` returned object as references to this inner `baseTodos` thing. And we *compose* that interface, by adding two more methods to the `Todos` returned object.

## Indexes vs Relationships

The next bit of structuring to consider is the *relationship* between the `Project` and the `Todo`. Each thing has a unique id assigned to it, and that is used as the key of its row in the data table.

And a project can contain any number of todo elements, but each todo can only belong to one project. This is, in relational database terms, a *one-to-many* relationship.

Typically, in a relational database, we might give the `Todos.model` a field for the `projectId`, some way of identifying with *which* project it is associated. So, for example, to find all the `Personal` project's todos, we could select all the todo elements with that `projectId`.

So note how, in the model of the Todo, we show a `projectId`:

```js
// Project:
{
  [id]: {
    title: 'Default Project Title',
    description: 'Default Description',
    created: '2023-02-02',
  }
}

// Todo:
{
  [id]: {
    projectId: [projectId], // <-- the key from `projects`
    title: 'Default Todo Title',
    description: 'Default description',
    priority: 'normal', // ['low','normal','high','critical']
    created: '2023-02-02',
    due: '2023-02-09',
    done: false
  }
}
```

> ***Indexes vs. Relationships***: 
> We discussed indexes above, and now we're discussing relationships. They are similar, and if you've worked with relational data, you may likely already know the difference, but here it is in a nutshell:
> * indexes are used within the context of a table, to facilitate searching for a particular property value within that table (for example, `priority==='high'`).
> * relationships are used within the dynamic of multiple tables, indicating a connection between one (a *local* table) and the other (the *remote*). In this case, the relationship would be `todo.projectId === project.id`.
> We're still comparing to something - but with indexes, we're typically getting information *about a table* while a relationship is giving us information *about multiple tables* through a common point.

In order to support relationships between data tables, we will need to provide the `Relationships` module:

```js
import { 
  createStore,
  createIndexes,
  createRelationships  
} from 'tinybase';

export const store = createStore();
export const indexes = createIndexes(store);
export const relations = createRelationships(store);
```

So we now have a `relations` module in which we can define our many-to-one relationship. As this is primarily the domain of the `Project`, we'll put it in the `Project.model.js` for now.

```js
// src/models/Projects.model.js
import { relations } from '../services/store';

import Model from './Model';
import Todos from './Todos.model';

relations.setRelationshipDefinition(
  'projectTodos',  // the id of the relationship
  'todos',         //  the 'many', or local side of the relation
  'projects',      //  the 'one', or remote side of the relation
  'projectId'      //  the local key containing the remote id
);

const Projects = (()=>{
  const baseProjects = Model('projects');

  return {
    ...baseProjects,
  }
})();

export default Projects;
```

Note that we import just the `relations` export, as we don't need an instance of the `store` itself. All we need to define the relationship is the relationship module itself. Also, we import the `Todos` module, as we want to add an array of todos to the project.

`relations.setRelationshipDefinition` defines the relationship between projects and todos, and gives that relationship the id `projectTodos`. The parameters for that function are:
  - `relationshipId`: a unique string to identify this relationship.
  - `localTableId`: the id of the local table for the relationship. Note that the local table is the 'many' side of the many-to-one relationship.
  - `remoteTableId`: the id of the remote table for the relationship. This is the 'one' side of that many-to-one, representing the unique project that can relate to zero or more todo rows.
  - `getRemoteRowId`: the name on this threw me for a bit, but it is the cell *in the local row* that contain the unique `id` of the *remote* row. So, in our case, this would be `projectId`, as that is the `todos` row reference to `projects`.`id`

Finally, we can *consume* that relationship within the definition of the `Projects` model itself:

```js
// src/models/Projects.model.js
const Projects = (()=>{
  const baseProjects = Model('projects');

  const byId = (projectId) => {
    const project = baseProjects.byId(projectId);
    project.todos = relations.getLocalRowIds('projectTodos', projectId)
      .map(Todos.byId);

    return project;
  };
  
  return {
    ...baseProjects,
    byId
  }
})();

```

Again, we expose the interface of the `baseProject`, and replace the stock `byId` method with a custom one.

## That's Nice and All, But... Why?

This post is about how to interface with and consume TinyBase's store, using plain vanilla javascript. And to this point, it's pretty darn okay. But if that was all there was, it wouldn't have much going for it.

In the next post, we will explore the **reactive** aspect of that store. We can set listeners on tables, rows, cells or data values, and we can respond when those points change. 

We will also look at *data storage and persistence*. TinyBase, in itself, includes some great mechanisms for both local and remote storing, and also describes how to write your own storage "hooks."

This is something I'm still playing with, something I'm still learning as I go - if y'all find something neat (or something I missed), lemme know!