Using Fauna's streaming feature to build a chat with Svelte
Tags: fauna, serverless, svelte
Published: 2021-01-22 18:43:22

Author: Pier Bover
Date: January 6, 2021

---

Today we're going to take a look into Fauna's new streaming feature by building a small chat using Svelte.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/pg7bnp45tfuh8liv76z0.png)

Streaming allows our applications to subscribe to changes in Fauna. At this time we can only subscribe to documents, but subscription to collections and indexes are planned for the future.

Note that streaming in Fauna is still in alpha stage. Bugs and changes in the API should be expected before the final release.

The finished project is [on this Github repository](https://github.com/PierBover/fauna-streaming-chat-with-svelte) which you can use to follow along.

One last thing. If you've never used Fauna or FQL before, it would be a good idea to at least take a quick look at this [introductory article](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-1).

# In this article:
* Setting up Fauna
* Configuring Rollup
* Fauna logic
* Application state
* The Svelte user interface

# Setting up Fauna
First, we're going to configure everything we need in Fauna.

If you haven't already done so, create a free Fauna user and login to the dashboard. Then create a new database with any name you prefer:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fqxllq7vbuzdb01g3ex3.png)
We're now ready to start writing FQL queries in the dashboard's shell:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/kko4ckh98x3jhbkvqrhz.png)
Our data model is very simple:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/vhfpkbla2uh1mlkh5lnm.png)
In terms of streaming, our chat application(s) will subscribe to the document of the chat room. We'll notify our users they have new messages by simply updating the **ts** (timestamp) of that document.

First, let's create our collections:
```
CreateCollection({
  name: "ChatRooms"
})
```
```
CreateCollection({
  name: "ChatMessages"
})
```
Let's also create a single chat room document. We're giving it an id of 1 so that we can reference it easily later on:
```
Create(
  Ref(Collection("ChatRooms"), "1"),
  {
    data: {
      name: "Fauna chat"
    }
  }
)

{
  ref: Ref(Collection("ChatRooms"), "1"),
  ts: 1608136745905000,
  data: {
    name: "Fauna chat"
  }
}
```
We're also going to need an index to retrieve messages:
```
CreateIndex({
  name: "ChatMessages_by_chatRoomRef",
  source: Collection("ChatMessages"),
  values: [
    { field: ["ts"] },
    { field: ["data", "authorName"] },
    { field: ["data", "authorColor"] },
    { field: ["data", "message"] },
    { field: ["ref"] }
  ],
  terms: [{ field: ["data", "chatRoomRef"] }]
})
```
As you can see in the **terms** settings, this index accepts a reference to be able to get messages from a single chat room.

This index also has a **values** setting that will return indexed data of the documents in the **ChatMessages** collection. By default, when no values are configured, indexes do not return documents but document references. This requires an extra read operation for each document we want to read.

Obviously, our chat application is going to be super popular and have millions of users. By having our index return some values instead of document references, we're reducing the number of read operations on the documents themselves. The drawback is that now each message will occupy more storage space, the document itself plus the indexed data.

To test our index let's create a new message:
```
Create(
  Collection("ChatMessages"),
  {
    data: {
      authorName: "Pier",
      authorColor: "tomato",
      message: "Hello there",
      chatRoomRef: Ref(Collection("ChatRooms"), "1")
    }
  }
)
```
As expected, if we query our index we will be getting the values from this document back:
```
Paginate(
  Match(
    Index("ChatMessages_by_chatRoomRef"),
    Ref(Collection("ChatRooms"), "1")
  )
)

{
  data: [
    [
      1608141339175000,
      "Pier",
      "tomato",
      "Hello there",
      Ref(Collection("ChatMessages"), "285089452953436680")
    ]
  ]
}
```
# Authorization rules
For simplicity's sake, we're not going to implement user authentication in this demo. Instead, we're going to create a custom role with limited privileges that our application will use.

Go to the security tab of the dashboard and create a new role with the name of **FrontEnd**. Add the two collections and the index we've created with these privileges:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/77xcdnokfmbatk61d3t6.png)
In plain English, our application won't be able to create new chat rooms but will be able to read existing ones. It will also be able to create and read chat messages. Finally, it will be able to read the results from our index.

We don't need to configure the membership settings for this role because we won't use authentication of users in our application. Instead, this role will be used by an application key. Check this article for more information on authentication and authorization with Fauna.

You can save your role now and go back to the security tab to create a new key using the **FrontEnd** role:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/7sqhjbyojqwdjqgmwcpn.png)
After creating the key, Fauna will show us the secret that we'll need later on to be able to execute queries from our application:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/fbk5mwaz4qvo3gx96dz5.png)

Save it somewhere safe. If you lose it you will need to create a new key to obtain its secret.

# Configuring Rollup
Again, [here's the repository with the final project](https://github.com/PierBover/fauna-streaming-chat-with-svelte) so you can check the final folder structure.

As with any modern front end project, we're going to need NodeJS and NPM. If you don't have them installed, I recommend downloading the LTS version from the [NodeJS official page](https://nodejs.org/en/).

Create a folder and open it with your terminal. Then init NPM with:
```
npm init -y
```
This will create a **package.json** file in your project folder.

We're going to use [Rollup](https://rollupjs.org/) as our JavaScript bundler. You could also use Webpack if you prefer. It should be straightforward to configure following the [official Webpack template](https://github.com/sveltejs/template-webpack) for Svelte.

Configuring bundlers is a deep topic in itself. For this article I won't go into every detail, but I've explained the broad strokes in this section.

Let's now install Rollup and Sirv, a static file web server which we'll use to run our application locally during development:
```
npm i rollup sirv-cli --save-dev
```
Then install these Rollup plugins:
```
npm i rollup-plugin-terser rollup-plugin-svelte rollup-plugin-node-polyfills rollup-plugin-livereload rollup-plugin-css-only @rollup/plugin-node-resolve @rollup/plugin-commonjs @rollup/plugin-replace --save-dev
```
Most of these plugins come from the [official Rollup template for Svelte](https://github.com/sveltejs/template). I've added two plugins that we're going to need later on:

* [@rollup/plugin-replace](https://github.com/rollup/plugins/tree/master/packages/replace#readme) will allow us to inject environment variables (such as our Fauna secret) into our front end app at compile-time.

* [rollup-plugin-node-polyfills](https://github.com/ionic-team/rollup-plugin-node-polyfills#readme) will insert polyfills for any needed NodeJS libraries. This is necessary to use the Fauna JavaScript driver in the browser.

If you're using Webpack, you can inject environment variables using [DefinePlugin](https://webpack.js.org/plugins/define-plugin/). Webpack v4 will include NodeJS polyfills by default, but with v5 you will need to configure that yourself.

We're now ready to create the **rollup.config.js** file at the root of our project. Copy the contents or download [this file from the Github repository](https://github.com/PierBover/fauna-streaming-chat-with-svelte/blob/master/rollup.config.js) which, for the most part, follows the official template.

Be sure to replace the Fauna secret with your own in the configuration of the Replace plugin:
```
replace({
  FAUNA_SECRET: '"fnAD9Ng2efACCJ-QS1FngbLOoPJ4JQfMgcv8Y5_I"'
})
```
I committed that secret to the repository to serve as an example. In a real world scenario you should **never** commit a secret to a Git repository. It would be a great idea to put the secret in a **.env** file like we saw in a [previous article](https://fauna.com/blog/getting-started-with-fauna-and-node-js-using-fastify).

As you can see from the config file, whenever Rollup bundles our code, it will create the resulting files in the **public/bundle** folder:
```
output: {
  sourcemap: true,
  format: 'iife',
  name: 'app',
  file: 'public/build/bundle.js'
}
```
Because we're using the **rollup-plugin-svelte** (which uses the Svelte compiler) Rollup will also output a **bundle.css** file with the CSS found in the Svelte components.

Now we need the **public/index.html** file with this content:
```
<!doctype html>
<html>
<head>
    <meta charset='utf8'>
    <meta name='viewport' content='width=device-width'>
    <title>Fauna chat</title>
  <link rel='stylesheet' href='global.css'>
    <link rel='stylesheet' href='build/bundle.css'>
</head>
<body>
<script src='build/bundle.js'></script>
</body>
</html>
```
And also a **public/global.css** file with any global CSS you might want. You can check the global CSS I used [in the repository](https://github.com/PierBover/fauna-streaming-chat-with-svelte/blob/master/public/global.css).

Finally, add these commands in the scripts section of the **package.json** file:
```
"scripts": {
  "build": "rollup -c",
  "dev": "rollup -c -w",
  "start": "sirv public"
}
```
This is what these commands do:

* **rollup -c** executes Rollup with the default config file (**rollup.config.js**). We'll use this to build our project in production mode.
* **rollup -c -w** adds the watch option which will rebuild our project in development mode automatically whenever we make changes.
* **sirv public** starts Sirv in our /**public** folder. Rollup will trigger this command automatically whenever it rebuilds our project during development.

With our project in place we can now start working on our front end code.

Remember to use the **dev** command which will fire up Rollup and Sirv for development:
```
npm run dev
```
Then open your browser at this URL:
```
http://localhost:5000
```
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xa4gsko249j67e9qo5xz.png)
# Fauna logic
Let's install our dependencies to be able to communicate with Fauna:
```
npm i faunadb mini-signals
```
[mini-signals](https://github.com/Hypercubed/mini-signals) is a signals library in JavaScript. Signals are an implementation of the observer pattern and are very common in event-driven programming.

Create the **src/fauna/index.js** file in your project folder with this code:
```
import faunadb from 'faunadb';
import FaunaStream from "./FaunaStream.js";

// We do this so that our FQL code is cleaner
const {Paginate, Match, Range, Index, Create, Collection, Ref, Do, Update, ToMicros, Now, TimeSubtract} = faunadb.query;

const client = new faunadb.Client({
  secret: FAUNA_SECRET
});

const chatRoomRef = Ref(Collection('ChatRooms'), '1');

export const chatRoomStream = new FaunaStream(client, chatRoomRef);

export async function getLatestMessages (afterTs) {
  const result = await client.query(
    Paginate(
      Range(
        Match(
          Index("ChatMessages_by_chatRoomRef"),
          Ref(Collection("ChatRooms"), "1")
        ),
        afterTs ? afterTs : ToMicros(TimeSubtract(Now(), 3, 'hour')),
        []
      ),
      {
        size: 10000
      }
    )
  )

  return result.data;
}

export async function createMessage (authorName, authorColor, message) {
  const result = await client.query(
    Do(
      Update(
        chatRoomRef,
        {
          data: {
            lastUpdate: Now()
          }
        }
      ),
      Create(
        Collection('ChatMessages'),
        {
          data: {
            authorName,
            authorColor,
            message,
            chatRoomRef
          }
        }
      )
    )
  )

  return result;
}
```
Note that for simplicity's sake we will not be handling Fauna errors in this demo. You can check my [previous article](https://fauna.com/blog/getting-started-with-fauna-and-node-js-using-fastify) about using Fauna with NodeJS to see one possible approach.

Let's examine the relevant bits of this file step by step.

#Initializing the client
We're only going to use a single secret (coming from our application key) so we can instantiate the Fauna client once and reuse it for all our queries:
```
const client = new faunadb.Client({
  secret: FAUNA_SECRET
});
```
As mentioned before, Rollup will replace the **FAUNA_SECRET** variable at compile-time with the configured secret.

# Writing modular FQL
This variable contains a reference to our chat room document:
```
const chatRoomRef = Ref(Collection('ChatRooms'), '1');
```
I admit it's not a very impressive example, but it showcases the modularity of using FQL in JavaScript.

Here's a better example:
```
function callMyIndex (param1, param2, param3) {
  return Match(
    Index('MyIndex'),
    [param1, param2, param3]
  ); 
}
```
By itself this function doesn't do much, other than returning a small FQL statement, but we can now use its return value into other FQL statements:
```
const result = await client.query(
  Paginate(
    callMyIndex('foo', 'bar', 'baz')
  )
);
```
This modular approach allows us to encapsulate and reuse our FQL statements, and also make them dynamic, validate input, write tests, etc. It's brilliant.

# Getting the latest messages
To get the latest messages we'll be using this function. It has a single parameter **afterTs** to be able to get the messages after a certain point in time:
```
export async function getLatestMessages (afterTs) {
  const result = await client.query(
    Paginate(
      Range(
        Match(
          Index("ChatMessages_by_chatRoomRef"),
          Ref(Collection("ChatRooms"), "1")
        ),
        afterTs ? afterTs : ToMicros(TimeSubtract(Now(), 1, 'hour')),
        []
      ),
      {
        size: 10000
      }
    )
  )

  return result.data;
}
```
In terms of FQL, we're using the **ChatMessages_by_chatRoomRef** index we created earlier to get up to 10,000 messages in a single page. Do note the use of the **chatRoomRef** variable (which is actually a [Ref()](https://docs.fauna.com/fauna/current/api/fql/functions/ref?lang=javascript) to a document) as the index term.

We're also doing a range query using [Range()](https://docs.fauna.com/fauna/current/api/fql/functions/range) to only get messages after a certain point in time. This can either be the value of the **afterTs** function parameter or, if it is undefined, the time in microseconds one hour before executing the query:
```
ToMicros(TimeSubtract(Now(), 1, 'hour'))
```
[TimeSubtract()](https://docs.fauna.com/fauna/current/api/fql/functions/timesubtract?lang=javascript) returns a [Time()](https://docs.fauna.com/fauna/current/api/fql/functions/time?lang=javascript) value, but we're using [ToMicros()](https://docs.fauna.com/fauna/current/api/fql/functions/tomicros?lang=javascript) to actually get an integer of microseconds so that **Range()** can compare with the **ts** values.

To dig deeper into filtering with Fauna, check this [article](https://fauna.com/blog/core-fql-concepts-part-4-range-queries-and-advanced-filtering) where we explored range queries and complex conditional statements with FQL.

# Creating new messages
To create new message documents we'll use this function:
```
export async function createMessage (authorName, authorColor, message) {
  const result = await client.query(
    Do(
      // First update the chat room document
      Update(
        chatRoomRef,
        {}
      ),
      // Then create a new message document
      Create(
        Collection('ChatMessages'),
        {
          data: {
            authorName,
            authorColor,
            message,
            chatRoomRef
          }
        }
      )
    )
  )

  return result;
}
```
We're combining two FQL statements with the [Do()](https://docs.fauna.com/fauna/current/api/fql/functions/do?lang=javascript) function. Because Fauna is completely ACID, should any of these two statements fail, all changes will be reverted back and an error will be returned.

Our first statement triggers an empty update to the document of our chat room. Anyone subscribed to this document will now be notified that it has changed, even though only its **ts** (timestamp) has been updated.

The second statement creates a new document in the **ChatMessages** collection with the provided data.

# Subscribing to a document
Let's now focus on the document streaming functionality of Fauna.

You may have noticed that in the **src/fauna/index.js** file there is an an instance of the **FaunaStream** class:
```
export const chatRoomStream = new FaunaStream(client, chatRoomRef);
```
This is a small class I created to simplify working with streams in Fauna.

Create the **src/fauna/FaunaStream.js** file with this code:
```
import Signal from 'mini-signals';

export default class FaunaStream {
  constructor (client, documentRef) {    
    this.documentRef = documentRef;
    this.client = client;    
    this.onUpdate = new Signal();    
    this.initStream();
  }

  initStream () {
    this.stream = this.client.stream.document(this.documentRef);

    this.stream.on('snapshot', (data, event) => {
      this.onUpdate.dispatch(data);
    });

    this.stream.on('version', (data, event) => {
      this.onUpdate.dispatch(data.document);
    });

    this.stream.on('error', (data, event) => {
      this.stream.close();
      setTimeout(() => {
        this.initStream();
      }, 250);
    });

    this.stream.start();
  }

  destroy () {
    this.stream.close();
    this.onUpdate.detachAll();
  }
}
```
The constructor accepts an instance of the Fauna client, and a reference to a document for the subscription.

To subscribe to a document we first need to create a subscription using the **client.stream.document()** method. We're doing that in the **initStream** method of our class:
```
this.stream = this.client.stream.document(this.documentRef);
```
Then we can add handlers to a number of events. For this demo we're only interested in these three events:

* **snapshot** which will return the full document whenever the stream starts.
* **version** which will return the updated document whenever there is change.
* **error** which will be triggered if anything goes wrong. If an error happens, it's necessary to close the stream and create a new one after waiting a bit.

The idea is that whenever we receive a document in the stream, either from a **snapshot** or **version** event, we simply dispatch an update to the rest of our application using our signal.

I've also added a **destroy()** method, which we're not using in this demo, but serves as an example on how to prevent memory leaks.

Here's an [article](https://fauna.com/blog/live-ui-updates-with-faunas-real-time-document-streaming) with more information on Fauna streams, including a complete list of events you could use.

# Application state
All the logic to communicate with Fauna is in place. We now need to work on the logic that will manage the state of our small application.

Svelte provides a number of reactive primitives we can use. Components can subscribe automatically to these reactive primitives and update the DOM when the data changes.

Because Svelte is actually a compiler and not a runtime, like say Vue or React, the dependencies between data and components are analyzed at compile time, resulting in very small and efficient applications.

Here's the code for the **src/store/index.js** file:
```
import {writable} from 'svelte/store';
import {chatRoomStream, getLatestMessages} from '../fauna/index.js';

const names = ['Cat', 'Lion', 'Dog', 'Lemur', 'Squirrel', 'Walrus', 'Octopus', 'Shark', 'Elephant'];
const colors = ['blue', 'orange', 'green', 'blueviolet', 'coral'];

export const messages = writable([]);
export const authorName = names[Math.floor(Math.random() * names.length)];
export const authorColor = colors[Math.floor(Math.random() * colors.length)];

let lastUpdateTs;

async function updateMessages () {
  const faunaResults = await getLatestMessages(lastUpdateTs);

  const latestMessages = faunaResults.map(item => ({
    ts: item[0],
    authorName: item[1],
    authorColor: item[2],
    message: item[3],
    id: item[4].value.id,
  }));

  if (latestMessages.length) {
    messages.update(array => [...array, ...latestMessages]);
    lastUpdateTs = latestMessages[latestMessages.length - 1].ts + 1;
  }
}

chatRoomStream.onUpdate.add(updateMessages);
```
Our store is exporting three pieces of data to our application

* **messages** which is a reactive [writable](https://svelte.dev/docs#writable).
* **authorName** which is decided randomly when the application starts from a list of animal names (it's a Fauna chat after all!).
* **authorColor** which is also decided randomly when the application starts from a list of CSS color names.

We're also importing the instance of our **FaunaStream** class and subscribing to the **onUpdate** signal using:
```
chatRoomStream.onUpdate.add(updateMessages);
```
In other words, whenever there is an update to the chat room document, the signal will execute our **updateMessages** function. This function, in turn, will fetch the latest messages from Fauna and update the **messages** reactive variable.

Whenever new messages are received, the timestamp (in microseconds) of the last message is stored into the **lastUpdateTs** variable:
```
lastUpdateTs = latestMessages[latestMessages.length - 1].ts + 1;
```
We're adding one microsecond to this value so that whenever we query Fauna we don't get the last message back:
```
const faunaResults = await getLatestMessages(lastUpdateTs);
```
Whenever a new user starts our chat, **lastUpdateTs** will be undefined, so instead it will receive the messages from the previous hour.

# The Svelte user interface
Our user interface has two components:

* **src/components/App.svelte** which serves as an application shell and displays the list of messages.
* **src/components/ChatForm.svelte** the form where users submit messages to the chat.

# App component
```
<script>
  import {afterUpdate} from 'svelte';
  import {messages, authorName, authorColor} from '../store/index.js';
  import ChatForm from "./ChatForm.svelte";

  let messagesDiv;

  afterUpdate(async () => {
    messagesDiv.scrollTop = messagesDiv.scrollHeight;
  });
</script>

<div id="App">
  <div class="messages" bind:this={messagesDiv}>
    {#each $messages as message}
      <div class="message">
        <strong style="color:{message.authorColor};">
          {message.authorName}
        </strong>: {message.message}
      </div>
    {/each}
  </div>
  <ChatForm/>
</div>
<div>
  You're writing as <strong style="color:{authorColor};">{authorName}</strong>
</div>

<style>
  #App {
    width: 30rem;
    margin-bottom: 1rem;
  }

  .messages {
    height: 20rem;
    overflow: hidden;
    padding: 1.5rem;
    overflow-y: scroll;
    box-sizing: border-box;
    background-color: white;
    margin-bottom: 1rem;
    border: 1px solid #aaa;
  }

  .message {
    margin-bottom: .75rem;
  }
</style>
```
This component is quite straightforward as it only displays the list of messages.

Remember that **messages** (defined in our store) is actually reactive. Because we're using the dollar sign in front of its name (**$messages**), Svelte will automatically subscribe to its changes and update the DOM accordingly. It will also unsubscribe from it when the component is destroyed.

We're also using the [afterUpdate](https://svelte.dev/docs#afterUpdate) hook so that, whenever this component is updated, the <div> containing the messages is scrolled to the end:
```
afterUpdate(async () => {
  messagesDiv.scrollTop = messagesDiv.scrollHeight;
});
```
Instead of using this component hook, we could have also manually added a callback to our reactive value like this:
```
const unsubscribe = messages.subscribe(() => {
  messagesDiv.scrollTop = messagesDiv.scrollHeight;
});

onDestroy(() => {
  unsubscribe();
});
```
# ChatForm component
Our **ChatForm** component is even simpler:
```
<script>
  import {createMessage} from '../fauna/index.js';
  import {authorName, authorColor} from "../store/index.js";

  let message;

  function onSubmit () {
    createMessage(authorName, authorColor, message);
    message = '';
  }
</script>

<form on:submit|preventDefault={onSubmit}>
  <input type="text" bind:value={message} required>
  <button type="submit">Send</button>
</form>

<style>
  form {
    display: flex;
  }

  input {
    width: 100%;
    margin-right: .25rem;
  }
</style>
```
It's essentially a **<form>** that triggers our **createMessage** function whenever it is submitted.

An interesting point to mention, is that component state in Svelte is always reactive. Whenever you use the assign operator (the equal sign) with one of the component variables, Svelte will update the relevant parts of the UI automatically.

For example, when we empty the message variable:
```
message = '';
```
Svelte will automatically empty our input field because we have bound its **value** with the **message** variable:
```
<input type="text" bind:value={message} required>
```
Remember, Svelte is a compiler. Before generating the final code of our app, it already knows which parts of the UI depend on which reactive variables. The compiler only needs to generate very simple and efficient JavaScript code to update the DOM in response to state changes.

# Conclusion
If you've made it this far, good job!

Here are some articles you could check to keep on learning about Fauna:

**Getting started with FQL**

Part 1: [Fundamental Fauna concepts](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-1)
Part 2: [Deep dive into indexes](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-2)
Part 3: [Modeling data with Fauna](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-3)
Part 4: [Running custom functions in Fauna](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-4)
Part 5: [Authentication and authorization in Fauna](https://fauna.com/blog/getting-started-with-fql-faunadbs-native-query-language-part-5)

**Core FQL concepts**

Part 1: [Working with dates and times](https://fauna.com/blog/core-fql-concepts-part-1-working-with-dates-and-times)
Part 2: [Temporality in Fauna](https://fauna.com/blog/core-fql-concepts-part-2-temporality-in-faunadb)
Part 3: [Data aggregation](https://fauna.com/blog/core-fql-concepts-part-3-data-aggregation)
Part 4: [Range queries and advanced filtering](https://fauna.com/blog/core-fql-concepts-part-4-range-queries-and-advanced-filtering)
Part 5: [Joins](https://fauna.com/blog/core-fql-concepts-part-5-joins)

Thanks for reading and, as always, if you have any questions don't hesitate to hit me up on Twitter: [@pierb](https://twitter.com/PierB)