Using Airtable as a database to store realtime messages
By: Srushtika Neelakantam
Tags: Chat, Vuejs, Realtime Database, Airtable

In terms of the folder structure you see in the GitHub project, the following are the main files of interest to us in this article.

ably-airtable-storage

| __ src

| __ | __ App.vue

| ______ | __ components

| __________ | __ infobox

| __________ | __ chatbox

| ______________ | __ ChatCard.vue

| ______________ | __ UsernameInput.vue

| ______________ | __ ChatMessage.vue

| ______________ | __ ChatInput.vue

server.js

The ably-airtable-storage folder holds the VueJS app whereas the server.js file in the root serves the VueJS app and issues auth tokens to the front-end app to authenticate with Ably. (More on this later)

As you saw in the live demo, we also have an information box on the side that shows the play-by-play of behind the scenes as you use the app. You can make use of that to understand what exactly is happening in each step and use the code snippets to try it out yourself. The code for this is in the infobox folder under the components folder. We won't be discussing much the information box in this article.

Let’s take a look at what’s going on in the rest of the files.

server.js

This is a super simple Express server that serves the index.html page from the dist folder of the Vue app. The dist folder is generated when you run the build command after you are done working on the Vue app. You can learn more about this in VueJS docs.

You’ll notice that we also have an /auth endpoint. As mentioned before, this is to issue tokens so the Vue app can authenticate securely with Ably's realtime service. Ably offers two ways of authenticating - Basic Auth and Token Auth. Basic auth uses the API Key directly whereas token auth expects auth tokens or JWT, making it a more secure way of authenticating the front-end applications. You can learn more about each of these types and trade-offs in Ably's documentation and best practice guide.

The VueJS chat app

1. App.vue

This is the main parent component for the entire app. So, a good place to instantiate and manage the connection with Ably.

We instantiate Ably in the created() lifecycle hook of this component and disconnect in the destroyed() lifecycle hook:

created() {

this.ablyRealtimeInstance = new Ably.Realtime({

authUrl: "/auth",

});

this.ablyRealtimeInstance.connection.once("connected", () => {

this.myClientId = this.ablyRealtimeInstance.auth.clientId;

this.isAblyConnected = true;

this.chatChannelInstance = this.ablyRealtimeInstance.channels.get(

this.chatChannelId

);

});

},

destroyed() {

this.ablyRealtimeInstance.connection.close();

},

The authUrl object sent to the Ably.Realtime instance prompts Ably that we are looking to authenticate via token auth via the given URL to automatically renew tokens just before they expire.

After the connection status becomes connected, we get an instance of the channel to subscribe to later. If you remember from the previous step, we’d need to use the chat-airtable channel name for publishing and subscribing to the chat messages as that is the channel we are using to trigger messages sent to the Airtable database. If you notice, the full name we specify, however, is [?rewind=2m]chat-airtable . The channel name is preceded by some meta-information enclosed in the square brackets. The channel param used there is Rewind with a value set to 2 minutes. This allows you to get any previously published messages in the last 2 minutes before successfully establishing a connection to Ably and attaching them to the channel. You can learn more about all the available channel params from Ably's docs.

2. ChatCard.vue

This is the parent component for the Group chat app, so we subscribe to updates on the chat channel here:

created() {

this.isReadyToChat = false;

this.chatChannelInstance.subscribe((msg) => {

this.handleNewMessage(msg);

});

},

We subscribe to the chat channel and call a new method to handle the new message every time the callback is invoked. More on this shortly.

This component has three child components:

UsernameInput.vue — accepts user’s name before they join the chat

ChatInput.vue — accepts users’ chat message if they’d like to send one

ChatMessage.vue — shows all chat messages in the group chat

The parent component has quite a few regular methods as well, here’s a break down of each:

i) The saveUsernameAndJoin() method

saveUsernameAndJoin(username) {

this.clientUsername = username;

this.isReadyToChat = true;

this.chatChannelInstance.presence.enter(username);

backgroundEventBus.$emit("updateBackgroundEventStatus", "join-chat");

}

This method is invoked from the UsernameInput.vue component and saves the username entered by the user. Ably's Presence feature allows you to see the realtime connection status of any client. This is useful to see which users are currently online. We make this user enter the presence set with their username in this method. The backgroundEventBus is a VueJS state management mechanism to emit various events to the infobox component.

ii) The handleNewMessage() method:

async handleNewMessage(msg) {

let messageContent = msg.data.records[0].fields;

let msgTimestamp = msg.timestamp;

await this.chatMsgsArray.push({

messageContent,

msgTimestamp,

msgType: "live",

});

if (this.$refs.chatMsgsBox) {

let divScrollHeight = this.$refs.chatMsgsBox.scrollHeight;

this.$refs.chatMsgsBox.scrollTop = divScrollHeight;

}

if (messageContent.clientId != this.myClientId && this.isReadyToChat) {

backgroundEventBus.$emit(

"updateBackgroundEventStatus",

"live-msgs-loaded"

);

}

}

Continuing from the channel subscription, this method is called for every new message pushed on chat the channel. We extract the required fields from the message and push them into the chatMsgsArray which is used to display messages in the chat screen. This is a live msg (vs one that is retrieved from a database).

iii) The loadPreviousMsgs() method:

loadPreviousMsgs() {

if (this.chatMsgsArray[0]) {

this.getMsgsFromDBWithMsgID();

} else {

this.getLatestMsgsFromDB();

}

}

This method is called when the ‘load previous messages’ popup is clicked. It checks if there is a previous message present in the chat array or not. Accordingly, calls other methods to retrieve messages from the database.

iv) The getMsgsFromDBWithMsgID method:

getMsgsFromDBWithMsgID() {

this.latestMsgId = this.chatMsgsArray[0].messageContent.msgId;

this.showLoadMoreBtn = false;

setTimeout(() => {

this.showLoadMoreBtn = true;

}, 500);

this.base = new Airtable({

apiKey: configVars.AIRTABLE_API_KEY,

}).base(configVars.AIRTABLE_BASE_ID);

let vueContext = this; this.base("Table 1")

.select({

view: "Grid view",

filterByFormula: "SEARCH('" + vueContext.latestMsgId + "',{msgId})",

})

.eachPage(function page(records, fetchNextPage) {

const latestRecordID = records[0].fields.ID;

vueContext.dbAutoNumber = latestRecordID;

if (latestRecordID) {

vueContext.getMsgsFromDBWithAutoID();

} else {

fetchNextPage();

}

});

}

This method is invoked when there’s a previous message present in the array. Note that all the records in the database are pre-ordered chronologically with an auto-incrementing ID field. We use msgId of the earliest message to find that record's ID in the Airtable database, then send another request to retrieve three records with an ID less than the previously retrieved record's ID. This is done in the getMsgsFromDbWithAutoID() method added next:

getMsgsFromDBWithAutoID() {

let vueContext = this;

this.base("Table 1")

.select({

maxRecords: 3,

view: "Grid view",

filterByFormula: "({ID}<" + vueContext.dbAutoNumber + ")",

sort: [{ field: "ID", direction: "desc" }],

})

.eachPage(

function page(records, fetchNextPage) {

records.forEach(async function(record) {

await vueContext.chatMsgsArray.unshift({

messageContent: record.fields,

msgTimestamp: 123,

msgType: "db",

});

backgroundEventBus.$emit(

"updateBackgroundEventStatus",

"db-msgs-loaded"

);

if (vueContext.$refs.chatMsgsBox) {

vueContext.$refs.chatMsgsBox.scrollTop = 0;

}

});

fetchNextPage();

},

function done(err) {

if (err) {

console.error(err);

return;

}

}

);

}

We add each of the retrieved records at the front of the chatsMsgsArray so they appear at the top of the chat list in the UI as the messages are ordered chronologically.

v) The getLatestMsgsFromDB() method:

getLatestMsgsFromDB() {

this.base = new Airtable({

apiKey: configVars.AIRTABLE_API_KEY,

}).base(configVars.AIRTABLE_BASE_ID);

let vueContext = this;

this.base("Table 1")

.select({

maxRecords: 3,

view: "Grid view",

sort: [{ field: "ID", direction: "desc" }],

})

.eachPage(

function page(records, fetchNextPage) {

records.forEach(async function(record) {

await vueContext.chatMsgsArray.unshift({

messageContent: record.fields,

msgTimestamp: 123,

msgType: "db",

});

backgroundEventBus.$emit(

"updateBackgroundEventStatus",

"db-msgs-loaded"

);

if (vueContext.$refs.chatMsgsBox) {

vueContext.$refs.chatMsgsBox.scrollTop = 0;

}

});

fetchNextPage();

},

function done(err) {

if (err) {

console.error(err);

return;

}

}

);

}

This method is invoked if there were no messages in the chatMsgsArray , meaning there was no earliest record to reference. We simply need the last three messages available in the database. The previous option can be combined with this as the filterByFormula field is the only differentiator, but it's added in two separate methods to make the two cases evidently clear.

3. ChatInput.vue

As mentioned before, this method manages the input box to add a new chat message. It has a single method that is invoked when the send button is clicked:

publishMessage() {

if (this.myMessageContent != "") {

const uniqueMsgId =

"id-" +

Math.random()

.toString(36)

.substr(2, 16); this.msgPayload = [

{

fields: {

clientId: this.myClientId,

msgId: uniqueMsgId,

username: this.clientUsername,

"chat-message": this.myMessageContent,

},

},

]; this.chatChannelInstance.publish("chat-msg", {

records: this.msgPayload,

});

backgroundEventBus.$emit("updateBackgroundEventStatus", "publish-msg");

this.myMessageContent = "";

}

}

In this method, we compute a random (unique) id to assign to the message and publish it on the chat channel with the message copy and other information like the clientId and username. As the echoMessages Ably client option is turned off by default, the same client also receives this message as a subscription update on the channel, leading to that message being added to the array and ultimately appearing in the UI.

As the UsernameInput.vue and ChatMessage.vue components are pretty much self-explanatory with minor data transformation and display, we'll skip explanations on those.

With that, we’ve closed the full loop of data transfer from publisher to subscriber to the database and back to the subscriber. Here’s the link again to the live demo so you can check it out again and piece the above information together: https://realtime-chat-storage.ably.dev/