Pagination & Sorting with AWS Amplify
Tags: aws, amplify, javascript, graphql
Published: 2019-08-09 16:18:38

---
title: Pagination & Sorting with AWS Amplify
published: true
description: Building a group chat app with AWS Amplify
tags: aws, amplify, javascript, graphql
cover_image: https://thepracticaldev.s3.amazonaws.com/i/u0wuxlj8rui7evnqt0dv.png
---

In my previous post, [Server-Side Rendered Real-time Web App with Next.js, AWS Amplify & Next.js](https://dev.to/rakannimer/server-side-rendered-real-time-web-app-with-next-js-aws-amplify-graphql-2j49) we went into the details of setting up a Next.js React app and connecting it to Amplify on both the server and client-side. 

In this more high-level post we'll be going through how to build a channel-oriented group chat app with Amplify that needs to fetch, listen to, paginate and sort lists of data.

Feel free to skip the write-up and check the final code [here](https://github.com/rakannimer/pagination-and-sorting-with-aws-amplify/) and a demo [here](https://master.d134reqxn9j89v.amplifyapp.com) built and deployed with the AWS Amplify Console.

- [Project Requirements](#project-requirements)
- [Defining Local State](#defining-local-state)
- [Defining the Data Model with GraphQL](#defining-the-data-model-with-graphql)
  - [Mutations](#mutations)
  - [Creating our Custom Queries](#creating-our-custom-queries)
    - [`getChannelList`](#getchannellist)
    - [`getChannelMessages`](#getchannelmessages)
    - [`updateChannel`](#updatechannel)
  - [Using our Custom Queries](#using-our-custom-queries)
  - [Subscriptions](#subscriptions)
- [Deploying](#deploying)
- [Wrapping it up](#wrapping-it-up)

## Project Requirements

The group chat app should consist of 3 routes :

- `channel?id={id}` A chat room identified by an id. A user can send and receive messages in real-time. Messages are sorted by descending message creation data (newest at the bottom).

![What the channel page should look like](https://thepracticaldev.s3.amazonaws.com/i/lgaed7j0wnf58makup8z.png)

- `channels` A list of channels sorted by descending last update date (newest always at the top). A user can add channels and see channels added in real-time.

![What the channels page should look like] (https://thepracticaldev.s3.amazonaws.com/i/430gsiipeghyl0ef7vq0.png)


- `me` The current user's profile route, used to edit the user's basic information. A form with 3 fields username, url, bio

![What the profile page should look like] (https://thepracticaldev.s3.amazonaws.com/i/x130wve4ud2twd7pofjj.png)


> The color palette is taken from @dabit3's excellent [conference-app-in-a-box](https://github.com/dabit3/conference-app-in-a-box/)

## Defining Local State

We won't go much into the implementation details of the app's offline functionality but you can do that by trying it [here](https://master.d134reqxn9j89v.amplifyapp.com/), or by checking the final code [here](https://github.com/rakannimer/pagination-and-sorting-aws-amplify/). If you have any questions about the code, please leave a comment here or ping me on [twitter](https://twitter.com/rakannimer) and I'll be happy to answer them !

A single state object is being used for the whole app.

Our state _without_ pagination data looks like this :

```typescript
export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: Array<{
    id: string;
    messages: Array<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};
```

To be able to paginate through the data we will be needing to store the list of elements and an additional piece of data, the `nextToken` string that our API returns when fetching a list of items.

We can store that token at the same level as the list (e.g. { channelsNextToken:string, messagesNextToken: { [channelId]: string } }).

However it's easier to follow the format our API uses and instead of having channels and messages as arrays we can define them as a custom List.

A list has 2 fields : items and nextToken.

The type of elements in the items array depends on the list we're querying.

So the state becomes :

```typescript
type List<T = unknown> = { items: T[]; nextToken: string };

export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: List<{
    id: string;
    messages: List<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};
```

## Defining the Data Model with GraphQL

We want the messages in a channel to be sorted by `createdAt` and the `channels` in our ChannelList to be sorted by `updatedAt`.

To do that we assign a `sortField` to our [connection directive] (https://aws-amplify.github.io/docs/cli-toolchain/graphql#connection).

```graphql
type Message @model {
  id: ID!
  text: String!
  createdAt: String
  senderId: String
  channel: Channel @connection(name: "SortedMessages")
  messageChannelId: String
  # Because every message belongs to a channel
  # a messageChannelId field is added when we create an instance
  # of Message to know to which channel this message belongs.
  # Note that when doing 1-to-n relations using connections with GraphQL Transformer 
  # between 2 types typeA & typeB where typeA has many typeB
  # then a field typeBtypeAId is added to typeB to link it to the right typeA instance
}

type Channel @model {
  id: ID!
  name: String!
  createdAt: String!
  updatedAt: String!
  messages: [Message]
    @connection(name: "SortedMessages", sortField: "createdAt")
  channelList: ChannelList @connection(name: "SortedChannels")
  channelChannelListId: String
  # Same as in message, channel will have the id of its owner (ChannelList)
}

type ChannelList @model {
  id: ID!
  channels: [Channel]
    @connection(name: "SortedChannels", sortField: "updatedAt")
}

type User @model {
  id: ID!
  name: String
  bio: String
  url: String
}

# Note here our custom subscriptions.
# Amplify will generate subscriptions by default but the generated ones are too "broad".
# For example we don't want to listen to every new message created if we're in a channel,
# we just need messages that belong to the current channel.

type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display the latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])
}
```

With this GraphQL schema, Amplify will :

1. Setup all the cloud resources we need for our app to work on any scale.
2. Generate code to CRUD the data

For customizing our data pagination and sorting, we will need to do a bit of extra work ourselves but for the rest we will just be using code generated by Amplify.

### Mutations

We won't need to write any query for our mutations, the ones Amplify created for us are all we need.

In `src/graphql/mutations.ts` we'll find all the different possible mutations we can do.

We will be using :

- `createUser`
- `createMessage`
- `createChannel`
- `updateChannel`
- `createChannelList`

For example when a user sends a message :

```typescript
import { API, graphqlOperation } from "aws-amplify";
import { createMessage as createMessageQuery } from "../graphql/mutations";
import { MessageType, Dispatcher } from "../types";

const addMessage = async (
  content: string,
  dispatch: Dispatcher,
  me: State["me"],
  channelId: string
) => {
  const message = {
    text: content,
    createdAt: `${Date.now()}`,
    id: nanoid(),
    senderId: me.id,
    messageChannelId: channelId
  };
  dispatch({
    type: "append-message",
    payload: message
  });
  setScrollDown(Date.now());
  try {
    await (API.graphql(
      graphqlOperation(createMessageQuery, { input: message })
    ) as Promise<unknown>);
  } catch (err) {
    console.warn("Failed to create message ", err);
  }
};
```

### Creating our Custom Queries

#### `getChannelList`

Let's create a new file in `src/models/` and call it `custom-queries.ts`.

Inside it, we will add functions that return a GraphQL query when called.

In `custom-queries.ts`:

```typescript
export type GetChannelListInput = {
  channelLimit?: number;
  channelNextToken?: string;
  messageLimit?: number;
};

export const getChannelList = ({
  channelLimit = 5,
  channelNextToken = "",
  messageLimit = 1
}: GetChannelListInput) => `
query GetChannelList($id: ID!) {
  getChannelList(id: $id) {
    id
    channels(

      # Number of channels to fetch on each request

      limit: ${channelLimit},
      
      # sorting direction by the sortField we specified in our schema: updatedAt

      sortDirection: DESC,

      # nextToken is a long string that our API sends back that we can use to
      # retrieve the next batch of entries (older channels in this case)
      # When nextToken is null, then we reached the end of the list

      ${channelNextToken !== "" ? `nextToken:"${channelNextToken}"` : ``}
    ) {
      items {
        id
        name
        createdAt
        updatedAt
        messages(

          # How many messages per channel to retrieve, in our case 1
          limit: ${messageLimit},
          
          # To get the latest first

          sortDirection: DESC,
          # No need for nextToken here
        ) {
          items {
            id
            createdAt
            senderId
            text
          } 
          
        }
      }
      nextToken
    }
  }
}
`;
```

Looking more closely at our query we'll notice that we are using 3 optional arguments to the channels and messages list fields, `limit`, `sortDirection` & `nextToken` explained above in the comments.

#### `getChannelMessages`

This one should be straight-forward to understand, it also uses `limit`, `sortDirection` & `nextToken`

```typescript
export type GetMessageListInput = {
  messageLimit?: number;
  messageNextToken?: string;
};

export const getMessageList = ({
  messageLimit = 10,
  messageNextToken = ""
}: GetMessageListInput) => `
query GetChannel($id: ID!) {
  getChannel(id: $id) {
    id
    name
    createdAt
    updatedAt
    messages(
      limit: ${messageLimit},
      sortDirection: DESC,
      ${messageNextToken !== "" ? `nextToken:"${messageNextToken}"` : ``}
    ) {
      items {
        id
        text
        createdAt
        senderId
      }
      nextToken
    }
  }
}
`;
```

#### `updateChannel`

The result of a GraphQL subscription with AppSync is the [mutation selection set](https://docs.aws.amazon.com/appsync/latest/devguide/real-time-data.html). 

In our case, the mutation is `updateChannel` and the subscription `onUpdateChannel`

The generated `updateChannel` looks like this : 

```graphql
mutation UpdateChannel($input: UpdateChannelInput!) {
  updateChannel(input: $input) {
    id
    name
    createdAt
    updatedAt
    creatorId
    messages {
      items {
        id
        text
        createdAt
        senderId
        messageChannelId
      }
      nextToken
    }
    channelList {
      id
      channels {
        nextToken
      }
    }
    channelChannelListId
  }
}
```

When a conversation is updated we want to receive the last message and some information about the channel.
However, by default, lists are sorted in ascending order, so we need to tell our AppSync API that we want them in descending order, and we'll limit the messages in the set to just one, because we're only interested in the last one.

So we write a custom update query (in `src/models/custom-queries.ts`) based on how we want the data to look like when a subscription fires an event.


```graphql
mutation UpdateChannel($input: UpdateChannelInput!) {
    updateChannel(input: $input) {
      id
      name
      createdAt
      updatedAt
      creatorId
      messages(limit: 1, sortDirection: DESC) {
        items {
          text
        }
        nextToken
      }
      channelChannelListId
    }
  }
```


### Using our Custom Queries

The queries above should give us everything we need to fetch both our messages and channels as lists in chunks of 10 or whatever we specify above in the limit.

For example in the channel route, when the component receives a valid channelId we run our query for the first time :

```typescript
import * as React from "react";
import { Flatlist, ActivityIndicator, View } from "react-native-web";
import { API, graphqlOperation } from "aws-amplify";

import { DispatcherContext } from "../state";

const getChannelMessages = (channelId: string, nextToken: string) => {
  try {
    const query = getMessageList({
      messageLimit: 10,
      messageNextToken: nextToken
    });
    const messages = await API.graphql(
      graphqlOperation(query, { id: channelId })
    );
    return {
      messages: messages.data.getChannel.messages,
      channel: messages.data.getChannel
    };
  } catch (err) {
    console.warn("Failed to get messages ", err);
    return { messages: { items: [], nextToken: "" }, channel: {} };
  }
};

const Channel = ({ channelId, messages }) => {
  const dispatch = React.use(DispatcherContext);
  const [isLoading, setIsLoading] = React.useState(false);
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    setIsLoading(true);
    // We start by sending an empty string as nextToken
    getChannelMessages(channelId, "")
      .then(({ messages, channel }) => {
        if (!isMounted) return;
        setIsLoading(false);
        // We store the new messages that contain the next batch of messages and update the nextToken giant string
        dispatch({ type: "append-messages", payload: { channelId, messages } });
        // And update the channel's updatedAt field
        dispatch({ type: "update-channel", payload: channel });
      })
      .catch(err => {
        console.warn(
          "Failed to retrieve channel messages for channel ",
          channelId
        );
        setIsLoading(false);
      });
    () => {
      isMounted = false;
    };
  }, [channelId]);
  return (
    <FlatList
      inverted={true}
      ref={flatlistRef}
      ListFooterComponent={() =>
        isLoading ? (
          <ActivityIndicator
            animating={true}
            color={colors.highlight}
            style={{ marginTop: 15, marginBottom: 15, height: 30 }}
          />
        ) : (
          <View style={{ height: 30 }} />
        )
      }
      keyExtractor={item => item.id}
      data={messages.items}
      renderItem={({ item }) => <Message key={item.id} message={item} />}
      onEndReached={() => {
        if (messages.nextToken === null) return;
        setIsLoading(true);
        // When the end is reached we fetch the next batch of messages if they exist
        getChannelMessages(channelId, messages.nextToken).then(
          ({ messages }) => {
            setIsLoading(false);
            dispatch({
              type: "append-messages",
              payload: { channelId, messages }
            });
          }
        );
      }}
      onEndReachedThreshold={0.01}
    />
  );
};
```

### Subscriptions

For our subscriptions we won't need to write any GraphQL queries. Amplify will generate all the ones we need.

In the GraphQL Schema input for the GraphQL Transformer we defined some subscriptions: 

```graphql
type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])

```

This will generate the queries in `src/graphql/subscriptions.ts` and the types we need in `src/API.ts`

For example let's look at the code needed to listen to new messages on a channel : 

```typescript
import { API, graphqlOperation} from 'aws-amplify'
import {
  onCreateMessageInChannel,
} from "../graphql/subscriptions";

const Channel = () => {
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    API.graphql(
      graphqlOperation(
        onCreateMessageInChannel,
        { messageChannelId: channelId }
      )
    ).subscribe(message => {
      const newMessage = message.value.data.onCreateMessageInChannel;
      if (newMessage === null || newMessage.senderId === me.id || !isMounted) return;
      // prepend instead of append because they are sorted in descending order by createdAt 
      dispatch({ type: "prepend-message", payload: newMessage });
    });
    () => {
      isMounted = false;
      onCreateListener.unsubscribe();
    };
  }, [channelId]);
  
  // Rest of the code
}

```

Simple enough, listening to a graphql subscription and turning it off on unmount.

## Deploying

The code is built and deployed by the AWS Amplify Console. To deploy your own, you can click on this button [![amplifybutton](https://oneclick.amplifyapp.com/button.svg)](https://console.aws.amazon.com/amplify/home#/deploy?repo=https://github.com/rakannimer/pagination-and-sorting-with-aws-amplify) or just connect your repository to the Amplify Console and that's it. 

As you can see in the picture below, Amplify builds and deploys every commit on the master branch.

## Wrapping it up

Most applications will need to handle lists of data and fetch progressively from it (chat, comments, history, feed). 

This post goes through the challenging parts of doing that with React & Amplify and provides a good starting point to build one yourself !