How to Implement Pagination with JavaScript
Tags: javascript, webdev, tutorial
Published: 2024-07-16 15:33:07

Pagination is the process of dividing a large set of data into smaller individual pages, making the information easier to process and digest when delivered to the user. In this tutorial, we are going to demonstrate how to implement a [JavaScript](https://www.thedevspace.io/course/javascript-introduction) pagination system in three different ways.

## Why you need JavaScript pagination

Creating a pagination system has several benefits. Imagine you have a blog with thousands of articles. It would be impossible to list all of them on one page. Instead, you could create a pagination system where the user can navigate to different pages.

Pagination also reduces server load, as only a segment of the data needs to be transferred every time a request is made. This enhances your application's overall performance, delivers a better user experience, and, as a result, improves the website's SEO.

## Project preparation

To get started, let’s initialize a fresh Node.js project. Go to your work directory and run the following command:

```bash
npm init
```

```bash
npm install express pug sqlite3 prisma @prisma/client
```

For this lesson, we are going to use [Prisma.js](https://www.thedevspace.io/course/javascript-prisma) as an example ORM, but you should remember that our focus is the logic behind pagination, not the tools.

Initialize Prisma with the following command:

```bash
npx prisma init
```

A `schema.prisma` file should be created. Open it and make the following edits.

```text
.
├── .env
├── index.js
├── libs
├── package-lock.json
├── package.json
├── prisma
│   ├── database.sqlite
│   ├── migrations
│   ├── schema.prisma   <===
│   └── seed.js
├── statics
│   └── js
│       └── app.js
└── views
 └── list.pug
```

```prisma
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "sqlite"
  url      = env("DATABASE_URL")
}

model Post {
  id          Int      @id @default(autoincrement())
  title       String
  content     String?
}
```

Line 5 to 8 specifies the type of database used, which is `sqlite` in this case, and `url` defines the connection string, which is pulled from the environmental variables stored in our `.env` file.

```text
.
├── .env   <===
├── index.js
├── libs
├── package-lock.json
├── package.json
├── prisma
│   ├── database.sqlite
│   ├── migrations
│   ├── schema.prisma
│   └── seed.js
├── statics
│   └── js
│       └── app.js
└── views
 └── list.pug
```

`.env`

```env
DATABASE_URL = "file:database.sqlite";
```

And line 10 to 14 create a new `Posts` table with a `title` and `content`.

For this tutorial, we will have to create a lot of posts to demonstrate how pagination works in JavaScript. To make things easier, instead of manually creating so many posts, let’s create a seed for our database. This ensures that the database will be filled automatically when we run database migrations.

Create a `seed.js` file under the `prisma` directory.

```text
.
├── .env   <===
├── index.js
├── libs
├── package-lock.json
├── package.json
├── prisma
│   ├── database.sqlite
│   ├── migrations
│   ├── schema.prisma
│   └── seed.js   <===
├── statics
│   └── js
│       └── app.js
└── views
 └── list.pug
```

`seed.js`

```javascript
const { PrismaClient } = require("@prisma/client");
const prisma = new PrismaClient();

async function main() {
  for (i = 0; i <= 99; i++) {
    await prisma.post.create({
      data: {
        title: `Post #${i}`,
        content: `Lorem ipsum dolor sit amet...`,
      },
    });
  }
}

main()
  .then(async () => {
    await prisma.$disconnect();
  })
  .catch(async (e) => {
    console.error(e);
    await prisma.$disconnect();
    process.exit(1);
  });
```

Then, you must tell Prisma where this `seed.js` file is located. Open the `package.json` file and add the following keys:

`package.json`

```json
{
  "name": "pagination",
  "type": "module", // Enables ES Modules, more info here: https://www.thedevspace.io/course/javascript-modules
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "prisma": {
    "seed": "node prisma/seed.js" // <===
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@prisma/client": "^5.16.1",
    "express": "^4.19.2",
    "prisma": "^5.16.1",
    "pug": "^3.0.3",
    "sqlite3": "^5.1.7"
  }
}
```

Finally, run the migration by executing the following command:

```bash
npx prisma migrate dev
```

## How to implement JavaScript pagination - the easy way

When you think about dividing items into pages, what is the easiest logic that comes to mind?

For example, you could retrieve all articles from the database as a single array and then split them into smaller arrays based on a certain page size using the `splice()` method.

![JavaScript Pagination](https://www.thedevspace.io/media/community/pagination-easy-way.excalidraw.png)

`index.js`

```javascript
import express from "express";
import { PrismaClient } from "@prisma/client";

const app = express();
const port = 3001;

const prisma = new PrismaClient();

app.set("views", "./views");
app.set("view engine", "pug");

app.use(express.urlencoded({ extended: true })); // for parsing application/x-www-form-urlencoded
app.use(express.json());

app.use("/statics", express.static("statics"));

// The easy way
// ===========================================================
app.get("/pages/:page", async function (req, res) {
  const pageSize = 5;
  const page = Number(req.params.page);
  const posts = await prisma.post.findMany({});

  const pages = [];
  while (posts.length) {
    pages.push(posts.splice(0, pageSize));
  }

  const prev = page === 1 ? undefined : page - 1;
  const next = page === pages.length ? undefined : page + 1;

  res.render("list", {
    posts: pages[page - 1],
    prev: prev,
    next: next,
  });
});

app.listen(port, () => {
  console.log(
    `Blog application listening on port ${port}. Visit http://localhost:${port}.`
  );
});
```

In this example, the page size is set to 5, meaning there will be five posts on every page.

Line 21, `page` is the current page number.

Line 22, `posts` is an array of all posts stored in the database.

Line 24 to 27, we split the array based on the page size. The `splice(index, count)` method takes two parameters, `index` and `count`. It splices and returns `count` number of elements from the array, starting from `index`. The remaining part of the array will be assigned to `posts`.

![JavaScript array splice](https://www.thedevspace.io/media/community/array-splice.excalidraw.png)

Line 29 and 30 each point to the previous and next page based on the current page number.

```javascript
const prev = page === 1 ? undefined : page - 1;
const next = page === pages.length ? undefined : page + 1;
```

If the current `page` is 1, `prev` will equal `undefined` because there is no previous page in this case. Otherwise, it equals `page - 1`.

`next`, on the other hand, will equal to `undefined` if the current `page` equals `pages.length`, meaning the current `page` is the last one. Otherwise it equals `page + 1`.

And lastly, the posts for the current page (`pages[page - 1]`), along with `prev` and `next`, will be sent to the corresponding view (`list.pug`).

`list.pug`

```pug
ul
    each post in posts
        li
            a(href="#") #{post.title}
    else
        li No post found.

if prev
    a(href=`/pages/${prev}`) Prev

if next
    a(href=`/pages/${next}`) Next
```

As you probably have realized, this solution has one problem. You have to retrieve all the posts from the database before splitting them into individual pages. This is a huge waste of resources, and in practice, it will likely take a very long time for the server to process this amount of data.

## How to implement offset-based pagination in JavaScript

So we need a better strategy. Instead of retrieving all the posts, we can first determine an offset based on the page size and the current page number. This way, we can skip these posts and only retrieve the ones we want.

In our example, the offset equals `pageSize * (page - 1)`, and we are going to retrieve the `pageSize` number of posts after this offset.

![offset based pagination in JavaScript](https://www.thedevspace.io/media/community/pagination-offset.excalidraw.png)

The following example demonstrates how this can be done using Prisma. The `skip` specifies the offset, and `take` defines the number of posts to retrieve after that offset.

```javascript
// Offset pagination
// ===========================================================
app.get("/pages/:page", async function (req, res) {
  const pageSize = 5;
  const page = Number(req.params.page);
  const posts = await prisma.post.findMany({
    skip: pageSize * (page - 1),
    take: pageSize,
  });

  const prev = page === 1 ? undefined : page - 1;
  const next = page + 1;

  res.render("list", {
    posts: posts,
    prev: prev,
    next: next,
  });
});
```

The frontend remains the same in this case.

`list.pug`

```pug
ul
    each post in posts
        li
            a(href="#") #{post.title}
    else
        li No post found.

if prev
    a(href=`/pages/${prev}`) Prev

if next
    a(href=`/pages/${next}`) Next
```

Of course, other ORM frameworks can achieve the same result, but the logic remains the same. At the end of this tutorial, we will provide some resources to help you create JavaScript pagination systems using other ORM frameworks.

## How to implement infinite scroll in JavaScript

Besides the offset-based pagination, there is a popular alternative called cursor-based pagination. This strategy is often used to create infinite scroll or the **Load More** button.

As the name suggests, the cursor-based pagination requires a cursor. When the user first visits a list of posts, the cursor points to the last item in the array.

![Javascript pagination cursor based strategy initial state](https://www.thedevspace.io/media/community/pagination-cursor-initial.excalidraw.png)

When the user clicks on the **Load More** button, a request is sent to the backend, which returns the next batch of posts. The frontend takes the transferred data and programmatically renders the new posts, and the corresponding cursor is updated to point to the last item of this new batch of posts.

![Javascript pagination cursor based next batch](https://www.thedevspace.io/media/community/pagination-cursor-next.excalidraw.png)

When it comes to actually implementing this cursor-based pagination, things get a bit more complicated, as this strategy requires the frontend and the backend to work together. But don’t worry, we’ll go through this step by step.

First of all, let’s create the root route (`/`). When the user visits this page, the first ten posts will be retrieved, and the `cursor` will point to the `id` of the last post. Recall that [`at(-1)`](https://www.thedevspace.io/course/javascript-arrays#accessing-an-array-element) retrieves the last element of the array.

```javascript
//Cursor-based pagination (load more)
// ===========================================================
const pageSize = 10;

app.get("/", async function (req, res) {
  const posts = await prisma.post.findMany({
    take: pageSize,
  });
  const last = posts.at(-1);
  const cursor = last.id;

  res.render("list", {
    posts: posts,
    cursor: cursor,
  });
});
```

Notice that the cursor will be transferred to the frontend as well. This is very important, and you must make sure that the cursor on both ends is always in sync.

`list.pug`

```pug
button(id="loadMore" data-cursor=`${cursor}`) Load More

ul(id="postList")
    each post in posts
        li
            a(href="#") #{post.title}
    else
        li No post found.

script(src="/statics/js/app.js")
```

The initial value of the cursor will be saved in the attribute `data-cursor` of the **Load More** button, which can then be accessed by JavaScript in the frontend. In this example, we put all the frontend JavaScript code inside `/statics/js/app.js`.

`/statics/js/app.js`

```javascript
document.addEventListener("DOMContentLoaded", function () {
  const loadMoreButton = document.getElementById("loadMore");
  const postList = document.getElementById("postList");

  let cursor = loadMoreButton.getAttribute("data-cursor");

  loadMoreButton.addEventListener("click", function () {
    fetch("/load", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        cursor: cursor,
      }),
    })
      . . .
  });
});
```

When the **Load More** button is clicked, a POST request will be sent to `/load` to retrieve the next batch of posts. Again, notice that you need to send the `cursor` back to the server, making sure they are always in sync.

Next, create a route handler for `/load`. This route handler takes the cursor and retrieves the next ten posts from the database. Remember to skip one so the post that cursor is pointing at will not be duplicated.

![Javascript pagination cursor based next batch](https://www.thedevspace.io/media/community/pagination-cursor-next.excalidraw.png)

```javascript
app.post("/load", async function (req, res) {
  const { cursor } = req.body;

  const posts = await prisma.post.findMany({
    take: pageSize,
    skip: 1,
    cursor: {
      id: Number(cursor),
    },
  });

  const last = posts.at(-1);
  const newCursor = last.id;

  res.status(200).json({
    posts: posts,
    cursor: newCursor,
  });
});
```

This handler will send a `200OK` response back to the frontend, along with the retrieved posts, which will again be picked up by the frontend JavaScript code.

`/statics/js/app.js`

```javascript
document.addEventListener("DOMContentLoaded", function () {
  const loadMoreButton = document.getElementById("loadMore");
  const postList = document.getElementById("postList");

  let cursor = loadMoreButton.getAttribute("data-cursor");

  loadMoreButton.addEventListener("click", function () {
    fetch("/load", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        cursor: cursor,
      }),
    })
      .then((response) => response.json())
      .then((data) => {
        if (data.posts && data.posts.length > 0) {
          data.posts.forEach((post) => {
            const li = document.createElement("li");
            const a = document.createElement("a");

            a.href = "#";
            a.textContent = post.title;

            li.appendChild(a);
            postList.appendChild(li);
          });
          cursor = data.cursor;
        } else {
          loadMoreButton.textContent = "No more posts";
          loadMoreButton.disabled = true;
        }
      })
      .catch((error) => {
        console.error("Error loading posts:", error);
      });
  });
});
```

> 🔗 [Download the demo project](https://github.com/thedevspacehq/pagination)

## Conclusion

Both the offset and cursor strategies have their pros and cons. For example, the offset strategy is the only option if you want to jump to any specific page.

However, this strategy does not scale at the database level. If you want to skip the first 1000 items and take the first 10, the database must traverse the first 1000 records before returning the ten requested items.

The cursor strategy is much easier to scale because the database can directly access the pointed item and return the next 10. However, you cannot jump to a specific page using a cursor.

Lastly, before we wrap up this tutorial, here are some resources you might find helpful if you are creating pagination systems with a different ORM framework.

- [Sequelize Limits and Pagination](https://sequelize.org/docs/v6/core-concepts/model-querying-basics/#limits-and-pagination)
- [TypeORM Pagination](https://typeorm.io/select-query-builder#using-pagination)
- [Objection.js](https://vincit.github.io/objection.js/recipes/paging.html#paging)
- [Mikro ORM Pagination](https://mikro-orm.io/docs/query-builder#pagination)

Happy coding!

## Further readings

- [What are the Data Types in JavaScript](https://www.thedevspace.io/course/javascript-data-types)
- [What are Maps and Sets in JavaScript](https://www.thedevspace.io/course/javascript-maps-sets)
- [What are Higher Order Functions in JavaScript](https://www.thedevspace.io/course/javascript-higher-order-functions)
- [JavaScript and Asynchronous Programming](https://www.thedevspace.io/course/javascript-asynchronous-programming)
- [How to Optimize Your Web App for Better Performance](https://www.thedevspace.io/course/miscellaneous-web-app-optimization)