Creating a Data Pagination Function in PostgreSQL
By: Gravity Well, Rob Tomlin
Tags: Postgres, Backend, Pagination, Database, Postgresql

Assumptions For This Article

I will assume that you have access to PostgreSQL and a tool such as pgAdmin to write queries.

You can use your own data, or use the scripts below to create a table and insert some data.

Create a database named mycompany. Run this command to create a customer table

CREATE TABLE customer

(

CustomerId INT NOT NULL,

FirstName VARCHAR(40) NOT NULL,

LastName VARCHAR(20) NOT NULL,

Company VARCHAR(80),

Address VARCHAR(70),

City VARCHAR(40),

State VARCHAR(40),

Country VARCHAR(40),

PostalCode VARCHAR(10),

Phone VARCHAR(24),

Fax VARCHAR(24),

Email VARCHAR(60) NOT NULL,

SupportRepId INT,

CONSTRAINT PK_Customer PRIMARY KEY (CustomerId)

);

3. The Insert Into statements containing the data is long and is at the end of the article. Scroll down and run those commands to insert customers.

4. Verify you have customers (There should be 59.)

Snapshot of customers

Working with LIMIT and OFFSET

LIMIT tells the query how many records you want.

OFFSET tells the query where to start.

LIMIT

When using LIMIT, it is important to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query’s rows. You might be asking for the tenth through twentieth rows, but tenth through twentieth in what ordering? The ordering is unknown, unless you specified ORDER BY.

To get the first 5 records type the following in a query window.

Select * from public.customer

Order By customerid

LIMIT 5;

LIMIT of 5 customers

2. Try it with other numbers for LIMIT.

OFFSET

To get the first 5 records with, OFFSET, type the following in a query window. We are explicitly telling the query to start at the beginning.

You will get the same result as in the last example.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET 0;

Notice that OFFSET is 0 based. So we do not start at 1. This will be important later.

2. To get customers 6–10, type the following.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET 5;

Notice the use of OFFSET to tell the query where to start. Again it is 0 based so we use 5 and not 6.

3. Customers 11–15 can be obtained with.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET 10;

So we are on our way to pagination! But wait! There’s more

Making This More Intuitive

This is a good start but not real intuitive if we think about it from a front-end developer’s standpoint. They would have to pass a start of 0 and number of records desired, 5. Then remember where they left off and request, 5 and 5; then 10 and 5, etc.

A more intuitive approach would be for the front-end developer to say they want page 1 and 5 records. Then page 2 and 5 records, etc.

Step 1 — Observation

For the first step, let’s just play with the numbers in OFFSET (yes I said numbers.)

In the script below, the OFFSET will be 0. We will get the first 5 records.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET (1 - 1) * 5;

2. In this example, we get page 2, containing 5 records.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET (2 - 1) * 5;

The OFFSET is (2–1) * 5 or (1) * 5 or 5. Start at record 5 (0-based.)

3. In this example, we get page 3, containing 5 records.

Select * from public.customer

Order By customerid

LIMIT 5

OFFSET (3 - 1) * 5;

The OFFSET is (3–1) * 5 or (2) * 5 or 10.

Notice what is changing. If we make the first number of the OFFSET a variable (of course LIMIT could be a variable also), we are specifying page 1, page 2, page 3, etc.

Step 2 — The Function

We will write our function to accept two parameters. One for the page number (to calculate the OFFSET) and one for the page size (LIMIT.)

Write the function below.

CREATE OR REPLACE FUNCTION udf_GetRowsByPageNumberAndSize(

PageNumber INTEGER = NULL,

PageSize INTEGER = NULL

)

RETURNS SETOF public.customer AS

$BODY$

BEGIN

RETURN QUERY

SELECT *

FROM public.customer

ORDER BY customerid

LIMIT PageSize

OFFSET ((PageNumber-1) * PageSize);

END;

$BODY$

LANGUAGE plpgsql;

PageNumber is the parameter for the desired page.

is the parameter for the desired page. PageSize is the parameter for the desired number of records per page.

is the parameter for the desired number of records per page. We use SETOF to return all columns. We could specify a subset of the columns.

to return all columns. We could specify a subset of the columns. LIMIT is the PageSize

is the PageSize OFFSET is calculated as before but using the parameter variables, PageNumber and PageSize.

2. Execute the query to create the function.

3. Run the function using the following to get pages 1, 2, and 3, each containing 5 rows.

Select * from udf_GetRowsByPageNumberAndSize(1,5) Select * from udf_GetRowsByPageNumberAndSize(2,5) Select * from udf_GetRowsByPageNumberAndSize(3,5)

Step 4 — Improving Our Function

We can improve our function in two ways. First we can do the OFFSET calculation in a variable named PageOffset.

CREATE OR REPLACE FUNCTION udf_GetRowsByPageNumberAndSize(

PageNumber INTEGER = NULL,

PageSize INTEGER = NULL

)

RETURNS SETOF public.customer AS

$BODY$

DECLARE

PageOffset INTEGER :=0;

BEGIN



PageOffset := ((PageNumber-1) * PageSize);



RETURN QUERY

SELECT *

FROM public.customer

ORDER BY customerid

LIMIT PageSize

OFFSET PageOffset;

END;

$BODY$

LANGUAGE plpgsql;

FETCH NEXT

The second way we can improve the function is to use the more readable FETCH NEXT operator. This allows us to fetch a specific number of rows. It replaces LIMIT but functionally it is no different from LIMIT.

CREATE OR REPLACE FUNCTION udf_GetRowsByPageNumberAndSize(

PageNumber INTEGER = NULL,

PageSize INTEGER = NULL

)

RETURNS SETOF public.customer AS

$BODY$

DECLARE

PageOffset INTEGER :=0;

BEGIN



PageOffset := ((PageNumber-1) * PageSize);



RETURN QUERY

SELECT *

FROM public.customer

ORDER BY customerid

OFFSET PageOffset;

FETCH NEXT PageSize ROWS ONLY;

END;

$BODY$

LANGUAGE plpgsql;

And there you have it! A nice database function for paginating.

Conclusion

In this article we have developed one of two back-end approaches to pagination. The back-end technique we did not cover is not that different. It simply involves passing the query, with the LIMIT, OFFSET or FETCH NEXT (or a variation of these if using an ORM such as sequelize) commands; as well as the values from the API to the database in a an HTTP GET, instead of building a function in the database.

Our function can obviously be expanded to include additional parameters such as a table name, or a value to query for in a WHERE clause.

It should be noted that the LIMIT/OFFSET approach is best used on static data. If there is data actively being inserted or deleted in the table, the OFFSET might not be accurate. A good resource for other options is here.

Also, although each use case if different, consider using front-end pagination if you have a few thousand records or less and server-side when fetching records numbering in the tens of thousands.

I encourage you to expand upon this function and experiment.

Causionary information about this approach

Thank you for reading and coding along!

Data for customer table.