Bitwise magic in PostgreSQL
By: Tamás Polgár
Tags: Postgresql, Sql, Database Development, Bitwise, Boolean

Hi, followers (all 15 of you), it’s been a while since I last had character diarrhea. This time the wisdom is: how to use integers to store Boolean switches in PostgreSQL?

Y tho?

Sometimes you need to store a lot of switches in your database. User rights is a good real life example. A single user may need 6–8 different flags describing their rights and privileges. While you can define a separate Boolean field for every single flag, it’s sometimes better to use a single integer, and use its bits as flags.

Now, why would you do that? It’s much easier to read a flag than to extract a bit from an integer. Granted, Boolean flags make easier reading and simpler code. But a good case can be made against this practice: cost. Let’s say you have a user with 8 flags. If you used Booleans, this is what you’d send from your REST backend to your frontend.

rights: {

banned: false,

confirmed: true,

user: true,

can_comment: true,

can_post: false,

can_message: false,

moderator: false,

administrator: false

}

Even if you minify this JSON structure, it’s 139 bytes. Still not too much. But let’s say you have to serve a few million requests every day. One million logins will set your traffic quota back with 132 megabytes. Apart from the cost, it’s a burden on the bandwidth too. There’s your reason to use a single byte.

rights: 14

Now one million queries will take 8 megabytes only. If it was only the single byte and no JSON node, it’d go down to 900 kilobytes.

(Side note: if anyone told you that 1 KB = 1000 bytes, don’t listen. That person is an impostor, a sore ass lamer, probably gay, and what’s even worse, a millennial. 1024 is the magic number, as the great John von Neumann once declared. If you disagree, go and live on a different planet, or preferably the Sun.)

Bitwise operations in PostgreSQL

A real gentleman isn’t scared of relational databases and frowns upon such modern ideas as “NoSQL” and abominations like Firebase. So let’s use PostgreSQL and see how can it help.

The easiest solution would be to do all bitwise calculations on your backend and use PostgreSQL to store the result. But that’d be an insult to such a versatile and powerful database engine, and also an unnecessary burden on your backend server. You’re probably paying for PostgreSQL as a service, or at least running it on a separate physical server or cluster, so just let these machines earn their pay. Let’s use stored PostgreSQL procedures!

First, let’s create a very simple user record.

CREATE TABLE IF NOT EXISTS users (

id serial NOT NULL PRIMARY KEY,

name character varying NOT NULL,

rights smallint NOT NULL DEFAULT 0

); INSERT INTO users (name, rights) VALUES ('John Doe', 255);

John Doe is born an almighty superuser with all 8 flags set to true as 255 is 11111111 in binary. The id field is an autoincrement one, and its value will be 1 as this is the first record. This is where we’d stop if we wanted to calculate things on the backend. After all, we can just retrieve John Doe’s record like this:

SELECT * FROM users WHERE id=1

>255

But that’s not good enough. Let’s write a PostgreSQL function which retrieves all the flags as a JSON structure!

Retrieving the value of a single bit from a number has a very simple magical formula in any programming language. It needs a bitwise right shift and a bitwise AND operator:

(byte >> bit_number) & 1 === 1

This also works in PostgreSQL, except you should use a single = instead of a triple one.

CREATE OR REPLACE FUNCTION getRights(flags int) RETURNS json as $$

BEGIN



RETURN JSON_BUILD_OBJECT(

'deleted', (flags >> 0) & 1 = 1,

'unconfirmed', (flags >> 1) & 1 = 1,

'user', (flags >> 2) & 1 = 1,

'can_comment', (flags >> 3) & 1 = 1,

'can_post', (flags >> 4) & 1 = 1,

'can_message', (flags >> 5) & 1 = 1,

'administrator', (flags >> 6) & 1 = 1

); END;

$$ LANGUAGE plpgsql; SELECT

getRights(rights) AS rights_json

FROM users

WHERE id=1;

This will produce the JSON seen in the example earlier. As we just set rights to 255, all JSON nodes will have a value of true .

While you may not want to send this JSON result to the frontend, you can use it for various operations on your backend. The procedure can be simplified to return just a single Boolean:

CREATE OR REPLACE FUNCTION hasRight(flags int, bit_no int) RETURNS boolean as $$

BEGIN

return (flags >> bit_no) & 1 = 1;

END;

$$ LANGUAGE plpgsql; SELECT

hasRight(rights, 7) AS is_admin

FROM users

WHERE id=1;

The result will be:

{

is_admin: true

}

Setting bits

Let’s say you want to take administrator rights away from John Doe. This means you should turn off bit 7 (also known as the most significant bit) of his rights field. How to do that? By subtracting a power of 2 from the current value. If it’s bit 7, then it’s 2⁷. Here’s a handy procedure to do just that.

CREATE OR REPLACE FUNCTION setFlag(flags int, switch int, value boolean) RETURNS smallint as $$

BEGIN IF (((flags >> switch) & 1 = 1) = value)

THEN

return flags;

END IF; IF (value)

THEN

return flags + 2^switch;

ELSE

return flags - 2^switch;

END IF; END;

$$ LANGUAGE plpgsql;

What does it do? It receives an integer value, a number indicating the bit we’d like to set, and then the value as a Boolean. If the bit already has the set value, the procedure simply returns the input value, otherwise it flips the bit.

SELECT setFlag(255, 8, false)

> 127

Flipping bits

What if you want to flip a bit, no matter its current value? Sure thing!

CREATE OR REPLACE FUNCTION switchFlag(flags int, bit_no int) RETURNS smallint as $$

declare

retval smallint;

BEGIN IF ((flags >> bit_no) & 1 = 1)

THEN

retval = flags - 2^bit_no;

ELSE

retval = flags + 2^bit_no;

END IF;



return retval;

END;

$$ LANGUAGE plpgsql;

This little routine will return a value with the specified bit flipped. Here’s how to flip bit 7:

SELECT switchFlag(255, 7); >127 SELECT switchFlag(127, 7); >255

So let’s take away John Doe’s privileges by flipping his bit:

UPDATE users

SET rights=switchFlag((SELECT rights FROM users WHERE id=1), 7)

WHERE id=1;

This will read the current value of users.rights for John, pass it to switchFlag() and update it with the return value. If you call it again, it will restore the bit, and thus John’s administrator privileges.