Angular, Node, and PostgreSQL
By: Zach Olson
Tags: Angular, JavaScript, Web Development, Postgres, Nodejs

AngularJS to Angular and more … my personal journey ❤️☕️

Prerequisites

This article will assume you already have PostgreSQL setup and installed on your machine and that you have knowledge of SQL. A nice resource is Getting Started with PostgreSQL and I’ve always enjoyed using DBeaver Community as my database administration tool.

Let’s get started …

From the story above, we have the ability to authenticate with the help of Firebase Authentication and we get a response back with some basic information. I capture this data into the IUser interface.

interface IUser {

uid: string;

email: string;

photoURL?: string;

displayName?: string;

emailVerified: boolean;

}

PostgreSQL or NoSQL

TLDR; I chose PostgreSQL over NoSQL

Now, what to do with this data? The first chunk of logic I wrote was to store the data as a Firestore document. The NoSQL data model intrigued me since you didn’t have to define your schema before adding data. It’s much easier to update as data and requirements change. Seemed like something that might be worth a try.

🤔 But, I started to think about the data I’d be dealing with and I don’t think NoSQL is going to work. Most of the data I will be dealing with is going to be relational. We are going to have users, as-well-as things related to those users. Like fuel logs, cost logs, events/locations, photos, followers, projects, articles. The solution NoSQL suggests for these types of things is data duplication, fan-out writes, etc.

For example, our users will be able to write articles and each article will have their profile picture and name attached. If we have to fetch a lot of those articles we also have to make a second query for each review to get the user profile picture and name. We’ve found ourselves in the N+1 problem, 1 query now becomes N+1 queries.

What happens if the user changes their photo? In NoSQL, we have to code a cloud function that watches for that change and dynamically searches through every article and changes the photo reference. Firebase has a quota on the number of reads and writes you make and charges for each one after you’ve exceeded the quota. This could easily add up to a large surprise bill.

I am sure there are solutions to these problems. Yet, I had to factor in the learning curve on my part and the development time. I would love to hear how others have solved these problems with NoSQL, so feel free to reach out to me. NoSQL still intrigues me and I will play with it more on a different project.

Project Setup with Node

We are going to use PostgreSQL and need a way of communicating with it via API calls. With the help of Node and node-postgres, we can spin up a server that can make those calls to PostgreSQL. Before we get into that we need to make some changes to our current project.

Client

The first thing I want to do is move my client and server logic into separate folders. At the root of the project, we will create a folder called client . Move all of the current files, except .editorconfig and .gitignore , into that folder. From the terminal, we can navigate to our client folder, run npm start and our client will start just like before 👍

Server

At the root of the project, we will create another folder called server . I like to have a node_modules folder for both client and server which helps prevent merging issues and module collisions. Inside our server folder let’s create a package.json file:

{

"name": "angular-journey-server",

"version": "0.0.0",

"scripts": {},

"private": true,

"dependencies": {},

"devDependencies": {}

}

We will use Express and Nodemon, so let’s terminal into the server folder and get those installed:

npm install --save express

npm install --save-dev nodemon

To start our server we need to add some scripts to package.json … this is what it should look like:



"name": "angular-journey-server",

"version": "0.0.0",

"scripts": {

"start": "node index.js",

"start:dev": "nodemon ./

"test": "echo \"Error: No Test Specified\" && exit 1"

},

"private": true,

"dependencies": {

"express": "^4.17.1"

},

"devDependencies": {

"nodemon": "^2.0.6"

}

} "name": "angular-journey-server","version": "0.0.0","scripts": {"start": "node index.js","start:dev": "nodemon ./ www.js localhost 3080","test": "echo \"Error: No Test Specified\" && exit 1"},"private": true,"dependencies": {"express": "^4.17.1"},"devDependencies": {"nodemon": "^2.0.6"

Finally, we need to add the www.js file which will create our server! 🎉

const express = require('express');

const bodyParser = require("body-parser");

const app = express(); app.use(bodyParser.json()); app.get('/api/status', (req, res) => {

res.json({info: 'Node.js, Express, and Postgres API'});

}); app.get('/', (req, res) => {

res.send('Hello World!');

});

const PORT = process.env.PORT || 3080;

const server = app.listen(PORT, () => {

console.log(`Server Running:

}); // Listen to the specified port, otherwise 3080const PORT = process.env.PORT || 3080;const server = app.listen(PORT, () => {console.log(`Server Running: http://localhost:${PORT}` );}); /**

* The SIGTERM signal is a generic signal used to cause program

* termination. Unlike SIGKILL , this signal can be blocked,

* handled, and ignored. It is the normal way to politely ask a

* program to terminate. The shell command kill generates

* SIGTERM by default.

*/

process.on('SIGTERM', () => {

server.close(() => {

console.log('Server Close: Process Terminated!');

});

});

From the terminal navigate to the server folder and run npm run start:dev . Our server will run on port 3080. Since we utilize nodemon the server will restart whenever we make changes to our file 😃. No more stopping/starting the server! 🎉

Nodemon restarting due to changes

Modify our Angular service to call our Node API

Finally, we are back to the topic of dealing with the IUser data we receive from Firebase Authentication. Our end goal is to save this information into PostgreSQL instead of AngularFirestore. Eventually, we will use our Node backend to make our calls to PostgreSQL, but for now, let’s take small steps by storing the information locally in Node.

Open the file server\www.js and setup a variable user as an array, so we can store the information and rootUrl so we can easily change the API path if needed. Then we need app.get to return that array and app.post to add data to the array. Below in bold is a snippet of the file and what I added.

const express = require('express');

const bodyParser = require("body-parser");

const app = express(); let user = [];

const rootUrl = '/api'; app.use(bodyParser.json()); app.get(`${rootUrl}/user`, (req, res) => { res.json(user); }); app.post(`${rootUrl}/user, (req, res) => {

const reqUser = req.body.user;

user = [];

user.push(reqUser);

res.json(user);

}); ...

Next, we need to modify our Angular service client\src\app\shared\services\authentication.service.ts so we can call our API. Let’s create a variable rootURL since we know our API will be at the /api path level. This also helps so that if we ever change the location we only have to modify one line of code.

We want to perform an HTTP request to communicate with our Node server, so let’s import { HttpClient } from '@angular/common/http' and add a private http: HttpClient to the constructor . We won’t be using AngularFirestore so we can remove those imports and constructor references.

Modify the method updateUserData to call our Node POST to /user .

async updateUserData({

uid,

email,

displayName,

photoURL,

emailVerified }: IUser,

registrationName?: string

) {

const data: IUser = {

uid,

email,

displayName: (registrationName) ? registrationName : displayName,

photoURL,

emailVerified: emailVerified

} const headers = { 'content-type': 'application/json'};

const body = JSON.stringify(data); return await this.http.post<IUser>(this.rootURL + '/user', { user: body }, { headers }).toPromise().then((result) => {

return result;

}).catch(error => { throw error });

}

Angular won’t know how to use HttpClient so let’s import them into our application. Go to app.module.ts and add the following:

import { BrowserModule } from '@angular/platform-browser';

import { BrowserAnimationsModule } from '@angular/platform-browser/animations';

import { NgModule, ErrorHandler } from '@angular/core';

import { HttpClientModule, HTTP_INTERCEPTORS } from '@angular/common/http'; ... @NgModule({

declarations: [

AppComponent

],

imports: [

BrowserModule,

BrowserAnimationsModule,

AngularFireModule.initializeApp(environment.firebase),

AngularFirestoreModule,

AngularFireAuthModule,

HttpClientModule,

SharedModule,

AppRoutingModule

],

providers: [

AuthService,

{ provide: ErrorHandler, useClass: ErrorsHandler },

{ provide: HTTP_INTERCEPTORS, useClass: HttpsInterceptor, multi: true }

],

bootstrap: [AppComponent]

})

export class AppModule { }

Setup Angular 🤝 Node API Communication

Our Angular app is running on port 4200 and the Node API on port 3080. How do we get them to communicate? Angular has a built-in proxying method. In the root of our client folder, create a file proxy.conf.json , and set up so that app paths starting with /api will be redirected to the target location, which we set to our Node API.

Then we need to go to our angular.json file and add the proxyConfig key under serve -> options

Now we can start two terminals, one for our server and another for the client. If we navigate to http://localhost:4200/api/status it should respond with {"info":"Node.js, Express, and Postgres API"} . So, instead of using Firestore, if we re-login to the application updateUserData will call our Node API POST and store our values in the user variable. You can verify this by using our GET method to view the stored values http://localhost:4200/api/user .

Okay, but every time I restart my server I lose those values 😢! This is where we need to persist that data into our database.

PostgreSQL

As I mentioned before we will be utilizing PostgreSQL to store our user data. I already created the table in my public schema

DROP SEQUENCE IF EXISTS public.users_id_seq;

DROP TABLE IF EXISTS public."users"; CREATE SEQUENCE public.users_id_seq

INCREMENT BY 1

MINVALUE 1

MAXVALUE 2147483647

START 1

CACHE 1

NO CYCLE; CREATE TABLE public."users" (

id int4 NOT NULL DEFAULT nextval('users_id_seq'::regclass),

uid varchar(128) NOT NULL,

displayname varchar(32) NULL,

email varchar(64) NOT NULL,

emailverified bool NOT NULL DEFAULT false,

photourl varchar(255) NULL,

disabled bool NOT NULL DEFAULT false,

CONSTRAINT users_email_key UNIQUE (email),

CONSTRAINT users_pkey PRIMARY KEY (id),

CONSTRAINT users_uid_key UNIQUE (uid)

);

To connect to the database from Node we will be using node-postgres so we can create a pool of connections. This way we don’t have to open and close a client every time we make a query. More information on pooling can be found here 📑

From the terminal navigate to the server folder and run npm install pg . We want to keep our configuration details in a separate file that is not accessible from our version control. We will store this information inside an .env file inside our server folder. Below is an example of the keys we will be using. Just replace the bold values with your own values.

DB_USER=your-db-user

DB_PASSWORD=your-db-password

DB_HOST=your-db-host

DB_PORT=your-db-port

DB_DATABASE=your-db-database

We don’t want this file to be checked into our version control, so let’s add the entry, .env to our .gitignore file.

To read the .env file we can use an npm package called, dotenv. This allows us to load a .env file in Node and access the values from process.env . Let's get this installed by running, npm install dotenv from the terminal inside our server folder.

As early as possible in our application, require and configure dotenv. Open www.js and at the top put the following require("dotenv").config(); Next, let’s configure our connection pool using our new process.env

require("dotenv").config(); const express = require('express');

const bodyParser = require("body-parser");

const app = express(); // Postgres Configuration

const { Pool } = require('pg');

const pool = new Pool({

user: process.env.DB_USER,

host: process.env.DB_HOST,

database: process.env.DB_DATABASE,

password: process.env.DB_PASSWORD,

port: process.env.DB_PORT

}); // the pool will emit an error on behalf of any idle clients

// it contains if a backend error or network partition happens

pool.on('error', (err, client) => {

console.error('Unexpected Error:', err);

process.exit(-1);

}); let user = [];

const rootUrl = '/api'; app.use(bodyParser.json()); ...

Why Pooling?

I couldn’t explain it any better than they did in the documentation for node-postgres 😊

“If you’re working on a web application or other software which makes frequent queries you’ll want to use a connection pool.”

Connecting a new client to the PostgreSQL server requires a handshake which can take 20–30 milliseconds. During this time passwords are negotiated, SSL may be established, and configuration information is shared with the client & server. Incurring this cost every time we want to execute a query would substantially slow down our application. The PostgreSQL server can only handle a limited number of clients at a time. Depending on the available memory of your PostgreSQL server you may even crash the server if you connect an unbounded number of clients. note: I have crashed a large production PostgreSQL server instance in RDS by opening new clients and never disconnecting them in a python application long ago. It was not fun. PostgreSQL can only process one query at a time on a single connected client in a first-in-first-out manner. If your multi-tenant web application is using only a single connected client all queries among all simultaneous requests will be pipelined and executed serially, one after the other. No good!

Enough explaining … back to coding. Let’s modify our GET method to query our Postgres users table with an async / await

... app.use(bodyParser.json()); app.get(`${rootUrl}/user`, (req, res) => {

;(async () => {

const { rows } = await pool.query('SELECT * FROM users')

res.json(rows);

})().catch(err => {

res.json(err.stack)

})

}); ...

Since our table is empty we expect to see [] returned when we hit the endpoint http://localhost:4200/api/user . 👍

Now we can modify our POST method to perform an INSERT into our users table.

app.post(`${rootUrl}/user`, (req, res) => {

const {

uid, displayName, email,

emailVerified, photoURL

} = JSON.parse(req.body.user); ;(async () => {

const client = await pool.connect();

try {

let results = await client.query(

`INSERT INTO public.users (

uid,

displayname,

email,

emailverified,

photourl

) VALUES ($1, $2, $3, $4, $5)

ON CONFLICT DO NOTHING

RETURNING uid, displayname, email, emailverified, photourl`,

[uid, displayName, email, emailVerified, photoURL]);

if (results.rowCount === 0) {

results = {

uid, displayName, email, emailVerified, photoURL

};

}

res.status(201).json(results);

} finally {

// Make sure to release the client before any error handling,

// just in case the error handling itself throws an error.

client.release();

}

})().catch(err => {

res.status(500).json({

"code": err.code,

"message": err.message

});

});

});

Success! ✔️ The user's information will be persisted in our Postgres database when they log in.

💡 Notice

For simplicity, I chose to have the SQL queries in my Node logic. However, I would NOT suggest this because it exposes your database structure. Export them from a file that is NOT checked into your repo or have some of them setup as views with Postgres

Repository Branch

Conclusion

This is an excellent place to take a break. Feel free to grab a snack or cup of ☕️. If you have any questions, comments, or suggestions for my stories, let me know.

Thank you for taking the time to read my stories, and if you’re a follower I am grateful ❤️. If you’ve enjoyed this article please consider holding down that applaud button 👏!