Proxy Server Setup (HAproxy)
Tags: linux, proxy
Published: 2023-02-28 06:14:51

## Introduction 

> **Proxy servers** are computer servers that act as intermediaries between clients (such as web browsers) and servers. When a client requests a resource from a server, the request is first sent to the proxy server, which then forwards the request to the server on behalf of the client. The server then sends the response back to the proxy server, which in turn sends it back to the client

👉 Why do we use it?

Mainly we would use a proxy server for load balancing. They can improve performance, provide anonymity, filter content, enable access to blocked resources, enhance security, control content, bypass geographic restrictions, protect privacy, aid in debugging and troubleshooting, and save bandwidth

There are 2 modes in Load balancing. L4 load balancing is based on transport layer while L7 load balancing is based on application layer. L4 load balancing uses IP addresses and port numbers while L7 load balancing uses content of the traffic like HTTP headers or cookies

✨ I will be using HAProxy that uses both modes of Load Balancing 

> Furthermore, I will be setting up 2 Proxy servers (one as the backup server). I already have 2 web servers prepared for testing purpose

---

## Installing HAProxy

I will install some initial required packages

```bash
yum -y install gcc openssl openssl-devel systemd-devel wget
```
👉 **gcc** is a compiler, **openssl** for ssl/tls setting, **systemd-devel** for including the source into systemd, **wget** for downloading the haproxy package from the website

Creating an empty directory

```bash
mkdir /HAproxy
cd /HAproxy
```

Downloading the HAproxy package,

```bash
wget http://www.haproxy.org/download/2.3/src/haproxy-2.3.10.tar.gz

# Unzipping
tar xvfz haproxy-2.3.10.tar.gz
cd haproxy-2.3.10

# Compiling & installing required files
make TARGET=linux-glibc USE_OPENSSL=1 USE_SYSTEMD=1
make install
```

Downloading and saving HAproxy service file into `/etc/systemd/system/haproxy-service`

```bash
curl "https://git.haproxy.org/?p=haproxy-2.3.git;a=blob_plain;f=contrib/systemd/haproxy.service.in;" -o /etc/systemd/system/haproxy.service
```

We can confirm the file after this 

```bash
ls -l /etc/systemd/system/haproxy.service
-rw-r--r--. 1 root root 1409 Feb 28 10:38 /etc/systemd/system/haproxy.service
```

Editing service lines inside this file

```bash
vi /etc/systemd/system/haproxy.service

ExecStartPre=/usr/local/sbin/haproxy -Ws -f $CONFIG -c -q $EXTRAOPTS
ExecStart=/usr/local/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE $EXTRAOPTS
ExecReload=/usr/local/sbin/haproxy -Ws -f $CONFIG -c -q $EXTRAOPTS
```

After this, we can try starting the haproxy service

```bash
systemctl start haproxy
Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.
```

👉 This means that the primary setup has been done as the control process actually tried to start the service

Now, we need to make some directories for HAproxy

```bash
mkdir /etc/haproxy
mkdir /etc/haproxy/certs
mkdir /etc/haproxy/errors
mkdir /var/log/haproxy
```

Copying errorfiles under haproxy errors

```bash
cd ./examples/errorfiles
cp ./* .http /etc/haproxy/errors
```
Adding a HAproxy service user

```bash
useradd -c "HAproxy Daemon User" -s /sbin/nologin haproxy

tail -1 /etc/passwd
haproxy:x:1001:1001:HAproxy Daemon User:/home/haproxy:/sbin/nologin
```

Setting up the log file configuration

```bash
vi /etc/rsyslog.d/haproxy.conf

$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514
local0.* /var/log/haproxy/haproxy-traffic.log
```

Setting up logrotate

```bash
vi /etc/logrotate.d/haproxy

  /var/log/haproxy/*.log {
        daily
        rotate 30
        create 0600 root root
        compress
        notifempty
        missingok
        sharedscripts
        postrotate
                /bin/systemctl restart rsyslog.service > /dev/null 2>/dev/null || true
        endscript
   }
```

---

## Main Configuration file

We need the following lines inside the main config file for HAproxy

```bash
vi /etc/haproxy/haproxy.cfg

global
	daemon
	maxconn 4000 
	user haproxy
	group haproxy
	log 127.0.0.1:514 local0

defaults
	mode http
	option redispatch 
	retries 3
        # Redispatching to the another web server if retries are over 3
	log global
	option httplog
	option dontlognull
	option dontlog-normal
	option http-server-close
	option forwardfor
        # These are the log formats

	maxconn 3000
	timeout connect 10s
	timeout http-request 10s
	timeout http-keep-alive 10s
	timeout client 1m
	timeout server 1m
	timeout queue 1m
	
	errorfile 400 /etc/haproxy/errors/400.http
	errorfile 403 /etc/haproxy/errors/403.http
	errorfile 408 /etc/haproxy/errors/408.http
	errorfile 500 /etc/haproxy/errors/500.http
	errorfile 502 /etc/haproxy/errors/502.http
	errorfile 503 /etc/haproxy/errors/503.http
	errorfile 504 /etc/haproxy/errors/504.http

listen stats
	bind *:9000
	stats enable
	stats realm Haproxy Stats Page
	stats uri /
	stats auth admin:haproxy1
        # Authentication for the Admin page

frontend proxy
	bind *:80
	default_backend	WEB_SRV_list
	

backend WEB_SRV_list
	balance roundrobin
	option httpchk HEAD /
	http-request set-header X-Forwarded-Port %[dst_port]
	cookie SRVID insert indirect nocache maxlife 10m
	server WEB_01 192.168.1.128:80 cookie WEB_01 check inter 3000 fall 5 rise 3
	server WEB_02 192.168.1.129:80 cookie WEB_02 check inter 3000 fall 5 rise 3
```

👉 HAproxy configuration file setup reference:
※ https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/
※ https://cbonte.github.io/haproxy-dconv/2.3/configuration.html#3.1

> The above sets rules for load-balancing incoming traffic across two backend web servers using round-robin, health checks, and cookies, and includes settings for connection limits, timeouts, error files, logging, and authentication. The "listen stats" section enables access to HAProxy statistics with authentication

Verifying the config file

```bash
haproxy -f /etc/haproxy/haproxy.cfg -c
Configuration file is valid
```
Starting and enabling haproxy

```bash
systemctl start haproxy
systemctl enable haproxy
```

Editing firewall settings

```bash
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-port=9000/tcp
firewall-cmd --reload
```

> 👉 From both Web Servers

We will open the configuration file

```bash
vi /etc/httpd/conf/httpd.conf

# Line 196
SetEnvIf Request_Method HEAD Health-Check        
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{x-forwarded-for}i %l %u %t \"%r\" %>s %b" common

# Line 219
CustomLog "logs/access_log" common env=!Health-Check
```

Now, we need to restart the web server service and set firewall

```bash
systemctl restart httpd
firewall-cmd --permanent --add-service=http
firewall-cmd --reload
```

👉 If we don't have any page running on web servers

```bash
cd /var/www/html
cat > index.html
WEB 1

# From Server 2
cd /var/www/html
cat > index.html
WEB 2
```

To test if our proxy server is working, we can navigate to our proxy server's IP address,


![Proxy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7jaah6x237eikb8lemba.png)


After deleting the cookies data, we can reload the page to see that the proxy server forwards the client to the other server as well


![Proxy2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k51pw7w2xb1r9iv3w0m2.png)


We can also check report for HAproxy by connecting to the port 9000


![Stats](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fr0s0bd6f0zwz1jb6quq.png)


---

## Applying SSL/TLS

👉 From the proxy server

```bash
rpm -qa | grep openssl
```

We needed to confirm that we have openssl package available

Creating the private key

```bash
openssl genrsa -out /etc/haproxy/certs/ha01.key 2048
```

Creating the authentication .csr file

```bash
openssl req -new -key /etc/haproxy/certs/ha01.key -out /etc/haproxy/certs/ha01.csr
```

Creating the public key cert

```bash
openssl x509 -req -days 365 -in /etc/haproxy/certs/ha01.csr -signkey /etc/haproxy/certs/ha01.key -out /etc/haproxy/certs/ha01.crt
```

We need a backup of the Public and the private key in one file

```bash
cd /etc/haproxy/certs
cat ha01.crt ha01.key > ha01_ssl.crt
mv ha01.* /backup # Create /backup if it isn't available
```

Finally need to edit the main haproxy config file


```bash
vi /etc/haproxy/haproxy.cfg

# Adding these lines below the 'global' block
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 # the hash cipher set for SSL/TLS
 ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets # Allowing only TLSv1.2 above versions

# Adding these lines below the 'frontend' block
bind *:443 ssl crt /etc/haproxy/certs/ha01_ssl.crt
 http-request redirect scheme https code 308 unless { ssl_fc }
# the `unless {ssl_fc}` prevents looping when redirecting to `https`
```

Now we need to verify the configuration and add firewall

```bash
haproxy -f /etc/haproxy/haproxy.cfg -c

firewall-cmd --permanent --add-service=https
firewall-cmd --reload

systemctl restart haproxy
```

We can use `https` to see the result


![Https](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cq25haocprvr2fpvz5ie.png)

---

## Setting up a Backup Server

👉 What if the proxy server goes down? We would need a backup proxy server to serve when the main server goes down. To make a backup server and to make it run when the main goes down we first need a duplicate of the Proxy server that we created.

> I have already created a duplicate proxy server (only the SSL key names are HA02 instead of HA01)

So on both Proxy servers,

```bash
echo net.ipv4.ip_nonlocal_bind=1 >> /etc/sysctl.conf
sysctl -p
net.ipv4.ip_nonlocal_bind = 1
```
👉 We are making the nonlocal_bind parameter to be active. 

From both of the Proxy Servers

```bash
yum -y install keepalived-*
```

Now we have to set up the config files for the `keepalived` service

```bash
# Main Proxy Server
vi /etc/keepalived/keepalived.conf

global_defs {
   router_id HA_01
}

vrrp_script HA_Check {
        script "killall -0 haproxy"
        interval 1
	rise 3
	fall 3
        weight 2
}

vrrp_instance HAGroup_1 {
    state MASTER
    interface ens32
    garp_master_delay 5
    virtual_router_id 51
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass test123
    }
    virtual_ipaddress {
        192.168.1.150
    }
    track_script {
        HA_Check
    }
}

# Backup Proxy Server
vi /etc/keepalived/keepalived.conf

global_defs {
   router_id HA_02
}

vrrp_script HA_Check {
        script "killall -0 haproxy"
        interval 1
	rise 3
	fall 3
        weight 2
}

vrrp_instance HAGroup_1 {
    state BACKUP
    interface ens32
    garp_master_delay 5
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass test123
    }
    virtual_ipaddress {
        192.168.1.150
    }
    track_script {
        HA_Check
    }
}
```

> ✔ How does `Keepalived` works
- The master device sends VRRP messages according to a set schedule
- The backup device receives the VRRP messages sent by the master device
- If the backup device does not receive VRRP messages from the master device, it changes its state to become the master device
- When the backup device becomes the master device, it sends a Gratuitous ARP (GARP) message, which includes its MAC address, to inform other devices on the network of the change
- The GARP message is used by the switch devices connected to the server equipment where VRRP is running
- The switch devices update their MacAddress Table by mapping the VIP (Virtual IP) address to the MAC address of the device that has become the master

👉 As VRRP is L3 protocol, we need to use iptable rules to set up the firewall

From both Proxy Servers

```bash
firewall-cmd --direct --add-rule ipv4 filter INPUT 1 -i ens32 -d 224.0.0.18 -p vrrp -j ACCEPT
firewall-cmd --direct --add-rule ipv4 filter OUTPUT 1 -o ens32 -d 224.0.0.18 -p vrrp -j ACCEPT
firewall-cmd --runtime-to-permanent
firewall-cmd --direct --get-all-rules
ipv4 filter OUTPUT 1 -o ens32 -d 224.0.0.18 -p vrrp -j ACCEPT
ipv4 filter INPUT 1 -i ens32 -d 224.0.0.18 -p vrrp -j ACCEPT
``` 

```bash
systemctl start keepalived
systemctl enable keepalived
```

Now to test if this works,

From the Main Proxy Server

```bash
ip address list | grep 192.168.1.150
inet 192.168.1.150/32 scope global ens32
```

From the Backup Proxy Server

```bash
ip address list | grep 192.168.1.150
```

Now we will try stopping the keepalived service from our main Proxy Server to test if the Backup Proxy server runs automatically

So now from Main Proxy Server

```bash
systemctl stop keepalived
ip address list | grep 192.168.1.150
```

From the Backup Proxy Server

```bash
ip address list | grep 192.168.1.150
inet 192.168.1.150/32 scope global ens32
```
👉 If the Keepalived daemon is stopped on the master device, the backup device will confirm that it has taken over the VIP (Virtual IP) associated with the stopped master device

When the Keepalived daemon is started again, it is necessary to confirm that the original master device has regained control of the VIP

--- 

## Conclusion

✍ Today I discussed regarding how to set up a proxy server in Linux CentOS7, apply SSL/TLS certificate to redirect traffic to HTTPS and finally configured a backup proxy server for the main server.