haproxy tuning for TLS termination
Tags: haproxy, openssl
Published: 2022-08-24 18:45:43

---
title: haproxy tuning for TLS termination
published: true
description: finding that sweet maxsslrate setting
tags: haproxy, openssl
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
---

I'm working on a platform which has been a little neglects from a performance tuning point of view. We had some issues recently which effectively caused denial of service, even though we were doing it to ourselves. As a result, I'm tuning some things and I've learned some things which I didn't know.

You can probably DoS your haproxy instance too, if you've not already tuned it. I'm using the wonderful [hey](https://github.com/rakyll/hey) tool to create connections to my haproxy.

My haproxy has multiple certificates with unique private keys associated with each. Some are 4096 bit keys. Most are 2048 bit keys. Which certificate and key is used depends on which hostname is used to connect to the server.

I see huge differences in the performance based on 2k or 4k PK used for signing. I've learned that this is expected and that openssl ships with a speed command so that you know what to expect. In my case, haproxy is running on nodes with 8 cores (c5.2xlarge) and is configured to use them (cpu-map 1- 0-).

I've seen the `openssl speed rsa` command, but I had to hunt down the `rsa2048`, `rsa4096`, and `-multi` commands and option.

In my case, there isn't a great solution because I'm mixing 4k and 2k PK. Don't do that. I'll be advocating to my team to use 2k as much as possible.

Here is example output from the commands and the resulting `sign/s` number is pretty much the value I suggest using for the `maxsslrate` setting in haproxy.

```
# openssl speed  -multi 8 rsa2048
Forked child 0
Forked child 1
Forked child 2
Forked child 3
Forked child 4
Forked child 5
Forked child 6
Forked child 7
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+R1:8096:2048:10.00
+DTP:2048:public:rsa:10
+R1:8202:2048:10.00
+DTP:2048:public:rsa:10
+R1:8279:2048:10.00
+DTP:2048:public:rsa:10
+R1:7769:2048:10.00
+DTP:2048:public:rsa:10
+R1:8329:2048:10.00
+R1:7351:2048:10.00
+DTP:2048:public:rsa:10
+DTP:2048:public:rsa:10
+R1:7355:2048:10.00
+DTP:2048:public:rsa:10
+R1:7363:2048:10.00
+DTP:2048:public:rsa:10
+R2:258668:2048:10.00
+R2:280384:2048:10.00
+R2:274594:2048:10.00
Got: +F2:2:2048:809.600000:25866.800000 from 0
+R2:262465:2048:10.00
+R2:279210:2048:10.00
+R2:255086:2048:10.00
Got: +F2:2:2048:832.900000:27921.000000 from 1
Got: +F2:2:2048:735.100000:25508.600000 from 2
Got: +F2:2:2048:827.900000:27459.400000 from 3
Got: +F2:2:2048:776.900000:26246.500000 from 4
+R2:266538:2048:10.00
Got: +F2:2:2048:735.500000:26653.800000 from 5
Got: +F2:2:2048:820.200000:28038.400000 from 6
+R2:297361:2048:10.00
Got: +F2:2:2048:736.300000:29736.100000 from 7
OpenSSL 1.1.1  11 Sep 2018
built on: Mon Jul  4 11:25:51 2022 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-wL7Fqk/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000159s 0.000005s   6274.4 217430.6
```

```
# openssl speed  -multi 8 rsa4096
Forked child 0
Forked child 1
Forked child 2
Forked child 3
Forked child 4
Forked child 5
Forked child 6
Forked child 7
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+R1:1170:4096:10.00
+DTP:4096:public:rsa:10
+R1:1144:4096:10.00
+DTP:4096:public:rsa:10
+R1:1195:4096:10.00
+DTP:4096:public:rsa:10
+R1:1152:4096:10.00
+DTP:4096:public:rsa:10
+R1:1070:4096:10.00
+DTP:4096:public:rsa:10
+R1:1178:4096:10.01
+DTP:4096:public:rsa:10
+R1:1195:4096:10.00
+DTP:4096:public:rsa:10
+R1:1145:4096:10.02
+DTP:4096:public:rsa:10
+R2:75477:4096:10.00
+R2:71013:4096:10.00
+R2:82989:4096:10.00
+R2:84313:4096:10.00
Got: +F2:4:4096:115.200000:8431.300000 from 0
+R2:77447:4096:10.00
+R2:76991:4096:10.00
Got: +F2:4:4096:117.682318:7699.100000 from 1
Got: +F2:4:4096:119.500000:8298.900000 from 2
Got: +F2:4:4096:107.000000:7744.700000 from 3
Got: +F2:4:4096:117.000000:7547.700000 from 4
+R2:75476:4096:10.00
Got: +F2:4:4096:119.500000:7547.600000 from 5
+R2:78033:4096:10.00
Got: +F2:4:4096:114.271457:7803.300000 from 6
Got: +F2:4:4096:114.400000:7101.300000 from 7
OpenSSL 1.1.1  11 Sep 2018
built on: Mon Jul  4 11:25:51 2022 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-wL7Fqk/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
                  sign    verify    sign/s verify/s
rsa 4096 bits 0.001082s 0.000016s    924.6  62173.9
```

6200 is much larger than 920.

2k keys and `maxsslrate 6200` it is.