Zabbix native HA:
Lessons learned and Tips & Tricks

oicts.com

Who am I?
Nathan Liefting
Zabbix Consultant / Trainer

19 April 2023

oicts.com

1• Zabbix support• Zabbix training• Zabbix consultancy• And more…

https://www.linkedin.com/company/opensource-ict-solutions/19 April 2023

oicts.com

2

Introduction

19 April 2023

• Zabbix server

Natively built by Zabbix• Zabbix frontend

Use industry best practices• Zabbix database

Look at database options• Zabbix proxies

Nothing available or is there?

oicts.com

3

Zabbix serverzbx-srv-node01

zbx-srv-node02• Two servers, but how will they know each other their states?
19 April 2023

oicts.com

4

Zabbix server < 6.0

zbx-srv-node01

PCS - Corosync

zbx-srv-node02• The old way, still an option but..
19 April 2023

oicts.com

5

Zabbix server > 6.0

zbx-srv-node01

zbx-srv-node02

The new way(highly recommended)

zbx-db19 April 2023

oicts.com

6

Zabbix server > 6.0

zbx-srv-node01

zbx-srv-node02

HANodeName=zbx-srv-node01

HANodeName=zbx-srv-node02

NodeAddress=192.168.1.210:10051

NodeAddress=192.168.1.209:10051

zbx-db19 April 2023

oicts.com

7

Zabbix server > 6.0
Table within the database‘zabbix_server -R ha _status’ command

19 April 2023

oicts.com

8

Zabbix server > 6.0
Session ID

UniqueID

19 April 2023

UniqueNAME

Address:port
For frontend

oicts.com

HeartbeatCurrent
Status9

Zabbix server > 6.0

zbx-srv-node01
activezbx-srv-node02
passive19 April 2023

oicts.com

10

Zabbix server > 6.0

Zzbx-srv-node01
activeZZ

zbx-srv-node02
passive

Heartbeat +usual zabbix data

Heartbeat only

zbx-db19 April 2023

oicts.com

11

Zabbix server: Notes
Active node went -> STOPPEDStandby node takes over instantly• Failover at restart• Failover at lost nodeDefault failover delay: 1 min

Active node went -> UNAVAILABLE
Standby node takes over after 1 minute

Failover delay can be changed

19 April 2023

oicts.com

Zabbix frontend
zbx-webzbx-dbZ

zbx-srv-node01
active19 April 2023

ZZ

zbx-srv-node02
passiveoicts.com

13

Zabbix frontend

IF status == 0 (active)

19 April 2023

oicts.com

14

Zabbix frontend
zbx-webzbx-dbZ

zbx-srv-node01
active19 April 2023

ZZ

zbx-srv-node02
passiveoicts.com

15

Zabbix frontend
zbx-webzbx-dbZZ

Z

zbx-srv-node01
passive19 April 2023

zbx-srv-node02
activeoicts.com

16

Doesn’t matter how many frontends!

Zabbix frontend

zbx-web-node02

zbx-web-node01

zbx-dbZZZ

zbx-srv-node01
passive19 April 2023

zbx-srv-node02
activeoicts.com

17

Zabbix frontend
Zabbix users

VIP: 192.168.1.100
zbx-web-node01
activeHeartbeat
Local app like keepalived

192.168.1.101

19 April 2023

zbx-web-node02
passive
192.168.1.102

oicts.com

18

Zabbix frontend
Zabbix users

192.168.1.100
Loadbalancer
Like HAProxyzbx-web-node02
activezbx-web-node01
active192.168.1.101

19 April 2023

192.168.1.102
oicts.com

19

Zabbix server + frontend
zbx-web-node01
activezbx-srv-node01
activezbx-dbZZ

Zabbix users

Z
zbx-web-node02
passivezbx-srv-node02
passive19 April 2023

oicts.com20

Zabbix server + frontend
zbx-web-node01
passivezbx-srv-node01
passivezbx-dbZZ

Zabbix users

Z
zbx-web-node02
activezbx-srv-node02
active19 April 2023

oicts.com

21

Zabbix server + frontend
ZZZ
zbx-web-node01
activezbx-srv-node01
passivezbx-db
Zabbix users

zbx-web-node02
passivezbx-srv-node02
active19 April 2023

oicts.com

22

Zabbix frontend: Notes• A Load balancer adds more complication to your environment• Another things that can break/cause issues• VIP via local application like Keepalived is simple but..
• It’s a less scalable solution as you only connect to 1 frontend at the time

19 April 2023

oicts.com

Zabbix database
Loadbalancer or VIP via local app like Keepalived

Active/Passive
zbx-db-node01

zbx-db-node01

19 April 2023

Active/Active

oicts.com

zbx-db-node02

zbx-db-node02

24

Zabbix database: Active/PassiveZ

zbx-web-node01
activezbx-db-node01

zbx-srv-node01
activeZ

Zabbix usersZ
zbx-web-node02
passivezbx-srv-node02
passivezbx-db-node02
19 April 2023

oicts.com

Not a fan…
25

Zabbix database: Active/Active

zbx-srv-node01
activezbx-web-node01
activezbx-db-node01“Instant” Replication
ZZ

zbx-srv-node02
passive19 April 2023

Add:
DB check!

Z

Zabbix users

zbx-web-node02
passivezbx-db-node02

oicts.com

26

Zabbix database: Active/ActiveZZ

zbx-srv-node01
passiveZ
zbx-web-node01
passivezbx-db-node01“Instant” Replication
Add:
DB check!

zbx-srv-node02
active19 April 2023

Zabbix users

zbx-web-node02
activezbx-db-node02

oicts.com

27

Zabbix database: Notes• A Load balancer adds more complication to your environment• Another things that can break/cause issues• More complicated to maintain and tune for a database, a lot of data will passthrough it• VIP via local application like Keepalived is simple but..
• It still adds an additional application to setup and maintain• Possibly less scalable than a load balancer• Direct database connections are super simple in Active/Active• At failover, simply use a different database as well
• Possibly less scalable than a load balancer
19 April 2023

oicts.com

Zabbix proxies/agents: Configuration

zbx-srv-node01
activeProxy

19 April 2023

ZZZ

zbx-srv-node02
passive

Zabbix Proxy(or Agent)oicts.com

Agent

29

NO OFFICIAL SUPPORTED SOLUTION

Zabbix proxies
zbx-px-node01
activezbx-srv-node01
activezbx-px-node02
active???
19 April 2023

oicts.com

30

NO OFFICIAL SUPPORTED SOLUTION

Zabbix proxies

Monitored by proxy determines the actual active node
19 April 2023

oicts.com

31

NO OFFICIAL SUPPORTED SOLUTIONZabbix proxies

Failover will be executed by using problems, actions and a custom Python script
19 April 2023

oicts.com

32

Zabbix proxies: Notes• In house solution, created by Opensource ICT Solutions• Not available publicly as of right now• No load balancing with this option• Official proxy HA + load balancing is on the roadmap

19 April 2023

oicts.com

Zabbix proxies: Notes

https://blog.zabbix.com/how-to-setup-redundant-zabbix-proxieswithout-complex-cluster-configurations/12092/
19 April 2023

oicts.com

Questions?

19 April 2023

35