Chapter 3. Services Configuration Reference This chapter outlines the configuration for each specific service in the overcloud that uses load balancing. Use this configuration as a guide to configure your external load balancer. For more information about parameters, see the "HAProxy Configuration Manual" located in /usr/share/doc/haproxy-*/configuration.txt on the Controller nodes (or any system where the haproxy package is installed). Note Most services use a default health check configuration: The interval between two consecutive health checks set to 2000 milliseconds (or 2 seconds). After two successful health checks, a server is considered operational. After five failed health checks, the service is considered dead. Each service indicates the default health check or additional options in the Other information section of each service. 3.1. aodh Port Number: 8042 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.2. ceilometer Port Number: 8777 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.3. cinder Port Number: 8776 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.4. glance_api Port Number: 9292 Binds to: storage, external Target network/server: storage on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.5. glance_registry Port Number: 9191 Binds to: internal_api Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.6. gnocchi Port Number: 8041 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.7. heat_api Port Number: 8004 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check This service uses HTTP mode instead of the default TCP mode HAProxy Example: 3.8. heat_cfn Port Number: 8000 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.9. heat_cloudwatch Port Number: 8003 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.10. horizon Port Number: 80 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check This service uses HTTP mode instead of the default TCP mode This service uses cookie-based persistence for interactions with the UI HAProxy Example: 3.11. keystone_admin Port Number: 35357 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.12. keystone_admin_ssh Port Number: 22 Binds to: internal_api Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.13. keystone_public Port Number: 5000 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.14. mysql Port Number: 3306 Binds to: internal_api Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check. However, the health checks use port 9200. This service is load balanced to only one server at a time. Each server is only used in load balancing when all other non-backup servers are unavailable. If the server is marked down, all connections are immediately terminated. Enable the sending of TCP keepalive packets on both sides. Enable HTTP protocol to check on the servers health. Configure a stickiness table to store IP address. This helps maintain persistence. Important The mysql service uses Galera to provide a highly available database cluster. While Galera supports an active/active configuration, we recommend using an active/passive enforced by the load balancer to avoid lock contention. HAProxy Example: 3.15. neutron Port Number: 9696 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.16. nova_ec2 Port Number: 8773 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.17. nova_metadata Port Number: 8775 Binds to: internal_api Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.18. nova_novncproxy Port Number: 6080 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check The default balancing method is round-robin. However, for this service, use a source method. This method hashes the source IP address and divides it by the total weight of the running servers. This designates the server that receives the request. This ensures the same client IP address always reaches the same server as long as no server goes down or up. If the hash result changes due to a change in the number of running servers, the balancer redirects many clients to a different server. HAProxy Example: 3.19. nova_osapi Port Number: 8774 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.20. nova_placement Port Number: 8778 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.21. panko Port Number: 8779 Binds to: internal_api, external Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example: 3.22. redis Port Number: 6379 Binds to: internal_api (redis service IP) Target network/server: internal_api on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check. Perform health checks using tcp-check send/expect sequences. The string to send is "info\ replication\r\n" and the response is "role:master" The Redis service uses a password for authentication. For example, the HAProxy configuration uses a tcp-check with and AUTH method and the Redis administration password. The director normally generates a random password, but you can define a custom Redis password. See Section 4.2.2, "Configuring Load Balancing Options" for more information. The default balancing method is round-robin. However, for this service, use a first method. This ensures the first server with available connection slots receives the connection. HAProxy Example: 3.23. swift_proxy_server Port Number: 8080 Binds to: storage, external Target network/server: storage on overcloud-controller-0, overcloud-controller-1, and overcloud-controller-2 Other information: Each target server uses a default health check HAProxy Example:

Commands:
listen aodh bind 172.16.20.250:8042 bind 172.16.23.250:8042 server overcloud-controller-0 172.16.20.150:8042 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8042 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8042 check fall 5 inter 2000 rise 2
listen ceilometer bind 172.16.20.250:8777 bind 172.16.23.250:8777 server overcloud-controller-0 172.16.20.150:8777 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8777 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8777 check fall 5 inter 2000 rise 2
listen cinder bind 172.16.20.250:8776 bind 172.16.23.250:8776 server overcloud-controller-0 172.16.20.150:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8776 check fall 5 inter 2000 rise 2
listen glance_api bind 172.16.23.250:9292 bind 172.16.21.250:9292 server overcloud-controller-0 172.16.21.150:9292 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.21.151:9292 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.21.152:9292 check fall 5 inter 2000 rise 2
listen glance_registry bind 172.16.20.250:9191 server overcloud-controller-0 172.16.20.150:9191 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:9191 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:9191 check fall 5 inter 2000 rise 2
listen gnocchi bind 172.16.20.250:8041 bind 172.16.23.250:8041 server overcloud-controller-0 172.16.20.150:8041 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8041 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8041 check fall 5 inter 2000 rise 2
listen heat_api bind 172.16.20.250:8004 bind 172.16.23.250:8004 mode http server overcloud-controller-0 172.16.20.150:8004 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8004 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8004 check fall 5 inter 2000 rise 2
listen heat_cfn bind 172.16.20.250:8000 bind 172.16.23.250:8000 server overcloud-controller-0 172.16.20.150:8000 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.152:8000 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.151:8000 check fall 5 inter 2000 rise 2
listen heat_cloudwatch bind 172.16.20.250:8003 bind 172.16.23.250:8003 server overcloud-controller-0 172.16.20.150:8003 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8003 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8003 check fall 5 inter 2000 rise 2
listen horizon bind 172.16.20.250:80 bind 172.16.23.250:80 mode http cookie SERVERID insert indirect nocache server overcloud-controller-0 172.16.20.150:80 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:80 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:80 check fall 5 inter 2000 rise 2
listen keystone_admin bind 172.16.23.250:35357 bind 172.16.20.250:35357 server overcloud-controller-0 172.16.20.150:35357 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:35357 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:35357 check fall 5 inter 2000 rise 2
listen keystone_admin_ssh bind 172.16.20.250:22 server overcloud-controller-0 172.16.20.150:22 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:22 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:22 check fall 5 inter 2000 rise 2
listen keystone_public bind 172.16.20.250:5000 bind 172.16.23.250:5000 server overcloud-controller-0 172.16.20.150:5000 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:5000 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:5000 check fall 5 inter 2000 rise 2
listen mysql bind 172.16.20.250:3306 option tcpka option httpchk stick on dst stick-table type ip size 1000 timeout client 0 timeout server 0 server overcloud-controller-0 172.16.20.150:3306 backup check fall 5 inter 2000 on-marked-down shutdown-sessions port 9200 rise 2 server overcloud-controller-1 172.16.20.151:3306 backup check fall 5 inter 2000 on-marked-down shutdown-sessions port 9200 rise 2 server overcloud-controller-2 172.16.20.152:3306 backup check fall 5 inter 2000 on-marked-down shutdown-sessions port 9200 rise 2
listen neutron bind 172.16.20.250:9696 bind 172.16.23.250:9696 server overcloud-controller-0 172.16.20.150:9696 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:9696 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:9696 check fall 5 inter 2000 rise 2
listen nova_ec2 bind 172.16.20.250:8773 bind 172.16.23.250:8773 server overcloud-controller-0 172.16.20.150:8773 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8773 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8773 check fall 5 inter 2000 rise 2
listen nova_metadata bind 172.16.20.250:8775 server overcloud-controller-0 172.16.20.150:8775 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8775 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8775 check fall 5 inter 2000 rise 2
listen nova_novncproxy bind 172.16.20.250:6080 bind 172.16.23.250:6080 balance source server overcloud-controller-0 172.16.20.150:6080 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:6080 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:6080 check fall 5 inter 2000 rise 2
listen nova_osapi bind 172.16.20.250:8774 bind 172.16.23.250:8774 server overcloud-controller-0 172.16.20.150:8774 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8774 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8774 check fall 5 inter 2000 rise 2
listen nova_placement bind 172.16.20.250:8778 bind 172.16.23.250:8778 server overcloud-controller-0 172.16.20.150:8778 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8778 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8778 check fall 5 inter 2000 rise 2
listen panko bind 172.16.20.250:8779 bind 172.16.23.250:8779 server overcloud-controller-0 172.16.20.150:8779 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:8779 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:8779 check fall 5 inter 2000 rise 2
listen redis bind 172.16.20.249:6379 transparent balance first option tcp-check tcp-check send AUTH\ p@55w0rd!\r\n tcp-check send PING\r\n tcp-check expect string +PONG tcp-check send info\ replication\r\n tcp-check expect string role:master tcp-check send QUIT\r\n tcp-check expect string +OK server overcloud-controller-0 172.16.20.150:6379 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.20.151:6379 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.20.152:6379 check fall 5 inter 2000 rise 2
listen swift_proxy_server bind 172.16.23.250:8080 bind 172.16.21.250:8080 server overcloud-controller-0 172.16.21.150:8080 check fall 5 inter 2000 rise 2 server overcloud-controller-1 172.16.21.151:8080 check fall 5 inter 2000 rise 2 server overcloud-controller-2 172.16.21.152:8080 check fall 5 inter 2000 rise 2