This blog post is the last part of a series of three tutorials dedicated to Rancher High-Availability.
We previously installed Rancher single-node using a external databases:
This time we will go full HA by installing Rancher on three servers.
You need an existing Rancher server (single-node) that uses an external database. You also need from 3 to 5 servers with at least 1GB RAM (8GB recommended) and a load balancer:
- Rancher Server One available at IP address 220.127.116.11,
- Rancher Server Two available at IP address 18.104.22.168,
- Rancher Server Three available at IP address 22.214.171.124,
- Load Balancer available at IP address 126.96.36.199.
All servers must have Docker installed.
You need a global access for each Rancher server on TCP Ports 22, 80, 443 and 18080.
You need access between Rancher servers :
- to UDP Ports 500, 4500
- to TCP Ports: 2181, 2376, 2888, 3888, 6379
Heap Size configuration
For larger deployments you may also need to increase the Rancher Heap Size.
That can be done by passing the environment variable
-e JAVA_OPTS="-Xms2048m -Xmx2048m" when running the rancher/server container.
Installing the Rancher Load Balancer
We need a load balancer in front of our 3 Rancher servers. The HA documentation gives us configuration samples for Nginx, Apache and HAProxy. Let’s give a try to HAProxy.
Copy the configuration file from Rancher documentation and edit it using you 3 nodes IP addresses:
global maxconn 4096 ssl-server-verify none defaults mode tcp balance roundrobin option redispatch option forwardfor timeout connect 5s timeout queue 5s timeout client 36000s timeout server 36000s frontend http-in mode tcp bind *:443 ssl crt /etc/haproxy/certificate.pem default_backend rancher_servers acl is_websocket hdr(Upgrade) -i WebSocket acl is_websocket hdr_beg(Host) -i ws use_backend rancher_servers if is_websocket backend rancher_servers server websrv1 188.8.131.52:443 weight 1 maxconn 1024 ssl server websrv2 184.108.40.206:443 weight 1 maxconn 1024 ssl server websrv3 220.127.116.11:443 weight 1 maxconn 1024 ssl
Connect to the load balancer server using SSH and paste the HAProxy configuration in a file named haproxy.cfg.
You may also need a SSL certificate. Generate it using the command:
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /root/rancher.key -out /root/rancher.crt
And create the .pem file using
cat rancher.crt rancher.key > rancher.pem
Then run a Docker container for HAProxy:
docker run -d --name haproxy -v /root/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro -v /root/rancher.pem:/etc/haproxy/certificate.pem:ro haproxy:1.5
You can display the logs using the command
docker logs -f haproxy.
Exporting the Rancher High-Availability script
Go to the Rancher High-Availability page:
- Copy/paste the IP address of your load balancer into the Host Registration URL field.
- Copy/paste the content of the rancher.key file into the Private Key field and the content of rancher.crt into Certificate.
- Click on Generate Config Script and on Download Config Script.
- Stop the Rancher container on Rancher Server One (
docker stop <containerID>).
- Copy the downloaded script on each node, and execute it:
sudo bash ./rancher-ha.sh rancher/server:v1.1.4
You can display the logs on each node using
docker logs -f rancher-ha.
Also you can access each Rancher UI at http://18.104.22.168:1800/ and view the Audit Logs (Admin > Audit Logs) and the HA Environment stack.
Wait for all the servers to start (this may take a few minutes). Finally the Rancher cluster UI should be available at the IP address of your load balancer https//22.214.171.124/ and should display the High-Availability status:
We managed to have multiple Rancher severs working together but the load balancer is still a Single Point of Failure. If it goes does the whole infrastructure would be unavailable.