HAProxy and Keepalived Setup for Ubuntu 16.04

High availability allows an application to automatically restart or reroute work to another capable system in the event of a failure, there must be a component that can redirect the work and must be a mechanism to monitor for failure and transition the system if an interruption is detected.

The keepalived daemon can be used to monitor services or systems and to automatically failover to a standby if problems occur.We will configure a floating IP address that can be moved between two capable load balancers.

These will each be configured to split traffic between two backend web servers.
If the primary load balancer goes down, the floating IP will be moved to the second load balancer automatically, allowing service to resume.

We are using servers from Digital Ocean. Assign a Floating IP for the Keepalived servers, please note that the servers must be in the same region.

Install Keepalived

apt-get install keepalived

Configure Keepalived for master server

vim /etc/keepalived/keepalived.conf ( for master server )

content :

vrrp_script chk_haproxy {
script "pidof haproxy"
interval 2
}

vrrp_instance VI_1 {
interface eth1
state MASTER
priority 200

virtual_router_id 33
unicast_src_ip 10.0.0.1
unicast_peer {
10.0.0.2
}

authentication {
auth_type PASS
auth_pass password
}

track_script {
chk_haproxy
}

notify_master /etc/keepalived/master.sh
}

Configure keepalived for backup server

vim /etc/keepalived/keepalived.conf ( for backup server )

content :

vrrp_script chk_haproxy {
script "pidof haproxy"
interval 2
}

vrrp_instance VI_1 {
interface eth1
state backup
priority 100

virtual_router_id 33
unicast_src_ip 10.0.0.2
unicast_peer {
10.0.0.1
}

authentication {
auth_type PASS
auth_pass password
}

track_script {
chk_haproxy
}

notify_master /etc/keepalived/master.sh
}

Add scripts file to track server ( both side )

vim /etc/keepalived/master.sh

content :

export DO_TOKEN='3757df50c19222de358be****************************************'
IP='100.1.1.23'
ID=$(curl -s http://169.254.169.254/metadata/v1/id)
HAS_FLOATING_IP=$(curl -s http://169.254.169.254/metadata/v1/floating_ip/ipv4/active)

if [ $HAS_FLOATING_IP = "false" ]; then
n=0
while [ $n -lt 10 ]
do
python /usr/local/bin/assign-ip $IP $ID && break
n=$((n+1))
sleep 3
done
fi

chmod 755 /etc/keepalived/master.sh

Install python requests

apt-get install python-requests

How to get assign-ip

cd /usr/local/bin

curl -LO http://do.co/assign-ip
python /usr/local/bin/assign-ip floating_ip droplet_ID

How to get Droplet_ID

curl -s http://169.254.169.254/metadata/v1/id

Install Haproxy

apt-get install haproxy

Configure haproxy log

vim /etc/rsyslog.d/haproxy.conf

content:

$ModLoad imudp
$UDPServerRun 514
local0.* /var/log/haproxy.log

service rsyslog restart

Configure haproxy

vim /etc/haproxy/haproxy.cfg

content :

global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon

ssl-default-bind-ciphers DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-CAMELLIA256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-
AES256-SHA
ssl-default-bind-options no-sslv3 no-tls-tickets
tune.ssl.default-dh-param 4096 #tune DH to 4096

defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

#stat page
listen stats # Define a listen section called "stats"
bind *:8989 # Listen on localhost:8989
mode http
stats enable # Enable stats page
stats hide-version # Hide HAProxy version
stats uri /haproxy_stats # Stats URI
stats auth statUSER:PASSWORD666 # Authentication credentials

frontend https_frontend
bind *:80
bind *:443 #ssl crt /etc/haproxy/test.com if u use https
mode http
option httpclose
option forwardfor
log-format "%ci:%cp \"[%tr]\" %ST %B \"%r\" \"%b\" \"%f\" \"%hrl\" \"%bi\" %si:%sp"
http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload;"
http-response replace-value Set-Cookie (.*) \1;\ Secure;
rspadd X-XSS-Protection:\ 1;\ mode=block
http-response set-header X-Frame-Options DENY
http-response set-header X-Content-Type-Options nosniff
http-response set-header Referrer-Policy no-referrer-when-downgrade

acl webserver1 hdr(host) -i www.test.com
acl webserver1 hdr(host) -i test.com
#default_backend web_server
use_backend webserver1-backend if webserver1

backend webserver1-backend
mode http
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
balance leastconn
option forwardfor
cookie SERVERID insert indirect nocache
#reqadd X-Forwarded-Proto:\ https
#redirect scheme https if !{ ssl_fc }
server s1 10.0.0.1:443 check cookie s1 ssl verify none

service haproxy restart