Idea is to create haproxy -configuration where it’ll try to make sure to always use a working backend server. Usually all the tutorials suggest using haproxy with “option mysql-check” – which is not enough this time, because MariaDB with Galera might answer correctly for this poll but still be unavailable in DONOR -state.
Prerequirements:
- Working MariaDB Galera Cluster
- Haproxy installed on the application server
- xinetd installed on the database server
In the application server, we create haproxy frontend which is listening at 127.0.0.1:3306 for database connections, has weighted loadbalancing and is checks database servers in their port 9200, and has an administrative interface (for debug) in port 9600. And always check configuration and scripts, never just copy & paste from the internet.
# cat /etc/haproxy/haproxy.cfg global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon node APP1 description APP1.macronet.fi maxconn 40000 spread-checks 3 quiet defaults log global mode tcp option tcp-smart-accept option tcp-smart-connect option dontlognull option redispatch timeout check 3500 timeout queue 3500 timeout connect 3500 timeout client 10000 timeout server 10000 userlist STATS group admin users admin user admin insecure-password adminpass user stats insecure-password userpass listen adminpage bind *:9600 mode http stats enable stats refresh 60s stats uri / acl AuthOkay_ReadOnly http_auth(STATS) acl AuthOkay_Admin http_auth_group(STATS) admin listen galeracluster bind 127.0.0.1:3306 mode tcp balance leastconn timeout client 60000 timeout server 60000 option tcpka option httpchk option allbackups server DB1 192.168.10.21:3306 check port 9200 weight 128 server DB2 192.168.10.22:3306 check port 9200 weight 64 server DB3 192.168.10.23:3306 check port 9200 weight 32
In database server, we need to create a solution which listens in port 9200 and tells HAProxy that it’s OK. Theres quite many scripts already going around the ‘net, this is just one mix and match of ’em. wsrep_local_state = 4 == SYNCED == OK!
# cat /opt/galeracheck.sh
#!/bin/bash
#
# This script checks if a mysql server is healthy running on localhost. It will
# return:
# "HTTP/1.x 200 OK\r" (if mysql is running smoothly)
# - OR -
# "HTTP/1.x 500 Internal Server Error\r" (else)
#
# The purpose of this script is make haproxy capable of monitoring mysql properly
#
MYSQL_HOST="127.0.0.1"
MYSQL_PORT="3306"
MYSQL_USERNAME="HAProxy"
MYSQL_PASSWORD="HAProxyPassword"
MYSQL_OPTS="-N -q -A"
TMP_FILE="/tmp/mysqlchk.$$.out"
ERR_FILE="/tmp/mysqlchk.$$.err"
FORCE_FAIL="/tmp/proxyoff"
MYSQL_BIN="/usr/bin/mysql"
CHECK_QUERY="show global status where variable_name='wsrep_local_state'"
preflight_check()
{
for I in "$TMP_FILE" "$ERR_FILE"; do
if [ -f "$I" ]; then
if [ ! -w $I ]; then
echo -e "HTTP/1.1 503 Service Unavailable\r\n"
echo -e "Content-Type: Content-Type: text/plain\r\n"
echo -e "\r\n"
echo -e "Cannot write to $I\r\n"
echo -e "\r\n"
exit 1
fi
fi
done
}
return_ok()
{
echo -e "HTTP/1.1 200 OK\r\n"
echo -e "Content-Type: text/html\r\n"
echo -e "Content-Length: 43\r\n"
echo -e "\r\n"
echo -e "<html><body>MariaDB Galera is OK!</body></html>\r\n"
echo -e "\r\n"
rm $ERR_FILE $TMP_FILE
exit 0
}
return_fail()
{
echo -e "HTTP/1.1 503 Service Unavailable\r\n"
echo -e "Content-Type: text/html\r\n"
echo -e "Content-Length: 42\r\n"
echo -e "\r\n"
echo -e "<html><body>MariaDB Galera is *down*!</body></html>\r\n"
sed -e 's/\n$/\r\n/' $ERR_FILE
echo -e "\r\n"
rm $ERR_FILE $TMP_FILE
exit 1
}
preflight_check
if [ -f "$FORCE_FAIL" ]; then
echo "$FORCE_FAIL found" > $ERR_FILE
return_fail;
fi
$MYSQL_BIN $MYSQL_OPTS --host=$MYSQL_HOST --port=$MYSQL_PORT --user=$MYSQL_USERNAME --password=$MYSQL_PASSWORD -e "$CHECK_QUERY" > $TMP_FILE 2> $ERR_FILE
if [ $? -ne 0 ]; then
return_fail;
fi
status=`cat $TMP_FILE | awk '{print $2;}'`
if [ $status -ne 4 ]; then
return_fail;
fi
return_ok;
As you must have noticed, it needs permissions on the database server:
MariaDB [(none)]> GRANT USAGE *.* to 'HAProxy' IDENTIFIED BY 'HAProxyPassword';
To make this listen in a port, use something like the following xinetd -service:
# cat /etc/xinetd.d/galeracheck
# default: on
# description: galeracheck
service galeracheck
{
flags = REUSE
socket_type = stream
port = 9200
wait = no
user = nobody
server = /opt/galeracheck.sh
log_on_failure += USERID
disable = no
# only_from = 0.0.0.0/0
# recommended to put the IPs that need to connect exclusively (security purposes)
per_source = UNLIMITED
}
And you’ll need to add the service to /etc/services and restart xinetd:
echo "galeracheck 9200/tcp # Galera clustercheck" >> /etc/services
After all of this, your result should be something like this:
# nc 192.168.10.21 9200 HTTP/1.1 200 OK Content-Type: text/html Content-Length: 43 <html><body>MariaDB Galera is OK!</body></html>
And your application should be able to query the database backend successfully through HAProxy.