Load Balanced Linux Web Cluster

Another example of a load balanced apache cluster, this time active-active using Pacemaker/Corosync/ldirectord.
This example is using CentOS.

10.11.1.40 – fixed IP of server1
10.11.1.41 – fixed IP of server 2
10.11.1.80 – apache site 1 (virtual IP)
10.11.1.50 – apache site 1 server 1
10.11.1.51 – apache site 1 server 2
10.11.1.90 – apache site 2 (virtual IP)
10.11.1.60 – apache site 2 server 1
10.11.1.61 – apache site 2 server 2

Make sure both servers have a sensible hostname and ensure their hosts files relate to this:

etc/hosts
127.0.0.1 localhost
10.11.1.40 webserver-01.mydomain.com webserver-01
10.11.1.41 webserver-02.mydomain.com webserver-02
10.11.1.50 site1-01
10.11.1.51 site1-02
10.11.1.80 site1-nlb
10.11.1.60 site2-01
10.11.1.61 site2-02
10.11.1.90 site2-nlb

Set up the fixed IP’s on each server in /etc/sysconfig/network-scripts/ i.e. server1 :
ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.11.1.40
NETMASK=255.0.0.0
GATEWAY=10.10.1.18

ifcfg-eth0:0
DEVICE=eth0:0
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.11.1.50
NETMASK=255.0.0.0
GATEWAY=10.10.1.18

ifcfg-eth0:1
DEVICE=eth0:1
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.11.1.60
NETMASK=255.0.0.0
GATEWAY=10.10.1.18

A loopback address is also needed for each VIP:

ifcfg-lo
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback

ifcfg-lo:0
DEVICE=lo:0
IPADDR=10.11.1.80
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback

ifcfg-lo:1
DEVICE=lo:1
IPADDR=10.11.1.90
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback

Update hostname in /etc/sysconfig/network
vi /etc/resolv.conf
nameserver 10.10.1.10
nameserver 10.10.1.12

Update /etc/sysctl.conf
net.ipv4.ip_forward = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.icmp_echo_ignore_broadcasts = 1

/etc/init.d/network restart
If you get an error SIOCADDRT: Network is unreachable ignore it.

Disabled selinux:
vi /etc/selinux/config
SELINUX=disabled

shutdown -r now

Set up SSH keys so that root can log onto each server. No passphrase is needed:

mkdir /root/.ssh
cd /root/.ssh
ssh-keygen -t rsa
chmod 600 id_rs*
scp id_rs* root@webserver-02:/root/.ssh
cat id_rsa.pub >> authorized_keys2
ssh root@webserver-02 "echo \`cat /root/.ssh/id_rsa.pub\` >> ~/.ssh/authorized_keys2"

Set up the correct repositories:
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
wget -O /etc/yum.repos.d/pacemaker.repo http://clusterlabs.org/rpm/epel-5/clusterlabs.repo

Install the HA stuff:
yum install -y pacemaker corosync openais heartbeat ldirectord

Install Apache and any web stuff needed
yum install httpd php mod_ssl

Disable httpd init scripts – pacemaker will control these
chkconfig httpd off
chkconfig ldirectord off

Make your web directories. In reality you would use a remotely mounted data store or rsync data etc. Local storage will be used in this example.

mkdir /var/www/html/site1
mkdir /var/www/html/site2
vi /var/www/html/site1/index.php
vi /var/www/html/site2/index.php

with something like:

<html><body><h1>Site 1 Test</h1>
< p>Site 1 test page.< /p>
< ?
$hostname=`/bin/hostname`;
echo "Running on ".$hostname.$_SERVER['SERVER_NAME'];
?>
< /body>< /html>

Update /etc/httpd/httpd.conf:

Listen 80
Listen 443

#Add your virtual hosts. They should listen on their local and the virtual IP:

ServerAdmin webmaster@localhost
ServerName site1
DocumentRoot /var/www/html/site1


SSLEngine On
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html/site1


ServerAdmin webmaster@localhost
DocumentRoot /var/www/html/site2


SSLEngine On
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html/site2

#Enable server status so Pacemaker can monitor apache

SetHandler server-status
Order deny,allow
Deny from all
Allow from 127.0.0.1

ExtendedStatus On

Add your config to /etc/ha.d/ldirectord.cf:
Something like wlc is a sendible scheduler but rr (round robin) is useful for initial testing
You might need to set checktype to connect for https to avoid invalid SSL errors failing the operation.

checktimeout=5
checkinterval=7
autoreload=yes
logfile="/var/log/ldirectord.log"
quiescent=yes
emailalert=me@domain.com
# Site 1 HTTP
virtual=10.11.1.80:80
fallback=127.0.0.1:80
real=10.11.1.50:80 gate 100
real=10.11.1.51:80 gate 100
service=http
scheduler=rr
protocol=tcp
checktype=negotiate
request="/"
receive="test"
Site 1 HTTPS
virtual=10.11.1.80:443
fallback=127.0.0.1:443
real=10.11.1.50:443 gate 100
real=10.11.1.51:443 gate 100
service=https
scheduler=wlc
protocol=tcp
checktype=negotiate
request="/"
receive="test"
Site 2 HTTP
virtual=10.11.1.90:80
fallback=127.0.0.1:80
real=10.11.1.60:80 gate 100
real=10.11.1.61:80 gate 100
service=http
scheduler=wlc
protocol=tcp
checktype=negotiate
request="/"
receive="test"
Site 2 HTTPS
virtual=10.11.1.90:443
fallback=127.0.0.1:443
real=10.11.1.60:443 gate 100
real=10.11.1.61:443 gate 100
service=https
scheduler=wlc
protocol=tcp
checktype=negotiate
request="/"
receive="test"

touch /var/log/ldirectord.log
chkconfig ldirectord off

(you want pacemaker to manage this)

Corosync configuration:

export ais_port=4000
export ais_mcast=226.94.1.1
export ais_addr=`ip address show eth0 | grep "inet " | tail -n 1 | awk '{print $4}' | sed s/255/0/`

Then, we check the data:
env | grep ais_

vi /etc/corosync/corosync.conf , enter your network address etc

compatibility: whitetank

totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 2.0.0.0
mcastaddr: 226.94.1.1
mcastport: 4000
}
}

logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}

amf {
mode: disabled
}
aisexec {
user: root
group: root
}
service {
# Load the Pacemaker Cluster Resource Manager
name: pacemaker
ver: 0
}

mkdir /var/log/cluster
touch /var/log/cluster/corosync.log

Make keys:
corosync-keygen
scp /etc/corosync/authkey webserver-02:/etc/init.d/corosync

chkconfig corosync on
/etc/init.d/corosync start

Time to configure the cluster:

crm configure edit
For apache, use the ocf script -lsb seems very unreliable, probably needs modifying.

node webserver-01 \
attributes standby="off"
node webserver-02 \
attributes standby="on"
primitive httpd ocf:heartbeat:apache \
meta target-role="Started" \
params configfile="/etc/httpd/conf/httpd.conf" statusurl="http://127.0.0.1/server-status" \
op monitor interval="20s"
primitive ldirectord ocf:heartbeat:ldirectord \
params configfile="/etc/ha.d/ldirectord.cf" \
op monitor interval="2m" timeout="20s" \
meta migration-threshold="10"
primitive virtual-ip-site1 ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.11.1.80" cidr_netmask="8" broadcast="2.255.255.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"
primitive virtual-ip-site2 ocf:heartbeat:IPaddr2 \
params lvs_support="true" ip="10.11.1.90" cidr_netmask="8" broadcast="2.255.255.255" \
op monitor interval="1m" timeout="10s" \
meta migration-threshold="10"
group load-balancing virtual-ip-site1 virtual-ip-site2 ldirectord
clone cl-httpd httpd
location prefer-node1 ldirectord \
rule $id="prefer-node1-rule" 100: #uname eq webserver-01
order IPs-up-first inf: virtual-ip-site1 virtual-ip-site2 cl-httpd
property $id="cib-bootstrap-options" \
dc-version="1.0.11-1554a83db0d3c3e546cfd3aaff6af1184f79ee87" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
no-quorum-policy="ignore" \
start-failure-is-fatal="false" \
stonith-enabled="false" \
last-lrm-refresh="1315990806"

Check its working!
crm_mon

You can leave a response, or trackback from your own site.

One Response to “Load Balanced Linux Web Cluster”

  1. […] SysAd Nonsensehttp://www.tomvernon.co.uk/blog/2012/03/load-balanced-linux-web-cluster/ Boludeces « Exploit de Joomla paso a paso    Briconsejos Gizmodo: el reloj […]

Leave a Reply