Posted by: kezhong | March 25, 2010

Setup Linux loadbalancer with LVS, ldirectord and heartbeat

As my last post, I had built a more healthy load balancer. But it is still not practical. Because if the linux director fails, the system will not work. The high availability should be considered. In this post, I list my steps to implement HA function.

I added another machine to the system. The environment of my lab is as below.

ha1: eth0: 192.168.26.201 eth1: 192.168.26.221(heartbeat)
ha2: eth0: 192.168.26.202 eth1: 192.168.26.222(heartbeat)
rs1: eth0: 192.168.26.211
rs2: eth0: 192.168.26.212
VIP: 192.168.26.200
gateway: 192.168.26.1

Configuration
On the ha1 system
Delete the following lines from /etc/rc.local.
ifconfig eth0:0 192.168.26.200 netmask 255.255.255.255 broadcast 192.168.26.200 up
echo 1 > /proc/sys/net/ipv4/ip_forward
/usr/sbin/ldirectord start

Stop the ldirectord daemon.
#/usr/sbin/ldirectord stop

Add the following lines to /etc/sysctl.conf.
net.ipv4.ip_forward = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2

Execute the sysctl command to load them.
# sysctl -p

Make ipvsadm not to automatically start at boot time
#chkconfig ipvsadm off

Add the following lines to the /etc/hosts file.
192.168.26.201    ha1
192.168.26.202    ha2

Configure the ha.cf file.
# cp /usr/share/doc/heartbeat-2.1.3/ha.cf /etc/ha.d/
Modify the /etc/ha.d/ha.cf file, use awk command to check the final result.
# awk ‘/^[^$]/&&/^[^#]/’ ha.cf
logfile    /var/log/ha-log
logfacility    local0
keepalive 2
deadtime 30
warntime 10

initdead 120
udpport    694
bcast    eth1        # Linux
ucast eth0 192.168.26.202
auto_failback on
node    ha1
node    ha2
ping 192.168.26.1
respawn hacluster /usr/lib64/heartbeat/ipfail
apiauth ipfail gid=haclient uid=hacluster

Configure the haresources file and check the result.
# cp /usr/share/doc/heartbeat-2.1.3/haresources /etc/ha.d/
# awk ‘/^[^$]/&&/^[^#]/’ haresources
ha1 IPaddr::192.168.26.200/24/eth0:0/192.168.26.255 ldirectord::/etc/ha.d/conf/ldirectord.cf

Configure the authkeys file and check the result.
# cp /usr/share/doc/heartbeat-2.1.3/authkeys /etc/ha.d/
# awk ‘/^[^$]/&&/^[^#]/’ authkeys
auth 1
1 crc

On the ha2 system
Install the heartbeat packages as my last post.

Add the following lines to /etc/sysctl.conf.
net.ipv4.ip_forward = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.eth0.arp_announce = 2 

Execute the sysctl command to load them.
# sysctl -p

Add the following lines to the /etc/hosts file.
192.168.26.201    ha1
192.168.26.202    ha2

Copy the /etc/ha.d/ha.cf, /etc/ha.d/haresources, /etc/ha.d/authkeys and /etc/ha.d/conf/ldirectord.cf files from the ha1 system.

Modify the /etc/ha.d/ha.cf file, change ‘ucast eth0 192.168.26.202’ to ‘ucast eth0 192.168.26.201’

Start the heartbeat service
Start the heartbeat service at the ha1 and ha2 system respectively.
# service heartbeat start

Make the service to automatically start at boot time.
# chkconfig –add heartbeat

Test
On the ha1 system
Use the ifconfig command, I could find the eth0:0.
Check the IPVS routing table, the result was as below.
# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.26.200:http rr
-> 192.168.26.211:http          Route   1      0          0
-> 192.168.26.212:http          Route   1      0          0

On the ha2 system
I was not able to find eth0:0 and the IPVS routing table.

On the ha1 system
Stop the heartbeat service.
# service heartbeat stop
I was not able to find eth0:0 and the IPVS routing table.

On the ha2 system
I found eth0:0 and the IPVS routing table.

On the ha1 system
Start the heartbeat service again.
I found eth0:0 and the IPVS routing table.

On the ha2 system
I was not able to find eth0:0 and the IPVS routing table.

During that period, I was able to launch the web site 192.168.26.200.


Responses

  1. In the “Direct Routing” case, 4 lines below should be put in real server machines and should not be put in heartbeat machines.

    net.ipv4.conf.all.arp_ignore = 1
    net.ipv4.conf.eth0.arp_ignore = 1
    net.ipv4.conf.all.arp_announce = 2
    net.ipv4.conf.eth0.arp_announce = 2

    • thank you for reminding

  2. what is the use of gateway machine. is it an external or internal gateway

  3. Excellent Post

  4. […] Setup Linux loadbalancer with LVS, ldirectord and heartbeat Install RAR Archiver on Fedora 12 […]

  5. […] StumbleUpon […]

  6. […] https://kezhong.wordpress.com/2010/03/25/setup-linux-loadbalancer-with-lvs-ldirectord-and-heartbeat […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: