3 Node DRBD + HA Installation

* server1 , server2 are the primary and secondary local nodes
* server-dr is the third node which is on a remote network
* Both server1 , server2 have interfaces on the 192.168.1.x network (eth0) for external connectivity.
* A crossover link exists on server1 and server2 (eth1) for DRBD data replication using 10.0.0.1 and 10.0.0.2
* Heartbeat provides a virtual IP of 192.168.1.10 and DRBD stacked to communicate with the disaster recovery node 192.168.1.20 located in a geographically diverse location.
* It is suggested DRBD and HA program is installed in CentOS5

[root@server1,2]# yum install heartbeat
[root@server1,2,-dr]# yum install kmod-drbd drbd
[root@server1,2]# chkconfig heartbeat on
[root@server1,2,-dr]# chkconfig drbd on

Install the heartbeat in server1,2 and drbd in server1,2,-dr

For Centos 5, edit in /etc/yum.conf and append this:
exclude=drbd82 kmod-drbd82*

[root@server1,2,-dr]# vi /etc/hosts

192.168.1.11 server1
192.168.1.12 server2
192.168.1.20 server-dr

Setup hostname in 3 node.

[root@server1]# vi /etc/ha.d/haresources

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
bcast eth1
ucast eth1 10.0.0.2
keepalive 2
auto_failback off
node server1
node server2

[root@server2]# vi /etc/ha.d/haresources

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
bcast eth1
ucast eth1 10.0.0.1
keepalive 2
auto_failback off
node server1
node server2

[root@server1,2]# vi /etc/ha.d/haresources
server01 IPaddr::192.168.1.10/24/eth0

[root@server1,2]# vi /etc/ha.d/authkeys
auth 3
3 md5 password

[root@server1,2]# chmod 600 /etc/ha.d/authkeys

Setup heartbeat in server1,2

[root@server1]# /etc/init.d/heartbeat start
[root@server1]# ifconfig |grep -b1 eth0:0
[root@server2]# /etc/init.d/heartbeat start

Heartbeat service start on server1 first , it is ok to be primary which hosted with VIP eth0:0 , then start server2's heartbeat.

[root@server1,2,dr]# fdisk /dev/sdb

Create the partition table /dev/sdb1 in 3 nodes for preparing DRBD setup(It is suggest they are same size)


resource drbd0 {

  protocol C;

  startup {
    degr-wfc-timeout 30;    # 30 sec.
    wfc-timeout 30;    # 30 sec.
  }

  disk {
    on-io-error   detach;
  }

  net {
    cram-hmac-alg "sha1";
    shared-secret "FooFunFactory";
  }

  syncer {
        rate 100M;
        al-extents 257;
  }

  on server1 {
    device    /dev/drbd0;
    disk      /dev/sdb1;
    address   10.0.0.1:7788;
    meta-disk internal;
  }

  on server2 {
    device    /dev/drbd0;
    disk      /dev/sdb1;
    address   10.0.0.2:7788;
    meta-disk internal;
  }

}


resource drbd10 {

  protocol A;

  stacked-on-top-of drbd0 {
    device    /dev/drbd10;
    address   192.168.1.10:7788;
  }

  on server-dr {
    device    /dev/drbd10;
    disk      /dev/sdb1;
    address   192.168.1.20:7788;
    meta-disk internal;
  }

}

Setup /etc/drbd.conf in 3 node (use scp to copy)

[root@server1,2]# drbdadm create-md drbd0
[root@server1,2]# /etc/init.d/drbd start
[root@server1]# drbdadm -- --overwrite-data-of-peer primary drbd0
[root@server1]# cat /proc/drbd

To build up the lower DRBD (res: drbd0) connection between server1(primary) & server2(secondary), After type "cat/proc/drbd" , you will see:

0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---
[==>.................] sync'ed: 15.9% (16057/19073)M

[root@server1]# drbdadm --stacked create-md drbd10
[root@server-dr]# drbdadm create-md drbd10
[root@server1]# drbdadm --stacked adjust drbd10
[root@server-dr]# drbdadm adjust drbd10
[root@server1]# drbdadm --stacked -- --overwrite-data-of-peer primary drbd10
[root@server1]# cat /proc/drbd

To build up the upper DRBD (res: drbd10) connection between server1(primary) & server-dr(secondary), After type "cat/proc/drbd" , you will see:

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent A r---
[=============>......] sync'ed: 74.4% (4900/19072)

[root@server1]# mkfs.ext3 drbd10
[root@server1]# mount /dev/drbd10 /home
[root@server1]# df -h
[root@server1]# umount /home

After it is synced, we will use the upper DRBD (drbd10) as our mounting point! But we need format it as ext3 filesystem first. Try mount it and test.

[root@server1,2]# vi /etc/ha.d/haresources
server01 IPaddr::192.168.1.10/24/eth0 drbdupper::drbd10 Filesystem::/dev/drbd10::/home::ext3

Revise haresource of server1,2 which we use "drbdupper" to serve.

[root@server2]# /etc/init.d/heartbeat stop
[root@server1]# /etc/init.d/heartbeat stop
[root@server1]# /etc/init.d/heartbeat start
[root@server2]# /etc/init.d/heartbeat start

Restart heartbeat to reload configure

[root@server1]# cat /proc/drbd
[root@server1]# df -h
[root@server1]# ifconfig |grep -b1 eth0:0

All are done, and take the final checking :]