In order to make troubleshooting, we need to understand the IP load balancing technology that we are using now in the cluster, and know the whole packet flow, such as how packets are received and sent out at the load balancer, and how packets are handled at real servers and sent out back to the clients.
To know more information about IP load balacing technologies implemented in the IPVS, check
- Virtual Server via Network Address Transaltion
- Virtual Server via IP Tunneling
- Virtual Server via Direct Routing
Then, we can use some packet capture tools, such as ethereal and tcpdump, to do troubleshooting at our cluster system. First, we can capture the load balanced traffic at both load balancer and real servers, in order to make sure that basic load balancing system works; Second, we can also capture packets of service monitoring among load balancers and real servers, to make sure cluster monitoring and high availability works.