Performance and Tuning

Performance Measurement

Tools

• testlvs: simple throughput testing tool for LVS from Julian, see the testlvs page
• ab: apache benchmark
• httperf: a tool for measuring web server performance from HP, see the httperf homepage
• Tsung: an open-source multi-protocol distributed load testing tool, see Tsung project homepage
• specweb: web server benchmark by spec.org, it is now specweb2005.
• webbench: a licensed benchmark program that measures the performance of Web servers, developed by VeriTest.

Performance Tuning

Connection Hash Table Size

The IPVS connection hash table uses the chaining scheme to handle hash collisions. Using a big IPVS connection hash table will greatly reduce conflicts when there are hundreds of thousands of connections in the hash table.

Note the table size must be power of 2. The table size will be the value of 2 to the your input number power. The number to choose is from 8 to 20, the default number is 12, which means the table size is 4096. Don't input the number too small, otherwise you will lose performance on it. You can adapt the table size yourself, according to your virtual server application. It is good to set the table size not far less than the number of connections per second multiplying average lasting time of connection in the table. For example, your virtual server gets 200 connections per second, the connection lasts for 200 seconds in average in the connection table, the table size should be not far less than 200x200, it is good to set the table size 32768 (2**15).

We can configure the size of IPVS connection hash table before compiling the Linux kernel. Here are the IPVS configurations in the 'make menuconfig' menu:

Networking Options -->
  IP: Virtual Server Configuration -->
    <M> virtual server support (EXPERIMENTAL)
    [ ]   IP virtual server debugging
    (12)   IPVS connection table size (the Nth power of 2)
    --- IPVS scheduler
    <M>   round-robin scheduling
    <M>   weighted round-robin scheduling
    <M>   least-connection scheduling scheduling
    <M>   weighted least-connection scheduling
    <M>   locality-based least-connection scheduling
    <M>   locality-based least-connection with replication scheduling
    <M>   destination hashing scheduling
    <M>   source hashing scheduling
    <M>   shortest expected delay scheduling
    <M>   never queue scheduling
    --- IPVS application helper
    <M>   FTP protocol helper

Netfilter Connection Track

IPVS uses its own simple and fast connection tracking for performance reasons, instead of using netfilter connection tracking. So, if you don't use firewalling feature at load balancer and you need an extremely fast load balancer, do not load netfilter conntrack modules into you system, because there is no need to do double tracking. Note that LVS/NAT should work too without the conntrack modules.

Julian compared the performance of IPVS with ip_conntrack and without ip_conntrack. See http://archive.linuxvirtualserver.org/html/lvs-users/2001-12/msg00141.html

Some Performance Data

While the LVS/DR cluster was pushing out 9.6Gbps traffic, the LVS load balancer was doing a negilgable ammount of work, which seems to indicate that LVS could push a great deal more traffic given sufficient real-servers and end-users.

• http://archive.linuxvirtualserver.org/html/lvs-users/2005-11/msg00266.html

External Links

• The C10K problem written by Dan Kegel, good notes on how to configure operating systems and write code to support 10K clients on a single server