Difference between revisions of "Building Scalable TFTP Cluster using LVS"
Jeffbearer (Talk | contribs) (→Introduction) |
Jeffbearer (Talk | contribs) |
||
| Line 10: | Line 10: | ||
And this is where the problem is, with a port based config, the LVS router doesn't know to listen on some random high port so the packet is dropped. The Real Server is listening on that port, but the client doesn't know anything about the real server. | And this is where the problem is, with a port based config, the LVS router doesn't know to listen on some random high port so the packet is dropped. The Real Server is listening on that port, but the client doesn't know anything about the real server. | ||
| − | This is where firewall marks come in to play, set up iptables rules on the lvs servers that mark all of your TFTP packets. Since TFTP uses any unprivileged port it's kind of like the carpet bombing of port selection. | + | This is where firewall marks come in to play, set up iptables rules on the lvs servers that mark all of your TFTP packets. |
| + | |||
| + | |||
| + | |||
| + | == Architecture == | ||
| + | |||
| + | |||
| + | == Configuration Example == | ||
| + | |||
| + | ===Adding Firewall Marks=== | ||
| + | |||
| + | In order to identify and group TFTP packets together use firewall marks. iptables can add the marks to your TFTP packets. Since TFTP uses any unprivileged port it's kind of like the carpet bombing of port selection, you will need to include them all. This will limit you to making TFTP the only UDP service that uses unprivileged ports for that particular Virtual Service IP. | ||
iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 69 -j MARK --set-mark 1 | iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 69 -j MARK --set-mark 1 | ||
iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 1024:65535 -j MARK --set-mark 1 | iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 1024:65535 -j MARK --set-mark 1 | ||
| − | + | Put those in an init script, or rc.local that runs on both of the LVS Directors. | |
| − | Then in the lvs.cf | + | ===lvs.cf Configuration=== |
| + | |||
| + | Then in the lvs.cf the config changes are simple, instead of the '''port''' option you use the '''fwmark''' option. In my example I left the port option for fun, but I think it's ignored. | ||
virtual tftp { | virtual tftp { | ||
| Line 45: | Line 58: | ||
weight = 1 | weight = 1 | ||
} | } | ||
| − | } | + | } |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| + | You may notice my check_tftp script, I took the [http://mathias-kettner.de/download/check_tftp Nagios Plugin] and it works great. | ||
== Conclusion == | == Conclusion == | ||
Revision as of 16:30, 8 June 2007
Contents
Introduction
TFTP is a bit tricky because of how the protocol works. The client sends Ack's to the same port from which the data came from the server. This is problematic with a port based LVS setup because the client ends up sending packets to the VIP on a port that it is not expecting packets. The answer to this is to use firewall marks.
TFTP Protocol:
- Client:12345 -> LVS: 69 (Request File)
- LVS:23456 -> Client: 12345 (Data)
- Client:12345 -> LVS: 23456 (Ack)
And this is where the problem is, with a port based config, the LVS router doesn't know to listen on some random high port so the packet is dropped. The Real Server is listening on that port, but the client doesn't know anything about the real server.
This is where firewall marks come in to play, set up iptables rules on the lvs servers that mark all of your TFTP packets.
Architecture
Configuration Example
Adding Firewall Marks
In order to identify and group TFTP packets together use firewall marks. iptables can add the marks to your TFTP packets. Since TFTP uses any unprivileged port it's kind of like the carpet bombing of port selection, you will need to include them all. This will limit you to making TFTP the only UDP service that uses unprivileged ports for that particular Virtual Service IP.
iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 69 -j MARK --set-mark 1 iptables -t mangle -A PREROUTING -i eth0 -p udp -s 0.0.0.0/0 -d <VIRTUAL IP> --dport 1024:65535 -j MARK --set-mark 1
Put those in an init script, or rc.local that runs on both of the LVS Directors.
lvs.cf Configuration
Then in the lvs.cf the config changes are simple, instead of the port option you use the fwmark option. In my example I left the port option for fun, but I think it's ignored.
virtual tftp {
active = 1
address = <VIRTUAL IP> eth0:1
vip_nmask = 255.255.255.0
fwmark = 1
port = 69
persistent = 45
expect = "OK - answer from server"
use_regex = 0
send_program = "/usr/local/bin/check_tftp --connect %h"
load_monitor = none
scheduler = wlc
protocol = udp
timeout = 6
reentry = 15
quiesce_server = 1
server ftp1 {
address = 10.0.0.2
active = 1
weight = 1
}
server ftp2 {
address = 10.0.0.3
active = 1
weight = 1
}
}
You may notice my check_tftp script, I took the Nagios Plugin and it works great.
Conclusion
|
"Building Scalable TFTP Cluster using LVS" is an LVS Example related stub. You can help LVSKB by expanding it |
