Difference between revisions of "Building clusterized proxy farms using LVS"
From LVSKB
| (5 intermediate revisions by 3 users not shown) | |||
| Line 7: | Line 7: | ||
*you need scalability because you want to be able to grow. | *you need scalability because you want to be able to grow. | ||
| − | In these cas, you've got plenty of answers to proceed. LVS is one of them (probably the best suited anyway). | + | In these cas, you've got plenty of answers to proceed. [[LVS]] is one of them (probably the best suited anyway). |
| − | But as you're doing extra-services over Squid, the base LVS ldirectord tests won't be sufficient. | + | But as you're doing extra-services over Squid, the base [[LVS]] [[ldirectord]] tests won't be sufficient. |
This page explains how to make some more advanced tests to be sure some extra-services will be checked too, and the corresponding realserver service will be dropped down if one of the extra service (or Squid itself) is not available. | This page explains how to make some more advanced tests to be sure some extra-services will be checked too, and the corresponding realserver service will be dropped down if one of the extra service (or Squid itself) is not available. | ||
| − | It present a proposed patch to ldirectord to do so. | + | It present a proposed patch to [[ldirectord]] to do so. |
== Architecture == | == Architecture == | ||
| − | The system architecture of web cache cluster using LVS is illustrated in the following figure. | + | The system architecture of web cache cluster using [[LVS]] is illustrated in the following figure. |
| − | |||
[[Image:sl-ha-lb-overview-ultramonkey3.png|center]] | [[Image:sl-ha-lb-overview-ultramonkey3.png|center]] | ||
| − | * it | + | |
| − | * it | + | * it uses [[LVS]] with Ultramonkey 3 settings for a [http://www.ultramonkey.org/3/topologies/sl-ha-lb-overview.html Streamline High Availability and Load Balancing] setting. |
| + | * it's based on the usage for Squid + DansGuardian (but could feet for any Squid + *). | ||
== Configuration Example == | == Configuration Example == | ||
| + | To set up this configuration with [[LVS]], just proceed to [http://www.ultramonkey.org/3/installation.html installation] and [http://www.ultramonkey.org/3/topologies/sl-ha-lb-eg.html configuration] as indicated on the Ultramonkey documentations. | ||
| + | Then : | ||
| + | * set up your Squid server, listening on port 3128 (example) and using DansGuardian on port 8000 | ||
| + | * set up your DansGuardian server, listening to port 8000 | ||
| + | * set up DansGuardian to reject (filter) the "http://proxy.testing.net/" URL (adding this domain/url to the blacklists. Be sure the "URL Filtering" will be returned by the proxy in case of filtering this URL. | ||
| + | * patch /usr/sbin/ldirectord with the FIXME patch | ||
| + | * configure your /etc/ha.d/ldirectord.cf with the following tests (example IP addresses): | ||
| + | |||
| + | # /etc/ha.d/ldirectord.cf | ||
| + | # Virtual Server for Proxy Service and Filtering Service | ||
| + | virtual=192.168.0.100:3128 | ||
| + | real=192.168.0.10:3128 gate | ||
| + | real=192.168.0.11:3128 gate | ||
| + | real=192.168.0.12:3128 gate | ||
| + | real=192.168.0.12:3128 gate | ||
| + | service=proxy_http | ||
| + | request="http://proxy.testing.net" | ||
| + | receive="URL Filtering" | ||
| + | protocol=tcp | ||
| + | scheduler=wlc | ||
| + | protocol=tcp | ||
| + | checktype=negotiate | ||
== Conclusion == | == Conclusion == | ||
| + | If you were using http test, you could only be able to check Squid is running. In case of DansGuardian issue, it could be a bad thing. This way, if the filtering engine is down/not responding, the whole Squid realserver will be dropped down for [[LVS]]. | ||
| + | |||
| + | Of course, you must still monitor all process to be warned anything is going wrong. But you will have time to resolve as your users won't notice anything :) Thank's a lot to [[LVS]] again! | ||
| + | == The ldirectord patch == | ||
| + | This patch is to be applied to release +ldirectord,v 1.77.2.32+, but is basic enought for you to report it to any newer release of [[ldirectord]]. | ||
| + | --- ldirectord_sav 2005-10-03 08:18:19.000000000 +0200 | ||
| + | +++ ldirectord 2005-11-13 14:39:51.000000000 +0100 | ||
| + | @@ -236,7 +236,7 @@ checking will take place and no real or | ||
| + | On means no checking will take place and real servers will always be | ||
| + | activated. Default is I<negotiate>. | ||
| + | |||
| + | -B<service = ftp>|B<smtp>|B<http>|B<pop>|B<nntp>|B<imap>|B<ldap>|B<https>|B<dns>|B<mysql>|B<pgsql>|B<sip>|B<none> | ||
| + | +B<service = ftp>|B<smtp>|B<http>|B<proxy_http>|B<pop>|B<nntp>|B<imap>|B<ldap>|B<https>|B<dns>|B<mysql>|B<pgsql>|B<sip>|B<none> | ||
| + | |||
| + | The type of service to monitor when using checktype=negotiate. None denotes | ||
| + | a service that will not be monitored. If the port specfied for the virtual | ||
| + | @@ -866,8 +866,8 @@ sub read_config | ||
| + | } | ||
| + | } elsif ($rcmd =~ /^service\s*=\s*(.*)/) { | ||
| + | lc($1); | ||
| + | - $1 =~ /(\w+)/ && ($1 eq "http" || $1 eq "https" || $1 eq "ldap" || $1 eq "ftp" || $1 eq "none" || $1 eq "smtp" || $1 eq "pop" || $1 eq "imap" || $1 eq "nntp" || $1 eq "dns" || $1 eq "mysql" || $1 eq "pgsql" || $1 eq "sip") | ||
| + | - or &config_error($line, "service must be http, https, ftp, smtp, pop, imap, ldap, nntp, dns, mysql, pgsql, sip, or none"); | ||
| + | + $1 =~ /(\w+)/ && ($1 eq "http" || $1 eq "proxy_http" || $1 eq "https" || $1 eq "ldap" || $1 eq "ftp" || $1 eq "none" || $1 eq "smtp" || $1 eq "pop" || $1 eq "imap" || $1 eq "nntp" || $1 eq "dns" || $1 eq "mysql" || $1 eq "pgsql" || $1 eq "sip") | ||
| + | + or &config_error($line, "service must be http, https, proxy_http, ftp, smtp, pop, imap, ldap, nntp, dns, mysql, pgsql, sip, or none"); | ||
| + | $vsrv{service} = $1; | ||
| + | if($vsrv{service} eq "ftp" and | ||
| + | $vsrv{login} eq "") { | ||
| + | @@ -1635,6 +1635,8 @@ sub ld_main | ||
| + | $$r{num_connects} = 0 if (check_http($v, $r)); | ||
| + | # my $req = new HTTP::Request(GET=>"$$r{url}"); | ||
| + | # $ua->register($req, \&http_received); | ||
| + | + } elsif ($$v{service} eq "proxy_http") { | ||
| + | + $$r{num_connects} = 0 if (check_proxy_http($v, $r)); | ||
| + | } elsif ($$v{service} eq "pop") { | ||
| + | $$r{num_connects} = 0 if (check_pop($v, $r)); | ||
| + | } elsif ($$v{service} eq "imap") { | ||
| + | @@ -1693,6 +1695,51 @@ sub ld_main | ||
| + | } | ||
| + | } | ||
| + | |||
| + | +sub check_proxy_http | ||
| + | +{ | ||
| + | + use LWP::UserAgent; | ||
| + | + use LWP::Debug; | ||
| + | + | ||
| + | + my ($v, $r) = @_; | ||
| + | + | ||
| + | + my $proxy_request = substr($$r{request},1); | ||
| + | + | ||
| + | + my $ua = new LWP::UserAgent(); | ||
| + | + | ||
| + | + $ua->timeout($$v{negotiatetimeout}); | ||
| + | + $ua->proxy(['http', 'ftp'], "http://$$r{server}:$$r{port}/"); | ||
| + | + | ||
| + | + my $h = new HTTP::Headers("Host" => $$v{virtualhost}); | ||
| + | + my $req = new HTTP::Request("$$v{httpmethod}", "$proxy_request", $h); | ||
| + | + | ||
| + | + my $res; | ||
| + | + { | ||
| + | + # LWP makes ungaurded calls to eval | ||
| + | + # which throw a fatal exception if they fail | ||
| + | + # Needless to say, this is completely stupid. | ||
| + | + local $SIG{'__DIE__'} = "DEFAULT"; | ||
| + | + $res = $ua->request($req); | ||
| + | + } | ||
| + | + | ||
| + | + my $recstr = $$v{receive}; | ||
| + | + | ||
| + | + if ($res->is_success && (!($recstr =~ /.+/) || $res->content =~ /$recstr/)) { | ||
| + | + service_set($v, $r, "up"); | ||
| + | + &ld_debug(2, "check_proxy_http: $proxy_request is up\n"); | ||
| + | + return 1; | ||
| + | + } | ||
| + | + | ||
| + | + service_set($v, $r, "down"); | ||
| + | + &ld_debug(3, "Headers " . $res->headers->as_string); | ||
| + | + &ld_debug(2, "check_proxy_http: $proxy_request is down\n"); | ||
| + | + return 0; | ||
| + | + | ||
| + | +} | ||
| + | |||
| + | More infos? mailto:christian.avramakis@siemens.com | ||
[[Category:LVS Examples|Cache]] | [[Category:LVS Examples|Cache]] | ||
Latest revision as of 04:40, 24 October 2010
Introduction
Sometimes, you just need to build a proxy farm with the following needs:
- you need high-availability, because of your SLA.
- you need load balancing, because you're serving numerous Kusers.
- you need caching with Squid.
- you need extra-services over Squid (URL filtering with DansGuardian/SquidGuard, Antivirus softwares, ...)
- you need scalability because you want to be able to grow.
In these cas, you've got plenty of answers to proceed. LVS is one of them (probably the best suited anyway).
But as you're doing extra-services over Squid, the base LVS ldirectord tests won't be sufficient.
This page explains how to make some more advanced tests to be sure some extra-services will be checked too, and the corresponding realserver service will be dropped down if one of the extra service (or Squid itself) is not available.
It present a proposed patch to ldirectord to do so.
Architecture
The system architecture of web cache cluster using LVS is illustrated in the following figure.
- it uses LVS with Ultramonkey 3 settings for a Streamline High Availability and Load Balancing setting.
- it's based on the usage for Squid + DansGuardian (but could feet for any Squid + *).
Configuration Example
To set up this configuration with LVS, just proceed to installation and configuration as indicated on the Ultramonkey documentations.
Then :
- set up your Squid server, listening on port 3128 (example) and using DansGuardian on port 8000
- set up your DansGuardian server, listening to port 8000
- set up DansGuardian to reject (filter) the "http://proxy.testing.net/" URL (adding this domain/url to the blacklists. Be sure the "URL Filtering" will be returned by the proxy in case of filtering this URL.
- patch /usr/sbin/ldirectord with the FIXME patch
- configure your /etc/ha.d/ldirectord.cf with the following tests (example IP addresses):
# /etc/ha.d/ldirectord.cf
# Virtual Server for Proxy Service and Filtering Service
virtual=192.168.0.100:3128
real=192.168.0.10:3128 gate
real=192.168.0.11:3128 gate
real=192.168.0.12:3128 gate
real=192.168.0.12:3128 gate
service=proxy_http
request="http://proxy.testing.net"
receive="URL Filtering"
protocol=tcp
scheduler=wlc
protocol=tcp
checktype=negotiate
Conclusion
If you were using http test, you could only be able to check Squid is running. In case of DansGuardian issue, it could be a bad thing. This way, if the filtering engine is down/not responding, the whole Squid realserver will be dropped down for LVS.
Of course, you must still monitor all process to be warned anything is going wrong. But you will have time to resolve as your users won't notice anything :) Thank's a lot to LVS again!
The ldirectord patch
This patch is to be applied to release +ldirectord,v 1.77.2.32+, but is basic enought for you to report it to any newer release of ldirectord.
--- ldirectord_sav 2005-10-03 08:18:19.000000000 +0200
+++ ldirectord 2005-11-13 14:39:51.000000000 +0100
@@ -236,7 +236,7 @@ checking will take place and no real or
On means no checking will take place and real servers will always be
activated. Default is I<negotiate>.
-B<service = ftp>|B<smtp>|B<http>|B<pop>|B<nntp>|B<imap>|B<ldap>|B<https>|B<dns>|B<mysql>|B<pgsql>|B<sip>|B<none>
+B<service = ftp>|B<smtp>|B<http>|B<proxy_http>|B<pop>|B<nntp>|B<imap>|B<ldap>|B<https>|B<dns>|B<mysql>|B<pgsql>|B<sip>|B<none>
The type of service to monitor when using checktype=negotiate. None denotes
a service that will not be monitored. If the port specfied for the virtual
@@ -866,8 +866,8 @@ sub read_config
}
} elsif ($rcmd =~ /^service\s*=\s*(.*)/) {
lc($1);
- $1 =~ /(\w+)/ && ($1 eq "http" || $1 eq "https" || $1 eq "ldap" || $1 eq "ftp" || $1 eq "none" || $1 eq "smtp" || $1 eq "pop" || $1 eq "imap" || $1 eq "nntp" || $1 eq "dns" || $1 eq "mysql" || $1 eq "pgsql" || $1 eq "sip")
- or &config_error($line, "service must be http, https, ftp, smtp, pop, imap, ldap, nntp, dns, mysql, pgsql, sip, or none");
+ $1 =~ /(\w+)/ && ($1 eq "http" || $1 eq "proxy_http" || $1 eq "https" || $1 eq "ldap" || $1 eq "ftp" || $1 eq "none" || $1 eq "smtp" || $1 eq "pop" || $1 eq "imap" || $1 eq "nntp" || $1 eq "dns" || $1 eq "mysql" || $1 eq "pgsql" || $1 eq "sip")
+ or &config_error($line, "service must be http, https, proxy_http, ftp, smtp, pop, imap, ldap, nntp, dns, mysql, pgsql, sip, or none");
$vsrv{service} = $1;
if($vsrv{service} eq "ftp" and
$vsrv{login} eq "") {
@@ -1635,6 +1635,8 @@ sub ld_main
$$r{num_connects} = 0 if (check_http($v, $r));
# my $req = new HTTP::Request(GET=>"$$r{url}");
# $ua->register($req, \&http_received);
+ } elsif ($$v{service} eq "proxy_http") {
+ $$r{num_connects} = 0 if (check_proxy_http($v, $r));
} elsif ($$v{service} eq "pop") {
$$r{num_connects} = 0 if (check_pop($v, $r));
} elsif ($$v{service} eq "imap") {
@@ -1693,6 +1695,51 @@ sub ld_main
}
}
+sub check_proxy_http
+{
+ use LWP::UserAgent;
+ use LWP::Debug;
+
+ my ($v, $r) = @_;
+
+ my $proxy_request = substr($$r{request},1);
+
+ my $ua = new LWP::UserAgent();
+
+ $ua->timeout($$v{negotiatetimeout});
+ $ua->proxy(['http', 'ftp'], "http://$$r{server}:$$r{port}/");
+
+ my $h = new HTTP::Headers("Host" => $$v{virtualhost});
+ my $req = new HTTP::Request("$$v{httpmethod}", "$proxy_request", $h);
+
+ my $res;
+ {
+ # LWP makes ungaurded calls to eval
+ # which throw a fatal exception if they fail
+ # Needless to say, this is completely stupid.
+ local $SIG{'__DIE__'} = "DEFAULT";
+ $res = $ua->request($req);
+ }
+
+ my $recstr = $$v{receive};
+
+ if ($res->is_success && (!($recstr =~ /.+/) || $res->content =~ /$recstr/)) {
+ service_set($v, $r, "up");
+ &ld_debug(2, "check_proxy_http: $proxy_request is up\n");
+ return 1;
+ }
+
+ service_set($v, $r, "down");
+ &ld_debug(3, "Headers " . $res->headers->as_string);
+ &ld_debug(2, "check_proxy_http: $proxy_request is down\n");
+ return 0;
+
+}
More infos? mailto:christian.avramakis@siemens.com
