With iptables you can use various modules to mangle and forward a connection, thus preserving the source IP address (which is one of the problems with balance - your apache nodes see the IP address of your balance machine, and not the client).

Now I’m no iptables expert, but I’ve been able to hold my own over the years. If you happen to know more then me and spot a goof, let me know.

In this example, the iptables the 192.168.1.10 server gets these rules:

/sbin/iptables -t mangle -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m statistic –mode nth –every 5 –packet 1 -m state –state new -j CONNMARK –set-mark 1
/sbin/iptables -t mangle -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m statistic –mode nth –every 5 –packet 2 -m state –state new -j CONNMARK –set-mark 2
/sbin/iptables -t mangle -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m statistic –mode nth –every 5 –packet 3 -m state –state new -j CONNMARK –set-mark 3
/sbin/iptables -t mangle -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m statistic –mode nth –every 5 –packet 4 -m state –state new -j CONNMARK –set-mark 4
/sbin/iptables -t mangle -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m statistic –mode nth –every 5 –packet 0 -m state –state new -j CONNMARK –set-mark 5
/sbin/iptables -t nat -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m connmark –mark 1 -j DNAT –to-destination 192.168.1.50:80
/sbin/iptables -t nat -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m connmark –mark 2 -j DNAT –to-destination 192.168.1.51:80
/sbin/iptables -t nat -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m connmark –mark 3 -j DNAT –to-destination 192.168.1.52:80
/sbin/iptables -t nat -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m connmark –mark 4 -j DNAT –to-destination 192.168.1.53:80
/sbin/iptables -t nat -A PREROUTING -d 192.168.1.10 -p tcp –dport 80 -m connmark –mark 5 -j DNAT –to-destination 192.168.1.54:80

If this is all greek to you - here is the explanation:

* Filter NEW connections that are destined to 192.168.1.10 port 80
* Mark each filtered packet with a number
* Each is marked in sequence, 1 through 5
* Each marked packet is sent to port 80 on a given IP address, depending on the marked number
* There are 5 apache nodes, 192.168.1.50 through .54

This was done using iptables 1.4.0 built on a CentOS 5 machine, on a local network using virtual machines running apache version 2.2. Above rules may vary depending on network and firewall setup. Turning on forwarding may also be necessary ( echo 1 > /proc/sys/net/ipv4/ip_forward )

Since I apparently missed the memo in my email… I didn’t find out it went through until tonight. So me pushing out the next release will have to wait till tomorrow. I have about 4 patches I posted on the site, I’ll consolidate them and put out version 1.5.3 of rootsh.

And since it’s now a project I maintain, I’ll be posting the source RPMs for it in my packages directory.

http://www.inlab.de/balance.html

Initial tests show it to work as advertised, and very well at that. I haven’t by any means been using it very long (I only found it this evening) so I can’t vouch for it’s long-term stability or talk about any quarks, I’ll be sure to post another update about it in the future.

Anyway, if all works out with this; looks like I’m going to have a definitive path for ravencore’s multi-server cluster plans. Over the past two years I have been debating how to set it up, and I’ve finally decided how it’s going to be.

Instead of setting up domain A on server X and domain B on server Y, all domains will be setup on all nodes. You’ll define the “storage” server (which mounts the vhost root nfs and all the slave nodes mount the vhost root to point to that share), You’ll define the “LB” server (which runs either the above mentioned software, or something else if it doesn’t work out), and on each non-storage, non-LB server you define what service(s) that node will use (http, smtp, imap, dns, etc).

This, combined with xen controls, should supercharge anyone using ravencore to increase their ability to handle larger and larger network loads by just adding more and more servers to the cluster, everything will “just work” after it’s plugged in, increasing capacity instantly.

http://ravencore.svn.sourceforge.net/viewvc/ravencore/branches/sqlub/

Oh gosh sourceforge is sooooo slow sometimes. I can’t wait to have sqlub setup on http://sharesource.org/ so my commits don’t take a whole damn hour to upload 276k worth of code.

This is the first time I built to be installed like a CPAN perl module, so I may have missed some things, but the general make and make install put the code and man pages in the right places. Soon I’ll get a .spec file built and post a source RPM as well, have the sqlub.pl be installed by the makefile as well, and have the sqlub.sql actually create the database upon install (you have to do those manually at this point).

More updates to come later.

/* TODO: Fix me, this function is beginning to resemble pasta */

Finding it rather humorous, I brought it up to him. This was his reply:

Well, there are varying degrees of pasta:
* you have linguini, easy to sort
* feutticini , a little harder
* spaghetti, annoying
* then angel hair pasta, impossible
right now, its at linguini

I wonder, though, because Tim made another comment: “but would a well written program be lasagna?”

We got to talking some more about this as it had the potential to get really silly…

Would an over abundance of perl modules mean risotto?

Are C libraries a bunch of tortellini? - Why yes, but you have cheese, beef … and vegan

Then you have ravioli , those are crypt libraries because you can never be too sure whats in them…

Of course there’s that whole diet libc thing … would that be ‘Adkins’ pasta?

Now, a rare pasta.. for c++ libs? - Tim couldn’t think of one, he doesn’t eat processed foods.

As for assembly… that’s the dough that eventually becomes angel hair pasta.

Since ‘flex’ is usually used to generate lexical parsers, that would be the rolling pin.

Pasta Salad: This is what happens when the angry fruit salad of an interface is more complicated than the pasta found in the code. We call this “Mayo”

Pasta with meatballs maybe is tainted kernel code? No, those are ‘blobs’, Linus put them there.

And if you ask me, parsing strings in C must be the pasta that was left on the counter overnight; it’s real hard and sticks badly. Tim disagrees, though couldn’t come up with an alternative.

Xen is the spaghetti resulting from when your Aunt Margret stole the recipie from Grandpa, a retired Colonel.

And there is no pasta crazy enough for m4; so we’ll just call it fucking psychotic:

AC_ARG_ENABLE(unsafe, [ --enable-unsafe          enable unsafe operation [[default=no]]],[
  if test "$enableval" = no; then
    ac_enable_unsafe=no
  else
    ac_enable_unsafe=yes
  fi
  ],[ ac_enable_unsafe=no])

All that for one goddamn switch!!!

Quite the revelation. These remarks remind me of this one I read a little while ago:

If a programming language was a boat

I agree with most of them… PHP itself isn’t a bamboo raft, just a lot of people’s PHP code ends up looking like one.

On the SRCE note, I started using it about a week ago. I really love it, and am very confident that I’ll be able to include it as an add-on to RavenCore for multi-server support not long down the road.

Anyway, in a battery of tests of SRCE I noticed something fishy and brought it to the attention to the developer, providing him with a patch for the issue. Less then 12 hours later he posted it as a security patch on the mailinglist which contains the patch I sent him, for srce-exec.c, along with a configuration change. Yay me.

This project will have three stages:

1) Build userspace tools to edit the sqlub database both manually and with a program API

2) Write a patch to the grub source to read the sqlub database instead of the grub.conf file, if it exists

3) Attempt to get this patch put into the mainstream grub program

Currently I am in phase 1, and at least have something that works (for me). For now, the code is posted in the ravencore subversion repo, here:

http://ravencore.svn.sourceforge.net/viewvc/ravencore/branches/sqlub/

I’ll be putting up a separate website for this project in the near future, as the code matures. The utility is written in perl for the moment, it’s likely to be ported to C (but also maintained in perl).

http://www.ravencore.com/packages/

The latest of them include SRCE, amavisd-new, dovecot, and sqlite. For now only the source RPMs are being posted because I don’t have the space or bandwidth on this tiny VM to do much else. Once I get a full machine colocated somewhere, I’ll be throwing them all into a YUM repository.

My goal is to provide stable versions of the latest software which matter in the realm of RavenCore. You can stick with an enterprise distro and still have all the nifty features of some select packages.

My latest accomplishment with the new mail configuration is getting postfix to work with amavisd-new-milter, so your mail detected as SPAM actually gets rejects at connect time. Honest mailers will get the rejection right away and know that their mail is being marked as spam, some spammers will not try your email address again (due to logging the 550 rejection) and other spammers will just continue to knock and never get through.

Short story even shorter, here is a list of RPMs I had to build from source to get all the new dependancies:

glib2-2.16.3-5.fc9.src.rpm
pygpgme-0.1-8.fc9.src.rpm
python-iniparse-0.2.3-3.fc9.src.rpm
python-setuptools-0.6c7-2.fc8.src.rpm
yum-3.2.14-10.fc9.src.rpm
yum-metadata-parser-1.1.2-8.fc9.src.rpm
yum-utils-1.1.13-2.fc9.src.rpm

As you might have noticed, they’re from fedora core 9. It hasn’t been released yet, but it’s so close to the release date that I’m not worried about them being that unstable.

I rebuilt these, installed their dependencies via yum to build / install them, and the upgrade to yum went perfect; I’m finding no conflicts so far. The only issue I had with rebuilding the source RPMs was the yum-metadata-parser package was trying to glob a non-existent file pattern so the build failed; a quick commenting out of this line in the %files section fixed the issue.

If I don’t encounter any problems with this upgrade between now and when I post all my RPMs from my packages directory, I’ll include pre-built RPMs for this yum upgrade on CentOS5 as well (and maybe even CentOS4).

postfix-2.5 built with the sqlite patch and dovecot sasl support
dovecot-1.0 with the sqlite extension
sqlite 3.5.x libraries

All of the above will be published in a yum repo, and will be marked as dependencies for the ravencore package. Just install this .repo file to your /etc/yum.repos.d/ directory and “yum install ravencore”, it’ll grab all of these for you and update them on the system. This will also make what distributions ravencore supports very clear; it will always be the 2 latest CentOS distributions, and the 2 latest Fedora distributions (CentOS 4 & 5, and Fedora 8 & 9)

Depending on how much help I get from other developers, I may or may not publish apt repos for debian packages. That is up in the air for the moment.

RavenCore will no longer use the one-file-per-line format for postfix and dovecot configuration files; it’ll instead use templates: main.cf.in, dovecot.conf.in, etc. You’ll be able to edit those to your liking, just make sure that when you upgrade you don’t miss any important changes in those files that may take place.

Dovecot will reference dovecot-sqlite.conf for user, password, and setup information. That conf file will reference a database in /etc/mail/vmail.sqlite. It’ll also have the dovecot-auth daemon listen for sasl auth inside a private postfix directory.

Postfix will contain a master.cf file that is full of references to other files such as: vmailbox-sqlite.cf, vmaildomain-sqlite.cf, etc. All of those configuration files will also reference the /etc/mail/vmail.sqlite file, each containing a different static query to gather the information needed for that option. SASL auth will be configured to point at the dovecot sasl socket inside of the postfix private directory.

In essence, all mail is controlled by the vmail.sqlite database, everything else is a static flatfile configuration. Since changes to the sqlite database don’t need to be postmapped, postfix won’t ever need to reload automatically. Same goes for dovecot.

To start with, the code will get a transaction lock on the vmail.sqlite database, drop everything, and recreate everything, and commit it all. Further optimizations will be made down the road to better support installations with thousands of email users.

The vmail.sqlite database will look much like what the ravencore database looks like now, as far as mail goes… a domain table and a user table. There may be more tables depending on how far I want to take the features at this point.