How to get the real accessing IPs behind a load-balancing proxy
If you are using a load-balancing proxy such as squid, apache itself or pound, you will only find the proxy’s IP in your backend apache’s access/error logs. I tested two apache modules which replace the proxy’s IP by the X-Forwarded-For-IP provided by your load-balancing proxy: mod_rpaf and mod_extract_forwarded.
If you are running Debian, the easier installation will be mod_rpaf since it is provided in the (stable) sarge tree as well as in the (unstable) sid tree. The sid version will run smoothly under (testing) etch. As long as you are loggin IP addresses, the mod_rpaf is doing a great job. As soon as you are setting HostnameLookups On though, you will find your proxy’s hostname in the logs again – which is not what I wanted.
On OpenInfo you can download (local copy) the other solution I found, mod_extract_forwarded. Just download it to your apache server and extract the tar-archive. As found in the module’s documentation you should comment out the line
#define USING_proxy_http_module 1
by preceeding it with two slashes (//). This will be necessary if your apache server doesn’t load mod_proxy at this time (which backend servers usually don’t). Just don’t forget to remove the slashes again should you need and load mod_proxy some time later.
Compile the module by issuing
apxs2 -c -i -a mod_extract_forwarded.c
and you will find your module in your apache’s modules directory. The -a option will add a line to your apache configuration files which will then load the module automatically on every server (re-)start. The line will look like this – in case you have to remove it again:
LoadModule extract_forwarded_module /usr/lib/apache2/modules/mod_extract_forwarded.so
Within your site configuration (somewhere above the DocumentRoot line) you have to enable the module. Be sure to enter only the IP address of YOUR load-balancing proxy, no other proxies or even all proxies, otherwise you could find many fake IP addresses in your logs. Here are the configuration lines, be sure to change 192.168.1.123 to YOUR load-balancing proxy’s IP address as it is seen by your apache-server (usually the IP you find for every access in your logs so far):
MEForder refuse,acceptMEFrefuse allMEFaccept 192.168.1.123
Restart your server and you should find the real IP addresses of your visitors in your logs.
Prevent apache from logging particular requests
If you don’t want apache to log every request in your access.log, maybe because you have thousands of requests from your monitoring system, you can prevent apache from logging these requests by two simple statements:
First you set an environment variable, second you don’t log when this variable is set:
SetEnvIf Remote_Host 192.168.1.123 no_logCustomLog /var/log/apache2/access.log combined env=!no_log
There are more than just IP address matches, look into the apache documentation!
How to enter ^M in vi
Maybe you sometimes wondered how to enter ^M (Control-M, STRG-M in German) in searches or substitutions in vi. Once you know, it is easy: enter ^V^M and you won’t end up as if you entered RETURN. So to remove all ^Ms in in a document which was originally edited on a DOS platform, you can use this command:
:%s/^V^M//g
The ^V will disappear immediately when you type the ^M – but the ^M will stay there 🙂
How to build your own Debian Etch Boot CD with recent 3ware-9xxx drivers
I wanted to install my latest server (Dual-Quad-Xeons and a 5.7TB 3ware 9650SE controller) with Debian Etch. Unfortunately at this moment there was no Debian installation CD with a recent 3w-9xxx driver – so my great RAID controller was not supported. Of course all Debian CDs didn’t find any harddisk 🙁
Now there were two possibilities – a network boot or a customized Debian installation CD. Here I will describe the second option, building your own installation CD.
The latest official Debian kernel at this moment was 2.6.18-3, so I downloaded exactly the following packages:
- linux-image-2.6.18-3-amd64
- linux-headers-2.6.18-3-amd64
This way I needed to simply exchange the included kernel module with my new version. To exchange the 3w-9xxx module, I downloaded the sources from 3ware and built the module. The resulting 3w-9xxx.ko has to be copied over the “old” file which does not know about the 9650SE controller yet. A simple
cp 3w-9xxx.ko /lib/modules/2.6.18-3-amd64/kernel/drivers/scsi/
did it in my case!
A Debian installer CD needs special kernel packages, so called “udebs”. You can build these udebs from an installed kernel, if you download the source of linux-kernel-di-amd64-2.6 with the command
apt-get source linux-kernel-di-amd64-2.6
then you enter the unpacked directory, in my case this was linux-kernel-di-amd64-2.6-1.14 . As I said, I wanted to build/update kernel 2.6.18-3, but the downloaded sources are probably for another/older version. In my case this was 2.6.17-2. To make everything work, you have to change every occurrence of 2.6.17-2 to 2.6.18-3 in the files debian/control and kernel-versions by opening the files with any editor you like. I was using vi and issued the command
:%s/2.6.17-2/2.6.18-3/g
Simply enter dpkg-buildpackage after this and you will find many .udeb files in the parent directory. If dpkg-buildpackage is not available, install the package dpkg-dev . If it breaks after it started, it could need the package kernel-wedge.
Keep the .udeb files, you will need them later!
Now you have to build a recent version of debian-installer. I used the developer version from the subversion repository. To get the sources, give the folllowing command and enter the installer directory afterwards:
svn co svn://svn.debian.org/svn/d-i/trunk/installer installer
Edit the file build/config/amd64.cfg and make sure the KERNELVERSION line shows YOUR kernel version. Also uncomment the word #monolithic in the first line by removing the #. Edit build/config/common and make sure SUITE is set to “testing” or whatever you wish to install later. Copy the .udeb files you created before into the build/localudebs directory,
You should probably copy the original kernel .deb from
/var/cache/apt/archives to /usr/src and also the newly built driver
module to /usr/src, so you can edit build/config/common and make sure
the following line exists:
EXTRAFILES = /usr/src/linux-image-2.6.18-3-amd64_2.6.18-7_amd64.deb /usr/src/3w-9xxx.ko
This way these files will be at the same place in installation environment –
and you should install /copy them before finishing the installation, otherwise the installer has found your
RAID disk, but after you boot into your new system the old kernel would
be active and not recognizing the controller. (To install you can open a shell by selecting one of the last installer menu entries. Your future system can be found in the /target directory. Don’t forget to add your necessary kernel module to be included in your initrd, otherwise the kernel will boot, but then wait endlessly for its root file system. I did that by adding 3w-9xxx to /etc/initramfs-tools/modules and then doing a “mkinitramfs -o initrd.img-2.6.18-3-amd64 2.6.18-3-amd64.)
Now enter the build directory and type
make build_monolithic
Should it break because of unmet dependencies, install them. In my case I had to enter this:
apt-get install grep-dctrl bc debiandoc-sgml libbogl-dev glibc-pic libslang2-pic libnewt-pic genext2fs mklibs mkisofs dosfstools syslinux tofrodos mtools bf-utf-source
Should you get an error saying something about weak symbols, try a “make reallyclean” and then a “make build_monolithic” again – it worked for me and I had an ISO image in dest/monolithic .
First post from iPhone
This is only a stupid test, if it is working!
