A few weeks ago, i attended a rather interesting Apache Security course in Birmingham taught by Gary Vaughan and some guy called Frank (whose last name escapes me). One of the key things i learned was that it was a good idea to compile packages straight from the source-code - that way it would be pretty easy to keep up to date with the latest security updates. It’d also be easy to cut out useless functionality (otherwise known as bloat).

And thus, i decided to think of a good way of automating the build + deploy process for a typical LAMP (Linux, Apache, MySQL, PHP) stack, so i’d essentially be able to quickly upgrade and tweak anything installed on my server, rather than typing everything in again and again.

After a short while of contemplating what to build the solution off, i decided to go with a combination of makefile’s and shell scripts. I also figured it would be a good idea to put the system on a virtual machine, so i could best reflect the environment of my server.

The Makefile

So to start off with i had a Makefile, which was essentially a hub to all the other build scripts and makefile’s, and also handled deployment. It looked something like this:

# Makefile of doom

# Login for the target server
SERVER=target-server.com
SERVER_USER=username
SSH_LOGIN=$(SERVER_USER)@$(SERVER)

# Prefixes
ROOT_PREFIX=/usr/local
APACHE_LOCATION=$(ROOT_PREFIX)/apache2
PHP_LOCATION=$(ROOT_PREFIX)/php

# Versions
APACHE_VERSION=2.2.4
PHP_VERSION=5.2.2

# Download url's
APACHE_URL=http://apache.rmplc.co.uk/httpd
PHP_URL=http://uk2.php.net/distributions

# List of configuration files which can be deployed
CONFIGURATION_FILES= config/httpd.conf config/php.ini

# ./configure options
include config/apache_build.cfg
include config/php_build.cfg

# Build scripts
include mk/apache.mk
include mk/php.mk

# Rules for required programs

deploy-config: $(CONFIGURATION_FILES)
scp -r $(CONFIGURATION_FILES) $(SSH_LOGIN):/home/$(SERVER_USER)/server/config/
ssh $(SSH_LOGIN) "ROOT_PREFIX=$(ROOT_PREFIX) sudo /home/$(SERVER_USER)/server/scripts/reload-config"

deploy-scripts:
scp -r scripts keys config $(SSH_LOGIN):/home/$(SERVER_USER)/server/
ssh $(SSH_LOGIN) "rm /home/$(SERVER_USER)/server/config/*.cfg"

deploy: php_deploy apache_deploy

# Typical generic targets
default: deploy-config

all: apache php deploy

clean-build:
-rm -rf build/*

clean-external:
rm external/*

clean-location:
-rm -rf "$(APACHE_LOCATION)" "$(PHP_LOCATION)"

clean: apache_buildclean php_buildclean

As you can probably see, relatively simple. I decided to move off the configuration options to separate files, so that they could easily be incorporated into targets (i.e. if you change them, everything related gets built again). I also split off the targets for each piece of software into separate files, to make things a bit more manageable.

And if your wondering why there is no mysql in there, i decided to just use the one that came with the distro - though of course if i was feeling very paranoid, i’d add in some mysql targets too.

The configuration files i used are as follows:

config/apache_build.cfg

APACHE_CONF = --enable-rewrite --enable-so --disable-userdir --enable-ssl --enable-auth-digest --with-mpm=worker

config/php_build.cfg

PHP_CONF= --with-mysql --with-zlib
# (FASTCGI is handy if you are thinking of using something other than apache)
BUILD_PHP_MODULE=true
BUILD_PHP_FASTCGI=false

Note that i could have also put all the *_VERSION defines in the configuration files so that if somebody was playing musical chairs with the version numbers it wouldn’t throw up, but i didn’t really think of that at the time.

The mk files

For each software package, i had a .mk file, which basically contains more Makefile targets. Its job is to download the tarball of the particular software package, extract it, configure it, build it, and then install it somewhere in $ROOT_PREFIX (otherwise known as “/usr/local”). In addition, it also had to provide a target to copy the built code to a temporary folder on the target server.

In this case, i had 2 things to build - apache and php:

mk/apache.mk

# Apache builder

APACHE_BUILD_ROOT=build/httpd-$(APACHE_VERSION)

# Sources
external/httpd-$(APACHE_VERSION).tar.gz:
curl -o $@ $(APACHE_URL)/httpd-$(APACHE_VERSION).tar.gz

# Targets
$(APACHE_BUILD_ROOT)/EXTRACTED: external/httpd-$(APACHE_VERSION).tar.gz
cd build && tar -xzf ../external/httpd-$(APACHE_VERSION).tar.gz && touch httpd-$(APACHE_VERSION)/EXTRACTED

$(APACHE_BUILD_ROOT)/Makefile: $(APACHE_BUILD_ROOT)/EXTRACTED config/apache_build.cfg
rm -rf $(APACHE_LOCATION)
-cd $(APACHE_BUILD_ROOT) && make clean
cd $(APACHE_BUILD_ROOT) && ./configure --prefix="$(APACHE_LOCATION)" $(APACHE_CONF)

$(APACHE_BUILD_ROOT)/httpd: $(APACHE_BUILD_ROOT)/Makefile
cd $(APACHE_BUILD_ROOT) && make

$(APACHE_LOCATION)/bin/httpd: $(APACHE_BUILD_ROOT)/httpd
cd $(APACHE_BUILD_ROOT) && sudo make install
mkdir $(APACHE_LOCATION)/conf/keys

# Deployment
apache_deploy: $(APACHE_LOCATION)
sudo rsync -avz -e ssh $(APACHE_LOCATION)/ $(SSH_LOGIN):/tmp/deploy-apache/
ssh $(SSH_LOGIN) "ROOT_PREFIX=$(APACHE_LOCATION) sudo /home/$(SERVER_USER)/server/scripts/do-deploy-apache"

# Cleanup
apache_confclean:
-cd $(APACHE_BUILD_ROOT) ; make clean; rm Makefile

apache_updateconf: apache_confclean $(APACHE_BUILD_ROOT)/Makefile

apache_buildclean:
rm -rf $(APACHE_BUILD_ROOT)

# Shorthand
apache: $(APACHE_LOCATION)/bin/httpd

mk/php.mk

# Php builder

PHP_BUILD_ROOT=build/php-$(PHP_VERSION)

ifeq ($(BUILD_PHP_MODULE), true)
# mod_php (also requires built apache)
PHP_EXTRA_REQS=$(APACHE_LOCATION)/bin/httpd
PHP_CONF+= --with-apxs2="$(APACHE_LOCATION)/bin/apxs"
else
# no extra requirements, great!
PHP_EXTRA_REQS=
endif

ifeq ($(BUILD_PHP_FASTCGI), true)
PHP_CONF+= --enable-force-cgi-redirect --enable-fastcgi
endif

# Sources
external/php-$(PHP_VERSION).tar.gz:
curl -o $@ $(PHP_URL)/php-$(PHP_VERSION).tar.gz

# Targets
$(PHP_BUILD_ROOT)/EXTRACTED: external/php-$(PHP_VERSION).tar.gz
cd build && tar -xzf ../external/php-$(PHP_VERSION).tar.gz && touch php-$(PHP_VERSION)/EXTRACTED

$(PHP_BUILD_ROOT)/Makefile: $(PHP_EXTRA_REQS) $(PHP_BUILD_ROOT)/EXTRACTED config/php_build.cfg
-cd $(PHP_BUILD_ROOT) && make clean
cd $(PHP_BUILD_ROOT) && ./configure --prefix="$(PHP_LOCATION)" $(PHP_CONF)

$(PHP_BUILD_ROOT)/sapi/cli/php: $(PHP_BUILD_ROOT)/Makefile
cd $(PHP_BUILD_ROOT) && make

$(PHP_LOCATION)/bin/php: $(PHP_BUILD_ROOT)/sapi/cli/php
cd $(PHP_BUILD_ROOT) && sudo make install

# Deployment
php_deploy: $(PHP_LOCATION)
sudo rsync -avz -e ssh $(PHP_LOCATION)/ $(SSH_LOGIN):/tmp/deploy-php/
ssh $(SSH_LOGIN) "ROOT_PREFIX=$(PHP_LOCATION) sudo /home/$(SERVER_USER)/server/scripts/do-deploy-php"

# Cleanup
php_confclean:
-cd $(PHP_BUILD_ROOT) ; make clean; rm Makefile

php_updateconf: php_confclean $(PHP_BUILD_ROOT)/Makefile

php_buildclean:
rm -rf $(PHP_BUILD_ROOT)

# Shorthand
php: $(PHP_LOCATION)/bin/php

You might have noticed that i touch an “EXTRACTED” file in the build folder after extracting the tarball. This is just to keep track of the time the tarball was extracted, and also provides proof it was successfully extracted.

It it important to note that if you are feeling very paranoid, it would be a good idea to add in a target that precedes the tarball extraction to verify it is genuine, using either an associated md5sum or a PGP signature. This should ensure that the source-code you have downloaded is a genuine release.

The scripts

In order to successfully deploy everything, i decided to make some shell scripts. Their job was basically to copy across any built copy of php or apache that had been uploaded to “/tmp” on the target server. They also had to copy across configuration files that had been uploaded to “~/server/config” to their corresponding locations on the target server.

Safe to say, there was quite a few of them. They are as follows:

scripts/reload-config

#!/bin/sh
# Copies configuration files from ~/server/config and reloads apache
CONFDIR=~/server/config

# Copy across configuration files
if [ -e $ROOT_PREFIX/apache2/conf/ ]
then
cp ${CONFDIR}/httpd.conf $ROOT_PREFIX/apache2/conf/
fi

if [ -e  $ROOT_PREFIX/php/lib/ ]
then
cp ${CONFDIR}/php.ini  $ROOT_PREFIX/php/lib/
fi

# Instruct server to reload config
if [ -e  $ROOT_PREFIX/apache2/ ]
then
/etc/init.d/httpd reload
fi

scripts/do-deploy-apache

#!/bin/sh
# Copies over apache to $ROOT_PREFIX from /tmp/deploy-php
SCRIPTDIR=~/server

# Check log directories
if [ ! -e /var/log/apache2 ]
then
mkdir /var/log/apache2
chown -R root:root /var/log/apache2
fi

if [ -e /tmp/deploy-apache ]
then
# Lets stop everything first
/etc/init.d/httpd stop

# Copy configuration back
cp ${SCRIPTDIR}/config/httpd.conf $ROOT_PREFIX/conf/

# Do a swap-around
mv  $ROOT_PREFIX /usr/old-apache
mv /tmp/deploy-apache $ROOT_PREFIX
rm -rf /usr/old-apache

# Setup the permissions
chown -R root:root  $ROOT_PREFIX
chmod 755 $ROOT_PREFIX
chmod 755 $ROOT_PREFIX/bin
chmod 755 $ROOT_PREFIX/conf
chmod 755 $ROOT_PREFIX/logs # preferrably a-r
# Assuming httpd is root:root
chmod 511 $ROOT_PREFIX/bin/httpd

# Start everything back up again
/etc/init.d/httpd start
else
echo "No deployment directory present. Cannot deploy!"
exit 1
fi

scripts/do-deploy-php

#!/bin/sh
# Copies over php to $ROOT_PREFIX from /tmp/deploy-php
SCRIPTDIR=~/server

if [ -e /tmp/deploy-php ]
then
# Do a swap-around
mv  $ROOT_PREFIX /usr/old-php
mv /tmp/deploy-php $ROOT_PREFIX
rm -rf /usr/old-php

# Copy configuration back
cp ${SCRIPTDIR}/config/php.ini $ROOT_PREFIX/lib/

# Setup the permissions
chown -R root:root  $ROOT_PREFIX
# Assuming php is root:root
chmod 511 $ROOT_PREFIX/bin/php
else
echo "No deployment directory present. Cannot deploy!"
exit 1
fi

It should also be noted that do-deploy-php should be run before do-deploy-apache, otherwise the http server might not be started correctly.

You might have noticed that i neglected to make a “/etc/init.d/httpd” startup script to be able to start apache on boot. I decided to leave this out considering the process to incorporate such a script varies across linux distro’s.

Putting it all together

Now i had all of these makefiles, shell scripts and whatnot, i needed to put them together in one place. Essentially my directory tree (stored in a “server” folder) looked like this:

• build
• config
  • apache_build.cfg
  • php_build.cfg
  • httpd.conf
  • php.ini
• external
• mk
  • apache.mk
  • php.mk
• scripts
  • do-deploy-apache
  • do-deploy-php
  • reload-config
• Makefile

First i needed to copy the “server” folder to my virtual build machine, like so:

mymachine $ scp -r ./server buildmaster@funkybuildmachine:

Then i needed to log into the virtual build machine and start to build everything:

mymachine $ ssh buildmaster@funkybuildmachine
...
funkybuildmachine $ cd server
funkybuildmachine $ make apache php

Finally, to deploy all i had to do was:

funkybuildmachine $ make deploy-scripts
funkybuildmachine $ make deploy

Note that the “deploy-scripts” target copies the scripts necessary to deploy both apache and php once they have been copied to the /tmp folder on the server, and thus only needs to be invoked once.

And finally, whenever i wanted to update the configuration of apache or php, all i had to do was:

funkybuildmachine $ make deploy-config

Which essentially copies the config/php.ini and config.httpd.conf files to the target server and then runs the scripts/reload-config script there.

To sum it all up…

In all, i got the solution to work. However, it still does feel a bit like a hack. In the future i hope to come up with something a bit better.

I suspect that i might be better off building .rpm’s or .deb’s. Of course, if anyone can come up with any better ideas, i’m all ears…