AUTO-BUILD(1) User Contributed Perl Documentation AUTO-BUILD(1)NAMEauto-build - run automated, unattended software build processes
SYNOPSIS
# auto-build [OPTION]...
Run build using latest code currently in source control
# auto-build--config /etc/auto-build.d/auto-build.conf
Run build using code committed prior to 12:30 on Dec 12th 2004
# auto-build--timestamp '12:30:00 Dec 12th 2004' \
--config /etc/auto-build.d/auto-build.conf
SUMMARY
"auto-build" is the command used to execute the Test-AutoBuild software
build automation framework. It is intended to execute a single
iteration of the build engine and then quit with its exit status
providing an indication of the build success / fail status (see the
"EXIT STATUS" section later in this manual page). To perform a
continous automated build this command would be launched on a frequent
basis from cron. Locking will ensure that two instances do not execute
concurrently, so it can safely be triggered every 5 minutes.
OPTIONS
The following command line options are valid:
--config=FILE
Specify an alternative path to the configuration file for the build
process. If ommitted, will default to looking for the file
$HOME/auto-build.conf. The configuration data is loaded by the
Config::Record module, so refer to that module's manual page for
details of permitted syntax. The file will also be pre-processed by
the Template module allowing dynamic generation of complex
configuration files
--log4perl=FILE
Specify an alternative file containing configuration settings for
the Log::Log4perl logging system. If ommitted, log4perl settings
will be loaded from the master autobuild configuration file.
--timestamp=DATE
Specify the timestamp to use when checking code out of the source
control repositories. For repository modules supporting timestamps
it is guarenteed that the code checked out will be the latest
version not newer than this timestamp. If ommitted, the timestamp
will default to the current timestamp. The argument can be either
an plain integer specifying number of seconds since the epoch, or
any of the text representations handled by the Date::Manip module
'today'
'1st thursday in June 1992'
'05/10/93'
'12:30 Dec 12th 1880'
'8:00pm december tenth'
--verbose
Increase the verbosity of warning messages and errors. This will
ensure that if the builder unexpectedly crashes, a complete stack
trace will be dumped along will full arguments.It will also cause
the post-processed configuration file to be dumped.
--help
Display a summary of the command line help options
SIGNALS
The build process provides special handlers for a couple of signals to
assist in debugging problems.
USR1
When receiving this signal the build process will send a stack
trace to STDERR, providing an indication of the current task being
performed. The build process will continue to run after receiving
and handling this signal.
USR2
When receiving this signal the build process will use Data::Dumper
to dump the entire runtime state of the builder to STDERR. The
build process will continue to run after receiving and handling
this signal
EXIT STATUS
The exit status codes of the build have the following meanings
0 The build process completed succesfully with no failures in any
module
1 The build process failed on one of more modules, but ran through to
completion of all stages
2 The build process aborted during processing, and did not complete
execution of all stages.
3 An invalid/unknown command line option was provided
4 The argument to one of the command line options was malformed or
invalid.
USER GUIDE
The notes which follow assume the software itself has been installed
into the /usr hierarchy with all common commands available within the
default $PATH. Some minor alterations may be neccessary if this is not
the case.
HOST SETUP
Before using the build engine a handful of things must be configured on
the host machine. This shouldn't take more than a few minutes to
accomplish if the example configurations are followed.
BUILD AREA CREATION
The first step after software installation is to create an area where
builds can be performed. The build engine expects that the build area
be populated with a small set of top level directories. To aid in this
setup, the command "auto-build-make-root(1)" can be run passing the
build area path as a command line parameter. The example configuration
file expects that the build area is at "/var/lib/builder", so assuming
this is the case, run the command:
# auto-build-make-root /var/lib/builder
NB, if the software was installed from RPM, this directory may have
been created on your behalf.
USER ACCOUNT CREATION
It is highly recommended that the build engine be run as an
unprivileged user. This is primarily intended to protect the host
machine against both delibrate and accidental flaws in a module's build
script. By convention the user to run the build as will be called
"builder". If only a single instance of the build engine is to be run
on a host, it is convenient to make this user's home directory would
point to the build area root. So to add a user to run the build engine
run the command:
# groupadd builder
# useradd -d /var/lib/builder -g builder builder
# passwd builder
WEB STATUS PAGE SETUP
The build engine will generate HTML status pages at the end of every
run summarizing the status of the build. There are 3 example
configuration files suitable for use with Apache located in
"/etc/auto-build.d/httpd". The standard "auto-build.conf" file assumes
that virtual hosting will be used on the web server, with a suitable
Apache config being the file "vhost.conf". Copy it to
/etc/httpd/conf.d, and edit it to set the server name of your build
host. If virtual hosting is not suitable, then the status pages can be
made to appear at a fixed URL "/builder", or as a user directory under
"/~builder". The sample config files "aliased.conf" or "user.conf" can
be used instead of "vhost.conf", although it will also be neccessary to
edit the "auto-build.conf" file to change the "httpPrefix" option.
AUTOMATION
As mentioned earlier, executing "auto-build" just runs a single
iteration of the build engine. To achieve continuous integration, this
command should be scheduled to run on a periodic (frequent) basis. On
UNIX, cron is perfectly designed to provide build scheduling, so 3
example crontabs are provided in the directory
"/etc/auto-build.d/cron". To run the build engine continous 24x7, copy
the "continuous.conf" file into "/etc/cron.d". There are also examples
for running an hourly, or daily build cycle.
NB. before setting up a scheduled build in cron it is a good idea to
run a cycle manually to ensure that your installation & configuration
is operating normally.
BASIC CONFIGURATION
The following notes describe how to add simple modules to the build
engine.
MODULE CONTROL FILES
To achieve complete independance from the language and tools used for a
project, the build stage of the autobuild cycle is considered a black
box. To perform the build, autobuild merely invokes an opaque "control
file" provided by the developer. This shell script will typically
perform four tasks - configure, build, install and package. The
autobuild will capture the standard output and error streams, saving to
the modules build log files. In common with standard UNIX behaviour, an
exit status of zero indicates a successful build, while non-zero
indicates failure.
By convention, the script should be called "autobuild.sh" and be placed
in the top level directory of the module's source. When invoked a
number of environment variables will be set for use by the control
file.
AUTOBUILD_MODULE
The name of the module which is being built
AUTOBUILD_INSTALL_ROOT
The path to a virtual root directory where software should be
installed. When a module is built, it is guarenteed that the
installed files from any dependant modules will be present in this
directory. This enables a module to depend on the output of
another module. The contents of this directory, however, are not
preserved across runs of the build.
AUTOBUILD_PACKAGE_ROOT
The path to a directory in which a module's build process will
create any binary packages it generates, for example RPMs, or
Debian packages. The packages are typically placed into a package
type specific sub-directory.Consider, for example, a module which
generates an RPM, of itself. The $AUTOBUILD_PACKAGE_ROOT directory
would be used to set the '_topdir' macro for the RPM build process
rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
AUTOBUILD_SOURCE_ROOT
The path to the directory in which modules are checked out. This
can be used in conjunction with $AUTOBUILD_MODULE to identify the
root directory for the module
AUTOBUILD_COUNTER
A counter identifying the current run of the build. Typically this
will be the number of seconds since the UNIX at the time the
current build cycle began, but may alternatively be a version
control changelist.
AUTOBUILD_TIMESTAMP
A counter identifying the timestamp taken at the start of the build
cycle. When checking out code, all version control systems are
synchronized to no later than this timestamp. For a given value of
$AUTOBUILD_TIMESTAMP the source code being built will always be
identical, thus this is suitable for use as a unique identifier for
a build.
The overall goal of the control file is to build the software, run unit
tests and then install the software into the autobuild install root.
The location of the latter is given by the "AUTOBUILD_INSTALL_ROOT",
and would typically be used to set the "prefix" when running
"configure" or an equivalent script. The control file should exit when
a non-zero exit status if an error occurs at any time. An exit status
of zero, indicates a successful build & install.
For example of creating a script for GNU AutoTools refer to latter part
of this manual page.
MODULE CONFIGURATION
ADVANCED TOPICS
MODULE DEPENDANCIES
PUBLISHING UNIT TEST RESULTS
As well as a number of environment variables, the control file is also
passed a single command line argument. This argument contains the name
of a file into which unit test results should be stored. There are
(currently) no requirements on what data format should be used when
writing to this file, so at its simplest one can just capture the
output from a 'make check' command. So, taking the control file written
earlier, one would add
test -n "$1" && TEST_RESULTS_FILE=$1 || TEST_RESULTS_FILE=results.log
rm -f $TEST_RESULTS_FILE
make check | tee $TEST_RESULTS_FILE
There are a couple of things to note here. First, if no argument is
given we default to writing results to a file called "results.log".
While this is not neccessary if running within the build engine, it
does make it easy for a developer to execute the control file
("autobuild.sh") manually. Second, rather than just redirecting output
into the results file, it is piped through the tee(1) command. This
simple lets one monitor the progress of tests when running the control
file manually.
When the HTML status pages are generated for a module, the contest of
the unit test results log will be made available for download.
CONTROL FILE EXAMPLES
A control file using GNU AutoTools (autoconf/automake)
With a few notable exceptions (Perl & Apache), C and C++ programmers
have been increasingly converging on GNU Auto Tools for configuring and
building their software. As with MakeMaker, the key task is to set the
installation prefix when running the configure script:
./configure --prefix=$AUTOBUILD_INSTALL_ROOT
The task of resolving inter-module build dependancies is somewhat less
well defined. There are a number of approaches that may work - some
programs may even need a combination of all of them!
Helper scripts (aka pkg-config)
Some libraries install a small shell script to the bin directory that
programs can use to determine the correct compiler and linker flags.
For example, the "pkg-config" script can print out the compiler flags
required when building against most libraries. It is usually just
sufficient to set the PKG_CONFIG_PATH environment variable to point to
the builder install root
PKG_CONFIG_PATH=$AUTOBUILD_INSTALL_ROOT/lib/pkgconfig
Configure flags
Another common approach to locating pre-requisite libraries is for the
program's configure script to have command line options for specifying
include and library search paths. For example, when building the Resin
Java servlet container, the configure script has --with-openssl-lib and
--with-openssl-include options. Thus when configuring Resin, as well as
setting the prefix, we'd set these two options:
./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
--with-openssl-lib=$AUTOBUILD_INSTALL_ROOT/lib
--with-openssl-include=$AUTOBUILD_INSTALL_ROOT/include
An alternative to specifying both library and include search paths is
to just tell the configure script the installation prefix of the
library:
./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
--with-openssl=$AUTOBUILD_INSTALL_ROOT
Compiler / linker environment variables
If there is no explicit support for specifying the location of pre-
requisite libraries then the final approach is to try setting compiler
environment variables. The two important ones being <code>CFLAG/code
and <code>LDFLAG/code:
CFLAGS=-I$AUTOBUILD_INSTALL_ROOT/include
LDFLAGS=-L$AUTOBUILD_INSTALL_ROOT/lib
Complete AutoTools example
#!/bin/sh
set -e
# Pull in config scripts
PATH=$AUTOBUILD_INSTALL_ROOT/bin:$PATH
export PATH
# Clean up build area
[ -f Makefile ] && make -k maintainer-clean ||:
# Re-generate autotools scripts
autoconf
automake -a
# Configure the build
./configure --prefix=$AUTOBUILD_INSTALL_ROOT \
--with-openssl=$AUTOBUILD_INSTALL_ROOT
# Make & install
make
make install
# Create source code dist
make dist
AUTHORS
Written by Daniel P. Berrange and Dennis Gregorovic, with contributions
from Richard W.M. Jones.
REPORTING BUGS
Report bugs to the bug tracker provided on the GNA! project site linked
from <http://www.autobuild.org>.
LICENSE
Copyright (C) 2002-2006 Daniel P. Berrange, Dennis Gregorovic, Red Hat.
Refer to individual source files for further details of copyright
holders.
Parts of this software are derived from Rolling builds version 2.0
Copyright (C) Richard W.M. Jones <rich@annexia.org>
Test-AutoBuild is distributed under the terms of the GNU GPL v2+. This
is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
SEE ALSO
For configuration information auto-build.conf(5); for setting up a
build root auto-build-make-root(1); for developers wishing to extend
the build framework Test::AutoBuild(3pm); for further project
information "http://www.autobuild.org"; for developer questions
"mailto:testautobuild-devel@gna.org".
perl v5.14.1 2011-09-01 AUTO-BUILD(1)
