CVS-DISTFILES(7) BSD Reference Manual CVS-DISTFILES(7)NAMEcvs-distfiles - fetch distfiles for MirPorts via CVS
SYNOPSIS
CVS_DISTREPO=:ext:cvs.example.com:/cvs
CVS_DISTDATE=2007-09-22
CVS_DISTTAGS=1_0_release
CVS_DISTMODS=src
SVN_DISTPATH=https://svn.example.com/projects/foo/trunk
SVN_DISTFILE=foo-svn
SVN_DISTREV=123
SVN_DISTDIR=foo
DESCRIPTION
This document describes how the "fetch distfile via CVS" (or Subversion)
method in MirPorts operates.
The problem
Often, software is not distributed as nicely as we would like to have it
- in a versioned tarball, gzip'd or compress'd. Sometimes, tarballs are
not versioned, broken, or just too old. But almost universally, source
code is available via a revision control system - such as CVS or Subver-
sion.
This manual documents a method to fetch sources via CVS, while retaining
control over correct checksums etc.
From a user's point of view: CVS
To enable the _CVS_DISTF fetch method, you have to put a few liness into
your port Makefile. As usual, learning by an example is "the way to go" -
prominent examples are the three ports essentials/pkgtools,
games/bsdgames, and devel/libtool.
The user interface consists of a few variables you can set:
CVS_DISTREPO the value of the -d option to cvs
CVS_DISTDATE (optional) value of -D option to cvs checkout
CVS_DISTTAGS (optional) value of -r option to cvs checkout
CVS_DISTMODS module(s) to check out, separated by space
CVS_DISTFILE (optional) file name prefix
Note that if CVS_DISTREPO does not start with a colon, CVSREADONLYFS is
set automatically, which is not compatible with the OpenBSD version of
anoncvssh.
Note: while both CVS_DISTDATE and CVS_DISTTAGS are optional, it is re-
quired to set one of them because HEAD checkouts are a moving target and
thus not supported benefitting checksumming.
Setting CVS_DISTREPO${_i} enables _CVS_DISTF operation on a specific
distfile. ${_i} can be empty or a single decimal digit, in case you need
more than one distfile.
Once operation has started, _CVS_DISTF${_i} is set to the name of the
file created in ${FULLDISTDIR} (/usr/ports/Distfiles), in the following
scheme: basename[-dateext][-tagext].mcz
The basename is CVS_DISTFILE${_i}, or (if unset) CVS_DISTMODS${_i} with
dashes removed. The dateext (only if CVS_DISTDATE is set) is exactly
this, but with all non-numerical parts removed. The tagext (if
CVS_DISTTAGS) is the tag, with all characters except digits, letters,
dash or underscore removed, and a capital T prepended. This ensures in
almost all cases that _CVS_DISTF is unique.
Then, _CVS_FETCH${_i} is set to a command to "retrieve the file" if it is
not yet there. If any _CVS_DISTF${_i} is set, archivers/mpczar is pulled
in as a fetch dependency automatically.
All _CVS_DISTF${_i} are added to _DISTFILES automatically, so there is no
need to do this. Now you only need to set a PKGNAME (unless the default
is fine to you) and you are done.
From a user's point of view: Subversion
To enable the _CVS_DISTF fetch method, you have to put a few liness into
your port Makefile. As usual, learning by an example is "the way to go" -
prominent examples are the two ports lang/llvm and sysutils/vesautils.
The user interface consists of a few variables you can set:
SVN_DISTDIR Name of the directory to check out into (top-level package
dir). Defaults to the basename of SVN_DISTREPO.
SVN_DISTFILE Optional file name prefix. Defaults to the basename of
SVN_DISTDIR.
SVN_DISTREPO Full path to the repository and subdirectory to check out.
SVN_DISTREV Revision number to check out.
Setting SVN_DISTREPO${_i} enables _CVS_DISTF operation on a specific
distfile. ${_i} can be empty or a single decimal digit, in case you need
more than one distfile.
Once operation has started, _CVS_DISTF${_i} is set to the name of the
file created in ${FULLDISTDIR} (/usr/ports/Distfiles), in the following
scheme: basename-revext.mcz
The basename is SVN_DISTFILE${_i}. The revext is the letter 'r' followed
by the numerical SVN_DISTREV revision. This ensures in almost all cases
that _CVS_DISTF is unique.
Then, _CVS_FETCH${_i} is set to a command to "retrieve the file" if it is
not yet there. If any _CVS_DISTF${_i} is set, archivers/mpczar and
devel/subversion are pulled in as fetch dependencies automatically.
All _CVS_DISTF${_i} are added to _DISTFILES automatically, so there is no
need to do this. Now you only need to set a PKGNAME (unless the default
is fine to you) and you are done.
More interesting possibilities
The infrastructure automatically defaults to setting the (first) distri-
bution directory:
WRKDIST?=${WRKDIR}/${CVS_DISTMODS}
WRKDIST?=${WRKDIR}/${SVN_DISTDIR}
If CVS_DISTMODS (without ${_i}) contains more than one module, you must,
consequentially, set WRKDIST yourself (also if you differ from what the
infrastructure thinks is a sensible default).
The package name is a requirement which still must be set manually, un-
less the new default of ${_CVS_DISTF:R}-${DASH_VER} (CVS) or
${SVN_DISTFILE}-${SVN_DISTREV}-${DASH_VER} (SVN) suits your port, which
will cover 95% of the cases. FOR CVS, THIS ONLY WORKS IF YOU USE
CVS_DISTDATE *AND* *NOT* CVS_DISTTAGS. Another example is to just set a
sensible one yourself, like libtool does (which pulls from the branch-1-5
tag):
PKGNAME=libtool-1.5.23a.${CVS_DISTDATE:S!/!!g}-${DASH_VER}
Of course, this needs more maintenance.
What is working behind the curtains
The infrastructure defines a target ${FULLDISTDIR}/${_CVS_DISTF${_i}} for
each file, which is invoked by 'mmake fetch' automatically and only if
the file does not exist. However, the target itself also checks for the
existence of the distfile, because in some rare cases all dependencies
are invoked.
First thing is to execute a 'mmake fetch-depends' so that mpczar is only
built if one of the _CVS_DISTF files is missing, and subversion is only
built if needed as well. This is done already inside the check for ex-
istence.
After that, _CVS_FETCH${_i} is invoked on the file and the size is
checked (cloned from a "normal" fetch process).
Checksumming, extracting and so on are processed with no difference; the
.mcz file extension yielded an addition to EXTRACT_CASES, of course.
Operation of _CVS_FETCH:
The script /usr/ports/infrastructure/scripts/mkmcz is invoked with a few
parameters (target, repo, date, tag, modules) and then creates said *.mcz
file, as follows:
Prepare a temporary directory (mkdtemp), change into it, invoke 'cvs
checkout', or 'svn checkout', call mpczar on the current directory with
an appropriate set of "ignore files" ('CVS' and '.svn').
mpczar(1) then archives all files (no directories, pipes, devices, sym-
bolic links, and other funky stuff) in ASCII sort order into an System V
CPIO archive with CRC, using the new -M option to cpio(1), which tells it
to "normalise" the archive: serialise inode numbers, zero out mtime, own-
ership and device information, and, for efficiency, store hard links'
file contents only once. This is safe in the scenario (it would not be
safe if one of the files to pack had a hard link to outside the area
which is packed, because its link count is increased by one then). The
CPIO archive is LZ compressed using the 'mpczar.z' helper (really just
compress(1) in disguise), which is a mathematical function.
This achieves that the same unordered set of input files always creates
the same archive (with the same checksum).
SEE ALSObsd.port.mk(5), ports(7), mpczar(1).
MirOS BSD #10-current November 22, 2009 2
