Apache Subversion 1.7 Release Notes

Improved output of svn update for multiple working copies ¶

Improvements have been made to the output of svn update when updating multiple working copies at once (see issue #3693). Below is an example of the new output:

  $ svn up subversion svncorp UNVERSIONED
  Updating 'subversion':
  At revision 1037740.
  Updating 'svncorp':
  At revision 288.
  Skipped 'UNVERSIONED'
  Summary of updates:
   Updated 'subversion' to r1037740.
   Updated 'svncorp' to r288.
  Summary of conflicts:
   Skipped paths: 1

svn diff now shows property changes as unidiff ¶

svn diff now shows textual property changes in unidiff format, except for svn:mergeinfo properties. Below is an example of the new output:

  $ svn diff
  Index: gamma
  ===================================================================
  --- gamma       (revision 23)
  +++ gamma       (working copy)

  Property changes on: gamma
  ___________________________________________________________________
  Modified: svn:externals
  ## -1,3 +1,3 ##
   ^/branches/gamma gamma-external
  -^/trunk/alpha alpha-external
  +^/trunk/beta beta-external
   ^/trunk/epsilon epsilon-external
  Index: beta
  ===================================================================
  --- beta        (revision 23)
  +++ beta        (working copy)

  Property changes on: beta
  ___________________________________________________________________
  Added: svn:eol-style
  ## -0,0 +1 ##
  +native

Error and warning numbers now reported ¶

The output of svn now contains an error number or warning number with every error. The following example illustrates these changes.

   % svn info
   svn: E155007: '/home/CIA-91' is not a working copy

These error values are useful as search keywords. Under the hood, these error codes correspond to the API error codes used by Subversion and APR. For programming to our API's, it's possible to convert a numeric error code to a symbolic one via the which-error.py script (first included in Subversion 1.3.0):

   % ./tools/dev/which-error.py 155007
   00155007  SVN_ERR_WC_NOT_DIRECTORY

Case sensitivity of authz access rules ¶

Prior to this release, the section headers in Subversion's authz access files—which contain repository names and repository filesystem paths—were parsed in a case-insensitive fashion. That's been fixed in this release. The practical fallout, though, is that your existing authz files might be depending (perhaps unintentionally) on the old behavior and the new behavior could result in access to items in your repositories being unexpectedly denied—or granted—as a result of upgrading to Subversion 1.7. Please review all authz access files for case correctness. (For details, see issue #3781).

Incompatible FSFS changes since 1.7.0-alpha3 for packed repositories ¶

Subversion 1.6 introduced support for packing FSFS revision files, and Subversion 1.7.x alpha releases (up to 1.7.0-alpha3) supported packing of revprops into an SQLite database. This support is not present in the final release (see issue #3952 for the reason). Any FSFS-backed repositories that were svnadmin created or svnadmin upgraded by svnadmin from a nightly build or from an alpha release of the 1.7.x line are not supported by the final 1.7.0 release. It is required to dump these repositories using an svnadmin built from the 1.7.0-alpha3 release (or to svnsync them using a source server running 1.7.0-alpha3) in order to upgrade them for the 1.7.0 release.

Subversion 1.7 will complain when it encounters such repositories, with the following error:

subversion/libsvn_fs_fs/fs_fs.c:1083: (apr_err=160043)
svnadmin: E160043: Found format '5', only created by unreleased dev builds;
   see http://subversion.apache.org/docs/release-notes/1.7#revprop-packing

We reiterate that this lack of upgrade path is within the latitude of our policy for pre-releases. We may provide in the future a script to downgrade a repository in-place to the format supported by both 1.6 and 1.7. (We will welcome contributions of such a script.)

We plan to reintroduce revprop packing in a future release; see issue #3944 for details.

'svn remove' now removes directories from disk immediately ¶

The svn remove command now removes directories from disk immmediately. In Subversion 1.6, directories were scheduled for deletion but kept on disk to retain meta-data stored in the .svn subdirectory. With the new working copy format introduced in Subversion 1.7 it is no longer necessary to keep the removed directory on disk. This also facilitates better handling of directory replacements in the working copy (see issue #3468 for details).

If the --keep-local option is used, svn remove will keep the removed directory on disk.

Centralized Metadata Storage ¶

A key feature of the changes introduced in Subversion 1.7 is the centralization of working copy metadata storage into a single location. Instead of a .svn directory in every directory in the working copy, Subversion 1.7 working copies have just one .svn directory—in the root of the working copy. This directory includes (among other things) an SQLite-backed database which contains all of the metadata Subversion needs for that working copy.

Even though the data is stored in a structured format, the relationships between the data are complex. We highly discourage external tools from modifying the data held in this database, as such modification is likely to result in working copy corruption.

Pristine Contents ¶

Subversion keeps a record of the unmodified contents of all files in the working copy. Historically, these have been called text-bases, but in Subversion 1.7, they have received a new name, and a new home. With the centralization of metadata, the text-bases have been renamed pristines, and are now located in the same .svn directory as the working copy metadata.

The pristines are stored by checksum in a sharded format. Any files in the working copy which have the same pristine content will now share references to the pristine store, making such working copies slightly smaller. Running svn cleanup will remove any pristines which are no longer needed by the current state of the working copy.

Note: In 1.7, we recommend to run svn cleanup periodically in order to claim back the disk space of unreferenced pristines. We expect a future Subversion release to purge unreferenced (and thus unused) pristines automatically; see issue #4071 for details.

Upgrading the Working Copy ¶

Subversion 1.7 introduces substantial changes to the working copy format. In previous releases of Subversion, Subversion would automatically update the working copy to the new format when a write operation was performed. Subversion 1.7, however, will make this a manual step. Before using Subversion 1.7 with their working copies, users will be required to run a new command, svn upgrade to update the metadata to the new format. This command may take a while, and for some users, it may be more practical to simply checkout a new working copy.

Note: Subversion 1.7 cannot upgrade working copies that a 1.6 client would have refused to operate upon before an svn cleanup was run (with a 1.6 client). In other words, before upgrading to 1.7, a 1.6 client must be used to run svn cleanup on all working copies that require cleanup. Likewise, Subversion 1.7 cannot upgrade corrupt 1.6 working copies. Unfixable problems can arise from missing or corrupt meta-data inside .svn directories. Such damage to the 1.6 working copy is permanent, and cannot be fixed even if svn cleanup is run prior to the upgrade.

We regret these limitations, but we had to introduce them in order for 1.7 to ship timely and without overcomplicating the internals. If your working copy does not upgrade cleanly, please check out a new one.

svnrdump load race condition ¶

Note that svnrdump load can suffer from a race condition in its locking algorithm when run against a 1.6 or earlier server. The race condition is only applicable if multiple svnrdump load processes may attempt to write concurrently to a single repository. This is the same problem that the new atomic revprop changes feature fixes for svnsync (see issue #3546), and the same server-side workaround is available.

Server administrators who would like to block their users from committing via svnrdump load may do so by installing the following pre-revprop-change script:

#!/bin/sh
PROPNAME="$4"
if [ "$PROPNAME" = "svn:rdump-lock" ]; then
  echo "'svnrdump load' disabled by the server administrator" >&2
  exit 1
fi
exit 0

This hook script suffices to protect repositories from accidental use of svnrdump load. It does not (and cannot) protect the server from users who intentionally recompile svnrdump in order to bypass this restriction.

svnrdump dump and libsvn_ra_serf editor driving don't mix ¶

The svnrdump dump command is more strict in its expectations from the network access library than the API permits it to be. This problem manifests itself in particular with the ra_serf HTTP access library, as documented in issue #4116. A workaround is to use the ra_neon HTTP access library (which is the default).

The problem does not affect other executables (such as svn and svnadmin). It also does not affect svnrdump when the default HTTP access library, ra_neon, is used. Third-party API consumers should not expect to be able to use all the orderings permitted by the delta editor API when the editor receiver is svnrdump dump.

This problem is related to, but not the same as, ra_serf's dishonouring of the delta editor ordering rules.

svn log can now print diffs ¶

svn log accepts the new --diff option which causes it to show changes committed in a revision as a unified diff. Below is example output:

  $ svn log --diff -r1033146 http://svn.apache.org/repos/asf/subversion/trunk
  ------------------------------------------------------------------------
  r1033146 | hwright | 2010-11-09 19:40:46 +0100 (Tue, 09 Nov 2010) | 3 lines

  * subversion/libsvn_wc/wc_db.c
    (svn_wc__db_op_copy): Remove unused variable.


  Index: subversion/libsvn_wc/wc_db.c
  ===================================================================
  --- subversion/libsvn_wc/wc_db.c	(revision 1033145)
  +++ subversion/libsvn_wc/wc_db.c	(revision 1033146)
  @@ -3061,7 +3061,7 @@ svn_wc__db_op_copy(svn_wc__db_t *db,
                      apr_pool_t *scratch_pool)
   {
     svn_wc__db_pdh_t *src_pdh, *dst_pdh;
  -  const char *src_relpath, *dst_relpath, *dst_op_root_relpath;
  +  const char *src_relpath, *dst_relpath;
     apr_int64_t dst_op_depth;
   
     SVN_ERR_ASSERT(svn_dirent_is_absolute(src_abspath));

  ------------------------------------------------------------------------

svn update now accepts the --parents option ¶

svn update now accepts the --parents option. This option removes the need to manually create missing parent directories when adding additional items to a sparse working copy.

For example, adding 3 files from different directories to a sparse working copy required creating each file's parent directories first:

   $ svn co --depth=empty ${PROJECT_ROOT_URL} wc
   $ svn up --depth=empty wc/A wc/A/D wc/A/D/G wc/A/D/H wc/A/B wc/A/B/E
   $ svn up wc/A/D/G/pi wc/A/D/H/omega wc/A/B/E/alpha

This becomes much easier with the new --parents option:

   $ svn co --depth=empty ${PROJECT_ROOT_URL} wc
   $ svn up --parents wc/A/D/G/pi wc/A/D/H/omega wc/A/B/E/alpha

New subcommand: svn relocate ¶

A new svn relocate subcommand has been added. It points a working copy to a different repository root URL, serving the same purpose as the old svn switch --relocate syntax which has now been deprecated.

svn switch now checks ancestry ¶

svn switch now checks that the working copy item being switched shares common version control history with the URL to which it is being switched.

This change was made to protect users from the unpleasant side effects of oft-mistyped switch commands, such as accidentally running svn switch ^/branches when you instead meant to run svn switch ^/branches/SOME-BRANCH. As of version 1.7, Subversion will check to see if the source URL and the target working copy item are ancestrally related and, if they do not, the switch operation will fail with an error.

   $ svn switch ^/branches
   svn: E195012: Path '.' does not share common version control ancestry
   with the requested switch location.  Use --ignore-ancestry to disable
   this check.
   $

What is meant by "ancestrally related"? In short, it means that if you were to compare all the paths and revisions that each of these two version controlled items has occupied over time — its current repository location plus all previous ones, back through all the copies and moves in its history — those two sets would overlap.

As noted in the previous example's error message, this ancestry check — which does come at some performance cost — may be disabled by passing the --ignore-ancestry option to the svn switch command.

svn diff can show copied files as if they were newly added ¶

When svn diff shows a copied file, it usually shows how the copied file differs from the file it was copied from. Sometimes this isn't the desired form of output. There are situations where a copied file needs to appear in its entirety, for instance when producing a patch file to be applied elsewhere with a patch tool (such as svn patch).

In Subversion 1.7, svn diff takes a new --show-copies-as-adds option which causes copied files to be shown in their entirety, just like newly added files are shown.

svn diff can show git-style diff annotations ¶

svn diff takes a new --git option which causes it to produce extra annotations denoting added, deleted, and copied files. This extended diff format was inspired by git-diff.

svn diff --git is intended to produce patches which are compatible to git-apply, but there are limitations due to differences between Subversion and git:

• svn diff currently cannot display rename information. Renames are always shown as a copy and a delete.
• All paths in patches are shown relative to the repository root to avoid ambiguity. It is possible that the numbers of leading path components on the left and right side of the patch differ, for instance if a diff between /trunk and /branches/mybranch is shown. This may prevent git-apply from applying such patches without modification.

In a future release of Subversion, svn patch will receive support for the extra annotations produced by svn diff --git, so that additions, copies, renames, and deletions can be handled explicitly rather than implicitly.

The same diff format extensions are also supported by Mercurial.

svn merge help text has been enhanced ¶

The help text for the svn merge command has been enhanced. It now explains common use cases and includes small examples, making it more useful for quick reference.

New subcommand: svnlook filesize ¶

A new svnlook filesize subcommand has been added. It returns the size of a given path in the repository, for a given revision or transaction. This is a constant-time operation regardless of the size of the file. In pre-commit hooks wanting to block commits with too large files, svnlook filesize can now be used instead of the much more costly workaround via svnlook cat.

JavaHL Updates ¶

Due to the move to the Apache Software Foundation, the JavaHL bindings have been similarly moved to a new package: org.apache.subversion. The old package still exists, and will continue to ship for backward compatibility reasons, but we recommend comsumers switch to the new package, as only it will continue to be updated.

Also, JavaHL now requires Java 1.5 to compile. In addition, many of the APIs in the new package have been switch to use generics, and other Java 1.5 language features.

Reduced subtree mergeinfo changes ¶

Merges no longer record mergeinfo (describing the merge) on subtrees (that have their own explicit mergeinfo), if the subtree was unaffected by the merge. This should greatly reduce the number of spurious svn:mergeinfo property changes for users who have large numbers of subtrees with explicit mergeinfo.

If a change being merged contains svn:mergeinfo modifications these will still be applied, just like any other property modifications. So if the change being merged was itself the result of another merge performed with a 1.5 or 1.6 client, excessive subtree mergeinfo changes are still possible. Best results will be achieved for new branches created and maintained exclusively with 1.7 clients.

Reintegrate merges are less restrictive ¶

Reintegrate merges now succeed in the case where all the prior 'sync' merges were done as subtree merges, but effectively all the changes were merged from the target to the source. Reintegrate also works with shallow targets, as long as none of the excluded subtrees are affected by the merge.

Improved notification about mergeinfo changes ¶

Merge-tracking aware merges now produce special notifications and headers when a merge records mergeinfo describing a merge or elides mergeinfo. This clearly differentiates between changes that are made due to application of a diff from the merge source and those that are simply merge-tracking 'housekeeping' changes.

Below is an example of the new output, showing notifications about application of a diff to the file lib/src/psi.c and mergeinfo changes resulting from this merge.

   $ svn merge ^/trunk -c3 --depth infinity
   --- Merging r3 into 'lib':
   U    lib/src/psi.c
   --- Recording mergeinfo for merge of r3 into '.':
    U   .
   --- Recording mergeinfo for merge of r3 into 'lib':
    G   lib
   --- Eliding mergeinfo from 'lib':
    U   lib 

Merges into mixed-revision working copies now disallowed by default ¶

To prevent unnecessary conflicts, svn merge will now fail with an error if the merge target is a mixed-revision working copy.

A mixed-revision working copy will need to be updated with svn update before a merge can be performed into it. This restriction has always existed during merges using the --reintegrate option and it is now the default behaviour for every type of merge.

For backwards-compatibility, there is a new option called --allow-mixed-revisions which disables the check for a mixed-revision working copy, except for reintegrate merges. Using this option is not recommended. Please use svn update instead.

The mergeinfo subcommand accounts for subtree and partial merges ¶

The svn mergeinfo subcommand now flags revisions wich are partially merged to a target with the '*' decorator. A revision may be partially merged due to non-inheritable mergeinfo on the target or because of explicit mergeinfo on some subtree of the target. The latter case is ignored by default, but may be considered by using the new --depth and -R options for svn mergeinfo.

Transitive record only merges ¶

Merges performed with the --record-only option now apply svn:mergeinfo diffs from the merge source.

Merges into sparse directories no longer cause tree conflicts ¶

Merges into shallow working copies used to cause tree conflicts on nodes which were affected by the merge but not present in the working copy. In 1.7, no tree conflict is flagged. Instead, non-inheritable subtree mergeinfo is created on the parents of missing nodes, so that later merges into working copies that are not sparse will pick up any missing changes for those nodes.

Data caching ¶

Per default, Subversion server processes will use a 16 MB memory block to cache file and folder content. This amount is being allocated by every server process, i.e., the maximum size of cache memory available per process can roughly be estimated as

cache limit = 0.8 * (RAM size / max. server process count) - 4 MB

A reasonable upper limit to that in-process cache size is the size of the active data set. This is usually the size of user files in /trunk plus the size of all differing files on active branches. So, two or three times the sum of all /trunk sizes or all active projects on that server is a reasonable indication for a performance-optimal cache size.

svnserve introduces a new optional --memory-cache-size / -M command line parameter to override the default cache size. Using --cache-fulltexts and --cache-txdeltas you may instruct the server what kind of data should be cached. The first should always be enabled unless data is only read once as during a svnrdump run. Text delta caching should be enabled unless your server memory is low.

For mod_dav_svn, a 10 GB cache configuration with maximum data coverage looks like this in httpd.conf

<IfModule dav_svn_module>
    SVNInMemoryCacheSize 10485760
    SVNCacheFullTexts on
    SVNCacheTextDeltas on
</IfModule>

As a side-effect, the memory consumption becomes more predictable in many usage scenarios since there is less need to gather and pre-process data. Please note that a cache size of 0 will deactivate the new caching facilities and cause the server to fall back to 1.6 caching mechanisms.

Network data compression level ¶

Subversion servers are often disk or network I/O limited. With the introduction of data caching, however, disk I/O can be widely eliminated and the CPU load created by data compression will become a bottleneck on fast network connections.

The default setting will allow for 10 to 15 MB/s net data throughput (5 MB to 10 MB compressed data) per client and per CPU core. On a multi-core server with a 1 Gb network connection or if clients are mainly connected with very limited bandwidth, you may want to select a higher compression rate to squeeze a little more data through the network at the expense of significantly higher server CPU loads. If the server's NIC is not a bottleneck, you may consider lowering the compression level to 1 or 2 for 100 Mb clients and to 0 (compression off) for a network with predominately 1 Gb clients.

To that end, svnserve accepts the new optional --compression or -c command line parameter. mod_dav_svn supports the feature as well but its impact is limited since over HTTP, network data compression is used only in certain cases to begin with. The optional module parameter SVNCompressionLevel controls it:

<IfModule dav_svn_module>
    SVNCompressionLevel 9
</IfModule>