mirrordir man page on Mageia

Man page or keyword search:  
man Server   17783 pages
apropos Keyword Search (all sections)
Output format
Mageia logo
[printable version]

mirrordir(1)							  mirrordir(1)

NAME
       pslogin - secure remote tcp login using strong stream cipher encryption
       and diffie-hellman key exchanges.

       forward - arbitrary tcp socket forwarding over a secure/encrypted chan‐
       nel.

       copydir,	 mirrordir - copy, mirror directory trees via a minimal set of
       changes, locally or over FTP, or over a secure tcp connection.

       recursdir - recurse through local or remote directories to command/find
       files or create tar files.

SYNOPSIS
       mirrordir
       [-a, --access-times]
       [-m, --strict-mtimes]
       [--no-mtimes]
       [--ignore-size]
       [-A, --mtime-threshold ext]
       [--time-offset [[+]|-][H]H[:MM]]
       [-A, --always-write]
       [-r, --restore-access]
       [--no-chown]
       [--no-chmod]
       [-D, --only-delete]
       [-b, -S, --backup-extension, --suffix ext]
       [-N, --num-backups num]
       [-O, --backup-outdate sec]
       [-B, --block-size bytes]
       [-M, --max-bytes num[K|M|G]]
       [-s, --starting-file path]
       [-i, --ignore-next-exclude]
       [[-i] -X, --exclude path] [[-i] -X, --exclude path] ...
       [[-i] -F, --exclude-from file] ...
       [[-i] -G, --exclude-glob expr] [[-i] -G, --exclude-glob expr] ...
       [[-i] -R, --exclude-regexp expr] [[-i] -R, --exclude-regexp expr] ...
       [-C,  --exclude-script  [expr|file]] [-C, --exclude-script [expr|file]]
       ...
       [-h, --help]
       [-v, --verbose] [-v, --verbose] ...
       [-V, --version]
       [-k, --keep-files]
       [-l, --no-hard-links]
       [--follow-symlinks]
       [-L, --strict-locking]
       [-p, --password password]
       [-P, --password-exact password]
       [--test-login]
       [--no-warn-first-login]
       [--read-password-from-stdin]
       [--allow-empty-ftp-dirs]
       [--no-allow-empty-ftp-dirs]
       [--netrc]
       [--no-netrc]
       [--proxy-host host]
       [--secure]
       [-z, --gzip]
       [--gzip-backups]
       [--case-insensitive]
       [--to-lower]
       [--to-upper]
       [--no-use-passive-connections]
       [-K, --key-size bits]
       [--download-scripts]
       [--tar-file filename]
       [--tar-block-size N]
       [-t, --dry-run, --test-only]
       [--nice num] control mirror

       mirrordir  [-c  |  --copy-mode  |  --recurs-mode	  ]   -[abBCdDFGhklMm‐
       NOopRrstvVX] src [src ...] dest

       copydir -[abBCdeFGhklMmNOopRrstvVX] src [src ...] dest

       recursdir -[abBCdeFGhklMmNOopRrstvVX] src [src ...]

       pslogin	  [--key-size	 bits]	  [mc://][username@]hostname[:portnum‐
       ber][/path]

	 But usually just
       mirrordir [--exclude path] control mirror
       copydir src [src ...] dest
       recursdir src [src ...] [-C program]
       pslogin [username@]hostname[:portnumber]

DESCRIPTION
       mirrordir is a set of useful utilities for manipulating	and  mirroring
       directories.  Included  is also the command pslogin - an alternative to
       ssh(1), and forward(1) for forwarding arbitrary TCP socket  connections
       over encrypted secure channels.

       mirrordir  copies files that are different between the directories con‐
       trol and mirror to the directory mirror. Files whose modification times
       or  sizes differ are copied. File permissions, ownerships, modification
       times, access times (only if --access-times is used), sticky bits,  and
       device types are duplicated. Symlinks are duplicated without any trans‐
       lation. Symlink modification and access times (of the  symlink  itself,
       not  the	 file  it  points to) are not preserved. Hard linked files are
       merely copied. Creation times cannot be set with Unix as far as	I  can
       see.

       mirrordir  is  a	 DANGEROUS  command  because files or directories that
       exist in mirror that don't exist in control are deleted. If control  is
       entirely	 empty,	 then  all  files  and	directories  in mirror will be
       deleted. If mirror is entirely empty, then all files and directories in
       control will be copied.

       In  short, mirrordir forces mirror to be an exact replica of the direc‐
       tory tree control in every possible detail  suitable  for  purposes  of
       timed  backup.  It  naturally descends into subdirectories to all their
       depths. mirrordir tries to be as efficient as possible  by  making  the
       minimal set of changes necessary to mirror the directory.

       Access time duplication is not usually required and creates unnecessary
       load. Hence it is given as an option.

       The directory control is left untouched. If --restore-access  is	 given
       then access times are reset to their original with each read.

       If  the	the  --strict-locking  option is on, files in control that are
       copied are locked for `shared reading´. This will  ensure,  if  another
       process	is  busy  writing to that file, that the file is not copied in
       its incomplete or corrupted state.

       Usually mirrordir will not exit, but will give error messages to stderr
       to report any problems, and then will continue.

       The directory mirror or dest must exist, even if it is empty.

       Before  erasing	all the files in a directory, mirrordir checks for the
       file *--keep-me (where * is zero or one characters). If	this  file  is
       present	it  will abort with an error message. Hence such a file can be
       created in all directories that you are fearful	of  being  recursively
       erased.

       copydir is equivalent to mirrordir -ck --no-erase-directories ...
       (although  -c implies -k anyway), so copydir is very much like a rigor‐
       ous version of cp(1) where filenames can also be URLs,  and  only  out‐
       dated  files  are  replaced.  Use copydir instead of mirrordir for most
       file transfers. Only use mirrordir, when	 you  really  want  to	delete
       things.

       recursdir  is  a further program that does nothing but descend into the
       directories  on	the  command  line.  It	 is  equivalent	 to  mirrordir
       --recurs-mode ...
       It  was	born  after the -C option was added, and can be used as a more
       rigorous version of find(1) and can also pack all the  files  it	 finds
       into a tar file.

       pslogin	is  yet	 a further program which has almost nothing to do with
       the previous three. It envokes a secure	login  session	using  secure-
       mcserv. It is equivalent to mirrordir --login-mode --secure ...
       pslogin should be called logindir. See --login-mode below.

       forward	is  yet	 a further program which has almost nothing to do with
       first three. It can do forwarding of arbitrary services over  a	secure
       channel. See forward(1) for details.

       The  importance	of  this  package is that you can use URL's instead of
       normal filenames, and hence manipulate files over a  network.  The  URL
       types  currently	 supported  are	 ftp://	 and  mc://  (http:// is not a
       filesystem and therefore is not supported).  mc:// is the Midnight Com‐
       mander filesystem and is served by the secure-mcserv daemon. It has the
       advantage of serving cryptographically strong secure file transfers and
       logins.

       You  can	 also  use glob expressions in filenames for the recursdir and
       copydir commands. These will be recursively expanded.

SECURITY AND ENCRYPTION
       mirrordir supports strong stream cipher encryption  and	Diffie-Hellman
       key  exchanges with several possible key sizes. Secure connections work
       with mc:// type connections.  See  the  options	--secure,  --key-size,
       --download-scripts.  See	 the EXAMPLES section for demo's and the FILES
       section for where public/private keys are stored.

OPTIONS
       --help Print out detailed help, then exit.

       --verbose
	      Specifies verbose output of file modifications made  to  mirror.
	      This  option  can be given multiple times for greater verbosity.
	      Output is written to stdout.

       --restore-access
	      Restore the access times of control with each read.

       --access-times
	      Duplicate even the access times of control.

       --always-write
	      Force rewrite of every file regardless of whether they appear to
	      be identical or not.

       --recurs-mode
	      This  is	set  by default with recursdir. Listed directories are
	      read recursively and nothing is done to  them.  This  option  is
	      useful with -C in order to execute shell commands and search for
	      files. Note the system(), exec() and popen() functions available
	      to the C interpretor.

       --login-mode
	      This  option  is	set by default with pslogin. This approximates
	      rlogin(1) using secure-mcserv  as	 a  server.  It	 is  a	secure
	      encrypted connection intended as a replacement to ssh(1).	 pslo‐
	      gin implies --secure as well. When using	this  option  or  when
	      using  pslogin,  only  one  path must be present on the command-
	      line. The path is of the form  [mc://][username@]hostname[:port‐
	      number][/path].
	      Immediately after logging in, a cd /path will be executed at the
	      shell prompt. To determine if a  prompt  is  available,  pslogin
	      searches	for  a #, $ or > character. Should these not be found,
	      pslogin blocks indefinately. To avoid  this  behaviour,  specify
	      /path as exactly /. This will leave you in your login directory.
	      Alternatively, change your shell prompt to include one of	 these
	      characters.

       --copy-mode
	      This  is	set by default with copydir. Approximate the behaviour
	      of cp(1) as regards source and destination  files.  This	option
	      implies  --keep-files. It overwrites any existing file or direc‐
	      tory with the same name as a source file or directory, but  does
	      not  delete  files or directories unnecessarily. Multiple source
	      files or directories may be given. The destination path must  be
	      a directory.

       --no-erase-directories
	      With  this option, if a target directory conflicts with a source
	      file, and the directory is not empty, then an error message will
	      be  printed  and the program will abort. This is the default be‐
	      haviour for copydir.

       --erase-directories
	      With this option, if a target directory conflicts with a	source
	      file,  then  the	target	directory will be recursively deleted.
	      This is the default behaviour for mirrordir.

       --allow-empty-ftp-dirs
	      Some ftp servers do not produce a . or .. directory. This	 makes
	      it  appear  as  though  you  did not have permission to read the
	      directory. This option overrides this by assuming that such com‐
	      pletely  empty  directories  are	merely	empty  and do not have
	      errors. If you get  unable  to  open  directory:	*:  Permission
	      denied  errors,  then  you  can  use this option. This option is
	      enabled by default (see next).

       --no-allow-empty-ftp-dirs
	      Because we now cd to the directory to check  for	permission  to
	      access it, the default behaviour is to allow empty directories.

       --only-delete
	      Do  not  make  any  changes  to mirror that will cause the total
	      space occupied by mirror to enlarge. This	 is  a	useful	option
	      when  backing  up	 onto  a  drive	 that has limited space, where
	      changes have been made to control that  might  cause  mirror  to
	      grow  larger  during  the duration of the transfer. Running mir‐
	      rordir once with this option and then once normally will	ensure
	      that the available space is not overrun.

       -i, --ignore-next-exclude
	      This  option  dictates that the next --exclude- type option must
	      completely overlook those files regardless of their existence or
	      non-existence  in	 the  mirror directory. It has the effect of a
	      return value of IGNORE for the  --exclude-script	option	below.
	      This  can	 be  used to cause certain files to never be modified,
	      for example if you would like /etc/named.boot to never be	 modi‐
	      fied,    use    mirrordir	   /mnt/1    /mnt/2    -i    --exclude
	      /mnt/1/etc/named.boot. Note that all paths must be specified  in
	      the  control  directory  and  not	 the  mirror  directory, hence
	      --exclude /mnt/2/etc/named.boot won't work. This has the idiotic
	      behaviour	 that if you want to avoid erasing a file, you have to
	      have that file present in the control directory, even if it  has
	      zero length.

       --exclude path
	      Exclude file or directory path. A large number of excluded paths
	      on the command-line will slow performance. An excluded path will
	      be  removed  from	 the  mirror tree if it already exists, in the
	      same way as any absent directory	or  file.  Use	 the  --ignore
	      option to ignore keep rather than delete these files.

	      If  you have a long list of files to exclude, use the --exclude-
	      from option.

       --exclude-glob glob
	      Exclude file or directory names matching glob  style  expression
	      glob.  Matches  the  file without its full path. For shells, the
	      expression should be enclosed in appropriate quotes  to  prevent
	      substitutions.

       --exclude-regexp regex
	      Exclude  full  pathnames of files or directories matching regex.
	      For shells, the expression should	 be  enclosed  in  appropriate
	      quotes to prevent substitutions.

       --exclude-script [expr|file]
	      Cause  execution	of  the script expr for each file before doing
	      anything with that file. The  script  is	a  C  style  statement
	      block,  terminating  with	 a  `return expression;´. The value of
	      expression can be one of INCLUDE,	 EXCLUDE,  UNKNOWN  or	IGNORE
	      (see  -i above) to explain what is to be done with that file. If
	      expr does not contain a semi-colon (;) then it is assumed to  be
	      the  name	 of a file - which is then loaded. In either case, the
	      text is byte compiled into reverse-polish notation for fast exe‐
	      cution.  This  option  can  be  specified multiple times and the
	      scripts will be executed in order until a script	returns	 some‐
	      thing  other  than UNKNOWN. A return value of UNKNOWN allows the
	      remaining --exclude- options on the command-line to take effect.

	      If you find that the interpretor incorrectly reports  errors  or
	      segfaults, please report the breaking script to me.

	      The scripting language itself is a subset of the full C program‐
	      ming language. For example, the following is a valid script:

	      /* PATH is the full name of the file including
		 its path, DIR is the directory, excluding
		 the trailing slash (/), CWD is the current
		 directory, and depth() returns the number
		 of forward slashes (/) less one. */
	      if (depth (DIR) - depth (CWD) > 3) {
		  printf ("%s: excluded\n", PATH);
		  return EXCLUDE;
	      } else
		  return INCLUDE;

	      The scripting language does not support the assignment operator,
	      and hence does not support user defined variables.

	      The  following  predefined  macros  are available. Note that the
	      expansion of the macros applies equally well to  directories  as
	      to files.

	      FILE   current file without its path

	      NAME   file  name	 without its path or extension or trailing dot
		     (.)

	      EXTENSION
		     file extension without its leading dot (.)

	      DIR    directory without file-name or trailing slash (/)

	      PATH   full file name with path

	      CWD    current working directory

	      TIME   current time in seconds

	      All of the logical, arithmetic and bitwise C operators are  sup‐
	      ported. These are ( ) >= <= > < != == && || !  - + * / % & ^ and
	      have the same meanings and precedences as in C.

	      The following further  predefined	 macros	 are  available.  Each
	      returns  an integer (type long int in C). These are based on a C
	      lstat (or stat if --follow-symlinks is used) on  the  file.  See
	      stat(2) for a detailed explanation.

	      stat.st_dev - device
	      stat.st_ino - inode
	      stat.st_mode  - permissions
	      stat.st_nlink - number of hard links
	      stat.st_uid - user id of owner
	      stat.st_gid - group id of owner
	      stat.st_rdev  - device type
	      stat.st_size  - file size in bytes
	      stat.st_blksize - block-size for file-system I/O
	      stat.st_blocks - number of blocks allocate
	      stat.st_atime - time of last accessed in seconds
	      stat.st_mtime - time of last modification in seconds
	      stat.st_ctime - time of creation

	      The following functions return boolean values:

	      strncmp(string1, string2, integer);
		     returns  an  integer less than, equal to, or greater than
		     zero if string1 is found, respectively, to be less	 than,
		     to match, or be greater than string2.

	      glob(glob, string);
		     returns zero if string matches glob expression glob.  Try
		     to use only one glob expression in your  code  for	 effi‐
		     ciency of the underlying implementation.

	      regexp(regexp, string);
		     returns zero if string matches regular expression regexp.
		     Try to use only one regular expression in your  code  for
		     efficiency of the underlying implementation.

	      strstr(string1, string2);
		     returns  the  first occurance of string2 in string1 up to
		     the length of string1, or zero if it did not occur.

	      The following functions also return  a  boolean  value  and  are
	      analogous	 to  the  corresponding	 macros	 explained in stat(2).
	      They return non-zero if the specified condition is true.

	      S_ISLNK(integer); - file is a sym-link
	      S_ISREG(integer); - file is a regular file
	      S_ISDIR(integer); - file is a directory
	      S_ISCHR(integer); - file is a character device
	      S_ISBLK(integer); - file is a block device
	      S_ISFIFO(integer); - file is an fifo
	      S_ISSOCK(integer); - file is a socket

	      The following functions manipulate strings:

	      strcat(string1, string2);
		     returns the concatenation of string1 with	string2.  Note
		     that the + operator also concatenates strings.

	      depth(string);
		     returns  one  less than the number of forward slashes (/)
		     in string.

	      printf(format, ...);
		     behaves like printf(3) with an important exception:  only
		     long  int format specifiers should be used. The behaviour
		     of anything that results in conversion of	other  than  a
		     long  int is undefined. For example, use "%ld" instead of
		     "%d". This function prints to stdout.

	      The following functions do system calls:

	      system(command);
		     executes /bin/sh -c command, but unlike the C version, it
		     returns  the exit code of the command. I.e. it executes a
		     single line of shell script, command.

	      exec(argv0, argv1, ...);
		     executes process argv0 with  arguments  argv1....	 argv0
		     must  be  a  full	path  name. This is faster than system
		     because it need not envoke sh.

	      popen([string, ] shell_command);
		     like system, but returns the output of shell_command as a
		     string.   If string is given, this writes string into the
		     standard input of shell_command and return zero  on  suc‐
		     cess.

	      The  following  further  integer constants are available and are
	      analogous to the macros  defined	in  stat.h  and	 explained  in
	      stat(2).

	      S_IFMT  S_IFSOCK S_IFLNK S_IFREG S_IFBLK S_IFDIR S_IFCHR S_IFIFO
	      S_ISUID S_ISGID S_ISVTX S_IRWXU S_IRUSR S_IWUSR S_IXUSR  S_IRWXG
	      S_IRGRP S_IWGRP S_IXGRP S_IRWXO S_IROTH S_IWOTH S_IXOTH

	      One  of the the following constants should be returned using the
	      return keyword, and imply to the caller as follows.  If  nothing
	      is returned, the return value is assumed to be UNKNOWN.

	      UNKNOWN
		     didn't  know  what	 to do, continue with other --exclude-
		     options

	      INCLUDE
		     include the file

	      IGNORE do nothing with the file regardless of its	 existence  or
		     non-existence in the mirror directory

	      EXCLUDE
		     consider  the  file  to  be  non-existent	in the control
		     directory and hence  must	be  removed  from  the	mirror
		     directory	(this  does  not  override  the option --keep-
		     files)

	      The following perform flow control analogous to C:

	      The if clause causes statement1, statement2, etc. to be executed
	      if  integer  is true (i.e. non-zero), or otherwise causes state‐
	      ment1, statement2, etc. to be executed. The else {...}  part  is
	      optional.

		  if (integer) {
		      statement1;
		      statement2;
		      .
		      .
		      .
		  } else {
		      statementA;
		      statementB;
		      .
		      .
		      .
		  }

	      The return clause gives a value back to mirrordir and causes the
	      script to exit:

		  return expression;

	      The exit function cause mirrordir to  exit  with	the  specified
	      exit code.

		  exit(integer);

	      C	 scripts  would	 typically  be used to exclude types of files.
	      Note that this is an excessive  implementation  of  a  scripting
	      language,	 and all the features are not meant for general use. A
	      typical script will, for example, do  nothing  more  than	 cause
	      device files to be excluded:

	      if (S_ISSOCK(stat.st_mode) || S_ISFIFO(stat.st_mode)
		       || S_ISBLK(stat.st_mode) || S_ISCHR(stat.st_mode)) {
		  return EXCLUDE;
	      } else
		  return INCLUDE;

	      C	 scripts  can  also  be	 used  to  search  for	files with the
	      --recurs-mode option (same as the recursdir command):

	      /* removes all core files */
	      /* This example has been moved to the EXAMPLES section. */

       --exclude-from file
	      Exclude from a list of paths listed  in  the  file  file.	 Empty
	      lines  and comment lines (with a # as the first character of the
	      line) are ignored.  This list of	files  is  sorted  and	binary
	      searched,	 so  if	 you  have lots of filenames to exclude, it is
	      best to include them here for performance.  This option  can  be
	      specified	 multiple  times  with	different files. BUG: the last
	      path in file must end with a newline.

       --backup-extension level
	      Create backups of	 files	before	deleting  or  replacing	 them.
	      extension	 is  a C style format string e.g. .ORIG.%d (be careful
	      of shell substitutions with the %). level is the highest	number
	      of  revision to keep. extension is appended to the filename, the
	      oldest file having the highest number.

       --backup-outdate sec
	      Delete backup files older than sec seconds.

       --nice num
	      Be nice to other processes by sleeping occasionally.  num	 is  a
	      small  integer.	--nice causes the process to sleep for as long
	      as it is active, times a factor of num. Hence a value of 1  will
	      (very  roughly)  double  the  time  it takes to do a copy, and a
	      value of 3 will quadruple the time. This can be used  where  one
	      would  like  timed backups to place less load on the CPU. --nice
	      may not be available on your system.

       --no-chmod
	      Normally	the  permissions  of  files  are  set.	If  you	  have
	      restricted  access  and cannot change permissions, than this can
	      be used to disable setting of permissions.

       --no-chown
	      Normally the ownerships of files are set. If you have restricted
	      access  and  cannot  change ownerships, than this can be used to
	      disable setting of ownerships.

       --mtime-threshold sec
	      This is the deviation in mtime that is allowed for a file before
	      it  is overwritten. If you have mirrored an ftp site, the mtimes
	      are accurate to within a minute only, thereafter an  nfs	mirror
	      will  cause  every  file to be copied: you can then use --mtime-
	      threshold 60 to fix this.

       --time-offset [[+]|-][H]H[:MM]
	      This sets the time offset of any vfs (i.e. non-local) directory.
	      For  instance,  I	 am 8 hours east of New York, so when I mirror
	      from New York, I use --time-offset -08:00.

       --test-only, --dry-run
	      Do not make any changes. If used with --verbose, this will  show
	      what  changes would be made. This is an effective way of compar‐
	      ing directory trees. This	 is  untested  -  i.e.	I  don't  know
	      whether this option actually makes any changes or not!

       --skip-symlinks
	      Symlinks	are  treated  as  though they were not read - hence if
	      they are found in the mirror directory then they are deleted.

       --keep-files
	      Don't remove files from mirror, even if they don't exist in con‐
	      trol. This makes mirrordir somewhat like cp(1).

       --no-hard-links
	      mirrordir mirrors hard-links properly unless this option is set,
	      in which case hard-links are copied as regular files.

       --follow-symlinks
	      mirrordir mirrors symlinks properly unless this option  is  set,
	      in  which	 case symlinks are copied as regular files. Useful for
	      mirroring the Debian tree. Note that this	 follows  symlinks  in
	      both  the	 control  and  mirror  directories,  hence if symlinks
	      already exist in the mirror directory, these  will  be  left  as
	      symlinks.	 Note  that  --follow-symlinks implies --no-hard-links
	      also.

       --strict-locking
	      Create shared read locks on files as they are  read.  This  pre‐
	      vents clashes especially when copying files from the mail direc‐
	      tory - mail programs will be trying to write to those  files  at
	      the  same time as mirrordir is trying to to read from them. This
	      option has no effect with virtual file systems.

       --max-bytes [[num[k|M|G]]|num]
	      Maximum number of bytes to  write	 before	 giving	 the  message,
	      `filled  up  all	blocks	-  first file/dir not mirrored: path',
	      which is written to stdout. The remaining files are deleted from
	      mirror,  but  in	their listed order - hence it is possible that
	      the archive will grow greater than num while mirrordir is	 still
	      running.	You should make allowance for this eventuality by mak‐
	      ing num smaller than the maximum available space. Also note that
	      some  filesystems	 will  give an error message `No space left on
	      device' before the device is completely full.  You  can  restart
	      mirrordir	 with  the  option --starting-file path and hence con‐
	      tinue with this file on another volume. This  enables  mirrordir
	      to back up across different devices. num can be appended by k, M
	      or G (case insensitive) to specify kilobytes, megabytes or giga‐
	      bytes respectively. If any single file is greater than this num‐
	      ber, then an error message will be given. See also --block-size.

       --password password
	      Specify the password for FTP and mc:// connections.  The	anony‐
	      mous  password  defaults to your login name @ your local machine
	      name. You will be prompted for other login passwords. As	usual,
	      you are warned that including passwords in scripts is a security
	      risk. It is much better to put the  password  in	your  ~/.netrc
	      file  and	 then  not  use the --no-netrc option; see man ftp for
	      details.

       --password-exact password]
	      Don't prepend a - to the anonymous password. With ftp  anonymous
	      passwords,  a  -	is usually prepended to the password string. I
	      don't know why the Midnight Commander vfs did this, but one user
	      had  problems  with  it,	hence this option is given to send the
	      password exactly as specified by password.

       --test-login
	      When using --login-mode or pslogin, you may want to test	access
	      non-interactively	 (eg.  in  a shell script). To do this you can
	      run pslogin with this option and then  check  its	 exit  status.
	      secure-mcserv  uses  this to verify if the user can login on the
	      password server.

       --no-warn-first-login
	      The first time you try a secure connection to a machine, no pub‐
	      lic  key exists on the local machine. There is hence no security
	      against a man-in-the-middle attack. A warning to this effect  is
	      printed  and the user is prompted if they want to continue. This
	      option disables this warning and goes ahead regardless.

       --read-password-from-stdin
	      Instead of specifying the password on the command line, you  can
	      write  it	 into  the  command via stdin. This is not the same as
	      typing the invisible password as one normally does,  because  it
	      can  be  used  even  if there is no terminal. This is useful for
	      usage within other programs using, say popen(3).	 secure-mcserv
	      uses  this  to  verify  if  the  user  can login on the password
	      server.

       --netrc
	      Scan the ~/.netrc file. By default, this option is on.

       --no-netrc
	      Turn off reading of ~/.netrc file.

       --proxy-host host
	      Set the proxy for ftp downloads.	Don't  know  how  or  if  this
	      works. Consult the mc(1) man page for info in proxy support.

       --secure
	      (This  feature is BETA) I have implemented a secure socket layer
	      for mirrordir. It is enabled with this  option  and  applies  to
	      connections  to secure-mcserv (i.e. using mc:// type URLs).  The
	      secure socket library consists of libdiffie.a and a header  file
	      diffie-socket.h.	 If  you  include  diffie-socket.h  after  you
	      include sys/socket.h you can  recompile  the  resulting  program
	      with  all	 normal	 sockets  turned  into	secure	sockets. (This
	      applies to any program that  uses	 Unix  socket  calls,  but  is
	      untested).  The  first  service supporting this is secure-mcserv
	      which compiles and installs by default. Hence you	 can  use  the
	      mc://  file system with the --secure option, provided the remote
	      host is running  secure-mcserv  (try  secure-mcserv  -h  to  for
	      help).  --secure	uses  a	 stream	 cipher	 (much more secure and
	      faster than block ciphers like DES) with discrete logarithm  key
	      exchanges	 with public key server authentication (Diffie-Hellman
	      and p-NEW schemes). For the full details, see the	 file  diffie-
	      socket.h in the source distribution. The default key size is 512
	      bits. Note that if you are using gcc  you	 should	 compile  mir‐
	      rordir  with  the	 options  -O3 -fomit-frame-pointer -s -Wall to
	      speed key generation.

       --key-size bits
	      The default key size is 512 bits. Sizes  require	primes	to  be
	      generated	 and  compiled into the file field.c, hence only those
	      primes listed in field.c are supported, which are at the	moment
	      512, 768, 1024 and 1536. 768 is the recommended value for medium
	      security if you have slow computer. Otherwise  1536  is  not  an
	      unreasonably  paranoid  size  for long term security. 512 can be
	      used if you are not worried about large corporations, well  con‐
	      nected  hackers,	or governments snooping your connections. Note
	      that the the stream cipher has a length of bits/2,  which	 means
	      that you have more chance of being hit by a meteor while cashing
	      in at the state lottery, than cracking it. Be aware that a  per‐
	      son  can	sniff  your connection and then keep the output for as
	      long as it takes to break it. In twenty years time  a  1536  bit
	      key  will be considered small. Note also that the discrete loga‐
	      rithm problem (used here) is considered to be more difficult  to
	      solve  than  the	factorisation problem, hence the key is effec‐
	      tively a little larger than for RSA.  This  is  all  my  (rather
	      uneducated) opinion.

       --download-scripts
	      Mirrordir	 comes in two versions, an International version and a
	      US version (see --version). The US version contains  no  encryp‐
	      tion code whatsoever. Instead it downloads the needed algorithms
	      from encrypt.obsidian.co.za (in South Africa).  These are	 writ‐
	      ten  in  a fast, native, C-style interpreted language. There are
	      four scripts: one for the server	Diffie-Hellman	key  exchange,
	      one for the client Diffie-Hellman key exchange, one for initial‐
	      ising the stream cipher and  one	for  actually  performing  the
	      encryption  using	 the  stream  cipher.  Mirrordir automatically
	      downloads these scripts when you try to use  the	security  fea‐
	      tures.  The  option  --download-scripts  however	can be used to
	      force a download at any time. The International version contains
	      compiled-in  versions  of	 the  stream  cipher,  hence  only two
	      scripts are used, which need never be downloaded since they come
	      with  the	 distribution.	No speed penalty is incurred by having
	      the Diffie-Hellman  exchanges  in	 scripts,  however,  you  will
	      notice  a	 significant speed different when using the encryption
	      scripts compared to compiled-in encryption.

       --version
	      Prints out the version number as well  as	 whether  this	is  an
	      International  or	 US  version  of  mirrordir.  See  --download-
	      scripts.

       -z, --gzip
	      Enable compression for mc:// connections. This actually  envokes
	      a	 compressed socket layer at a lower level than the encryption.
	      Compression uses the libz library of gzip(1). The degree of com‐
	      pression	is  dynamically set to minimise transfer time. It will
	      drop to no compression for fast ethernet connections,  and  will
	      go  to maximum compression for slow modem connections. The algo‐
	      rithm adjusts the compression level so that the time for	a  TCP
	      write  call is between 2% and 5% of the time for the same amount
	      of data to be deflated (i.e. compressed).

       --gzip-backups
	      Backups are usually just a copy of the file.  With  this	option
	      they are compressed and a .gz is added to the default extension.
	      Note that if you specify	your  own  extension  using  --backup-
	      extension, then it must have a .gz at the end for compression to
	      work.

       --case-insensitive, --for-Robert-Seese
	      Ignores case in comparison  of  filenames	 and  linknames.  This
	      option  is useful when interacting with certain brain-dead oper‐
	      ating systems. I am not sure if this  option  behaves  correctly
	      under all circumstances.

       --to-lower

       --to-upper
	      Convert all new filenames to upper or lower case. When used with
	      --case-insensitive, it will apply only when creating new	files.
	      When  used  without  --case-insensitive,	all files, existing or
	      not, will be converted to uppercase in the  inefficient  way  of
	      deleting the old file and then copying the new file. This option
	      is useful when interacting  with	certain	 brain-dead  operating
	      systems.	I  am  not sure if this options behave correctly under
	      all circumstances.

       --no-use-passive-connections
	      If you get an error message could not setup passive mode	it  is
	      likely  because  you  need to enable this option. I don't really
	      understand what `passive' means, so don't ask me.

       --tar-file filename
	      This is to be used only with recursdir. It creates a tar archive
	      in  the  same  format  as	 GNU tar(1) and stores it in filename.
	      Leading special prefixes and leading slashes are	removed,  i.e.
	      ftp://machine/dir/file  becomes dir/file. If the first character
	      of the filename is a | character, the remaining text is  consid‐
	      ered as a command through which the output is to be piped. Hence
	      a gzipped archive can be created for example with
	      recursdir ftp://machine/dir --tar-file '| gzip -d > foo.tar.gz'

       --tar-block-size N
	      Sets the block size for tar output to 512 *  N.  These  are  the
	      units  in	 which	data is written to the archive. The default is
	      20. This is significant only when writing to block devices. This
	      must not be confused with --block-size.

       --block-size bytes
	      The  default block size is 1024 bytes. File sizes are rounded up
	      to the nearest block when calculating the total number of blocks
	      consumed.	 If the actual block size is greater than assumed then
	      it is possible for fewer blocks to be counted than are  actually
	      consumed	during	writing.   Hence  it is essential to specify a
	      block size greater than or equal to the actual block  size  when
	      using the option --max-bytes.

       --strict-mtimes
	      When  copying  regular files, mirrordir normally only overwrites
	      the mirror file if it is	older  than  the  control  file.  This
	      option  forces  a copy if there is any discrepancy at all in the
	      modified times of the files.

       --no-mtimes
	      Copy files only if their sizes differ. Ignore the file modifica‐
	      tion time.

       --ignore-size
	      Copy files depending on mtime, but ignore size differences.

       --starting-file path
	      path  can	 be  a file or directory. Until path is read, files or
	      directories will be processed as though they were excluded files
	      (i.e. if they exist in mirror they will be deleted). Directories
	      that contain path will be created if  they  don't	 exist.	 After
	      path is read, files are mirrored as usual. path itself will also
	      be mirrored. If path  or	any  of	 its  subdirectories  are  not
	      present,	then  mirrordir	 exits	immediately.  This is the only
	      time mirrordir exits prematurely. This is to prevent the	entire
	      filesystem  from	being  erased  on  account of path never being
	      found.

FTP SUPPORT
       Ftp transfers are supported using the Midnight Commander's Virtual File
       System (VFS), see mc(1). In short, this means that full URLs as well as
       local directories are supported. The following example demonstrates:
	   mirrordir --verbose \
	    ftp://lava.obsidian.co.za/pub/mirrordir \
		/home/mirrordir
       or alternatively,
	   mirrordir --verbose /home/mirrordir \
	    ftp://psheer@lava.obsidian.co.za/home/ftp/pub/mirrordir
       also works, but will prompt me for a password first. If you are upload‐
       ing  to	an ftp server, you should not have the --strict-mtimes options
       on, because modification times cannot be set via ftp, and hence	every‐
       thing will be copied.

       You will notice (with --verbose) that mirrordir repeatedly tries to set
       modification and access times over ftp. I am going to leave these  mes‐
       sages  in  to  remind  users of the limitations of the VFS type in use.
       These attempts do not appreciably detract from performance. You can use
       mc://  instead  although	 performance  is poor when uploading with this
       protocol. Downloading is always preferable to uploading.

       In general you should NOT have an ftp upload in your cron jobs  or  use
       ftp  upload to keep directories in sync. Always download from the other
       end when trying to keep directories in sync. Uploading is good for once
       off uploads only.

EXAMPLES
       Here are some nifty things you can do with mirrordir.

       Pedantic minimalist copy
	      I	 have  two  source  trees. I like to keep an old copy before I
	      apply a patch. I just do mkdir tree.OLD and then

		    mirrordir -v tree tree.OLD

	      If I run mirrordir again, then only a minimal  change  is	 made,
	      i.e.  only  the  updated files are copied. (cp(1) can in fact do
	      this).

       System backups
	      Some systems used timed backups onto tape archives. Others still
	      use  RAID devices which constantly maintain an identical copy of
	      a partition.  mirrordir provides a further alternative. You  can
	      install two drives in a system - one for use and one for backups
	      - and place mirrordir in your cron(8) tables. The various backup
	      options  can  be set to make backups of files that have changed.
	      The backup directory might also be readable  by  users  so  that
	      they  can	 see their backed up files. Previous versions of files
	      would also be available for users that might  like  to  retrieve
	      any  older  instances  of	 the file.  Because mirrordir executes
	      only the absolutely minimal set  of  changes,  it	 is  extremely
	      fast.  It	 can  be  run several times a day or even continuously
	      with the --nice option.

	      Backups can also be made by FTP onto a remote machine for	 addi‐
	      tional security against the destruction of the machine.

       Backing up hourly between two machines:
	      On one machine called dar2 I have a cron job that runs every six
	      hours:

	      #!/bin/sh

	      # (this is just in case of any bugs I don't know about,
	      # but I don't think it is necessary)
	      killall -9 tee
	      killall mirrordir >& /dev/null
	      sleep 2
	      killall -9 mirrordir >& /dev/null

	      (								      \
	      date							  ;   \
	      echo "mirrordir says (if it said nothing it is bad):"	  ;   \
		  mirrordir mc://dar1:12346/ -p abcdefg /mnt/dar1/	      \
		  -i --exclude-regexp '^mc://dar1:12346/var/lock/subsys/atd'  \
		  --exclude-regexp '^mc://dar1:12346/proc/'		      \
		  --exclude-regexp '^mc://dar1:12346/mnt/[^/]*/.*$'	      \
		  -i --exclude-regexp '^mc://dar1:12346/boot/'		      \
		  -i --exclude-regexp '^mc://dar1:12346/etc/lilo.conf'	      \
		  -C							      \
	      ´
	      if (S_ISDIR (stat.st_mode)) {
		  if (!regexp ("^mc://dar1:12346/[^/]*$", PATH))
		      printf ("Backing up: %s0, PATH);
	      }
	      ´							  ;	      \
	      date						  ;	      \
	      echo "Done"					  ;	      \
	      ) 2>&1							      \
	      | tee --ignore-interrupts --append /var/log/mirrordir.log	      \
	      | mail -s 'dar1 backup results' psheer@obsidian.co.za

       Secure transfers and logins
	      On the machine turing.co.uk execute
		  secure-mcserv -p 12345 -d
	      On some foreign machine
		  copydir --secure -K 512 -z \
		  mc://alan@turing.co.uk:12345/usr/src/linux/.config .
	      to securely copy a file using a security key  of	512  bits  and
	      using compression, or
		  pslogin mc://alan@turing.co.uk:12345/
	      to login to the machine securely.

       Mirroring FTP sites
	      Ftp  sites often disable their -R option on ls so that a	tradi‐
	      tional mirror(1) will fail (?). mirrordir does not  suffer  from
	      this limitation.

		  mirrordir -v ftp://metalab.unc.edu/pub /home/ftp/pub

       FTP transfers
	      Typical  FTP  transfers can be made easily with a single command
	      using the --copy-mode option. You can  copy  multiple  files  in
	      both  directions	and  even  between two ftp sites (albeit indi‐
	      rectly), just like with cp(1). I use
		  copydir -v mirrordir-0.9.15.tar.gz \
		  mirrordir.lsm ftp://metalab.unc.edu/incoming/Linux
	      to upload mirrordir to the sunsite.

	      Passwords for non-anonymous ftp transfers are best placed in the
	      ~/.netrc	file  according	 to  standard  ftp convention, and the
	      option --netrc used.   Alternatively  use	 ftp://myname@machine/
	      instead.

       Finding files
	      recursdir / -C
	      'if (!glob ("*.c", FILE)) printf ("%s\n", PATH);'
	      will print out all the C files on your system.
	      recursdir / -C
	      'if (S_ISCHR(stat.st_mode)) printf ("%s\n", PATH);'
	      will print out all character devices on your system.

       Backup FTP sites onto tape
	      Use
		  recursdir ftp://user@remote.machine/ \
		  --exclude-regexp '//[^/]*/proc/' --tar-file /dev/mt
	      to back-up a remote site onto tape.

       Removing core files
	      This removes all core files from your system:

	      recursdir / -C ´
	      long l;
	      if (strncmp (PATH, "/proc", 5)) {
		  if (S_ISREG (stat.st_mode) && !strcmp ("core", FILE)) {
		      if (strstr (popen ("file " + PATH), "ELF 32-bit LSB core")) {
			  l = l + stat.st_size;
			  printf ("removing: %s, cumu. total = %ldkB\n", PATH, l >> 10);
			  exec ("rm", "-f", PATH);    /* could also use system() */
		      }
		  }
	      }
	      ´

ENVIRONMENT VARIABLES
       TMPDIR The directory where you would like temporary files to be stored.
	      The ftp filesystem downloads files first into the this temporary
	      directory,  and then copies the file into its correct place. See
	      BUGS below.

	      If TMPDIR is not specified, then it defaults to the directory of
	      the current file in progress.

RETURN VALUE
       mirrordir returns the following:

       0      Success.

       1      Some  kind  of  error occurred like a write error, a permissions
	      error or the like. In this case, the precise  error  would  have
	      been written to stderr.

       2      A	 file was in use and therefore could not be copied, but other‐
	      wise a success. In this case the error `unable to	 open  control
	      file for writing' would have been written to stderr. If you grep
	      for these in stderr, you can then run mirrordir with just	 these
	      files. See grep(1).

BUGS
       Some  versions  of  the	atd daemon create a lock (or pid?) file with a
       lock on the file that cause secure-mcserv to block  indefinately.   You
       will  have  to kill atd or exclude (?) this file before hand to prevent
       this.

       Mirroring over ftp where there is  some	daylight  savings  correction,
       seems  to produce a one hour time offset. Use --time-offset as a tempo‐
       rary work-around. I don't know if this is mirrordir's fault though.

       A bug that caused mirrordir to eat CPU and  appear  to  halt  has  been
       fixed.

       A  large number of command-line --exclude expressions will slow perfor‐
       mance. This is noted for the purposes of those  wanting	to  exclude  a
       large  list  of	files.	This  list should be placed in a text file and
       excluded with the --exclude-from option.

       Modification and access times of symlinks (not the files they point to)
       are not duplicated.

       Note  that the previous limitation that hardlinked files are treated as
       regular files has now been alleviated. The  option  --no-hard-links  is
       provided to emulate the behaviour of versions prior to 0.9.8.

       No check is made to see if a hard-link would be created across devices,
       in which case an appropriate error will be reported.

       The ftp file-system downloads files first into the temporary directory,
       as  a  file  with the name *ftpfs*. This is a waste of space but is the
       default behavior of the vfs library. If there is not  enough  space  in
       this  directory,	 then  mirrordir  will	probably hang. See ENVIRONMENT
       VARIABLES above.

       The C scripting language used to exclude files is an  excessive	imple‐
       mentation.

FILES
       ~/.netrc
	      List  of	machines  and  corresponding passwords. See the option
	      --netrc.

       /etc/ssocket/accept.cs
	      This script performs the key exchange and	 signature  generation
	      on the server side of the connection.

       /etc/ssocket/connect.cs
	      This  script performs the key exchange and signature authentica‐
	      tion on the client side of the connection.

       /etc/ssocket/arcinit.cs
	      This initialises the stream cipher encryption. (Not  present  in
	      International versions.)

       /etc/ssocket/arcencrypt.cs
	      This  performs  the  stream  cipher  encryption. (Not present in
	      International versions.)

       /etc/ssocket/private/
	      This directory contains private keys for the host. Each key will
	      be  held	in  a  file:  filenames	 are  512, 1024 etc. This is a
	      preferable alternative to databases of keys as it	 obviates  the
	      need  for	 a  key management utility. Soon the Reiser (spelling)
	      filesystem will be standard and database files will become obso‐
	      lete anyway.

       /etc/ssocket/public/
	      Analogous to /etc/ssocket/private/ for public keys.

STANDARDS
       mirrordir  is  an  invention  of	 its author and does not belong to any
       operating system standard (although it should!).

AVAILABILITY
       The latest version of the program can be found  at  either  ftp://meta‐
       lab.unc.edu/pub/Linux/system/backup,	   or	     ftp://lava.obsid‐
       ian.co.za/pub/linux/mirrordir.

AUTHOR
       Paul Sheer  <psheer@obsidian.co.za>  <psheer@icon.co.za>

SEE ALSO
       mirror(1), pavuk(1), cp(1), scp(1),  find(1),  mc(1),  ftp(1),  ssh(1),
       tar(1), rlogin(1), rlogind(8), forward

Linux				1998 November 1			  mirrordir(1)
[top]

List of man pages available for Mageia

Copyright (c) for man pages and the logo by the respective OS vendor.

For those who want to learn more, the polarhome community provides shell access and support.

[legal] [privacy] [GNU] [policy] [cookies] [netiquette] [sponsors] [FAQ]
Tweet
Polarhome, production since 1999.
Member of Polarhome portal.
Based on Fawad Halim's script.
....................................................................
Vote for polarhome
Free Shell Accounts :: the biggest list on the net