mm_data man page on DigitalUNIX

Man page or keyword search:  
man Server   12896 pages
apropos Keyword Search (all sections)
Output format
DigitalUNIX logo
[printable version]

MM_DATA(5)							    MM_DATA(5)

       mm_data - NetWorker media multiplexor data (tape and disk) format

       This  documents	the  data  format that the NetWorker media multiplexor
       daemon, nsrmmd(8), writes to long term storage media such as tapes  and
       optical disks.  See nsr_device(5) and nsrmm(8) for a discussion of sup‐
       ported device families and types.  The format described here applies to
       any  fixed  record  device,  such  as  raw  disks, or fixed record tape
       devices with file marks.	 NetWorker uses the eXternal Data  Representa‐
       tion  (XDR)  standard  to write media which can be interchanged among a
       wide variety of machines.  Only the mechanism used  to  multiplex  save
       set  streams  onto  the storage media is described here; the formats of
       save set streams depend on  the	type  of  NetWorker  client,  and  are
       described in nsr_data(5).

       A  volume  is  one  physical piece of media such as a tape reel or disk
       cartridge.  A tape volume is made up of multiple media files, and  each
       media  file  may	 contain several media records.	 These media files and
       records should not be confused with a client's  (for  example  UNIX  or
       DOS) user files or records; the two do not necessarily correspond.  For
       example, a given media file or even a single media record  may  contain
       many  small client user files. On the other hand, a single large client
       file may be split across several media files, and even  across  several
       volumes.	  Media	 files	do  not span volume boundaries.	 Save sets may
       span media files and even volumes.

       On most tapes, media files can be skipped very quickly by the  device's
       hardware	 or  associated	 device	 driver software, and the hardware can
       detect when an end of a file has been reached.  On some tapes,  records
       can also be quickly skipped forward.  Otherwise, access to the media is

       Media records are described by the mrecord  structure.	Label  records
       are  fixed in size, MINMRECSIZE bytes. Other records can potentially be
       a larger size that must be some constant for the rest  of  the  volume.
       NetWorker  always  writes,  reads and skips data in units of full-sized
       media records.  Each  mrecord  contains	zero  or  more	mchunks	 These
       mrecords	 are used for storing one or more client save sessions or used
       by NetWorker for synchronization and labelling.	The XDR	 format	 of  a
       media file's mrecords and mchunks are as follows:

       const MINMRECSIZE = 32768;      /* minimum media record size */
       const MMAXCHK = 2048;	       /* maximum number of chunks in record */
       const MHNDLEN = 120;	       /* private area length for handlers */

       enum mrec_version {	       /* mrecord version */
	   MREC_VER5 = 0,	       /* older format mrecord */
	   MREC_VER6 = 6	       /* current format mrecord */

	* For media record format version 5, the data types lgui_t, lg_off64_t,
	* and lg_time64_t are defined as:
       typedef struct lgui_t unsigned long;
       typedef struct lg_off64_t unsigned long;
       typedef struct lg_time64_t unsigned long;

	* For media record format version 6, the data types lgui_t, lg_off64_t,
	* and lg_time64_t are defined as:
       typedef struct lgui_t {	       /* XDR encoded Unique Id. */
	   char _bytes[20];
       } lgui_t;
       typedef struct lg_off64_t unsigned long long;
       typedef struct lg_time64_t unsigned long long;

       typedef lgui_t ssid_t;	       /* save set id */
       typedef lgui_t volid_t;	       /* key for the volume database */

       struct  mchunk {
	   ssid_t mc_ssid;	       /* owning save set id */
	   lg_off64_t mc_low;	       /* 1st byte, relative to save stream */
	   opaque mc_data<MINMRECSIZE>;/* chunk's data */

       struct  mrecord {
	   opaque mr_handler[MHNDLEN]; /* private to media handler */
	   mrec_version mr_version;    /* Media record version number */
	   u_long mr_orec;	       /* record size */
	   volid_t mr_volid;	       /* encompassing volume's id */
	   u_long mr_fn;	       /* encompassing file number */
	   u_long mr_rn;	       /* record number within the file */
	   u_long mr_len;	       /* record byte length */
	   mchunk mr_chunk<MMAXCHK>;   /* chunks of save streams */

The first field of an mrecord, mr_handler, is reserved for media-specific data
(currently it is not used by any implementation).  The mr_version field is the
version number of the media record format.  The size of the rest of the fields
in the media record depends on the version number.  The mr_orec field  is  the
size  of the current record.  A media record's header fields, mr_volid, mr_fn,
and mr_rn, are used to check the tape position and  the	 data  read  from  the
record.	  The  file  numbers  and  record  numbers start at zero and increment
sequentially.  The record number is reset each time the file number is	incre‐
mented.	  On  disks,  file  numbers  are always zero.  The mr_len field is the
actual number of valid bytes in this record, as opposed to  the	 size  of  the
device's read or write request.

If  file  or  record  skipping is unreliable, NetWorker can still recover from
isolated errors, at worst by rewinding and reading the tape  from  the	start.
If a volume can be physically unmounted or mounted without notice to the media
management daemon, then the volume identifier in each record provides a	 quick
way  of	 verifying  when  this happens, without the need for a full rewind and
reading of the label in most cases.

The mchunks within an mrecord contain client data from one or more  save  sessions.	The mc_ssid and mc_low values are used to reconstruct the save streams
from the chunks within the records.  The mc_data field holds the  actual  data
of each chunk.	For a given save set, mc_low plus the length of mc_data should
equal the following chunk's value for mc_low.  Save sets may  by  intermingled
arbitrarily within media records.

The  first  chunk  of  the  first record of the first media file on the volume
encapsulates the volume label information; for some media,  the	 second	 chunk
contains additional volume information, for example, the media pool the volume
belongs to: Subsequent data in the first file is reserved  for	future	expan‐
sion.	The  label  may be duplicated in a second file for redundancy, in case
the first copy of the label gets accidentally overwritten.  The formats of the
volume	label  and additional label information are described by the following
XDR data structures:

const MVOLMAGIC = 0x070460;	/* volume magic number */
const NSR_LENGTH = 64;		/* length of several strings */
const RAP_MAXNAMELEN = 64;	/* maximum length of attribute name */

struct mvollabel {
    u_long  mvl_magic;		/* medium volume verification number */
    lg_time64_t mvl_createtime; /* time at which volume labeled */
    lg_time64_t mvl_expiretime; /* time for volume to expire */
    u_long  mvl_recsize;	/* expected size of mrecords */
    volid_t mvl_volid;		/* medium volume id */
    string  mvl_volname<NSR_LENGTH>;/* medium volume name */

struct vallist {
    vallist *next;
    string value<>;		/* attribute value */

struct attrlist {
    attrlist *next;
    string name<RAP_MAXNAMELEN>;/* attribute name */
    vallist *values;		/* attribute values */

 * Additional information may includes the following attributes
 * (listed by the name they are stored with):
 * "volume pool" : the media pool
struct mvolinfo {
    struct attrlist *mvi_attributes;/* any other information */

The mvl_magic field must be equal to MVOLMAGIC in order for the chunk to  represent	a  valid volume label.	If the volume label changes in the future, the
new format will have another ``magic'' number, but the format  described  here
must  still  be allowed.  The mvl_volid is an internal identifier assigned and
managed by the media manager.  The mvl_volname is  the	volume	name  that  is
assigned when the media is first labeled.  The time fields are in UST format -
the number of seconds elapsed since 00:00 GMT, January 1, 1970.	 The  mvl_rec‐
size is the size of all subsequent media records found on the tape.

The  mvp_pool  is  the	pool  name  that  is  assigned when the media is first
labeled.  Different media pools allow administrators to segregate  their  data
onto  sets  of	volumes.   Media  cannot  be reassigned from one media pool to
another.  Pool names are a maximum of NSR_LENGTH characters long.

Synchronization marks, called schunks, are also written	 periodically  to  the
media  for  each save set.  Synchronization chunks are used by scanner(8) when
verifying or extracting directly from a volume.	 They are also used by	nsrmmd
when  trying to recover from media errors during file recovery.	 The following
XDR data structure describes a synchronization chunk:

typedef lgui_t clientid_t;

struct				   ssclone_t {
    lg_time64_t ssc_cloneid;	   /* unique UST stamp wrt ss_ssid */
    u_long ssc_flag;		   /* lots of status buried here*/
    u_long ssc_frag;		   /* not used, always 0 */

 * Synchronization chunk of the newer MREC_VER6 media format.
struct schunk {
    u_long ssi_gen;		   /* Not used. */
    ssid_t ssi_ssid;		   /* save set identifier */
    ssid_t ssi_prev;		   /* non-zero iff continuation */
    u_long ssi_level;		   /* backup level*/
    lg_time64_t ssi_time;	   /* save time on client */
    lg_time64_t ssi_create;	   /* creation time on server */
    lg_time64_t ssi_insert;	   /* insertion time on server */
    lg_time64_t ssi_complete;	   /* completion time on server */
    clientid_t ssi_clientid;	   /* client name identifier */
    u_long ssi_flags;		   /* more details about this ss */
    string ssi_host<>;		   /* client name - save set owner */
    string ssi_name<>;		   /* symbolic name, for example "/usr" */
    uint64_t ssi_size;		   /* actual number of bytes saved */
    uint64_t ssi_nfiles;	   /* number of client files saved */
    u_long ssi_browse;		   /* browse time offset */
    u_long ssi_recycle;		   /* recycle time offset */
    struct attrlist *ssi_al;	   /* generic RAP attribute list */
    ssclone_t ssi_clones<>;	   /* information about this clone */

 * Synchronization chunk of the older MREC_VER5 media format.
struct old_schunk {
    opaque ssi_host[NSR_LENGTH];   /* save set host */
    opaque ssi_name[NSR_LENGTH];   /* symbolic name */
    u_long ssi_time;		   /* save time */
    u_long ssi_expiry;		   /* expiration date */
    u_long ssi_size;		   /* actual size saved */
    u_long ssi_nfiles;		   /* number of files */
    ssid_t ssi_ssid;		   /* ssid for this save set */
    u_long ssi_flag;		   /* various flags, see below */
    u_long ssi_info;		   /* volid or ssid, see below */

#define SSI_START	1	   /* start of a save set */
#define SSI_SYNC	2	   /* synchronization point */
#define SSI_CONT	3	   /* continued from another volume */
#define SSI_END		4	   /* end of this save set */
#define SSI_SSMASK	0x0000000f /* save set sync chunk type */
#define SSI_LBIAS	0x10000000 /* the level is included in the flags */
#define SSI_LMASK	0xff000000 /* mask to cover bits for level */
#define SSI_LSHIFT	24	   /* shift amount for the level */
#define SSI_INCOMPLETE	0x00010000 /* not finished (aborted) */
#define SSI_CONTINUED	0x00800000 /* continued save set series */

The ssi_ssid is the save set identifier of this save set.  The ssi_time	 field
contains  the  create time of the save set in UST based on the client's clock.
The ssi_create field contains the create time of the save set in UST based  on
the  server's  clock.  The ssi_insert field contains the time the save set was
inserted into the media database in UST based  on  the	server's  clock.   The
ssi_complete  field  contains the completion time of the save set in UST based
on the server's clock.	The ssi_clientid and ssi_host are the  client  identi‐
fier  and  name of the index which contains this save set.  Traditionally this
is the client identifier and name of the client where the save set originated.
The ssi_name is the save set name to be presented to the user.	These are both
null-terminated strings, even though the fields are fixed length in the	 older
version	 media	records.   The ssi_size and ssi_nfiles are the number of bytes
and number of files saved so far for this save set.   The  ssi_browse  is  the
time  offset in seconds from the save set insertion time to the time this save
set is no longer browsable.  The ssi_recycle is the  time  offset  in  seconds
from the save set insertion time to the time this save set becomes recyclable.
The ssi_al is the generic save set attribute.

The ssi_flag indicates the type of this synchronization chunk and other information	about  the save set.  In the older version synchronization chunk, this
field also contains the level of this save set.	 There are four basic types of
synchronization	 marks that can be found from examining ssi_flag & SSI_SSMASK.
SSI_START is used to mark the beginning of a save set.	SSI_SYNC marks a peri‐
odic  synchronization  point  and is only written at an exact file boundary in
the save set.  SSI_CONT indicates that this is the continuation of a save  set
that  started  on a different volume.  When ssi_flag & SSI_SSMASK is SSI_CONT,
ssi_prev or ssi_info contains the volume identifier for the save set's preceding  volume.   These  synchronization  chunks are used when a save set spans a
volume boundary.  SSI_END marks the end of a save set.

On the new version of synchronization chunk, the ssi_level field contains  the
save  set backup level.	 On the older version of synchronization chunk. Should
the SSI_LBIAS bit be set then ssi_flag & SSI_LMASK shifted to the right by the
value  of  SSI_LSHIFT specifies the level of the save set.  The SSI_INCOMPLETE
bit indicates that this save set did  not  finish  properly.   This  could  be
caused by a user interrupting an in progress save.

The  SSI_CONTINUED  bit indicates that this save set is logically continued to
or from another save set.  These continued save sets are used to  handle  very
large save sets.  If the SSI_CONTINUED bit is set and ssi_flag & SSI_SSMASK is
SSI_START, then ssi_prev or ssi_info gives the previous save set id that  this
save  set  was continued from.	If the SSI_CONTINUED bit is set and ssi_flag &
SSI_SSMASK is SSI_END, then ssi_prev or ssi_info gives the next	 save  set  id
that this save set is continued to.

The  ssi_expiry field is the expiration date, in UST, for this save set.  This
field is zero if an explicit save set expiration time was not  specified  when
the save set was created.  This field no longer exists in the new synchronization chunk.

       nsr_device(5), nsr_data(5), nsrmm(8), nsrmmd(8), nsrmmdbd(8), nsr(8),

       RFC 1014 XDR: External Data Representation Specification

NetWorker 7.3.2			  Aug 23, 06			    MM_DATA(5)

List of man pages available for DigitalUNIX

Copyright (c) for man pages and the logo by the respective OS vendor.

For those who want to learn more, the polarhome community provides shell access and support.

[legal] [privacy] [GNU] [policy] [cookies] [netiquette] [sponsors] [FAQ]
Polarhome, production since 1999.
Member of Polarhome portal.
Based on Fawad Halim's script.
Vote for polarhome
Free Shell Accounts :: the biggest list on the net