Introduction to AF's Backup system
Client

Full and Incremental backup, Backup levels

A full backup means, that everything requested is stored on the server. An incremental backup stores only these files and direc- tories, that have changed or are newer than the start of the previous full or incremental backup. For determining, what has changed, the mtime in the inode is used. Why this is done, can be read from the HOWTO.FAQ.DO-DONT, Q2. Problems may arise, if mechanisms are used that create files and directories while resetting the mtime to an earlier date and time (e.g. using tar -x or cpio -i). Then these files are not saved during the next incremental backup. If a backup level is supplied, when running incr_backup, then everything is stored, that has changed since the most recent run with a higher or equal level. Note, that other backup systems define levels the other way round: the lower the level number, the more is written to backup. The full backup can be divided into several parts. This is useful for example, if the duration for backing up everything is too long for the intended time frame. If the full backup should run during the weekend, but the about 60 hours from friday evening to monday morning are not sufficient, this is a case for several parts. These can be configured listing the files and directories for each part in different clientside configuration parameters. Before the parameter NumBackupParts must be set appropriately.

Packing and Unpacking

The client packs the files and directories and sends them to the server or receives a data stream from the server and unpacks it. If compression is requested, a configurable compression program can be used to process each packed file individually. The name of the appropriate uncompress-program is packed into the stream, so it needs not be supplied during restore. Nontheless it must be insured, that compress and uncompress-programs are within the search path of the client processes or supplied with the full path.

Filelists

For each stored filesystem entry the client writes a line to a file containing the cartridge number, where it can be found for restore, also the tape file number and (since 2.9) the ID of the user owning the filesystem entry. This listfile resides locally in a filesystem on the client. It serves for determining, where the data for restore can be found on the server and to determine, whether a user is allowed to restore the corresponding file, if this possibility is configured (see HOWTO.FAQ.DO-DONT Q7). Each time a full backup is done, a new filelist is started. Incremental backups result in appending lines to this file. The filelist files are identified by the number of the full backup, that is increased continuously each full backup. A configurable number of such filelist files is kept for later use. The restore utility needs these lists, when certain files are requested for restore. If these filelists get lost, you are in trouble. You should use the mechanism described in the next section to avoid this.

Minimum restore information

The minimum information to restore everything (the stored data and the filelists) is, to which cartridges and - on those - in which tape files the data can be found. This information is passed each start of a full or incremental backup to a configurable program, that can read this information (a line of a special format) as the standard input. It should write this to a place where it is safe from crashes of the local machine. One possi- bility is a NFS-mounted directory, another one is to send this information via email. This minimum information can be passed to the restore utility for restoring either everything or only the filelist files. The minimum restore information is not cumulative i.e. only the latest one is required. Without this minimum restore information, a tape scanning is necessary to do emergency recovery (get everything back after a complete loss). This takes notably longer and can be avoided having this little string available, when needed.

Clientside program structure

There is a two-layer program hierarchy on the client side. The program "afclient" in the client's binary directory is performing the packing, unpacking and the communication with the server side. It does not read the configuration file and does not maintain the filelists described above. This is done by programs with higher functionality, that in turn call the "afclient"-program setting it into different modes. The entry "afbackup" in the same directory is identical with "afclient".