Introduction to AF's Backup system
Server

Storage Devices

Any kind of streamer can be used with this backup system. The only restriction is, that it must have a block-device entry in the /dev-directory. The blocksize of the device must be set correctly in the serverside configuration. Besides of streamers also filesystems can be used to save the data to. The configuration to do this is a little special and can be taken from the HOWTO.FAQ.DO-DONT 3. With streamers several tape cartridges can and should be used. There are two basic cases: Either you have a robot or have not. If you have no robot, you must change the cartridges manually. The server supplies you with e-mail, whenever another cartridge should be put into the drive. Having done this, a special command (cartready) must be run on the server to let him know, that he can continue now. If you have a robot, but no special software (commands) to tell it, which cartridge to insert into the drive, you must set the robot to sequential mode. That is, the robot puts the next tape into the drive once the actual one is ejected. Such a mode is usually supported and then the server ejects the cartridges out of the drive until the right tape is found. If you have commands to set the cartridge directly, you can configure them in the configuration file.

Cartridges

The cartridges are identified by numbers beginning with 1. Once they are in use, they automatically get a magnetic label, that they keep until overwritten with a special command (label_tape). It is a good idea also to use adhesive labels and to write the numbers onto the tapes, so they can't be mixed up. The server recognizes, if there is a tape in the drive, that has the wrong label. What then happens, is depending on the cases mentioned above. If you have no robot, you get another email, that points you to the problem. If you have a robot in sequential mode, the server tries all cartridges, he has, to find the right one. If it finds a tape with the right label, this is used. If it does not, it stops and writes errors to the serverside error logfile. If you have a robot and commands to set the cartridge and the robot inserts the wrong one, this is considered a severe problem and the server stops, again writing errors to it's error log, waiting for a maintainer to solve the problem and to allow the server to continue by issuing a command, he receives in a mail. Once all tapes of a set (for cartridge sets see the next section) have been used up, the server tries to start over with the first of the set overwriting it. The server tracks, what tapes are needed for clients to restore all their data. When full, these tapes are not overwritten until explicitely permitted or the data on them is no longer in any index file on a client. Tapes can be set to read-only state. The server will never write to tapes in this mode, even not in the variable append operating mode. In variable append mode, any tape of the correct cartridge set can be supplied to the streamer device and it will be accepted for writing. If the data on that tape is still needed by clients, the next data will be appended to the already written area on tape, Otherweise the tape will be overwritten. If not in variable append mode, the next writing position is, where writing stopped before. This can be overridden using the cartis command or the clientside option -G. More cartridges than the robot can juggle can be used. For this case see the HOWTO.FAQ.DO-DONT Q6.

Cartridge Sets

You can devide your cartridges into sets for several purposes. A set comprises of several cartridges, e.g. the cartridges number 1-3. There may be cartridges, that are not in any set and a cartridge must not be in more than one set. This would be a configuration error. Nonetheless if you have a robot, you must tell the backup system the number of cartridges, the robot is handling. The configuration parameter is named Cartridge-Sets. If you do not configure sets of cartridges, all of the available cartridges will be used as the one and only existing set. Which set to use, can be configured in the client side configuration file or overriding this setting by using commandline options (-S). Attempts of answers to the question, why to use sets, can be found in the HOWTO.FAQ.DO-DONT Q4.

Storage units

This is a term i use for the combination of a backup server's hostname, the port number to connect and a valid cartridge set number on this server. Several storage units can be configured on each client. When starting a backup, the client side first checks the servers for availability one after the other and starts the backup to the first available one found. Running a restore the user or administrator does not have to remember, to which storage unit the backup went. This is all handled transparently. For each of the three clientside configuration parameters (server hostname, port number and cartridge set) several entries are allowed now. They are associated by position. For further information, see the HOWTO.FAQ.DO-DONT HOWTO Q9.

Read and Write

The read position can be requested by clients without any restriction, assumed, the client is granted access to the tape in question. With the write position it is different. The server by default continues to write at the position, where the most recent write operation ended. This handling is maintained individually for each cartridge set. For each set the writing position is stored. Writing position means not only the cartridge used before, but also the number of the file on tape. Thus not another tape is started for each and every backup. The default behaviour can be changed to variable append mode. In this mode any tape of the right cartridge set with remaining space and not in read-only state can be supplied to the streamer device and be accepted for writing. If the tape is containing data needed by clients for restore, the next data is appended, otherwise the tape is reused and completely overwritten. The data stream is written to the tape in pieces (tape files) of a configurable maximum size. This is done to find the position faster, where certain data has been stored. The client knows for every file, on which cartridge the file is and in which tape file it can be found. Thus a restore of certain requested files is much faster than starting one only file for each backup in the worst case using up a whole cartridge.

Locking

The server process generates a lockfile when starting to prevent several server processes to access the same tape drive. Nontheless if several processes start, the one started later waits a confi- gurable time for the device to become available sending email after configurable intervals. Another lock is set for access to a possibly used autochanger.