MicroBackup - short overview

This is backup manager utility for Linux/Unix, written on Perl. It makes backups of various data using external programs: find, egrep and archiver.

Why MicroBackup is better for you?

The main goal for creating "yet another program..." are features missing in existing backup managers:

So, first, source files are selected not only by names (and other properties supported by find), but by content wildcards used by egrep.

My sample uses this feature for archiving files in /etc directory containing label of any member of wheel group only, i.e. only files which contents is manually changed after installation (and is accurately commented by editor!) are selected.

This feature causes backup will be significanly smaller. The another goal is preventing conflicts at restore backup stage between unchanged configuration files stored in backup, and upgraded versions of them.

Another important criteria is that file is not found in any package, or modification time for actual file on disk and file record in package is mismatched. rpmuncovered does this checking. MicroBackup contains this utility with appropriate policy sample for archiving /etc directory. This policy attaches rpmuncovered to find output using add_filter option.

Your own filters are allowed. For example, lot of /etc files is not included by any RPM, but is generated at installation time. We don't need backup them. Well, exclude them:

find_point  /etc
add_filter  rpmuncovered -
add_filter  egrep -vf /etc/microbackup/ignore_etc_files

Second, MicroBackup can partially delete outdated backups. Consider we need to track system state with daily precision for last week, weekly - for last month, monthly - for last year, and then with yearly precision. Wasted excrescent backups are removed by MicroBackup when stay outdated. Take look to following policy example:

backup_type  snapshot
snapshot_policy  1 30  7 48  60 999
archive_name  /var/mbackup/something/some
find_point  /etc
This means:

Additionally to snapshot mode (may be, will be better to term it as "exhaustic"?), two modes are provided: plain for unconditional archiving all files and incremental for archiving files with modification time newer than modification time of latest found backup. In both modes old backups are not removed and keep unchanged.

Do not store extraneous files with path matching archive_name -- MicroBackup treates them as backups!!! Do not change archives after they are created!!! Both situations causes errors in incremental (when next archive is created, lot of source files may be older than one of mistakely changed archives or extraneous files and will be ignored) and snapshot mode (mistakely changed archive or extraneous file will be matched in strange time interval and will be removed or mistakely inspires removing of another archive).

Third, MicroBackup is NOT replacement of another good-working tools. List of source files is created by find, egrep and additional filters. The next step is archivation. You may select any archivation way you need. For example, archivation may be specific at stage of reading source files as for Samba tdb-files using tdbdump, or may be specific at stage of writing backup (copy to another host using SFTP, signing by PGP, compressing, etc.).

As good example of external archiver, MicroBackup contains small diffbackup utility, that produces line-by-line difference report between actual files and last archived tree, then replaces last archived tree by actual for comparison at next time. In appropriated policies diffbackup is used for tracking of /etc directory, for tracking list of installed RPM packages and for tracking list of [/usr]/[s]bin files.

Before and after archiving may be needed to execute some additional commands, for example, stop and restart service. For this purpose policy can contain preexec postexec options. Each may be given multiply times.

In my samples preexec is used for creating temporary file with list of wheel members (then passed to egrep), or list of installed RPM packages, or list of [/usr]/[s]bin files (passed to diffbackup). In postexec temporary files are removed.

All configuration settings are available in subcommands as environment variables, for example:

postexec /usr/bin/pnupg --sign ${backup_name}${backup_suffix}

Subcommand started by "@" is executed as Perl instruction in context of caller module:

postexec @chdir("/tmp")
postexec echo $PWD

Fourth, all configuration settings are collected to simple for editing configuration files -- policies. Each policy contains now about 20 settings, almost of them are trivial. Fully commented sample named microbackup_policy.full_sample is included to package. Use compact sample microbackup_policy.mini_sample as template for your own policies.

Now, MicroBackup receives policy filename in command line. Support of global policy file that contains default values for most options is planned. Options passing directly in command line is planned too.

This package contains script for running daily be cron. This script executes all policies stored in /etc/microbackup/policy directory.

Security

By default, archives are created with permissions 750. Policy and global configuration file have umask parameter (by default is 027). For additional security, you may change permissions of directory where backups are stored.

Configuration directory /etc/microbackup has permissions 750 root:root. Backups directory /var/microbackup has permissions 2750 root:root (2750 = rwxr-s---), and directory /var/microbackup/saveroot used by diffbackup for storing last filesystem state, has permissions 750 root:root.

Be careful, when set backups permissions! Liberal permissions opens backdoor to your passwords! This is true both for backups and policies because backup contain /etc files and policy can contain passwords for connecting to network resources etc.

Distribution

Distribution license is GNU GPL.

Now only RPM package format is supported, and only for Alt Linux Master, Mandrake clone. Porting to another platforms is welcomed.

What needed for running?

  1. Cron -- calls daily script for executing all policies found in /etc/microbackup/policy.
  2. Perl -- is language on which MicroBackup and rpmuncovered are written. Additional modules POSIX::strftime and perl-RPM are required. Of course, both are already in Alt Linux.
  3. Bash -- is language on which all other utilities are written: group_members for printing members of given group in egrep "(value|value|...)" format (uses getent command), and DiffBackup (uses diff ;-)
  4. find and egrep -- for creating list of files to be placed to backup. egrep will be used only when policy contains grep_mask option.
  5. Any archiver. By default, zip.

Of course, this list is needed for manual overview only, not for installation: all checking at installation time will be automated by RPM+APT ;-)

Debugging and testing

Most of my scripting stuff supports -d switch. When -d given, script produces lot of tracing messages, and displays critical commands (files deletion, running commands, etc.) instead of executing them. Note: "-d" must be first option in command line!

Additionally, main microbackup module supports --msglevel nn switch, where nn is detaility level between 0 (quiet) and 100 (verbose).

Bugs & Todo's

Author!

Хостинг от uCoz