Mageia forum

I have seen many complaints about problems with the Sep 1/2 KDE upgrade. Well, fact is that sometimes things go wrong with upgrades, and this was a big one. Now, I myself had some trouble with this upgrade, but not to nearly the same extent as some people are reporting; I solved my problems with a bit of work (and a lot of grumbling). However, even if I had needed to roll back, I could have rolled back to the previous system with no difficulty and only a few minutes work.

Basically, once a day at around 4AM my system does a backup of the important stuff to other hard drives on my computer. I back up / and I back up /home separately to different drives. In the current situation, had I not been able to fix my KDE installation I would have rolled back using the backup of /.

This backup mechanism is not original with me; I found it on the net probably a decade ago. But I have used it ever since, and I have set up many other systems with this scheme. It works very well.

Essentially, I have two scripts in /root that are run as cron jobs. One script, called dosysbackup, does the nightly backup. The other script, called doweeklysysbackup runs once a week on saturday night.

Here is dosysbackup:

Code: Select all

rm -rf /mnt/sda6/back/backup6
mv /mnt/sda6/back/backup5 /mnt/sda6/back/backup6
mv /mnt/sda6/back/backup4 /mnt/sda6/back/backup5
mv /mnt/sda6/back/backup3 /mnt/sda6/back/backup4
mv /mnt/sda6/back/backup2 /mnt/sda6/back/backup3
mv /mnt/sda6/back/backup1 /mnt/sda6/back/backup2
cp -al /mnt/sda6/back/backup0 /mnt/sda6/back/backup1
rsync -a --delete --exclude /dev --exclude /tmp --exclude /home --exclude /media --exclude /proc --exclude /sys --exclude /mnt --exclude /var/tmp  --exclude /var/cache  / /mnt/sda6/back/backup0

Here is what happens. This backup resides on /mnt/sda6/backup on my machine. Within backup, there are the subdirectories backup0 through backup6 and the subdirectory weeklybackups.

When dosysbackup runs, the first thing it does is delete the directory backup6. Poof...gone. Then it renames backup5 as backup6, backup4 as backup5, and so forth. Finally, it copies using hard links the contents of backup0 to backup1. Since this copy is done using hard links, the actual files are not copied, only a new directory structure is created which points to the same files.

Finally, the script runs rsync on / with the indicated exclusions and with the flag --delete set, comparing the contents of / with the contents of backup0 and updating backup0 with only the changes that have occurred since the last backup.

Now, once a week, a cron job runs the script doweeklysysbackup, before doweeklybackup runs. Here is that script

Code: Select all

#!/bin/bash
dirname=`date`;
mv /mnt/sda6/back/backup6 /mnt/sda6/back/weeklybackups/"$dirname"

This script merely moves backup6 into the weeklybackups subdirectory with a directory name that is derived from today's date. Now, backup 6 is one week old when this happens, so to get the actual date of the backup, I need to subtract one week. I was too lazy to do that in the bash script, since I know what I have.

So I wind up with a daily backup going back one week, and a weekly backup going back as far as I choose to let it remain on the drive - usually 6 months. A couple times a year, I take the most recent weekly backup, tar/gzip it, and archive it on a backup drive. Then I delete the weekly backups.

I do the same thing with /home, but I do it to a different drive.

Now, had my KDE upgrade been hosed, forcing me to roll back, I could have rolled back by replacing all of /usr with last night's backup, like this:

(as root)
rsync -a --delete /mnt/sda6/backup/backup0/usr/ /usr

Simple, and saves a LOT of aggravation when something gets fouled up.

For laptops, you do this backup to a USB drive...which, of course, you have to plug into the laptop. Even if you don't do this daily, you can do it from time to time and you should always do it before accepting an upgrade. If you only have one drive, just set up a backup directory on the same drive. In this event, it isn't a true backup, just a duplicate copy you can use to get you out of trouble.

Thanks jiml8. This sounds almost exactly like rsnapshot which is also packaged for mga.
It saved me many times already. Almost always from my fault .

Interesting. I never heard of rsnapshot.

Like I said, I found this on the net a long time ago and have used it ever since. Best backup mechanism I have ever used.

I guess others thought it was a good idea too.

After looking around for a long while this still appears to be one of the simplest and cleanest approaches.

My question is, instead of using cp to make a copy of the rsync'ed file system, would using tar work, so that (in your example) /mnt/back/backup1 is actually a compressed tarball? Or is there a better approach?

Could be made to work, but then you can't directly sync the contents back with rsync. If you want to save some space, you can surely do that - but you will exchange it for longer compression and extraction times, so recovery will take longer and be slightly more complex (in the script).

While it could be made to work, it would take A LOT more space.

Remember that the copy from backup0 to backup1 is done using hard links. This means that only the directory structure is copied; the contents of the directories are the exact same locations on the hard drive. Thus, the copy increases usage on the disk by very little...only enough to store the new directory structure. Then, after rsync runs, any file that was not changed remains available in that one location on the hard drive, linked to by as many different directory structures as are relevant. Only the changed files have new entries on the hard drive.

If you tar/gzip it, you wind up creating a new file that is the tgz file, and it will take its own space on the disk. Further, given that there will likely be small changes in / (and in /home) from day to day, each time the backup is run, you'll wind up with a new and different tgz file - each occupying as much drive space as it needs.

So, not only will you increase your scripting complexity and your time to backup/restore, you'll massively increase your space utilization on your backup hard drive.

Sorry, overlooked the cp -al

Now that removable devices get mounted under /run/media, where should I look at excluding from? Should we just exclude the entire run directory, or are there things in there that we should backup also?

I cannot believe it has been 4 years since I started this thread.

I still use the same backup mechanism but it has evolved. Specifically, I now have a NAS up and running, and I have created a 9 TB iscsi share on that NAS which I mount on my workstation. I now back up to both a hard drive on the workstation, and to that share on the NAS if it is available (it is always supposed to be available, but being a remote device, I have to check for it).

I also have been responding to a growing threat environment by considering whether or not my workstation is under attack by ransomware or other malware at the time that I am doing the backup. I don't want to run a backup if the system is currently under attack, which is as yet undetected by me. To do this, I have placed a small file called okay.txt in the root of every partition (including that iscsi share on the NAS) and before backing up I check these files on both the NAS and the local backup drive to make sure they are unchanged. If either of them are changed, there is no backup. Also, the backup directories require root permission to enter them. I don't expect this putative malware to have root permission, and if it does, I am basically screwed anyway.

So, here is the current version of my "dosysbackup" script:

Code: Select all

#!/bin/bash
fileOK=1
mounted=0
mnts=`mount -t ext4`
if [ -n "$mnts" ]; then
    for mnt in $mnts
    do
        if [ "$mnt" == "/mnt/NAS" ]; then
            mounted=1;
            break;
        fi
    done
fi
for usefile in /mnt/sda1/okay.txt /okay.txt
do
  if [ -r $usefile ]; then
    while IFS='' read -r line || [[ -n "$line" ]]; do
      if [ "$line" != "file is OK" ]; then
        fileOK=0;
      fi
    done < "$usefile";
  else
    kdialog --msgbox "Dosysbackup: File $usefile not found or not readable."
    fileOK=0;
  fi
done
if [ "$mounted" -eq 1 ]; then
    if [ -r "/mnt/NAS/okay.txt" ]; then
        while IFS='' read -r line || [[ -n "$line" ]]; do
            if [ "$line" != "file is OK" ]; then
                fileOK=0;
            fi
        done < "/mnt/NAS/okay.txt";
    else
        kdialog --msgbox "Dosysbackup: File /mnt/NAS/okay.txt not found or not readable."
        fileOK=0;
    fi
fi
if [ "$fileOK" -eq 1 ]; then
    if [ "$mounted" -eq 1 ]; then
        rm -rf /mnt/NAS/back/backup6
        mv /mnt/NAS/back/backup5 /mnt/NAS/back/backup6
        mv /mnt/NAS/back/backup4 /mnt/NAS/back/backup5
        mv /mnt/NAS/back/backup3 /mnt/NAS/back/backup4
        mv /mnt/NAS/back/backup2 /mnt/NAS/back/backup3
        mv /mnt/NAS/back/backup1 /mnt/NAS/back/backup2
        cp -al /mnt/NAS/back/backup0 /mnt/NAS/back/backup1
        rsync -a --delete --exclude=/dev --exclude=/run --exclude=/tmp --exclude=/home --exclude=/media --exclude=/proc --exclude=/root/final --exclude=/sys --exclude=/mnt --exclude=/var  / /mnt/NAS/back/backup0
        rsync -a --exclude=/var/log --exclude=/var/tmp --exclude=/var/cache /mnt/sdc5/var /mnt/NAS/back/backup0
    fi

    rm -rf /mnt/sda1/back/backup6
    mv /mnt/sda1/back/backup5 /mnt/sda1/back/backup6
    mv /mnt/sda1/back/backup4 /mnt/sda1/back/backup5
    mv /mnt/sda1/back/backup3 /mnt/sda1/back/backup4
    mv /mnt/sda1/back/backup2 /mnt/sda1/back/backup3
    mv /mnt/sda1/back/backup1 /mnt/sda1/back/backup2
    cp -al /mnt/sda1/back/backup0 /mnt/sda1/back/backup1
    rsync -a --delete --exclude=/dev --exclude=/run --exclude=/tmp --exclude=/home --exclude=/media --exclude=/proc --exclude=/root/final --exclude=/sys --exclude=/mnt --exclude=/var  / /mnt/sda1/back/backup0
    rsync -a --exclude=/var/log --exclude=/var/tmp --exclude=/var/cache /mnt/sdc5/var /mnt/sda1/back/backup0
fi


Interesting changes. Thanks for the update Jim

Interesting read: http://www.mikerubel.org/computers/rsync_snapshots/

BTW I used rsnapshot years ago, via long USB cable through a hole in the concrete floor to a disk safely hidden in basement, which automatically powered up when i connected USB. However the filesystem screwed up after a while - i blame the usb connection via hub and IDE bridge...

About this procedure...

Say a file have not changed for a while, so it is in all backup folders /backup6, /backup5, /backup4 etc as hard links to the same actual content.
Nice it saves space.

Then say the next time backup runs, the file to be backed up have changed content (same name).
Don't that new file version show up in all /backup6, /backup5, etc immediately?

No.

The file in the filesystem consists of the data in a physical location on the disk, and one or more links to that physical data that is in the filesystem directory structure.

When you copy a new version of the file into a directory, that new version is given its own location on the hard drive - it does not overwrite the earlier copy. After it is written, it is then linked into the filesystem in the appropriate directory. As for the old copy, so long as there is at least one link to it (and there are all those hard links from earlier days), the space on the hard drive remains listed as occupied. If your new copy resulted in the only link to the old copy being moved to the new copy, then the space occupied by the old copy is marked as free, and depending on the filesystem and the device might be zeroed out.

So in this backup mechanism, the new version gets a new link, and the old version retains all the links from earlier backup dates.

Symlinks would work, effectively, as you described. Hard links do not.

Ah, thanks

The key is rsync always unlinks before overwriting ... as described in the link i posted a few posts up three years ago and i have now read again...

It isn't just rsync that does this; cp does it too. I would expect the filesystem would require this of pretty much any command that writes data to the filesystem. It is a safety thing; we don't unlink the old file until the new file is fully written, against the possibility that the write will fail for some reason.

jiml8 wrote:cp does it too

Before i wrote my question, i had just tested with plain cp on single files, which did not unlink.

Just laborated, modifying a test backup script similar to your first post from rsync to cp:
cp -a does unlink
cp -r not.

As always the devil is in the details...
Always test And backup (nondestructively...) (oh well that is what i try...)