Advertisements

Ubuntu 9.10 Partition Backup: ext4 vs partimage vs dd

Ubuntu 9.10 uses ext4 filesystems by default and it’s usually a Good Thing to not mess with defaults early in the installation.

That precept worked up to the point where I wanted to make a full-partition backup before doing something potentially catastrophic… at which point I discovered that the current version of System Rescue CD has a version of partimage that doesn’t know about ext4 filesystems. While FSArchiver looks promising, all I really wanted was a quick-and-simple backup and (possibly) restoration.

So. Once again, dd to the rescue.

This being a fresh installation, there’s not much other data to contend with. In fact, the installation uses under 3 GB, but it’s in a 32 GB partition. Wouldn’t It Would Be Nice If we could back up just the 10% of useful data and skip the rest?

Reboot to System Rescue CD (hereinafter, SRC) and, while that’s happening, plug in a spare hard drive using a USB-to-SATA converter. That will be the “backup drive”.

Mount the backup drive (which appeared as /dev/sdf1 and will be different for you) at /mnt/backup, which is conveniently provided by SRC:

mount /dev/sdf1 /mnt/backup

Mount the partition to be backed up (which is /dev/sda8 and will be different for you) at /mnt/custom, another existing mount point.

mount /dev/sda8 /mnt/custom

Zero out all the unused space by creating one honkin’ big file, then erase it leaving all those highly compressible zeros behind:

dd bs=1M if=/dev/zero of=/mnt/custom/zero.bin
sync
rm /mnt/custom/zero.bin

Unmount the partition, make the backup, and save the MBR while you’re at it:

umount /mnt/custom
dd bs=1M if=/dev/sda1 | gzip -c -3 > /mnt/backup/boxname-sda8-ext4-32GB.bin.gz
dd bs=512 count=1 if=/dev/sda of=/mnt/backup/boxname-mbr.bin

The bs=1M option sets a decent blocksize for the read operations, which gets dd trundling along at a pretty good clip by reducing the per-read overhead.

The -c option tells gzip to pipe the output to stdout and not mess with the input file. The -3 says to not waste a lot of time trying to compress the data; much of the partition consists of raw binary executables, so there’s no point. The whole process will be limited by disk I/O speed, most likely.

As it happened, the partition squeezed down into about 1.8 GB worth of gzipped backup file.

Unmount the backup drive, reboot, and do risky things…

As it turned out, I actually had to restore the partition.

Once again, boot into SRC with the backup drive plugged in, mount the backup drive. Restoration is straightforward:

gunzip -c /mnt/backup/boxname-sda8-ext4-32GB.bin.gz | dd bs=1M of=/dev/sda8

Warning: if you bungle the target of that dd, you are so screwed.

You’re both saving and restoring from a specific partition (/dev/sda8, in my case) within the drive, not the whole drive (which would be /dev/sda). Pay attention to what’s on the screen, check twice, and have a full-drive backup lurking in your fireproof safe…

Advertisements
  1. #1 by Travelinrob on 2010-06-17 - 12:23

    Thank you for your tutorial. It is appreciated.

    • #2 by Ed on 2010-06-17 - 15:40

      Glad to help… turns out it’s one of the more popular posts.

      When partimage includes ext4 support, I’ll be glad to have it become obsolete!

  2. #3 by me on 2010-06-19 - 00:13

    dd bs=1M if=/dev/sda1 | gzip -c -3 > /mnt/backup/boxname-sda8-ext4-32GB.bin.gz

    this whole command is pretty redundant.
    -c is gzip’s default mode when no file arguments are given!
    also, that you need dd to read raw disk devices (on linux at least) it’s a common misconception that i will never understand.

    you can do the whole thing as easy as:
    gzip /mnt/backup/boxname-sda8-ext4-32GB.bin.gz

    greetings from your hardcore linux user/admin…

    [Edit: I took the liberty of moving your follow-up note here and wrapping it in a source code structure to make it more obvious.]

    wordpress removed the redirection from my command…
    it should look more like:

    gzip </dev/sda1 >/mnt/backup/boxname-sda8-ext4-32GB.bin.gz
    
    • #4 by Ed on 2010-06-19 - 07:11

      As the saying goes: “Let us not say we have found the truth. Rather, let us say we have found a truth.”

      I know that dd uses a default 512-byte blocksize. Does a read-from-device redirection default to something sensible?

      There’s always something new to learn… thanks for the explanation!

    • #5 by Travelinrob on 2010-06-19 - 13:42

      So… Does this mean that you do not need to use dd at all? Also, I’m confused on the redirection. Is there a supposed to be a space before the greater than sign? Can you please explain the command?

      Thank you.

      • #6 by Ed on 2010-06-19 - 14:28

        It makes use of the “in Unix everything’s a file” principle: redirect standard input from the raw disk partition, run it through gzip, and redirect standard output to the compressed file. I’d include the -3 compression control, but that’s in the nature of fine tuning.

        Using dd gives control over the blocksize read from the drive, but probably doesn’t actually add anything else to the process; that’s what I just learned.

        Everything’s space-delimited, as usual for command-line stuff; the command in the source-code block at the bottom of that comment is what you’re looking for. WordPress munched on the previous line, at “as easy as”.

  3. #7 by me on 2010-06-21 - 08:31

    you could have just corrected the command in the first post and cut the reply text alltogether. :)

    i checked (with strace, on my system) – gzip reads 32k blocks.
    that should be OK for the disk, and it’s what it would read from a pipe anyway,
    and in any case, removing the extra pipe also improves performance. (a pipe is not just syntactical glue – it means all the data has to be passed from dd through the kernel to gzip, likely copying it a few times on the way)

    the restoring part can be done the same way, just have gunzip write directly to the device.

    also, redirections in the shell don’t need space delimiters, you basically only need spaces after command names (including shell keywords) and between arguments, any others are just for readability.
    (for extra fun, redirections need not go at the end, you can put them anywhere, even before the actual command)

    • #8 by Ed on 2010-06-21 - 09:26

      just corrected the command in the first post

      The next thing you know, I’d be rewriting entire comments: there’s no urge more powerful than the urge to meddle with someone else’s text!

      gzip reads 32k blocks

      I did some crude tests a while ago that put the knee of the curve somewhat higher, but any block size over 512 bytes was a vast improvement. Given that I was writing the compressed data back into a file on a different partition on the same drive (the worst choice!), the whole affair was painfully I/O bound.

      I think the kernel does some fancy caching to eliminate copying during piping, but for an I/O bound process that doesn’t hit the page file it makes no difference.

      In any event, omitting the whole dd thing is a net win.

      you basically only need spaces after command names … and between arguments

      Olde Fartes such as I need all the help we can get and, verily, sprinkling a few extra spaces here and there won’t do my wrists any further damage! [grin]

      Oh: my daughter wonders if you know Enoch Root?

  4. #9 by Stephan on 2011-02-08 - 17:51

    Handy How-To. Rather than do a ‘zero’ file, you can also set about using gparted (from a bootable live distro) to shrink the partitions you want to back up, and then pipe the remaining data through gzip as you suggested earlier. The only data easier to compress than a 0 is -no- data ;) Of course, you run a slight risk of losing said data in the resize, so your circumstances and mileage may vary.

    • #10 by Ed on 2011-02-08 - 19:41

      I try really hard to avoid doing anything involving partition manipulation before a backup.

      Most of the time, of course, it works perfectly and life is good.

      Once in a while, though, the process goes toes-up and there’s no way to undo your way back to a working system… which is not what you want to have happen immediately before a backup!

  5. #11 by m on 2011-12-18 - 06:06

    OK, I backed up my patition with this

    gzip >/media/BACKUP/Bak/w7.n.bin.gz </dev/sda2

    can you pls confirm my perception that at a later stage

    gunzip >/dev/sda2 </media/BACKUP/Bak/w7.n.bin.gz

    will work for me to restore everything in place?

    (Sorry, but as you can see beginners also may feel bad need for backing up a full partition… and also to restore)

    thank you

    [Ed: I took the liberty of inserting the redirection markers where I think you had them and putting the commands in code blocks.

    • #12 by Ed on 2011-12-18 - 07:13

      That should work, as long as you’re restoring to the same partition; the filesystem will fit into the original space. If you have a different partition (perhaps on a new drive), then the filesystem won’t expand or shrink to match the new space available.

      I recommend using partimage for Windows partitions, because it automagically handles data compression, cuts the output file into 2 GB chunks, and generally knows what it’s doing. It seems ext4 is the only common filesystem it doesn’t handle…

  1. Backup partición Ubuntu con dd | Blog de Raul DOE-UPV