CIT 470 Advanced Network and System Administration: Lab #9: System Recovery

Lab #9: System Recovery
Date: March 3, 2008

Follow the instructions for this lab very carefully, and review your commands before you hit ENTER. Small typing mistakes with the commands in this lab can destroy your entire hard disk, forcing you to reinstall your operating system before being able to continue.

Disk Repair with fsck

While modern journaling filesystems like ext3 prevent filesystem corruption from a sudden crash or power loss, there are other sources of disk corruption. It's still important to understand how to use fsck to resolve these issues. In order to make creating a corrupt filesystem simple, we'll create a new ext2 partition and suddenly power off the computer.

Create a new ext2 filesystem.

fdisk /dev/sda                  # Create a new 2G partition
mke2fs -v /dev/sdaX             # Make a filesystem on new partition #X
mkdir /ext2                     # Create new mount point
vim /etc/fstab                  # Add ext2 partition to filesystem table
mount /ext2                     # Mount your new partition

Run the following loop to copy files, so there is some data on the partition.

for i in 0 1 2 3 4 5 6 7 8 9
do
    mkdir /ext2/etc$i
    cp -a /etc /ext2/etc$i &
done

Record a list of all files on the disk for future comparison.
```
ls -lR > /tmp/ls-lR
```
Destroy a small part of the filesystem metadata of the filesystem you created above by overwriting it with zeros.
```
dd if=/dev/zero of=/dev/sdaX bs=512 seek=12 count=8
```
Umount the filesystem and run a filesystem check on it.
```
umount /ext2
e2fsck -v /dev/sdaX
```
Mount the filesystem again and record a list of all files on the disk after the repairs.
```
mount /ext2
ls -lR > /tmp/ls-lR.fsck
```
Compare the two lists of files to verify that no filenames or other file metadata has changed.
```
diff /tmp/ls-lR /tmp/ls-lR.fsck
```

MBR Recovery

It can be extremely frustrating when an event renders your computer unbootable. One common source of this problem is overwriting of the master boot record (MBR), which happens when you install Microsoft Windows. That particular problem can be fixed by re-installing the GRUB bootloader.

However, re-installing GRUB won't help you if you've lost the partition table too. However, all hope is not lost. After all, you followed my directions and backed up everything, so you have a copy of the MBR of all your computers, right? We'll examine that scenario, as well as what options remain if you haven't backed up your MBR.

Backup the MBR. You should always do this for every system you have, though it's best to store the backup someplace other than your current hard disk. You should store an extra copy of the MBR on your kosh account to ensure that you can recover it.
```
dd if=/dev/sda of=/mbr bs=512 count=1
sftp username@kosh.nku.edu
```
Destroy the MBR. You should never do this, but this is a recovery lab.
```
dd if=/dev/zero of=/dev/sda bs=512 count=1
```
Reboot and verify that the system won't boot.
```
reboot
```
Boot system with Kanotix live CD.
```
<Select> acpi - dma - English
unionfs
```
Start a terminal and become root.
```
su
```
Verify that the partition table is corrupt.
```
fdisk -l /dev/sda
```
Guess the partitions. Let's assume you haven't backed up the MBR and see what you could do. Manual guessing won't help, but Kanotix comes with the gpart program for automatic guessing. Read the manual before using gpart. Note that it doesn't always work and has known problems with some common configurations, like extended partitions. The argument /dev/sda is repeated twice in the command below: the first time after the -W indicates where to write the partition table, the second time indicates which disk to search the partition table for.
```
man gpart
gpart -W /dev/sda /dev/sda
```
If gpart fails to retrieve your partition table, try to recreate the partition table manually for just the root partition (which is where the MBR backup is stored.)
```
fdisk /dev/sda
```
Mount your root partition.
```
mount /media/sda1
```
Restore the MBR. If you cannot find the MBR file, use sftp to retrieve your saved MBR from kosh.
```
dd if=/media/sda1/mbr of=/dev/sda bs=512 count=1
```
Reboot and verify that the system will boot again.
```
reboot
```

Hard Disks

In many situations, you need to recover data from a non-booting computer. This part of the lab will show you how to access both standard disk partitions and logical volumes using your live CD.

Boot system with Kanotix live CD.
```
<Select> acpi - dma - English
unionfs
```
Start a terminal and become root.
```
su
```
View list of partitions that Kanotix found.
```
cat /etc/fstab
```

Mount and view regular volumes.

mount /media/sda1
ls -l /media/sda1
mount /media/sda2
ls -l /media/sda2
mount /media/sda3
ls -l /media/sda3
df -k

Scan for logical volumes.
```
vgscan
```
Activate all logical volumes.
```
vgchange -a y
ls -l /dev/cit470
```

Mount and view logical volume.

mkdir /users
mount /dev/cit470/users /users
ls -l /users
df -k