Installing a bootable software SATA Raid

Why the trouble? Let me count the ways: 1) the on-board SATA RAID controllers are not "true" RAID controllers - they do not present the disk array as a single device. The controllers simply offer a hardware assist to the software RAID. (This was very confusing at first.) 2) FC3 doesn't complete the RAID installation. RAID install must be completed manually before the GRUB install can be completed. 3) the GRUB installer needs a bit of help ensuring that both drives in the RAID array are bootable.

Hardware: AMD64 3200+, ASUS Kv8 SE Deluxe mother board. The Kv8SE comes with two on-board SATA RAID controllers: Promise PDC20378 and a VIA VT8237. I was installing four drives, two WD360GD on the VIA controller, and two WD160 7200 rpm drives on the Promise controller. Both arrays were constructed as RAID1 (mirrors) using the BIOS utilities.

Goals: bootable RAID1 drives, remaining disk space set up as LVM on RAID1.

Overall approach - Here are the high-level steps to get the beast running smoothly:

1) Install FC3, creating bootable RAID and second RAID LVM group,
2) Partially complete GRUB install,
3) Complete RAID install,
4) Complete GRUB install,
5) Test, test, and more test

DETAILS:
--------

1) INSTALL FC3, CREATING BOOTABLE RAID AND SECOND RAID LVM GROUP. Keys to success: both drive pairs must be partitioned exactly the same, with the appropriate partitions matched into RAID1 arrays. On the primary bootable array, I reserved 100MB for use with md0 (on both drives), 2GB for swap (both drives), and allocated the remaining space as md1. On the second drive pair, I allocated all the space to md2. I then mapped /boot to md0, and joined md1 and md2 into a LVM volume group, and mapped / to the new volume group.

At completion of the install, I tried a reboot, and the machine froze at GRUB.

2) PARTIALLY COMPLETE GRUB INSTALL - keys to success: boot from your CD using "linux rescue" option to get a command prompt. Change root to the newly created system (>> chroot /mnt/sysimage), and run grub. Below are the commands I used (you would substitute for your first SATA hard disk - mine was /dev/sda)

grub
> device (hd0) /dev/sda
> root (hd0,0)
> setup (hd0)

These commands make the first hard disk of your RAID array bootable. However, the second can't be made bootable until RAID is up and running properly. See step 3.

3) COMPLETE RAID INSTALL - Keys to success: boot the new hard disk, and check to see that your RAID arrays are working properly, using > cat /proc/mdstat.

[root@jigsaw ~]# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb2[1] sda2[0]
33945280 blocks [2/2] [UU]

md2 : active raid1 sdc1[0] sdd1[1]
156288256 blocks [2/2] [UU]

md0 : active raid1 sdb1[0] sda1[1]
104320 blocks [2/2] [UU]

This shows a properly working array. Note md0 - both partitions are listed. My first time through, I found that I was missing the other half of my RAID pair. SO - I needed to add them "hot" and rebuild the RAID. Here is the trick:

>> mdadm /dev/md0 --add /dev/sdb1

This will "hot add" /dev/sdb1 back into /dev/mdO and rebuild the RAID pair. For larger drives, this syncing make take a while. You can check the status with "cat /proc/mdstat"

4) COMPLETE THE GRUB INSTALL - Keys to success: once the /boot partition is properly synced (see step 3), then you can tell GRUB to boot from either disk. Let's repeat step two, activating the MBR for the first partition on each drive of the RAID array on /boot.

grub
> device (hd0) /dev/sda
> root (hd0,0)
> setup (hd0)
> device (hd1) /dev/sdb
> root (hd1,0)
> setup (hd1)

If you try this without completing the RAID install, you'll get "ERROR 15: file not found". Why? the /boot partition isn't properly mirrored across the disks.

5) TEST, TEST, and TEST - Keys to success: patience. a) power down, b) unplug a drive, c) power up and watch the bios, d) ensure you get a boot, e) check to see if all data is accessible, f) power down, g) replug the drive, h) power up and watch the bios, i) rebuild the array using "mdadm /dev/mdx --add /dev/sdyy"