We are in the process of moving our HIS servers from Windows 2000/2003 to Ubuntu Linux 9.10 (Karmic Koala). We are using IBM servers of the xSeries (with a little bit of variation here and there) -probably about 8-10 servers in all. Only two of them are rack mount and one of them Running RAID5. The others are all running RAID1 (hardware) on the Adaptec AIC-7982 Dual RAID controller with the suppplied 73 + 73 Gb hot swappable SCSI drives.
This install was tried on IBM’s xSeries 226 server. Creating the array from the RAID setup utility and then installing Windows 2003 has always been easy and I expected it to be easier on Ubuntu server as I’m now a veteran of innumerable installs of Ubuntu server using software RAID. I was excited as Ubuntu detected the RAID array by name and expected everything to fall into place as usual. However I ran into a bottleneck almost immediately.
If I let it “activate” and use the detected RAID array, the installer would loop when installing the grub boot loader. If I tried reinstalling the boot loader, it brought me back to the install menu so I could retry the Grub installation. If I clicked skip or cancel, the installation would complete cleanly and leave me with an unbootable server. If I chose not to allow it to use the detected array, it would see through it and present me with two discrete hard disks. It seems to allow over riding the array setup done using the controller’s own software. Setting up software RAID was out of question. Why pay the performance penalty that software RAID levies on the system when you have already paid IBM for hardware RAID? On the assumption that probably the Grub bootloader was unaware of the ext4 file system, I created a separate /boot partition and formatted it with the ext2 file system. After about 8 tries at reinstalling and checking and rechecking that the array was bootable, and whether I had set the appropriate partition as bootable, I finally gave up. Early next morning, after an extended hot shower and 3 cups of coffee I began plotting against Ubuntu. I was foxed all along as I had not expected this issue as on the same server, Windows 2003 always saw a single hard disk.
[ 2nd March 2010: Did a Windows 2003 Server install today and noticed that Windows 2003 loads a driver called IBM ServeRAID... That probably explains it... ]
Then I remembered that our systems were hot swappable and removing and replacing a drive caused the array to be rebuilt automatically. I don’t know whether this is true for all servers or whether this might work with some modification to for other levels of RAID but this is what worked for me. Life is full of workarounds. Sigh!
[ 6th December 2010: This workaround will only work for a RAID 1 configuration]
As your server boots up, press ctrl A or the appropriate key combination to enter the inbuilt RAID controller configuration tool. Create your RAID 1 array.
It took the server two hours to build the array. Reboot the server and break the array by pulling out one of the drives (you might need to open the side/panel and/or unlock the drives). If your drives are not hot swappable, turn off the power and disconnect the power to one of them. Pop in the Ubuntu server CD, complete the installation and reboot the system. Login and shut down the server.
Replace the hard disk you removed/disconnected and enter the RAID controller utility by pressing ctrl A or whatever you do to get there on your server. It should report your array as degraded. Press the appropriate key to rebuild the array and loaf around for another two hours. Reboot and you’re done! Congratulations 
Your array is now degraded
I am not sure of why it wouldn’t rebuild automatically. Maybe that feature was broken as the server is quite old or it needed to be activated by installing IBM’s/Adaptec’s Windows based RAID configuration utility or from the bootable serveRAID CD. However I shall find out in the coming months and add it to this post as I have 8 more servers to go!
Rebuild the RAID array after Ubuntu installs and you're good to go!
This issue might crop up on other servers too. This probably happens because Ubuntu does not have the driver for the RAID controller and thus sees the individual hard disks instead of the single array.
Notes:
[13th May 2010] Had to upgrade from Karmic to Lucid today. Ran into the same problem at the end of the upgrade… Grub saw through the array and asked me which disk I wanted to install the boot loader to.. in a hurry I randomly chose a disk as this server was now live and not experimental as before and had users and patients waiting for the upgrade to finish. The safe option would have been to disconnect the second drive, finish the upgrade, push back the disk and mirror the disks again. I hope Canonical is doing something to fix this issue? I have a server on RAID 5 in Landour, Mussoorie to convert to Ubuntu before June 15th and I’m already worrying… Hopefully Lucid has some new magic…
[18th June 2010] Finally got the chance to install Lucid LTS on our server in Mussoorie. The server is a higher version than the one on which the post was written on. It is an IBM x3400 tower server with an Adaptec SAS RAID controller. Besides it has 3 - 134GB hot swappable hard drives setup in RAID5. Installing Lucid was pretty straightforward with no issues. It saw the RAID5 array as a single disk and the install completed smoothly. It is difficult to figure out whether Lucid fixed the issue (but the upgrade in the above case failed too) or more likely Ubuntu probably does not have any issues with this version of Adaptec’s RAID controller. 8 more servers to go by December so I will keep you guys posted -all the rest are RAID1.
[21st Jan 2011] In all the problems that we faced with our poorly written software, I came back to Delhi without realizing that the server was running on a broken array. I was also scared to do it over the phone with the IT support guys on site as if the wrong disk was referenced to complete the mirror, I’d end up with a nightmare on my hands. It had to be fixed however so yesterday evening over a skype video call, we successfully broke and rebuilt the array.
We noticed something interesting during the rebuild. While it had taken about 2 hours when we first mirrored the array (about 8 months back) It now took 4 1/2 hours to reach around 40%. We also observed that it was not really necessary to wait for the complete rebuild. We could start off the rebuild and immediately abort the rebuild and then reboot the server into normal operating mode. The rebuilding continued in the background while the server became available for use. Adaptec’s RAID controller also gave us the option of choosing the drive which contained the current data to be used as the master (the disk with the updated data). Of course making a mistake here and choosing the wrong drive would have set us back by about 8 months data. I had also taken backups of all the databases and applications before the rebuild.
[23rd August 2011] Bought a new IBM x3650 M3 -7945IES rack server for our Central Office in Delhi. We bought a ServeRAID M5015 SAS/SATA RAID controller for it. Our server has 12 2.5″ SAS drives connected to it. Two in RAID1 (Running VMware ESXi 4.1) and the rest in RAID 5. No problems here and Ubuntu saw the virtual drives as they were supposed to be seen. This brings me to the conclusion that Ubuntu didn’t have the driver required for the Adaptec card and was hence seeing discrete disks instead of a single virtual drive or RAID array.
If you would like to make a comment, please fill out the form below.
