Greg's Tech blog

Amahi Drive Replacement

Sunday 13 of February, 2011
<p>My <a class="wiki external" target="_blank" href="http://www.amahi.org/" rel="external nofollow" style="text-decoration: none; ">Amahi HDA</a>&#160;<a class="wikicache" target="_blank" href="http://linux2.gmartin.org/tiki/lib/fckeditor/editor/tiki-view_cache.php?url=http://www.amahi.org" style="text-decoration: none; ">(cache)</a>&#160;server&#160;was reporting two SMART drive warnings. The boot drive and one of the storage pool drives had more bad sectors than SMART preferred and I've been looking for replacement drives for several weeks. I found a pair of 1TB seagate drives at <a class="wiki external" target="_blank" href="http://newegg.com" rel="external nofollow">NewEgg</a> for $55 ea. Their free shipping cinched the deal.</p> <p>Amahi <br /> My plan was this:</p> <ul> <li>Replace the bad (sata) disk in the pool</li> <li>Move the data to one of the new disks</li> <li>Remove the second drive (sata) form the pool</li> <li>Use this good sata drive to replace the failing ATA boot drive</li> <li>Add the second new sata drive to the pool</li> </ul> <p><br /> The storage pool in Amahi is handled by <a class="wiki external" target="_blank" href="http://code.google.com/p/greyhole/" rel="external nofollow">greyhole</a> <a class="wikicache" target="_blank" href="tiki-view_cache.php?url=http://code.google.com/p/greyhole/">(cache)</a>, a truly ingenious technology similar in idea to Microsoft's now defunct Drive Extender. But, using greyhole makes swapping out a pool drive a non-trivial operation.<br /> <br /> First step in changing a drive is to get the data off the drive. The tricky part was figuring out which physical drive had the error and then which drive that represented in the pool. Fortunately, i had two different drive manufacturers in the pool. Looking at the /dev/disk/by-id info allowed me to determine it was the WD drive in my pool. From there I determined the WD drive was mounted on /var/hda/files/drives/drive4.<br /> <br /> The command for telling greyhole to move the data from this drives is:</p> <div class="code"><tt> greyhole --going=/var/hda/files/drive/drive4/gh</tt></div> <p><br /> Greyhole will tell you to wait while it goes off to move the data. &#160;I had 150GB of files on the disk. It probably ran a hour or so (I didn't time it). Once finished, I could verify with the greyhole -s command that there was no data on the disk.<br /> <br /> Next step is to remove the disk from fstab. I always choose to comment out the line by adding a leading #. That way I could put it back without issue. &#160;Of course, I failed to make the fstab change prior to shutting off and removing the disk. Fedora would not boot afterwards telling me there was a disk error and dropping me to the shell to fix it. Oddly I could not edit fstab from that shell, so I put the disk back in, booted, edited fstab and shutdown again. At this point, all the storage pool data was on a single drive<br /> <br /> Next step was to get the replacement drive in place. I followed the instructions for <a class="wiki external" target="_blank" href="http://wiki.amahi.org/index.php/Adding_a_second_hard_drive_to_your_HDA" rel="external nofollow">adding a second drive to my system</a> <a class="wikicache" target="_blank" href="tiki-view_cache.php?url=http://wiki.amahi.org/index.php/Adding_a_second_hard_drive_to_your_HDA">(cache)</a>. It comes down to partitioning the disk, adding a file system (I use ext4) and mounting it to the right place. &#160;The hda-diskmount command will find the drive once formatted and suggest a location and fstab entry. This is handy and it worked as advertised.<br /> <br /> One thing of note: Amahi mounts drives under /var/hda/files/drives/driveX. It doesn't seem to detect whether old mount points are still in use so it always creates a new mount point. This is safe, but messy. When I added the new 1TB disk, it was assigned to 'drive5' even though there are 3 unused directories already.</p> <p>I added the fstab command and rebooted. I then added the new disk to the storage pool and everything was good. I got to thinking that all the data was still on the first disk and wondered if there was a way to balance the data across drives. A review of the greyhole help and a question in the #amahi IRC channel pointed me to the balance option. I ran</p> <div class="code">greyhole --balance&#160;</div> <p>and monitored the greyhole.log file to see greyhole making a bunch of data moves. It ran for awhile.</p> <p>Once that was complete, I had to tackle replacing the drive that contained the boot partition. &#160;For this, Clonezilla was the obvious choice. &#160;I don't have any experience with the tool. &#160;Since the disks were different sizes (the ata disk was 160GB and sata 250GB),I started with cloning only the Fedora boot and root partitions. This left me with an unbootable sata disk. I then tried a full disk to disk clone and this worked. &#160;I wound up with a 200MB boot partition, a 160GB root and 83GB unallocated space. &#160;The disk worked as expected.<br /> <br /> I wasn't satisfied with the unallocated space, so I turned to gParted Live CD to resize the root partiion to add the additional space. &#160;Again, this was flawless.<br /> <br /> At this point I had only one remaining task - Adding the second 1TB disk into the system and to the strorage pool. &#160;This went off without a hitch. &#160;I did one thing as a test. &#160;I removed the now unused /var/hda/files/drives/drive4 directory and when I ran hda-diskmount, sure enough it allocated that unused directory for the drive mount. &#160;I again ran greyhole to balance the data and when it completed, had ~80GB on each disk.</p> <br /> <br type="_moz" />