{"id":209,"date":"2014-08-06T18:54:54","date_gmt":"2014-08-06T23:54:54","guid":{"rendered":"https:\/\/makecoolthings.com\/?p=209"},"modified":"2015-07-28T13:45:54","modified_gmt":"2015-07-28T18:45:54","slug":"week-32-2014-well-that-escalated-quickly","status":"publish","type":"post","link":"https:\/\/makecoolthings.com\/?p=209","title":{"rendered":"Week 32, 2014 – Well, That Escalated Quickly…"},"content":{"rendered":"

In the middle of the night on July 31st, the motherboard in my workstation\u00a0burst into flames<\/em>. \u00a0Things just sorta went downhill from there…<\/p>\n

That is not hyperbole, by the way. \u00a0A slight exaggeration, maybe– it was more of a candle than a conflagration –but in any case I had to ditch my plans to buy more screenprinting supplies (Which would have included a proper t-shirt platen, a poster platen, and some larger screens)<\/em> with all of my disposable income for August, and fix my computer instead.<\/p>\n

Well, with all my money spent, and everything I need to fix my computer– hopefully, I actually still don’t know if the CPU and RAM are okay –my 12 terabyte RAID5 array, where all of my media, as well as everything I’ve worked so hard on for my\u00a0entire adult life<\/em> is stored, decided to suffer a multi-disk failure.<\/p>\n

15 years of work, gone. \u00a0Poof! All of my writings, including two series that I’ve been developing since the turn of the millennium, something like 30,000 photographs, all of my product designs, electronic circuits, CAD<\/acronym> drawings, artwork… \u00a0Gone. I don’t know what to do, I’m stuck. \u00a0Without the RAID<\/acronym> array, all my work is gone…I don’t even have anywhere to put NEW work. \u00a0I’m not even sure I can put an OS on my machine when I fix it, because all our Windows license keys were in a file on the RAID<\/acronym> array… \u00a0I guess poor people just aren’t supposed to even try being productive.<\/p>\n

If I can just fix this one drive, I can get the RAID<\/acronym> back… \u00a0I’ve seen the prices for data recovery… It’s more than the rent on my apartment. \u00a0And I don’t think it covers this situation, because I need the disk or an exact clone of it, for the RAID<\/acronym> to recognize it…not just the ‘disk contents’. \u00a0And paying for a full RAID<\/acronym> recovery is totally out of the question. \u00a0That would cost more than I get in several months put together! \u00a0I may have to resort to swapping the platters myself, which I should be fully capable of doing, given the right tools<\/a>.<\/p>\n

Drive Problem Details:<\/h2>\n

The array is RAID<\/acronym> level 5 running on a HighPoint RocketRaid 2340, and it consisted of seven 2TB HDDs. (Five WD Greens, and two much newer Seagates)<\/em> \u00a0Of those, 1-2\u00a0(Array#-Disk#)<\/em>\u00a0had been dropped due to the typical Green timeout bullshit, and then timed out at the end of its rebuild…usually it succeeds on the second try, but apparently it wasn’t taking this time, and during rebuild attempt #3 disk\u00a01-5 failed outright.<\/p>\n

When I put 1-5 in an external dock on another computer, it showed up as\u00a0134217728.00 GB of ‘Unallocated’ space. (Yes, that’s 134 petabytes)<\/em> \u00a0I’ve been led to understand that this absurd figure is simply the 48-bit LBA address space limit, and usually means that it can’t read the first sector of the disk. \u00a0After a few more tries, the disk stopped showing up in the Drive Manager at all when I turned the dock on.<\/p>\n

After swapping the PCB from an identical model\/stepping drive (WD20EARS-00MVWB0)<\/em> it worked again, but gave the same figure. \u00a0Given that the drive doesn’t make any strange noises and it certainly seems to spin up and I think I hear the head un-park, but then I don’t hear it begin to read, I surmise that the fault is in the head assembly. \u00a0It is, after all, a WD Green that has been in continuous use in a RAID<\/acronym> for almost 4 years.<\/p>\n

It would seem that the most straightforward course of action to recover 1-5 would be to transplant the platters (Using the appropriate tools from\u00a0eBay.)<\/em> into a NOS drive of the same model\/stepping…which also seem abundant on eBay. \u00a0With luck, this will get the RAID<\/acronym> array back long enough for me to successfully rebuild a drive, then replace drive 1-5.<\/p>\n

Updates:<\/h2>\n

August 7th
\n<\/strong>Well, at least I can install Windows. \u00a0I found the official Microsoft download link for the ISO<\/acronym> (I always lose the discs I burn when I install…)<\/em>, and I was able to get my Win7 Ultimate N key off my primary machine’s HDD with a key recovery tool. Added the ‘drive problem details’ and ‘next step’ sections. \u00a0Maybe I can get some freebie professional advice?<\/p>\n

August 8th
\n<\/strong>IT’S\u00a0<\/strong>ALIIIIVE!<\/strong> \u00a0HighPoint support provided me with a DOS tool that let me arbitrarily construct arrays without actually affecting the disk contents. \u00a0(As well as backing up the original array info)<\/em> \u00a0And I was able to get it back up and running by sticking a spare 2TB drive in the 1-5 slot to pad the array out to the full size. (HighPoint support later praised my resourcefulness, because I shouldn’t have been able to recover the array without all the member disks.)<\/em> It was short 2TB of data, of course, and I did something clever to find any corrupt files\/directories:<\/p>\n

robocopy \"K:\" \"C:\\TMP\" \/E \/CREATE \/R:0 \/TEE \/LOG:\"C:\\corruptionlog.txt\"<\/pre>\n
Which copied all the contents of the array that it could (some 1.4 million files in 200k directories)<\/em> to C:\\TMP as 0-byte files with zero retry attempts, while logging all console output to a log file. (It took just over an hour.)<\/em> \u00a0Scanning the log file turned up any instances of files\/folders coming up ‘Corrupt\/Inaccessible’.<\/p>\n
And with the exception of a single Inventor project (Which consisted almost entirely of parts from other projects)<\/em> I didn’t lose any work. \u00a0In fact, the damage was almost entirely contained to some<\/em> items from the past 3 weeks, and unsorted torrent\/newsgroup download directories. \u00a0(Those last two alone added up to over a terabyte…)<\/em><\/p>\n
Another nice thing about using robocopy like that is, when I eventually take a crack at restoring 1-5 to working order, and reintroduce it to the array, I’ll be able to run that again, and compare the two logs to get a list of files that weren’t present without the missing disk.<\/p>\n
So, after almost 72 hours without sleep, I finally got the array back up and running and can begin recovery! (Though I took a little nap instead.)<\/em> Which of course means I need to come up with money to buy some WD Reds to make a new– and actually reliable –array. \u00a0(As well as probably upgrade to a RAID<\/acronym> card that supports RAID6, just in case.) <\/i>Not that I have any idea where that money will come from…<\/p>\n
But in the meantime, I’m backing up everything of importance to some spare 1.5TB drives I have, and then backing up the photos and work to 25GB BD-R discs.<\/p>\n
August 10th<\/strong>
\nComputer works! \u00a0The new motherboard works like a charm. \u00a0It took some bit-wrangling to get the old Windows install working as a temporary solution, but I plan to do a reinstall once everything has calmed down. (I still have a god damned headache from the caffeine I ingested to keep my ass up for three days… I’m getting too old for this shit.)<\/em><\/p>","protected":false},"excerpt":{"rendered":"
In the middle of the night on July 31st, the motherboard in my workstation\u00a0burst into flames. \u00a0Things just sorta went downhill from there…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"bgseo_title":"","bgseo_description":"","bgseo_robots_index":"","bgseo_robots_follow":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1,28],"tags":[40,25],"wf_post_folders":[79],"class_list":["post-209","post","type-post","status-publish","format-standard","hentry","category-blogposts","category-status-report","tag-everythings-ruined-forever","tag-hard-drive"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6NlLW-3n","_links":{"self":[{"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/posts\/209","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=209"}],"version-history":[{"count":10,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/posts\/209\/revisions"}],"predecessor-version":[{"id":424,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=\/wp\/v2\/posts\/209\/revisions\/424"}],"wp:attachment":[{"href":"https:\/\/makecoolthings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=209"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=209"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=209"},{"taxonomy":"wf_post_folders","embeddable":true,"href":"https:\/\/makecoolthings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fwf_post_folders&post=209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}