IT Professional and The Amazing Powers of Incompleteness
IT Professionals and accountants
I heard sometime that accountants are rated at the top of the stressful jobs list.
I think that is dead wrong… It is the IT Professional, who’s under huge stress.
Both professions require accuracy, keeping a hawk’s eye on details and
ever making sure no one is hurt by what they are doing. When they do a mistake,
it could have huge impact on many people…
IT PRO and The Story of 2 failed disks.
Once upon a time there were two small little disks.
They rotated happily and spitting out data and from time to time swallowed some bits as well.
They were not alone.
Together with them there were few other disks running and throwing bits and bytes at each other.
They were all in the same secret group – the RAID5 group.
One day bad electricity surge boy came along and hit one of them…really hard and so the disk failed.
The other disk was shocked so much, that he could not spin no more.
And so the whole RAID group dropped dead that very moment.
FIX FIX FIX – Completely FIX!
Since the 2 failing disks could not be fixed, a service technician was called upon.
He rushed away bringing new disks to replace with the failed ones.
Actually, the technician was so happy to help that he brought a whole new set of disks rather than just two.
He was worried about the way the two disks failed and wanted to make sure COMPLETELLY
that there would be no chance for a failure on the other disks in the group.
As he was getting ready to walk away with the bad disks, I said “STOP!”
The tech guy said “Why? What happened? Aren’t you happy?”
I said – “Put your hands up in the air and give me all the bad disks!”
The reason why I wanted the bad disks.
Why on earth would I want to keep the bad disks?
I should be restoring my data from tapes to the new set of disks – right?
Why should I care about the bad disks, their data is gone anyway – right?
Well…maybe not…
What happens if I find out the data on our backups excludes very important items…
critical items…which I can’t find anywhere…except on the bad disks…
How on earth can you recover data from a failed disk, which is part of a RAID group?
As an IT Pro, you probably know that each file stored on a RAID5 disk set,
will be spread across the group’s disks.
And so COMPLETE recovery would be almost impossible, specially if 2 of the disks in
the group have failed , resulting in the miss of the chunks of the files they had stored – right?
Well, actually if you know a bit about Computer Forensics you are already guessing that
even thought one cannot COMPLETELLY recover the data in this failed group, he can
still recover some files…maybe the critical files that are missing.
IT Professional and The Amazing Powers of incompleteness
We IT professionals always want to Completely fix things.
We don’t like loose ends.
We want clean data, situation resolved, close all holes, failing disks trashed, COMPLETELY.
That’s great. However, I found out that many times when I accept incompleteness
it opens up better or new possiblities.
So before you run around fixing everything, solving all the issues, consider that maybe
not all issues have to be COMPLETELLY resolved. Sometimes a workaround is just fine.
Sometimes it is good to keep the failed stuff around…it may be better than the
new shiny stuff you put in place 
P.S.
Do let me know your thoughts…
P.P.S.
If you enjoyed this post, please take the time and
download one of more of our white papers and reports in the vault.
You’d enjoy them and the contributors of those reports will be happy…and so will we.
Yours,
Jack.
Tagged with: it pro • IT Professional Career • IT Professional Technical Tips
Filed under: data backup • IT Courses • IT Professional Career • Success Stories
Like this post? Subscribe to my RSS feed and get loads more!
You are definitely on the right track here. In my opinion, the question is about recognizing the real problem or problems. I see two principal problems here. First, there is the issue of data protection. A disk, by its very purpose, contains data; when a disk fails, for whatever reason, that data doesn’t cease to exist, and it can be recovered by whoever has the means to do it. There are failure modes where the data is intact and can be very easily recovered. So, broken disks must be treated just like any data: you either store them properly or dispose of them properly.
The other problem is that of potentially unreliable backups: you may want to keep the failed disks as a last resort in case the backups turn out unreadable.
Solving the first problem provides a solution to the second as well.
As to your original subject, knowing how far to go with a solution is a skill in and of itself. We’ve all had our share of recurring problems that won’t go away without a clean install, and nobody wants to create new ones. Then again, it’s a waste of effort to fix something that isn’t a problem. That’s why it’s so crucial to ask “What is the problem?”
Juhana Siren – consultant – Apertas Solutions Oy
Juhana, my friend,
Always good to look at issues with like-minded IT PROs
as you…
Indeed I forgot to mention that one has to get rid of
the data on bad disks, even when they are a total waste…
I always either pay for keeping the failed disk on site
and de-comissioning it using our own procedures
(disk cruncher…).
I also resolve complex situations by digging into
the question – “What is the right problem we should
be solving now”.
Yours,
Jack.
I agree. Having a forensics background. That data can be recovered. In the case you have stated, something as simple as replacing the circuit board of the drive may recover all the data. I would put a new “set” of disks in the RAID, taken the set. And had a disk repair company repair the disk. If you got a week, I’m sure you could have recovered most if not all the data. Since the problem was more electrical in nature. Second, your data could have walked out the door. Even a few sectors could give a hacker information that could be valuable.
Replacing the circuit board on a failed drive doent even require high teck lab. Just the willingness to “break” apart a good drive of like model.