Main

storage Archives

May 7, 2007

Not living up to your potential

My middle school social studies teacher frequently gave me a hard time for not living up to my potential. "Not firing on all cylinders" was how he phrased it. Today's storage buses are in the same situation. Or are they?

During construction of my last system I agonized over the fact that the Western Digital Raptor drives (10K RPM, 4.6ms seek time) were only SATA-150 and not SATA-300. as a friend pointed out to me, the concern was moot. There was no way the drive could saturate the SATA-150 pipe let alone a 300 pipe. The faster seek times would more than anything else set the winner apart.

Yet system builders continue to clamor for a SATA-300 Raptor. Why? Is it the planning for future expansion? I think it is more to due with the goal of eliminating as many bottlenecks as possible. I know I have felt the frustration at being hamstrung by a certain bus or interface, so it is clearly a case where emotion can trump reason.

August 4, 2007

On Storage

Ladies and Gentlemen, I bought a Drobo. A Drobo is a cool new gadget billed as a "data robot", an intelligent, scalable storage device that takes the fuss and fury out of reliable, scalable hard disk management.

There are several downsides to owning a Drobo. For starters, it costs five-hundred wing-wangs and that's just for the housing device. Drives, which, in case you haven't caught on are really important when using a Drobo, cost extra. I found some nice cheap SATA drives on Newegg.com for $65 a pop.

Once you get it out of the box, you'll find that your Drobo only connects to your system via USB 2.0. For exactly the same reasons that washing your car angers Aquamec the rain god, my purchase of this device is a sure sign that USB 3.0 will be out in a matter of minutes. The Drobo also only support NTFS and HFS+ partitions, so you're only going to have luck using the device on a Windows machine or a Mac. ext3 support or network connectivity is something that Data Robotics, Inc. is "working on".

So with all these minuses working against it, why did I buy it?

Well, let's look at the alternative. The Ultimate Perfect Forever-Free Storage Solution is going to involve buying a new system, setting up hardware or software RAID on multiple drives, and installing Linux in order to create an LVM configuration for your multiple drives, stitched together with RAID and formatted with a Big Bad filesystem like ReiserFS, JFS, or XFS that will never lock your data into some proprietary format that is going to require you to pay through the nose for the privilege of keeping your data handy.

If you put your data on a system like this, it is safe for as long as all your drives work. If one drive dies, you power the machine off, replace it, and rebuild your array. This solution is scalable, open-source, and gives you unlimited options for how you shape your storage. It can be maintained forever and easily transferred to another setup five years down the road when the hardware starts to flake.

This solution is also going to cost you a minimum of $500 in hardware, and you will constantly wage the risk of asking yourself "Even if I only pay $25 for a RAID controller, do I really want to entrust all of my most important data to a $25 RAID controller?" So my opinion is that you should not scrimp on your Ultimate Perfect Forever-Free Storage Solution. Trying to low-ball your hardware is only going to cause headaches at best, and data loss at worst.

So even once you've taken the risk of getting some cheap gear together, you now need to assemble it. Plan on at least one evening spent building your box, putting SATA drives in, screwing them in, taking them out, turning them right-side-up, then putting them back in again. Once everything POSTs, you still have to install CentOS, fdisk the drives, set up RAID devices, set up LVM, and choose your file system. If you want to get to your data from across the network, that's going to require an NFS or Samba config, too. How much is your time worth to you?

I would love, positively love, to do all of this. It's good experience and builds character. I want to be able to trust that my data is safe because I've built every last layer of protection into the system with my own two hands. But you know what? I have, at this exact moment in time, precisely 18.1 GB of free space on my C drive, and this precipitates a rather urgent need for a scalable storage solution that isn't going to cost me both $500 and an entire weekend.

The Drobo, by comparison, is pretty much a "plug it in, go have a beer" kind of technology. It's pricey and it only works correctly when directly attached to a Windows system. But you know what? So are consultants.

August 11, 2007

The Drobo Story: Setup

My Drobo and the 3 drives I bought from Newegg arrived simultaneously on Wednesday. I had it out of the boxes, installed, and completely functional in under 15 minutes. I put a gigabyte of MP3s on it immediately. I took it home on Thursday night, forgetting the AC power cord in the process, and after a trip back to the office I had a working Drobo connected to my home PC.

Using a combination of SyncToy for the heavy lifting and rsync for accuracy, I began moving my data over. Thursday night, Drobo was at 0% capacity. Today, on a Saturday morning, it's at 67%. At 85% Drobo is programmed to start complaining at you, and above 90% it deliberately limits its throughput in order to nag you into upgrading its capacity.

Basically, this is just a roundabout way for me to say that I bought a 4th drive from Newegg this morning.

I used Junction to create a mount point (an "NTFS reparse point" in Microsoftese) from C:\Downloads to the corresponding Drobo directory. This way, I can drag files from my Desktop to a shortcut to C:\Downloads and have Explorer.exe assume that I mean "move" instead of "copy". By default, dragging a file from your C drive to an E drive would be a copy operation, which in this specific case I don't want to have happen.

So I've moved most (not all) of my data to a little black box no bigger than the little black storage box where I used to keep 5.25" floppy diskettes. And already I'm nervous enough to warrant maxing out the 4 drive bays with another drive.

It's been totally painless, save for a few bullsh*t SyncToy errors that rsync handled like a champ. I have some extra IDE drives laying around that I still need to migrate, and after that I expect my Drobo space consumption rate to level off logistically.

About storage

This page contains an archive of all entries posted to Electric Sheep in the storage category. They are listed from oldest to newest.

networking is the previous category.

system building is the next category.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.35