Drobo issues

[bob carnie]

I have a Drobo about 5 years old with a second time one of the drives going, It is costing me a new drive and 1 1/2 hour to put back in shape,

I was told the other drives are same vintage and can fail in no predictable order.

Storage for me is important, but not as critical as lets say a working commercial photographer or art photographer capturing info 100 mb at a time.

In a perfect world I need to keep about 20 - 40 terabits of stored info for my personal and customer projects. 40 being really the top end.

Can some of you with good digital experience suggest alternatives to Drobo or suggest some methods of long term storage that is not susceptible to a 4-6 year replacement.
I think I paid about 1200- 1500 hundred dollars , and each time I replace a drive it seems to be about $250

So every 4 years about $1000 bucks, is this a normal expectation to continue with safe storage?

[jp]

That's very reasonable ongoing expense for that amount of storage. It's probably time for a new drobo too. I only store a few terabytes of data and upgrade to a newer bigger drive about every three years for $250 and another $250 to upgrade the backup on a similar schedule.

I use a WD Mycloud for backup which is quite affordable for backup but not high performance enough for working storage.

[bob carnie]

That's very reasonable ongoing expense for that amount of storage. It's probably time for a new drobo too. I only store a few terabytes of data and upgrade to a newer bigger drive about every three years for $250 and another $250 to upgrade the backup on a similar schedule.

I use a WD Mycloud for backup which is quite affordable for backup but not high performance enough for working storage.

Yes I guess you are right , I look at my enlarger and only have to replace the lightbulb, but storage of data is quite key to me right now so I should bite the bullet.

[Preston]

Bob, I suggest you purchase 'enterprise grade' HD's such as the Western Digital Red or Western Digital RE. You can get either 5400 or 7200 RPM drives in capacities up to 2 TB for the RE and up to 6 TB for the Red. The 5400rpm drives will tend to run a bit cooler, but are not quite as efficient (Peak read/write MB/s) as the 7200 rpm drives.

The Drobo are pretty good. If you are looking at a new platform, take a look at the offerings from Synology, QNap, Western Digital, and Netgear. You can see reviews of most of these at Anandtech (http://www.anandtech.com/tag/nas)

Drives that last five years under heavy use is actually pretty decent, so I would say you got a good return on your investment.
--P

[Kirk Gittings]

This is the best source I know of for drive testing. testing 61k+ drives-yes that is sixty one thousand plus drives.
They buy them in 10k lots.
https://www.backblaze.com/blog/hard-drive-reliability-stats-q1-2016/

[DennisD]

Bob, Preston's advice regarding enterprise hard drives is right on target especially for a server or NAS device.

They will outlast and outperform standard drives. Further, the enterprise drives are generally guaranteed for 5 years which helps protect your investment. Manufacturers have the ability to design and build life expectancy right into the drive. I can't tell you how many drives I've seen go south a few months after the warranty ends - especially drives with 2 or 3 year warranties.

I've had excellent results with enterprise drives, in particular Seagate. However, I'm sure there are other people who swear by WD, etc.

I recently replaced my business NAS with an 8 drive QNAP product for about $1300 plus drive cost. Depending on the drive size you select, such a machine should meet the capacity you mentioned. This type server can grow in capacity as needed because you don't need to fill all the drive slots initially. You can add drives and expand capacity as needed.

Capacity will NOT equal drive size X # of bays. Depending on your RAID configuration and whether you have any"hot" spares you may lose capacity of 2or 3 drives. The unit we selected will accomodate drive sizes up to 8TB. All drives should be same capacity. We installed 6 drives, 6TB each at a cost of $250 per drive. We have about 25TB useable storage. Recently added one more drive to provide a little more capacity for comfort. The transition to the new drive was seamless and easy. We can still add one more drive. As a point of interest, 8tb drives will run about $380.

While the total investment is not insignificant, there are not many alternatives. You can always go to the 'cloud', but that introduces a whole new set of considerations, esp. with transfer times/performance.

One final thought. If you are using the NAS device as your primary means of storage, you should consider how you will back up the device. Some people have a second NAS to back up the first, so double the cost !

Good luck,
Dennis

Note: I have no relationship or financial interest in any of the companies mentioned above.

[Light Guru]

suggest some methods of long term storage that is not susceptible to a 4-6 year replacement.

No such thing. There are 2 kind of hard drives those that have failed and those that will fail.

[Graham Patterson]

I run an 8 bay QNAP at home for my wife's music and artwork. That's at 16TB at the moment, using RAID 6. At work I have a similar unit as a second tier backup for a 9TB enterprise file server (that's a whole story in itself). I budget about one drive per year replacement (maybe $0.60 USD a day), and at the moment I am several drive's worth ahead on the deal 8-) I would expect the chassis unit to run maybe 8 years if it survives the first year - this is a rule of thumb based on running servers and file stores as part of my job.

Once you start operating multiple terabytes, the transfer time becomes an issue. At home I would not consider 10 gigabit fibre channel connections - gigabit ethernet and eSATA is about the cost/performance limit.

You have other options - LTO tape is still available, but not suited to normal home use. Cloud services are fine if you pay the rent, but awful if you want to pull a lot of stuff back in a hurry. And then you may have local service data caps.

[williaty]

Keep in mind that RAID IS NOT A BACKUP. Many, many people and companies have lost critical data because they falsely believed RAID helped protect their data. RAID protects availability. RAID is really designed to keep things like a critical database available for constant access while any single HDD fails and gets replaced. As drive sizes have gotten ever larger, the situation has gotten even worse as the probability of a second drive failing and taking out the whole array while trying to rebulid the first drive grows more and more likely due to the increasing length of time it takes to rebulid the array when the new drive is added.

If you want read-write access to your archival data, no, there's no viable alternative to HDDs at this point. For read-only storage, the HTL-type Blu-Ray Discs are pretty damned stable, giving you 25 years without much worry and some vendors are claiming as much as 100. It seems likely that the hardware and software to read that BRD and present the image files on screen is likely to be lost before the disc becomes bad given that I can't read discs I created as a kid due to the computer not having a slot for me to stick an 8" floppy disc in anymore.

For first-line read-write data storage, if you're done with the Drobo, the QNAP NASes offer a ton of functionality for not very much money. I have a TS-219P and whatever the model that replaced it is. The older one is getting close to 10 years and still hasn't given me a problem. Sure, the drives tend to die every 6 years or so, but HDDs just do that.

[Jim Andrada]

Yes, LTO tape is certainly available. The latest version (Generation 7) has a native capacity of 6TB per cartridge. Cartridges are still somewhat pricey though. Generation 6 drives are still being sold and have native cartridge capacity of 2.5TB - Gen 6 is probably the current price performer re capacity and speed. Gen 8 should be out around the end of next year at 12 TB per cartridge. Gen 9 and 10 have been announced as part of the LTO Roadmap, with capacity doubling each time. Compressed capacities are around 2.5X native, but this is of course somewhat dependent on the type of data.

I don't think there is any better medium for backup. The drives are enterprise class and extremely durable. The cartridges are also very robust. They're designed from the start as long term "on the shelf" media and are much better than hard drives or optical for long term backup. I believe the official rating is 5000 load/read-write/unload cycles, but we've seen them last 20,000 to 30,000 cycles on the drive test lines in Japan and China. I believe the drives have been tested to around 1 million load/unload cycles. All currently sold versions of LTO support a mode of operation that implements a file system structure on the cartridge so files can be dragged and dropped to tape. They also support Write Once Read Many technology using WORM mode cartridges.

The drives also have the ability to read and write previous generation media and to read two generation older media - ie Gen 7 can read and write gen 6 cartridges and read gen 5 cartridges.

All these good things having been said, whether a tape based approach fits your work flow or not is something only you can decide. By the way. there is a closely related higher-end tape technology that's proprietary to IBM and between LTO and the IBM technology, all the major cloud vendors back up to tape. Except for a small market share in the high-end automation space by Oracle all Gen 7 and future enterprise tape products are designed and built by an IBM/NEC (Japan) partnership.

I've been involved with the program since its inception.

[Peter De Smidt]

Bryan (Corran) posted about a system not that long ago. It sounded really good, but I don't remember the specifics.

[Fred L]

A friend's Drobo 5D turbo gave up the ghost and he's now looking for a better solution. basically I think he's looking at Qnap NAS, which another friend has been running for years with no issues.

Software RAID seems to be great idea but failures are catastrophic I've learned.

[Corran]

Bryan (Corran) posted about a system not that long ago. It sounded really good, but I don't remember the specifics.

unRAID (https://lime-technology.com/)

This is NOT a traditional RAID system, it's a JBOD (just a bunch of disks) with the added capability of protecting your data in case of a drive failure with a parity drive.

A basic system is free but with your storage needs you'll need the full version - but it's cheap!! If you have an old computer sitting around you can build an unRAID system with nothing more than fresh disks and a USB drive for the custom OS, assuming the motherboard has plenty of SATA inputs (most modern boards do, made in the last half decade). As already mentioned, get "enterprise" level disks for a bit more reliability, but also as already stated you WILL have failed disks. But my enterprise disks seem to hold up better/longer.

If you don't know about building a computer have someone do it for you, but it's really simple. You could get ten 4 TB drives and have 36 TB of storage (one disk is the parity drive, perhaps get a 6 TB instead to future-proof yourself). I think it also supports a double parity system but I'm not sure it's been a while since I looked. Built my system several years ago and it's rock-solid, same as the day I built it. I'll probably start replacing disks here or there to give me piece of mind, and with larger disks (I did 2 TB disks which were pretty middle of the road back when I built it, so I'll start replacing them with 4 TB drives).

Western Digital drives are the best in my opinion (least amount of failures).

[bob carnie]

I just got it fixed, and sure as shit a second drive is now warning me to replace...

Question, for those who obviously know tons more than me on this.

The work I am storing is projects done for myself and others, which I do not access on a daily basis but have it in ARCHIVE STATUS.

If I downloaded to a portable drive and just put it in storage until possible use later , can I expect less failures this way. I can have my clients pay for a portable drive
so the cost of multiple large drives is not problematic for me...
I am constantly accessing the Drobo now which has current work and Archive work, so by putting Archive work to sleep so to speak am I able to sleep better not thinking about lost data??

[williaty]

HDDs fail because of starts/loads and because of total run time. In this case, keeping the drives powered down should extend overall livespan. However, I certainly wouldn't trust a drive to (mechanically) survive sitting for 5-10 years and then working the moment you plug back in. Some of them will start back up (probably most) and some of them won't (probably only a few). To me, that's just too chancy to trust. You'd also run the risk of not realizing some upgrade had made your old drives unreadable in the meantime (either no longer have the right port to connect to or no longer have the software to read the data).

Long term archival storage is a mess. At work, my wife generates about 4TB a month and federal law requires it to be stored permanently. It's a mess.

ETA: If you want to try off-line storage and you don't want to look at burning BRDs (which really look like a good solution for this) take a look at solid state storage. I'd need to google to check this out myself but if the data is long-term stable when powered down, at least they wouldn't have any problems starting back up again in the distant future.

[bob carnie]

HDDs fail because of starts/loads and because of total run time. In this case, keeping the drives powered down should extend overall livespan. However, I certainly wouldn't trust a drive to (mechanically) survive sitting for 5-10 years and then working the moment you plug back in. Some of them will start back up (probably most) and some of them won't (probably only a few). To me, that's just too chancy to trust. You'd also run the risk of not realizing some upgrade had made your old drives unreadable in the meantime (either no longer have the right port to connect to or no longer have the software to read the data).

Long term archival storage is a mess. At work, my wife generates about 4TB a month and federal law requires it to be stored permanently. It's a mess.

I am beginning to think we are revisiting the Kodak Colour for life situation, remember when all the negatives and prints faded from 60's onward, it is the reason I moved to Black and White .
Some projects I work on start at set date and do not get printed until 5- 10 years later, with black and white negatives we never have issues but I am beginning to wonder now.

I have scanned over 2 thousand colour and Black and White negatives for Alternative Work, took me over two years to do this, I am very much creeped out that I could lose these scans, and
I do not see me printing these all within the next 5 years... a MESS is right.

I do have the option to make BW separation silver negatives to print size and am actually starting to do this , as I know the film if processd right will last over 100 years, long enough for me to print .

The problem is the roll of film is expensive and unless I win the lottery its going to take me 5 years to do this option. I am starting now as Pictorico does not last well so Ortho Ilford is the plan.

[Corran]

I just looked and Google Drive has cloud storage packages up to 30 TB ($300 per year for that). If you wants a hands-off solution with no maintenance costs or headaches honestly that's probably the best option for you. And personally I would imagine that Google is going to be safe and available in the future, barring some cataclysmic solar flare event or worse, at which point it doesn't really matter.

Other than that, personally if some of this is client projects that are finished, I would seriously consider making long-term storage and protection their problem, or charge them appropriately for storage if you think that's paramount. I used to be a "free cloud" for my clients and after amassing many TBs of useless data I had to purge it all and make sure my clients understood that the onus was on them. Burn them some high-quality discs of the work, archive it for them for up to 6 months or whatever you feel comfortable, and then purge. I understand your situation may be different but just throwing that out there.

EDIT: one more thing - is a lot of your data raw scan TIFFs or similar? These can be losslessly compressed for storage. And not WinZIP which is old and not as efficient as newer programs. An oldie but goody that I use a lot is WinRAR.

[bob carnie]

Hi Bryan

I am thinking exactly like you state to put the onus on them and not me as I cannot control the issue, so why take the risk.
As I said for my personal work I may consider Google Cloud storage , not a bad price to pay until I get all my images converted back to separation printing film.

thanks

Bob

[Fred L]

I would ditch Drobo and get cloud storage (multiple if you're really paranoid) and keep duplicate hard drives in town. One at the lab or wherever and another off site. I'd also get into the regimen of copying onto new drives every couple of years. I use Western Digital portables and desktops and, (knock on wood) so far they're been issue free.

[DennisD]

+1 to what Fred L says above.

The problem with any single electronic storage unit, hard drives in particular, is they can be good today and mysteriously unreadable or dead when turned on months or years later.

For that reason, backing up your backup is really important. Perhaps the best is a combination of cloud and something else, hard drive, blue ray, or any other alternative you deem reasonably reliable.

[bob carnie]

Thank you all for your great responses, I am talking with a good friend in Seattle about how to go about this and, for sure come up with a plan.

One thing is for sure , I need to take the responsibility of storage away from myself as I am not as tech savvy as many of you here and I could run into a few major problems.

I think Cloud Archive with minimal back and forth looks promising , for me.

I will continue with the Drobo for day to day access but not for archiving.


Bob

[seezee]

Bob, I suggest you purchase 'enterprise grade' HD's such as the Western Digital Red or Western Digital RE.

I 2nd that recommendation. These drives are designed for NAS environments. They are more isolated to vibration, which is important when you have so many spinning disks in close proximity. That's why they typically last longer and carry the extended warranty. The price keeps dropping, too, so by the time you expand capacity it shouldn't cost much more than your last round.

I don't know the Drobo software, but it probably has a health-check feature to give you some idea of whether any of the drives is about to fail. Always keep a spare.

[seezee]

I think Cloud Archive with minimal back and forth looks promising , for me.

I will continue with the Drobo for day to day access but not for archiving.

Don't rely on remote backup as your only backup. A backup plan must include redundant on-site and off-site archives; otherwise, it's not a backup plan. All storage media (flash storage, hard drives, tape archives) fail eventually. Your house could burn down. Your off-site provider could go bankrupt. At a minimum, you want RAID 1 (mirroring) + off-site backup (e.g., portable drive in a safe-deposit box). Ideally, you should have RAID 5 (fully redundant storage) + off-site + cloud backup. The cloud is the last resort since it takes forever to restore from.

Don't believe your provider could fold? Listen to this story (https://gimletmedia.com/episode/71-the-picture-taker/).

[Peter De Smidt]

This is what I do.
1) I have a back up disk in my computer. So all of my data files are on two drives. Using a Drobo/Raid....is similar.
2) I have a couple of large external hard drives. Every so often, I back up everything to them. I should store one off-site, but I don't, as I'm not that industrious.
3) On line storage. I use Dropbox, but other options might be better. Good ones allow you to send them a drive/drives full of your images so that you don't have to upload a tremendous amount of data. They also will send you a drive in the case where you need all of your files.

[nonuniform]

I think you're lucky to get 5 years out of a drive these days. I use Backblaze, a Drobo, several external drives, and a raid 0 drive for files I'm currently working on (1gb scans). Haven't had a catastrophic failure in 20 years. Drives die all the time, but the data has multiple backups.

[bob carnie]

Here is a rating system a friend sent me

http://www.toptenreviews.com/services/internet/best-online-data-backup-services/