Anybody using Network Attached Storage / RAID Arrays for their files?

[Corran]

I've been thinking lately about building a NAS system with some old computer parts I have here. I'm not really sure what I'm looking for though since I haven't ever used a RAID array.

I also don't know if it's really going to do what I want. Right now I have a 2TB drive that's 90% full, a 750GB that's getting there, and my 1.5TB "Storage" drive decided to quit on me earlier this year. I am looking for a redundant storage system that I can keep all of my audio, video, and photo work. LF scans have really started to eat into my storage, but 36mp D800 files and 50-80GB of video files for a commercial shoot are killing me too.

I suppose that my work flow will be scan/shoot -> RAW / TIFF files edited and then filed away onto the NAS and one copy of my JPEG files will stay on my computer for easy access. What's important is the redundancy - if a HDD goes bad I want the RAID array to be able to recover itself. I don't remember what RAID type that is. Probably looking at putting 3-4 1.5TB or 2TB drives in there, and maybe the whole thing in a fire-proof cabinet if possible.

Anybody have one of these systems, or built one? I'll probably cross-post this to a pro audio site but I figured I'd ask y'all as well!

[Yo' Vinny]

Everyone will have a lot of opinions here, but I eventually ran into this problem and got tired of the constant hard drive swapping, USB/Thunderbolt drives, etc. Ended up standardizing on an internal fast 7200rpm RAID 0 striped array for pictures within the last two years. Moving everything else offline to a Synology NAS device - highly recommended! Don't waste your time or money on Drobo. My offsite backup plan will be cloud storage for the most critical images, but I'm still working through those options Amazon Glacier, Crashplan, etc.

[Light Guru]

Sounds like you need a Drobo.

[Corran]

Well the thing is, I have a mid-ATX box, motherboard, and quad-core Pentium just sitting around from my last computer build, so rather than buy a ready-made NAS box, I want to build my own. I think I just need a RAID controller and NAS OS like FreeNAS that I've read about.

Vinny, which Synology do you have though?

Oh - I also already have a power conditioner and all of that. Disastrous power-fluctuations should not be an issue.

[polyglot]

* DO NOT RAID-0, it increases the chance of data loss. If you want speed (for editing, swap-space, tile-caches, root fs), get an SSD.
* RAID-5 should be your minimum option, e.g. a 3+1 configuration (6TB from 4 of 2TB drives), consider also RAID-6 (3+2 with hot spare)
* thermal & vibration management makes the difference between 1-year and 15-year drive longevity
* if you can use a Unix variant, they generally include excellent software-RAID functionality. You can get PCI 4-port SATA cards for peanuts and most motherboards now have heaps of SATA onboard.
* hardware RAID can be nice but the card becomes your single point of failure. You also can't throw the discs in USB boxes and recover them from a different PC just using software.
* If you're doing NAS, the network connection is the bottleneck anyway so there's no performance gain from HW RAID controllers
* Consider ZFS
* RAID is never a substitute for backups. Backups must be offsite or the thief/fire/whatever will take them too.
* Cloud backups are a nice theory, but how long will it take to pull 2TB down your DSL line with a 100GB/month quota!? Forget it for the bulky stuff like scans & video.
* a couple of 2TB hard drives in a drawer at your work/friends/parents' house is pretty secure, don't forget to update them monthly or whatever
* rsync is your friend!

the longer version of this post...

[Daniel Stone]

Power conditioner is one thing, a decent battery backup with an integrated conditioner (ala APC) is a better option IMO...
It allows you to have enough time to finish gocopying, transferring or whatever you need to do, without a hard shut down.
I would not recommend leaving it running unattended, not unless you have a generator wired into your homes electrical system.

Dan

[Corran]

Thanks polyglot! Lots to chew through. I am definitely leaning towards RAID 5 or 6 with 4-5 2TB drives.

Have not heard of ZFS before. Will investigate.

I've built computers and been at the forefront of technology for most of my life, but somehow or another all this RAID mess has never come up. Grateful for your input!

[Corran]

Daniel - I have a battery backup system for my location audio gear. Probably will get another one for this system when it's completed and running.

[Preston]

I recently read some reviews of Synology products at AnandTech

Here are the search results for Synology at AnandTech.

If I were considering an NAS, I would be giving their products a real close look.

--P

[polyglot]

No problem; PM me if you have any questions.

UPSes are a good idea too but if you're using a good operating system with a reliable journalling filesystem (e.g. ext4) then a power failure is not a data-loss event. It's best to keep the journal on a separate fast device though, like your SSD; journal linearity in conjunction with RAID can be a dangerous assumption. And make sure you create your filesystem (block stride etc) to match the RAID settings.

Second-hand UPSes are often very cheap because they have dead batteries (I got a 3kVA unit for $150, it just needed $120 of batteries and was good for another 3 years). Buy more batteries off you go!