RAID archiving software

[Hollis]

I am about to set up a RAID system to archive and protect my files. I was wondering if anyone had any use using Lightroom or iView with a RAID setup and what the limitations/benefits are. Any help is appreciated.

Hollis Bennett

[Mike Boden]

There are many different types of RAID setups. What you need to ask yourself is why you want a RAID setup and how you plan to use it. Read the information at the following link to decide what works best for you:

http://en.wikipedia.org/wiki/RAID

From there, you'll need to make a decision as to whether you want a software or hardware setup. There are pros and cons to both. After that, create the RAID. Once that's done, it'll act just like any other hard drive on your system, irregardless of what application you use...i.e. Lightroom, Photoshop, etc.

Now if you're looking into an application to archive and backup your files, that's entirely another topic.

[Daniel Geiger]

I second Mike's points. RAID is a way to combine multiple drives for various reasons (larger disk, partial redundancy [e.g., RAID5], to full redundancy).

The second question is physical arrangement: within main computer, using multiple drive bays, or an external approach either through firewire, USB, or Gigabit ethernet and NAS-box.

The top two go a bit together, as your CPU may have some RAID controller software, but with a NAS box, it will be driven by the NAS.

Third, do you plan on accessing the working files directly from the RAID array (so possibly rather go with internal drive bay set-up for faster access), or do you use the RAID as an additional back-up? If the latter, then question as to back up software arise.

What I do is, use a couple of Buffalo Technologies 1&2 TB NAS boxes through GB ethernet and use Retrospect as backup scheduling software. Retrospect is an awful program as the terminology used is completely incomprehensible and infuriating, but I'm not aware of anything better on the Mac.

[mrladewig]

Like Mike mentioned, RAID will configure your drives. Your application will just see it as a disk. The bigger difference will be whether you maintain the photos on your PC or on a NAS. If on a NAS or file server, there may be some issues with Lightroom or other file browsers related to how fast the data can be transferred over the network.

I do something similar to Daniel.

I have a single working disk on my PC for photos. I run an automated backup to my Buffalo terrastation which happens to be configured as RAID 1. I have an additional copy of the files on an external disk kept at my office. I'm on PC and use Syncback SE for backup.

[Daniel Geiger]

Forgot to mention, I run my terrastations as RAID 5, so a compromise between more storage (75% of total 4-disc space vs. 50% with mirroring) and data security (only parity bits distributed, vs. full disc images).

[Hollis]

What I would like to do is run a mirrored, external 2 bay RAID array in conjunction with my internal drive which will be my work disk. I produce alot of work (very large files and I shot a ton of digital - upwards of 50gb a job) so I need alot of space. My ideal setup would be to use my internal drive to do the work and quick access type stuff that will be faster than on an external and then backup to the RAID. When I fill the RAID, I will eject the two drives, keep one at home in a fire-proof box and the other will go to the safe deposit box. I just need to know if Lightroom or iView will 'see' the file (through the preview catalog - which will be hosted locally) and when I go to access it, tell me what drive it is on (i.e. RAID set A, B, C, etc.) so that I can mount that drive, retrieve the file to my active, internal drive and be on my way.

[Anthony Lewis]

I have a RAID drive because we do video work - Digital Betacam and HD, several terrabyts. Whether it is mirrored, redundant, or striped, it is still nothing but ONE hard disc drive. I've had mine go down a couple of times and it is one of the most well known brand names. (X-Serve RAID) Each time, I am offered the option of wiping everything and reinstalling the software, or if I really need all the information it can be sent off for hardware disc recovery 'whatever', and that will cost me several thousand dollars without guaranteed results.

Admittedly my unit is an early model, still only a few years old, but it is unreliable particularly for archiving.

Someone wrote on this forum that the words 'digital' and 'archiving' are an oxymoron. The more I look into this situation, even though these words are an exaggeration, archiving digitally is problematic.

In the video world the only way to archive reliably is to archive onto tape - even to this day - and it must be in a uncompressed format such as Digi Betacam or HD (not HDV). As far as I can see that same goes for 'stills'. It needs to be archived onto a CD in an uncompressed format such as TIFF or Adobe. Even then if you wish the CD to last more than 10 years it will need to be glass mastered - lots of money.

I would forget about any form of hard disc drives or even solid state drives for any form of archiving. I know others may know technically more than me, and they may be right, but this is my opinion.

All I can say is thank goodness I originate everything on film, and that tiny little piece of celluloid is carefully stored away for safe keeping for a hundred years or so. It is not perfect, but I believe it is still the best we have going for us - good old technology. Stick with what is there blindly in front of us - FILM - and don't be blinded by technology!

[Steve Kefford]

What I would like to do is run a mirrored, external 2 bay RAID array in conjunction with my internal drive which will be my work disk. I produce alot of work (very large files and I shot a ton of digital - upwards of 50gb a job) so I need alot of space. My ideal setup would be to use my internal drive to do the work and quick access type stuff that will be faster than on an external and then backup to the RAID.....

I was thrown by the title of the post, as RAID is not sw and it is not an archiving system. WADR, you are combining your requirements (archiving sw) with a solution (RAID), i.e. you want an archiving system, that you think could be solved by iusing RAID. It is best to isolate requirments from solutions.

I would doubt if a RAID mirror is your best solution. RAID can cause a few problems, and mirroring is usually used in "mission critical" apps, where a system can continue when a disk goes south. I would look at a software based synching solution - this would then be a sw archiving solution.

Steve

[Lenny Eiger]

I have been building RAID's for, well since the early 90's, however long that is. RAID's are usually faster, especially striped scenarios, and often are what is needed when saving large amounts of data. I just built a 5 Terabyte RAID for about 2K. Hard drives die, this has to do with the 2nd Law of Thermodynamics, (or just basic Physics) and you can't fight it.

The cardinal rule when building a RAID is that for every bit of storage you create, you must also have a backup. If you do that, when a drive fails, you simply use your backup for a few days and then get a new drive and reinstall. RAID's are great, they make things possible....

As to Lightroom or Aperture, they are great if you are using small file (less than 100 megs or so). They are useless if you are scanning or printing with large files that extend into the gigabytes. I also don't like that they have proprietary formats that mean once you start with their idea of how your business works, you are stuck with it. If it matches, great. If not... it stinks. I am at odds with Adobe these days as I like to use large files. They refuse to fix their memory issues and instead focus on what beginning users need. Nothing wrong with helping out the beginners but I actually need the advanced features to work properly. As such, Adobe is a poor software vendor, not taking care of its base. It opens up the possibility of another vendor coming in... Just my 2 cents....

Lenny

[Daniel Geiger]

... I just need to know if Lightroom or iView will 'see' the file (through the preview catalog - which will be hosted locally) and when I go to access it, tell me what drive it is on (i.e. RAID set A, B, C, etc.) so that I can mount that drive, retrieve the file to my active, internal drive and be on my way.

Although I am not familiar with Lightroom and IView, I have a hard time to understand how these programs could generate previews of files, which are located on disconnected drives (the one in your fire proof vault). You could easily test that by burning a CD-ROM put the images of the CD into the program, eject the CD and see whether the program will give you a location.

I use a database driven system (FileMaker) where I have a relational system for each digital file with CD-ROM number, so with the thumbnail and info on the image, I have also the location of all digital assets come up (many to one relationship). You could do the same with numbering hard drives. Input of the image file records from the CDs (in my case) is done with an apple script I wrote. Took a bit of fiddling as I am not a programmer.

[Gary Nylander]

I have been using iView for a couple of months now, it makes a preview ( smalls size photo ) of everything that you want copied, even files that are on external drives or discs, that's what I like about it, I can search for a file name on iView and it will show me the picture(s) and tell me what drive or disc that particular image is on.