• Login
  • No products in the cart.

How to protect your images: RAID, DVD, Online Storage or something else?

Several people have asked me how I protect my important files, so I decided to share my answers in this article, summarizing everything on one place.

In my opinion, storing your files in a back-up is a must not only for a professional photographer, but for a serious amateur. Who wants to lose photos or videos of children, parents and grandparents, or friends? Each passing year these images and videos become more and more valuable, and storing them securely is important.  Most PC (or even Mac users, I guess :-) had some experience with loss of data:  HDD may just stop working, file system can be corrupted, PC can be infected by a virus and data erased.

The task is easy when you have 10-100 GB of data:  burn a DVD (better to use double layer, up to 8.5 GB on one disk) and store them in your fireproof safe or friend’s home.

However, if you have Terabytes of images (like me), DVD is not going to work for you very well.  Here is why:

1.            DVD is easily damaged by heat or physically.

2.            Plastic can degrade with years, making DVD non-readable.

3.            It’s not easy to manage hundreds of discs.  Instant access isn’t possible.

The better option is to use hard drives (HDD),  similar to using a diskette  5 years ago.  With current HDD prices of $90 for 1 Tb disc, and docking stations for another $40, this does not sound crazy, right? Plug in the docking station to you PC, insert a HDD, back-up everything you need, remove a drive and store it on a shell with the label, same as you did with a diskette. But this will be 1Tb diskette! You can swap these disks on a fly:  pull out  one and insert another.

Usually,  for a serious amateur, 1 TB disc will be enough to store images from a whole year, most of professional photographers may not have more than 1-2 Tb per year of  the images they keep.

Next, more advanced option is to build a RAID array(s) as storage. RAID is a redundant array of independent disks (by Wikipedia), which appears as a single physical drive to OS.  You would need to use RAID1   or RAID 5, both will give you a 100% data protection even if one HDD will fail. For more on RAID types, please read   Wikipedia.  There are also available external RAID storage solutions with Ethernet or USB interface (examples).  Usually they cost more than internal build-yourself RAID arrays,  slower (by the interface limitation) , but more convenient in use.

Storing valuable photos and videos on a back-up HDD drive is a very good option.  However, it still does not guarantee your files will be safe. Unexpected disaster can crash all your hardware, virus or RAID controller malfunction can wipe out everything.

Recently I have discovered another solution:  online distributed storage.  It was there for a while, but only recently prices dropped significantly.

Currently I am using two services: Amazon Simple Storage Service (aws.amazon.com/s3/)  and Mozy (www.Mozy.com).  There are few more, if you Google it.

Both services are quite affordable:  Mozy costs  $56 per year for unlimited storage plan,  Amazon costs around $0.150 per GB.

Advantages:

°              Much more reliable compared to in-office data storage.  Just think what should happen to Amazon if data centers lost even one byte of data.

°              Accessible from any Internet connection.

°              Silent monitoring of your folder,  backing up the data, while you are not working.

Disadvantages:

°              Probably only one:  Slow data transfer rate. It is really limited by speed of your internet connection, but imagine how long it would take to back up 2 Tb of photos I have. At least a few months!

This is what I have for today :-)  I would be glad to hear any other options and experience related to the subject.

UPDATE 11-19-2009:

Thanks to people who contributed comments for this article I have to highlight  few things I have overlooked:
1.  You can’t really store your data in on-the-shell HD, as it will loose everything, if you do not run it at least every few months.
2. Mozy.com has one big disadvantage: if you won’t go online for more than 30 days, all your stored files will be deleted. So, it is good for day-to day protection, but not for a “store and forget” type of use.

UPDATE 11-23-2009:

Ok, now about Drobo. The difference between Drobo and a regular RAID NAS storage is that Drobo is not using RAID.  They use what they call   BeyondRAID”,   this is (as I understand) some sort of software manageable solution which allows  you to have different sizez HDD installed, wihout thinking how they will work together. Here is what they say:
Drobo utilizes the revolutionary BeyondRAID storage technology that protects data against a hard disk crash, yet is simple enough for anyone to use. As long as you have more than a single disk in Drobo, all data on Drobo is safe no matter which hard disk fails. There’s no need to worry about anything else“.
So, you can have 4x500Gb disk arrays, then start replacing 500Gb disck with 1Tb, one after another, and everything will work fine.  Sounds quite impressive!


16 responses on "How to protect your images: RAID, DVD, Online Storage or something else?"

  1. Hi guys, I see this conversation is from 2009. Any new suggestions for backups?
    I’ve come to problem with the TB’s of storage on multiple HD’s for photography strictly and I’ve come to the problem of having to backup personal things like iPhone, iPad, and three different laptops in the house. Cloud based backup is to slow for transfer speed. I”m looking into some kind of central network backup solution.
    I’m starting to see a couple different options like WD Mycloud backup up to 8TB with 1Raid or the Seagate Central device. Has anyone looked into these? I’m trying to keep a budget of around $500.
    Thanks,
    Cory

  2. Love my DROBO
    Three years running and not a glitch.
    With the price of memory so low and getting lower I can’t any reason not to use this system and a off site system.

  3. Storage Protection is not easy,

    DVD’s degrade over time but are a good solution if you copy or reburn them time over time, make sure that you check them regularly and reburn them.
    Portable HDD’s are better, but as stated above you need to spin them. also after being used often. when you do not use them for quite some time the possibility of hdd failures are a big risk.
    A RAID-1 or RAID-5 Storage solution that is available in a NAS (network attached storage) or a drobo (RAID-1 or RAID-5) through USB is a lot better, HDD failures will happen. Because of moving parts and heat the HDD’s tend to fail over time. but using a RAID solution with hot spare the data is well protected. In this the sky is the limit. and you can buy a fairly cheap conceptronic with 2 disks in raid 1 but you can also buy a promise V-trak with 7.2TB of raw storage in a RAID-5 or Raid-6 set.
    The only drawback of this is that is safe in 1 place but all the photo’s can be lost by theft or fire.
    that’s why it is better to use 2 storage appliances. and replicate the data between them. for example use 2 qnaps appliances. and put 1 in your home and the other one in the office, house of a friend, parent, work or a datacenter. and replicate them so that they stay the same. it will make sure that when 1 is lost the other one is still there. and the data is not lost.

    for every budget is a solution. portable hdd’s are a good solution. for 45 euro’s you will have 500GB external 2,5inch hdd. if you have 2 and exchange them regularly then data protection is already quite good. decide on what you want and need.

  4. Hi
    After reading Luke’s article, I will use my backup HD’s more often….. Didn’t know that these HD’s need to spin so often to keep their data. Thanks for the info!

    Next to my coputer is a small 20 minute fire/water-proof safe with 2 portable USB HD’s. In a wall-mounted vault on the attick is another USB HD. These HD’s grew from 300GB to 500GB, to 1TB and growing.

    Here is my schedule (in theory, I still have to do a backup very urgently):
    -Shoot photos during the day and store them during the day on the datatank
    -Normally I have enough CF for 1 day work, so the datatank is not needed
    -Download from cameras to computer.
    -Only after a backup of computer to the close-by portable HD, I erase my CF cards.
    -Only after a backup of computer to both close-by HD and attick HD, I erase the datatank.
    Of 5 possible locations, at least on 3 locations my photos are stored.

    What I still need to arrange is an out-of-house solution, in case a plane would drop down :-( A HD with my parents is not an option because that would be used only 1-2 times per year.
    I will look at the online options too.

    Bye Joris

  5. Thanks all for the posts and info. I just went to the Drobo site and will be purchasing one in the very near future, this handles mirroring your data without the limitations of a Raid system. I will also keep two separate backup sets on external hard drives. I am still working out a way to use optical discs that will work with my filing system. An excellent resource is “The DAM Book” by Peter Krogh – O’Reilly Media. http://thedambook.com this is a must have and read for serious photographers.
    Cheers!
    Dan Holmes

  6. Luke,
    Thanks for sharing, great info!
    I was not taking into consideration physics of HD you have pointed on, and you are so right..
    I can’t just use HD to store data on a shelf. I’ll probably look into something like Blue Ray drives as for a optical alternative to HD or DVD. I need to be able to store the whole photo-session on a single disk, and 8Gb won’t be enough.
    Alex Koloskov

  7. Guys, my world is that of high end broadcast & Film Post Production – so if you think that you might never run to the end of 4Tb of storage … the guys I consult to, are talking about Petabytes of daily managed storage! Yeah, crazy, right? But, they, like you, are working with an absence of shooting media that is a recording source (remember that funny organic stuff called film?) and have pretty much all moved to digital file based media. I really want to throw my two cents worth in here just because this area is something that has been the focus of my world for the last seven or eight years at least. What I’d like to offer is the pros and cons of some of the systems available to you.
    Hard Drive (HD) vs Optical Disk (OD) approaches.

    HD is cheap and gets cheaper. It also gets corrupted and can fail. What’s worse, it will, unlike most forms of storage, fail more, the less you use it – strange right? Well, this is because, the disks are iron oxide (fundamentally) baked onto plastic disks. If you don’t spin them, make the heads access the information on them on a regular basis, the iron “forgets” to stay magnetic, and the magnetism of the disk is how all the information stays recorded. So, one thing you have to promise yourself that you will do is to get out your backups, put them into the caddies and do a read/write operation once every three months or so – the rust stays magnetic, the information stays intact. Good. Now, the other thing that you should contemplate is way more storage than you will ever need! if you have 5 DVDs worth of storage now – bear in mind that you have over 20 gigabytes of storage – and the chances are that the camera you shot those files with was only recording half the information that the camera you have now or are lusting over now can record, right? Go bigger than you think you need and be prepared to upgrade it. And seriously think about going to a minimum of RAID 5 technology – that system has one redundant disk that with failure can be removed, replaced with a similar drive and the rebuilding of your library takes place in the background. RAID 6 will let that happen with two drives failing – and that really is a pretty hard to imagine situation (short of a power failure).

    Now, I’d like to offer up some opinions on Optical Drives.

    The start of the thread suggested that there were some points of failure possible with the media type. While there is a great deal of material out there on both sides, I can only offer up my own experiences. As of now, I’ve been authoring DVDs for almost 10 years now. I still have my original 1X (single speed) burned DVD-Rs and they still read, they still are accessible. Seriously, the one thing you CAN NOT do with them is use them as drink coasters and they will offer you years of storage potential!
    Unlike magnetic disks, there is nothing for them to corrupt them if you store them on a shelf away from excessive humidity, heat and too much light. They are very very cheap, you can set up your archive system to span across many disks if you need a big back up set. That is to say maybe you have more than the 4Gb of material that can be recorded readily to one DVD, then you can tell your Nero or Toast program to burn it to two or three or more DVDs and you simply put your disks back in the sequence that the program recorded the material.
    Or, even better, make more use of one of the simple units of your operating system, Folders, when you break down your images, so that you only need to record the contents of one folder to a DVD.

    Now, the DROBO unit that many of you said very positive things about I have to add that while I don’t use one, thats more about, I don’t use one now. The system offers very high level management of your drives and will permit you to add more storage when and as you need it. It also takes away the need for you to remember when to run your hard drives to make sure the rust stays magnetic, it tells you when there is a chance of your hard drives fail so your peace of mind factor is quite high. If you are more IT savvy, the same effort can be done with off the shelf tools and a little “IT OCD” determination. So that will probably let 97% of the rest of us enjoy the extra time to shoot pictures with a clear conscience!

    But seriously, keeping your material for the ages is in many ways easier than ever, but more about making it a regular routine then ever. And for what it’s worth, while the ideas expressed here are from my own experience, study and research – you have to understand that I could be entirely wrong and that the moment you try the technologies they could fail – that’s the thing about this stuff – it works like this for me, it may not for you. Whatever works for you, is the best system possible.

    Luke
    l’enfer, c’est l’autre

  8. Good morning,

    I use a Thecus N5200PRO unit (www.thecus.com) that can have up to 5x drives in different RAID configurations. Their support is exceptional (needed them when I expanded my RAID 5 config and had a DOA new disk). They even fix my problem remotely and they made sure that I didn’t lose any data while in recovery mode and was able to salvage 100% of my data.

    It’s networked on a Gb switch so I can work my pics and videos from any computer (included my wireless laptops) in my office.

    Rock solid performance and stability after more than one year of use. (backups are done on 2x 1.5TB external Seagate eSata drive)

    This has been my best investment in hardware in a long time.

    Hope it helps.

  9. I’m using the HP MediaSmart home server. It can duplicate files across drives, create a disaster recovery drive, your own web address for access from anywhere, vendor supplied (incremental) backup program. Lots of videos on YouTube to give you an idea of what it looks like and how it works.

  10. The Drobo is essentially a stack of 4 generic drives managed in such a way that two copies of your data are automatically disbursed over the 4 drives. If any drive starts to fail or actually fails, you receive a notification. The drive can be swapped out and the new drive automatically loaded from the redundant data on the three other drives. This also allows you to upgrade to higher capacity drives as you need them. I currently use a Drobo with 4 2TB drives which give me almost 4 TB of useable storage with the remaining capacity used for the backup and management. I have had no problems with the Drobo. It may be the most reliable piece of computer equipment I have. Also, there is an accessory that allows you to hook the Drobo to a wireless router so I can access my entire library on my laptop from anywhere in the house.

    • William, the same features has every other external RAID solution, plus, usually built-in Ethernet controller (the thing you hookup to your router) included, not as optional accessory.
      I am not saying Drobo not good, probably it is as good as others (may be better?), but why do you guys prefer Drobo over the others? I was surprised, 3 people recommend Drobo, and I”ve never heard about this brand before.
      So, any special (besides nice look) about Drobo?

  11. I use RAID-1 and then an external drive.

    I just wanted to add a note on RAID, with all the noise out there, some people tout RAID-0 as a redundant storage system. IT IS NOT. RAID-0 was designed for performance but spreads your data on 2 disks. If one is unreadable YOU LOSE IT ALL!

    The way to go is either RAID-1 or RAID-5.

    • Huba, I 100% Agree. I use RAID 0 as my workstation OS drive only, it gives you up to 50% faster read/write speed comparing to a single drive, close to SCSI drives. This gives you a real advantages while editing a huge TIFFs, especially while creating HDR images from 3 or 5 exposures, when it starts swapping RAM memory to a HDD.
      But you should not keep anything important on such drive.

  12. Drobo looks nice, but.. Not sure how reliable they are. I’d prefer Buffalo. Looks solid and serious:-)

  13. I want a Drobo. They have the ability to hot-swap RAID type hard drives without losing any data. The catch is, they start at $500. I think your option above of HDD and put them on a shelf is great. As a pro shooter, I shoot about a TB a year and the idea of being above to rotate them off site is very appealing. That way if the building burns down or some other disaster, I am still covered!

  14. An interesting alternative to a RAID-setup is the “DROBO”
    It provides an interesting grow-path as storage space becomes cheaper over time and flexibility combined with simplicity. The website http://www.drobo.com give clear info.

Leave a Message

Copyright © Photigy Studio Photography All Rights Reserved
EXTENDED! Zero Gravity Photography course: 97% sale  Get It Now
X