Storage.

[braeside]

Braeside - Just curious, what is your total investment in the (4) 1TB drives and the USB2 Drobo unit? What does it cost to do a 4TB firewire Drobo 2? FWIW, it costs me about 200 bucks to store 4TB on DVDs. Yeah, I could use better DVDs, but to this point I have not had a problem with any of the name brand DVD media I am using. I would guess some folks may have more $$ invested in Drobo units than I have invested in my two main systems, monitors, and storage media combined. I may not be using the greatest system in the world, but it works for me, and I do own and have archived 200,000 Minolta/KM/Sony image files over past 6 years.

-Sonolta

That's a huge number of DVDs to store.

You are right - it was more than $200 - Bare 1TB hard drives SATA are about £120 at present, probably less in the States. With the Drobo you can start off with the cheapest per GB drive available at this time and as your storage requirements increase, swop out to bigger drives.Drobo FW800 is about $499 in the states, but you need to add your drives to that - not cheap. It is however an elegant system that is expandable in capacity, sort of like RAID, but without the configuration required. - RAID for Dummies?

My use of the Drobo is not primarily for archive of photos, but for is as a Time machine backup media for my whole computer, so that I can easily get back something that I have lost/deleted/corrupted. In addition to my Time Machine data, I still have bags of space for keeping less important stuff in another partition on the Drobo (Movies etc) and they are always available, and protected by Drobo's data redundancy.

I use a separate (offsite) FW800 HD (1TB) as a clone of my photos/HD so that I can get back up and running if the computer is stolen/fire/lightning.

I do also stick some stuff on DVDs as well as you might have guessed I'm belt and braces. Especially after having read "The DAM Book" by Peter Krogh.

I don't know anyone that uses the exact same system for archiving / data protection - some go overboard, some do absolutely nothing

[Alain]

Hi

Using DVD's for backup is cumbersome and it's a safe practice to always burn two identical sets.

HD's are probably cheaper per GB and far easier to use. I my experience backups are made more often.

Using write once DVD's has one advantage, they are protected against accident -user error- overwrite.

Oh and try to get at least one copy off site (family/work/...).

As a DAM (Digital Asset Management) solution I use IdImager and I'm quite happy about it.


Alain

1) DVD's are not that cumbersome. One of my file cabinet drawers can hold 6.58 TB....the entire cabinet will hold about 26.32 TB.

2) HD's are not far easier to use...I have never had a CD or DVD fail and I have lost 3 local hard drives and one server drive in the past few years.

3) HD's are not cheaper than DVD's...I am spending maybe $25 for 470GB of storage.

4) HD's are less reliable than CD/DVD

5) DVD's have way more advantages than that one. Lose a 1TB hard drive and you have lost as many as 300,000 files. Lose a DVD and you have lost 1/3 of 1 shoot, maybe a couple hundred files.

6) Some external HD's start acting funny when a few of the same models are mounted on the same system, an older model Maxtor drive comes to mind. I own over a half dozen external hard drives and they are not immune to problems.

Maybe you like using hard drives only for storage and backup, I prefer using both disks and drives. Again, I have had 4 hard drives go down in six years, and I have never found a backup CD/DVD that had failed on me yet.

-Sonolta

1) My experience is that burning DVD's is postponed, HD backup not. DVD + automatic verify takes some duration to do (but something else can be done in the meantime.)

2) I did some tests on brand name CD's after a few years and there where several defect --> Always burn two Well I do keep several backup HD's to, with one at a different location.

3) I'm at 0.11 euro/GB with HD's, for that price I can't get decent DVD's. But there's a quite substanial tax on DVD's here, before that I could buy Taiyo Yuden DVD-r for about 0,50 euro/piece, which is about the same price as HD's.

4) If kept in a good storage place both are rather safe, but not save enough for one copy (well you need two locations anyway.)

5) that's correct

6) I never keep my external drives connect to my system, only when needed. I also keep most of them in a another room. Actualy even in two seperate rooms.

A dual system : several HD backups and DVD backup will offcourse give more protection.

[HUM469]

Sonolta, looks like you have had a lucky run with no DVD failures. I was reading an interesting article a couple months ago that said that optical media isn't as time proof as once thought. A number of old CD's failing has told me there might be some truth to it (and no the CD's had no scratching what so ever). Apparently the foil layer can degrade over time thanks to the air quality of the storage location, proximity to heat, light, and anything that might give off any kind of smell. The article recommended re-burning all important disks onto new media every 5 years, which is ironic because I have been into digital photograph for 5 years now. It sounds like time to start re-burning some of my stuff I suppose.

That being said though, I use all of the above. I have all photos and important data on a pair of network drives on opposite ends of the house. There are also 4 machines in my possetion, all of which happen to keep different types of photos, and again occupy different parts of the house. All the photos I really care about are uploaded off site to a couple of different online storage options, and every photo is burned to a pair of disks, one set in the house firebox, the other off site. Of course I am not as dilligent as I should be about backing up, and sometimes stuff does not make it to disk for a month, but still with all the hard drives running around that update daily, I still think some really freaky things would have to happen for me to find myself completely shorted. This I think is one of only a couple cases where I hope we are all throwing our money away though, and will never need a massive recovery effort. Good luck to us all on that end!

[Hawk]

Actually, few years ago CDs quality was way better than today. Also slower burn speeds were better as slower burning is far more precise. Today only brand CDs/DVDs offer good quality. Personally I use TDK or Verbatim media for long time storage + some cheap no name media copies for daily usage. As for CDs/DVDs that get defective as time passes, yes, thats 100% true. However when it will happen depends on disc quality. I remember reading about some laboratory tests where they used kind of artificial method to speed up media aging. Most well known brands lasted for 30-50 years at least, some discs even 100 years equivalent. Cheap and/or no name discs were dying at various age. Some lasted 5-7 years, some only 3, some 1 and some were unreadable after 3-6 months. Why it is that way? Two reasons:

1. Most cheap media are in fact rebranded TDKs and other well known brands that didn't passed quality tests in factories.
2. Almost all discs are using organic based substances for dye/layers. It is normal that any organic substance will degrade as time passes.

[Hawk]

OMG, you've posted quite big piece of informations. Now... let me read it and write something here in the meantime First of all I know about media classes, I know that brand doesn't matter (theoretically) but factory where discs were made does matter. However with CD/DVD identifier you may check that specific brands are usually selling discs made in same factories. Thus buying Verbatim media for example I may be almost 99% sure they were made by Mitsubishi Chemical Corporation which is known producer of 1st class media. So... by choosing brand (Verbatim) I'm choosing known good quality discs because I know in which factory they were made.

Now some information about writing speed vs final disc quality. Sorry, no web page and no person will convince me that slower burn speeds doesn't improve quality. Why? Because I've performed following tests some time ago:

1. Write an copy of CD audio disc at 1x or 2x, then write second one at 48x or 52x. Now put both in high-end audiophile CD player (the one that cost way more then 500 US dollars). While disc written at low speed plays perfectly you should be able to notice constant hiss when playing disc written at high speed.

2. Write some medium quality DVD disc at low 2x or 4x speed, write second one at high 12-16x. Then run Nero CD/DVD or any other application able to do quality scan of disc. When they're finished compare number of C1 and C2 read errors on both media. How come that discs written at high speed get much more read errors? And yes, most modern CD/DVD drives will correct them so you won't even notice them in daily use and may ignore the problem. However let me have different opinion in that matter

3. Get some old CD or DVD player. Burn audio CD or DVD at low and high speeds again. While discs written at low speed usually plays well the other ones may cause player to do excessive error correction or not read disc at all. That test however may not show difference sometimes, depending how good error correction player has.

BTW, I'm writing data DVDs usually at 8x, data CDs at 24x but audio CDs at 4x max

Dye color and type... Yeah, I know all of that. I have some discs burned 12 years ago. Most of cyanine based discs have read errors now. Sometimes few, sometimes more. Few discs doesn't read at all (those usually have different color than other ones, they somehow got darker when compared to working ones burned in same time).

Burner + firmware matters. Yeah. I know that too. Even my oldest Yamaha 2x CD writer had specific media data included in firmware that was affecting burn quality. Which reminds me of converting this 2x writer into 4x one (by removing one resistor and flashing modified firmware from 4x model)

Anyway, enough about media here. Thread is getting offtopic

[Hawk]

Dude I have burned up a half dozen burners or more...

So do I and that doesn't mean anything except my experience is different than yours. At least I don't blindly believe in what I've read on the web/forums/manuals/technical specifications/documentations/wikipedia/whatever and that saved my butt many times.

Sure you do, you think you know everything

Its your right to think anything you want, its free world. No more comments from me about CD/DVD media. I prefer to get my camera and go practicing instead of arguing in some pointless flame. Peace!

[Hawk]

Of course lab tests and the cold hard facts mean nothing to you, Hawk.

I've just seen too many lab tests being inaccurate in last few years. Mainly PC hardware tests, but it was enough to make me suspicious about any lab tests

[David Kilpatrick]

Cool it here down the 'weakest link' route folks! It's a complex subject, and the only test we ever did was to hang a DVD out on a washing line for 3 months.

Nothing was readable (in fact the entire item delaminated and the surface fell off in tatters). So we concluded that DVDs are not archival

But I've got everything from the very first CDs ever written on desktop systems, nearly 20 years ago, to the first ever Kodak Photo CDs, still filed away. So far they all remain readable. The least successful archive medium I have used has been DDS tape. My entire archive 1993-2000 is unreadable despite keeping old machines to run the software and buying additional drive units. It just proved to be an unworkable archive medium dependent in complex indexing to even read parts of any tape. At least my earliest CDs, SyQuests, MagOpts, Zips and Jaz files remain (just) readable. 'Just' because for all except the 100Mb Zip and CDs I need a SCSI equipped Mac with an old system.

Our original 20MB SCSI HD from 1984 (external SCSI HD for the Mac SE) has been kept, and remains readable. Richard has a room full of Mac 128, Mac SE and various generations of stuff and sometimes he'll fire one up and run some of the original software; it's great to return to the simplicity and speed of that monochrome GUI and truly tiny program and data files!

David

[David Kilpatrick]

Just a rather appropriate thing - my 640GB Iomega Ultramax, a high speed striped RAID 0 twin disk unit, which acts partly as a backup (and then gets backed up again) has had a failure today due my UPS interrupting the power and my failing to switch the drive off before resetting (I have finally disconnected the UPS since it is the only thing in my system causing problems). A couple of other drives were also not switched off but are unaffected. The problem looks like a damaged power supply not HD damage but I'm waiting for their support to come back to me. As far as I can see it contains two fairly large keyworded final processed file catalogues which are not on my library backup, though all the raws with XML sidecars are separately stored (I keep raw archives on a different HD to processed work). It also has a lot of old raw files (more than a year) but the processed files from those are separately archived again. If I do eventually need to recreate a period of work, Alamy will send me an .XLS output of my keywording etc which is fairly easy to apply back to a new set of Lightroom exported files. Nothing is disastrous as this unit does not contain any of my business/work archives, it was just for photos.

One thing which comes out well is Lightroom. Lightroom has saved a full catalogue of one of the shoots which could be lost, with the ratings of the files, the conversions, etc all intact - and that is in another location. By opening that catalogue again all the files can be re-created very rapidly, unattended. However, I'm hopeful that Iomega will confirm a power supply problem - normally, when you plug this unit in, the front light is dead - you press the button, it lights blue when the drive is powered up, and then flickers during access slightly. What's happening now is that when you plug the power supply in, the blue button light flashes/flickers almost regularly without being pressed, and pressing it does nothing.

David

[Javelin]

I used ACDSEE to make a bunch of DVD's with all my pictures on them, Seems like a nice automated system. there doesn't seem to be a verify routine for it though which would be nice. So I took 2 of the DVD's and imported them into lightroom to see how that goes and it worked except if the DVD isn't i the drive it hasn't got a way to tell you what Disk it's on. on one DVD it used the volume label in the library as long as the disk was in place but on the other it name the drive instead. So if your media isn't on line how do you find which media to use to find a particular photo ?

[Javelin]

Theres a lot of good things about LR. but for the life of me I can't add my DVD based pictures into it. the instructions say to hit the import button and just follow it all through. well it works fine until I try to add a second disk of pictures. I even tried importing by adding a folder. what happens is I get a new list (this is a group of folders with it's own divider and online indicator) with the DVD name on it and all the pics from that DVD in there nice and neat. but as soon as I put a second DVD in the name of that first lists changes to the second and the import routine wants to lump all the 2nd disk's pictures in with the first instead of making a new list of folders which is what I expect. any DVD's beyond the first will import OK except in the same folders as the first but there is no way to tell at that point what disk certain pictures are on. I have my most recent pictures on my hard drive and that has it's own group with the label and online indicator I just can't get it to differentiate between the DVD's so as they are recognized as separate disks with subfolders and pictures on them. it goes wrong as soon as I put the second disk in the drive. I just blew 2 hours trying tomake it work.

[David Kilpatrick]

Here's an update on my Iomega drive issue. I believe the power supply is broken - the AC adaptor is supposed to deliver 5v/12v, and tested, it's delivering 5v/18v. I do not think the twin RAID 0 striped drives have been affected at all, just the PSU/controller in the casing (this is confirmed by the USB hub not working although the Firewire hub still does, in the dead drive). Iomega are very generous and will simply replace my entire unit with a new one if I return it, FOC, under warranty - but they refuse to rescue the HDs from it and put them in a new casing. They would just throw away my data unless paid to 'recover' it. They also tell me that if I dismantle the drive, I void the warranty.

So - I've bought a new 1TB Iomega identical to the failed drive. First I'll just try the new AC adaptor with the old drive, if that does not work, the disks will be swapped over. Opening the drive only involves four conventional screws and they should not be able to detect this. Hopefully I can then copy the missing content of this backup disk (about 500 out of 7500 image files, plus a stack of old original raw image folders - all converted in the past, but I don't like losing them). Then put the drives back as they were, configure the new 1TB as a RAID 1 drive (meaning it will only be worth 500GB of space, but very safe space), and put that on the system for current work backup. I've also bought another Iomega 1TB drive, this one Gigabit Ethernet network attached so it will not be part of the main computer configuration. This will be configured to do overnight incremental backups.

When the replacement 640GB drive arrives, it will be configured like the last one RAID 0, or maybe two plain HDs, for high speed access on Firewire 800. It will then become my primary Photoshop scratchdisk and 'work in progress' store. That will free up an existing 250GB USB drive, which will become a second copy of my picture library.

A warning about Microsoft/iView Media Pro - before creating a large catalogue, get your folder/directory heirarchy organised. I copied all my library files to a new 500GB HD I bought for the purpose in February (this is why I did not lose all my PPd final images, just some work from March and April). Yesterday I added May-July finished folders to this, but I just dumped them on the drive with the other files. I then catalogued the drive at just under 7000 images using iView Mediapro. This was a tedious process, trying to do it automatically in one go resulted in repeated crash/quits for MediaPro. Older TIFFs and JPEGs (2003-5) took a long time for thumbnails to rebuild. JPEGs created by CS2 onwards had their thumbnails appear instantly and MediaPro only had to import the metadata; obviously these later JPEGs have better embedding of thumbnails and previews. Eventually I had to add the folders one at a time, saving the catalogue between each update. This eliminated the crashes, and produced a reliable fully searchable photo library which is what I needed to recreate.

But I decided I wanted to move all these folders into a new overall folder to keep the disk tidy, so I could back up some other stuff into a different folder. MediaPro won't have it! It has used absolute paths to all the files, not relative, and if they are placed anywhere except on the root directory of the HD, the catalogue just hangs for ever on opening. I had this problem once before when trying to copy my whole library to new HD; to keep the iView catalogue usable, I had to give the new HD the same name as the old one. If there's a prefs setting in iView to switch over to relative paths I have not found it.

Every one of my library folders has a small iView MediaPro catalogue inside the folder with the images, just cataloguing that folder. These catalogues still work if the folder is backed up on to a DVD. If the iView catalogue was created outside the image folder, it won't work, even if you copy it on to a DVD and leave it outside a corresponding image folder. I am not an expert in filesystem heirarchies so my use of absolute/relative here may be confused - part of what iView does seems to be relative, but the catalogue must living in a directory which does not change name, and the folders or files it catalogues must be in the same relationship to that directory.

Despite all this I still find MediaPro the best solution for creating a searchable catalogue of all my finished images.

David

[Javelin]

on your power adapter issue. usually those power supplies aren't regulated and a lot of the 12v one will put out 20v with no load on them. once there is a load and the load is sized properly it'll give you a nominal 12v. if your reading was 18 volts with it hooked to the unit and it's supposed to be 12 then the unit isn't drawing any current and the fault is likely upstream of the adaptor.

A lot of those boxes are just that empty boxes wth a circuit board and mounting for the hard drives, if you swap the drives over be sure the order the drives are in remains the same. if there are cables in there it may be confused and don't take it for grated that the first drive you see in side correspods exactly with the older unit. I'd just be cautions about confusing the raid array when it starts up and the drive order is out of wack. It shouldn't in theory make a difference but you never know if there is some proprietary things going on in there.

For my stuff I think i'm just going to ditch lightroom. it's already been way more trouble than i'd bargained for.

PS. How in the world did adobe reader reach 34mb in size!?!

[David Kilpatrick]

on your power adapter issue. usually those power supplies aren't regulated and a lot of the 12v one will put out 20v with no load on them. once there is a load and the load is sized properly it'll give you a nominal 12v. if your reading was 18 volts with it hooked to the unit and it's supposed to be 12 then the unit isn't drawing any current and the fault is likely upstream of the adaptor.

I got my new 1TB UltraMax - which is being configured as RAID 1 much though I hate wasting 500GB, and will replace my drive for all current work. Tried its power supply and the 'damaged' drive of course fired up perfectly and worked. So it was the adaptor, not the drive. Stupid of Iomega Europe support, they would have persuaded me to risk losing my data to get a free replacement drive when only a power supply was needed.

I was able to copy the few missing folders around to make sure I have my usual two backups, but found I had done something very stupid and erased 130 finished images taken my my daughter in the USA - along with the raw files, and the iView catalogue, all the captions and keywords. This happened because I was getting tight on disk space and I thought I had copied them to the backup. Now I have had to ask Alamy whether they can send me back the 130 files (fortunately all uploaded in a single session) and .XLS spreadsheet for the metadata, if I send them a suitable USB pendrive. The total data should be well under 2GB. As one of these shots from May just sold for $350, I don't want to lose the ability to file them with other libraries - or my own future on-line collection. But I won't tell Alamy that.

David