• Welcome to Camera Craniums: The Photography Community for Enthusiasts.
 
Members
Stats
  • Total Posts: 62,411
  • Total Topics: 5,704
  • Online today: 239
  • Online ever: 856 (January 21, 2020, 09:07:00 AM)
Users Online
  • Users: 0
  • Guests: 247
  • Total: 247
bhphotovideo
Amazon Spring Deal: SanDi...🌸🌼 Get Ready to Blossom w...Marantz Professional MPM-...Google Pixel 7a and Pixel...JasmineSanDisk Ultra 64GB USB Fl...SanDisk 512GB Extreme PRO...GiaDo You Shoot Photos With ...Which eye do you use with...SanDisk 256GB Extreme PRO...Duracell Plus Alkaline 1....RØDE VideoMicro Compact O...I must be one of the rare...Learning ResourcesPhotography and Time of D...

Got backups?

Started by Jonathan, November 10, 2009, 06:46:42 PM

Previous topic - Next topic

chris@seary.com

Quote from: Hybridphotog on November 11, 2009, 08:59:01 AM
My 'strategy', which has worked for a number of years...
"If the relevant directory/folder is nearing 650MB/4.4GB in capacity, dump the contents to a CD/DVD. Use a cataloging application to catalog the new discs contents."

Why make "backup strategies" complicated?

If you're clearing the folder on the computer after saving it to disc, then you're not doing a backup. You're archiving. As Hevans has explained, you still have a single point of failure. You've just moved the folder structure around a bit.

The cloud is becoming a popular solution for storage and applications.You have experts giving advanced technical solutions, which they can professionally manage. If I had something especially commercially valuable, I wouldn't try to produce the backup strategy myself. However, it depends on the value of the item.

anglefire

At work we have a 6 disk array which can loose a disk or two before there are issues, plus a tape backup - and we are just about to install a NAS with a 1.5Tb drive to back all the roving laptops onto and anyones "My Document" - plus its going to be used to backup the array - possibly with an immediate backup so that its easier to restore the odd file - not seccure from fire - but the tapes are.

We are also looking into cloud backup/offsite server once we get our 10M network connection installed - which would be switched up to 100M if that route was followed!
----------------------------------
Mark
* A HISTORY OF THE UNIVERSE - THE SHORT STORY* 'Hydrogen is a light, odourless gas, which, given enough time, turns into people.'

CPS Gold Member
My Website

Current Bodies:
Canon 1Dx
Canon R3
Canon R5

Sold Bodies:
Canon 350D
Canon 1DMk3
Canon 5D
Canon 1Dx Mk3

Hybridphotog

Quote from: chris@seary.com on November 17, 2009, 08:56:15 PM
If you're clearing the folder on the computer after saving it to disc, then you're not doing a backup. You're archiving. As Hevans has explained, you still have a single point of failure. You've just moved the folder structure around a bit.
Therefore, creating a duplicate disc and storing the duplicate somewhere safe (*not* on a spindle!) and out of sunlight is making a backup.

As to 'point of failure', they're everywhere. A hard drive will inevitably fail, regardless of RAID arrays or whatever. CD/DVD readers will start mis-reading. CD/DVD media will eventually wear out. And I'm sure solid state media has a finite number of writes, before a beloved USB pen conks out.

Cloud?? Those things are currently giving Clacton a grey sky... I don't do buzzwords.

chris@seary.com

Quote from: Hybridphotog on November 18, 2009, 01:33:25 PM
Quote from: chris@seary.com on November 17, 2009, 08:56:15 PM
If you're clearing the folder on the computer after saving it to disc, then you're not doing a backup. You're archiving. As Hevans has explained, you still have a single point of failure. You've just moved the folder structure around a bit.
Therefore, creating a duplicate disc and storing the duplicate somewhere safe (*not* on a spindle!) and out of sunlight is making a backup.
Yip, once again I'm eating my words after jumping to conclusions....    :)
Quote from: Hybridphotog on November 18, 2009, 01:33:25 PM
As to 'point of failure', they're everywhere. A hard drive will inevitably fail, regardless of RAID arrays or whatever. CD/DVD readers will start mis-reading. CD/DVD media will eventually wear out. And I'm sure solid state media has a finite number of writes, before a beloved USB pen conks out.

This is very true, and probably the most salient point in the discussion. You're never going to be able prevent data loss completely. There's always the possibility of a terrifying accident at the firework factory at the end of the street, which will likely wipe out your RAID array and the building that it sits in.

The place to start is not 'which technology should I use to make sure I don't lose data', but rather 'what could I afford to lose?'

In the commercial world, you look at threats and the impact that they have. Data held by the UK government has a greater threat of a malicious person attempting to destroy it than the jpeg files that I keep on my PC. Therefore, I don't hire a firm to store my backups in a site that's impervious to most terrorist threats. Also, the cost would be so much greater than the value of my data.

Next, you look at what you're prepared to lose, bearing in mind it's impossible to give 100% protection (as Dave has outlined). For instance, loss of the last 24 hours of data would not be a problem to me, but to a photographer such as Jonathan this could mean the loss of yesterday's wedding photos. What could you bear to lose? This generally comes under a service level agreement.
Quote from: Hybridphotog on November 18, 2009, 01:33:25 PM
Cloud?? Those things are currently giving Clacton a grey sky... I don't do buzzwords.
Whether it's cloud computing or a vendor taking responsibiltiy for your backups, it all comes down to letting someone else who has the expertise in business continuity manage your data. That means they'll be able to more accurately match your service level requirements to the appropriate technology.

If my data was really valuable, I'd rather have an expert take care of it. It's cheaper, as they do it for lots of people, so you'll get the economies of scale, and they'll also pay out compensation to you if they don't achieve what they say.

Darren1978

At present I don't back-up any of my files  :o

I guess I've never really felt the need to; I've never had any laptop/PC/EHD fail on me....yet.  I'm beginning to think that now might be a good time to start getting into the habbit of backing up my files.  I'm mainly interested in my image files as I don't have too much in the way of important personal files, but those that I do have I would like to back-up as well.

With regard to my photos, this is my current workflow if you like (laptop with portable EHD connected at time of downloading photos):

1. Download photos and import to Lightroom Catalogue which is located on the EHD
2. Process photos and keep any changes/duplicates in same folder on EHD
3. Delete files from camera memory card

Thats it.  Quite shocking really!  I think now that I've got to the stage where I have alot of images, I would really hate to lose them.  I never keep any photos on the hard drive of my laptop at any stage of the photography process.  The main reason being that the laptop has a small HDD and it just gets clogged up.  I only keep software and Word/Excel files on the HDD.

I think I would like to stick to this method as its something I have got used to, and it works for me with the resources I have got.  Apart from the alarming fact that I don't back anything up, I have no idea if the general way in which I work is good or not?  If it is seriously bad then I may consider reviewing my methods in line with any suggestions that you guys may put forward.

Right, now moving on to the actual process of backing files up...

I quite like the idea of this Cloud thing.  I have a rough understanding of how it works (upload files to a remote server/locatation which then means those files can be accessed from anywhere in the world - right?), but don't actually know how to go about it.

I'm guessing that the 'Cloud' is a generic term and htat there are many companies out there who offer this facility?  Do they charge for this service?  What can/can't you upload?  How secure is your information?

Sorry for the long post...looking forward to hearing your advice :)

Forseti

I don't particularly like the idea of you having both images as well as the Lightroom catalogue on your EHD. In the event of the EHD failing then you've lost just about everything. Have you considered placing Lightrooms catalogue file (.lrcat) onto the laptops drive, where incidentally it installs by default so you must have changed it. Unless you're a mega snapper lol this catalogue file isn't really going to use up that much space. You could then set up Lightroom to backup the catalogue to the EHD. Should the EHD fail then at least you've got a copy (the original) sitting on the laptop so at least you've not lost everything. But really, backing up the images is a must and a second EHD doesn't cost that much nowadays.
Canon 7D,  Canon SX1 IS, EF100 f/2.8 USM Macro, EF70-200 f/4 L IS USM, EF17-40 f/4 L USM, Sigma 50mm f/1.4 EX DG HSM, Canon Speedlite 580EX MkII

"Everyone can take a great picture with digital, the knack is to take two" - David Bailey

Darren1978

Thanks Forseti.

I deliberately re-directed the location of the Lightroom Catalogue after I had installed it, thinking that the catalogue would take up alot of space on the HDD of my laptop, which is what I don't want to happen. 

I don't know too much about the in's and out's of LR and how it actually works and manages data etc and probably need someone to explain to me (or point me in the right direction) in simple terms how LR actually stores/manages image files.

I probably have a couple of thousand images, and these are all located on the EHD in the LR catalogue.  This is where I believe the physical image files are being stored, correct?  If so, I would think that all of those images are going to take up a fair amount of space (I've got RAW, TIFF & Jpegs) on the HDD of the laptop?


Forseti

Quote from: Darren1978 on November 19, 2009, 10:20:25 AM

I probably have a couple of thousand images, and these are all located on the EHD in the LR catalogue.  This is where I believe the physical image files are being stored, correct? 

Wrong Darren. Lightroom is a DAM (Digital Asset Management) application. Amongst other things such as registering edits made to your images, it doesn't import those images into the catalogue (.lrcat) - it only saves information as to where those files are to be found on your computer. Here's a work around to demonstrate this. First locate your catalogue file on the EHD and make a copy of it onto your computers desktop. Now disconnect the EHD so that it is offline to Lightroom. Now, going back to the desktop double click the catalogue file which will open up Lightroom. When it does you'll notice that all your images have question marks next to them in the top right hand corner which is an indication that Lightroom is unable to locate the images. This is obvious really because the EHD containing the images is offline. You will never have noticed this before because both your present catalogue AND the images are located on the EHD and when this is fired up Lightroom has a direct link to your image files. Whilst your at it - hover your mouse over the catalogue file whilst it's on the desktop or right click on it and choose Properties. You'll notice how small the catalogue file is compared to the actual number of images that you have. Again this demonstrates that Lightroom is not actually importing images but simply providing a link to where they are to be found on the system ie. DAM.


So just to reiterate - imho you should be locating the catalogue file (.lrcat) on the computer itself (the Preview file will also default to here as well) whilst leaving the images on and a backup copy of the catalogue file on the EHD. You set this up from within Lightroom itself. Now having the catalogue on the computer and the backup on the EHD at least you are protected, not only from losing the catalogue in the event of an EHD failure but also in the case of a catalogue corruption. I always keep at least 3 catalogue backups deleting older ones. Now all you need is a method of actually backing up the images themselves and for this my preferred method is a second EHD whilst using a free application called SyncBack to backup the contents of one EHD to the second. This also backs up the catalogue file once again of course because the original backup is on the first EHD. Believe you me, in this hobby you can't have enough backups.

Indirectly related but the next version of Lightroom (v3) will have the ability to backup the catalogue prior to closing down instead of as at present upon first starting up which is much better.
Canon 7D,  Canon SX1 IS, EF100 f/2.8 USM Macro, EF70-200 f/4 L IS USM, EF17-40 f/4 L USM, Sigma 50mm f/1.4 EX DG HSM, Canon Speedlite 580EX MkII

"Everyone can take a great picture with digital, the knack is to take two" - David Bailey

Darren1978

Thanks again Forseti...I had just been doinf a spot of reading on a LR website and discovered that my assumption was wrong (I had an inkling that it was when I first posted it).

What you have just written i.e using two EHD's, was also mentioned on this particular website too, although the author on that site seemed to promote the idea of having all your images and LR catalogue file(s) on an EHD, which is what I currently do.

Food for thought....

Forseti

#24
Quote from: Darren1978 on November 19, 2009, 11:02:48 AM
What you have just written i.e using two EHD's, was also mentioned on this particular website too, although the author on that site seemed to promote the idea of having all your images and LR catalogue file(s) on an EHD, which is what I currently do.

Food for thought....

Yes, food for thought indeed.  ::) The main disadvantage to that imho is that the drive head is tearing backwards and forward reading data from the image files and writing metadata etc to the catalogue file which can slow things up a bit. It could also be the case that the author is using multiple computers and that by simply having the catalogue and images on one EHD makes it easier for him/her. The main thing though, whichever method chosen, is to always have a backup of the catalogue and your images. When I'm away from home using only the laptop with it's own LR catalogue and a small portable EHD I simply use the option within LR to 'Export the Catalogue' and then save the resulting file on this portable EHD. Back at home, from within the desktop PC's version of LR I choose the option 'Import from Catalogue' which then imports the laptops catalogue into my main catalogue. Each to his own though, and whatever works of course.
Canon 7D,  Canon SX1 IS, EF100 f/2.8 USM Macro, EF70-200 f/4 L IS USM, EF17-40 f/4 L USM, Sigma 50mm f/1.4 EX DG HSM, Canon Speedlite 580EX MkII

"Everyone can take a great picture with digital, the knack is to take two" - David Bailey

Darren1978

Sorry, I should have added that the author was saying that this method is mainly aimed at people using multiple PCs/laptops etc.

I see where your coming from though :)

Forseti

Yes I thought so. One other advantage that I didn't mention regarding having the catalogue file located on the PC/laptop is that even when the EHD containing the images is offline you have the ability to work on your images (thumbnails will be shown) adding such things as keywords, rankings etc etc. - the only thing you can't do is edit the images themselves. This saves on wear and tear to the EHD because it doen't have to be connected unnecessarily.
Canon 7D,  Canon SX1 IS, EF100 f/2.8 USM Macro, EF70-200 f/4 L IS USM, EF17-40 f/4 L USM, Sigma 50mm f/1.4 EX DG HSM, Canon Speedlite 580EX MkII

"Everyone can take a great picture with digital, the knack is to take two" - David Bailey

picsfor

Quote from: Forseti on November 19, 2009, 10:49:08 AM
Indirectly related but the next version of Lightroom (v3) will have the ability to backup the catalogue prior to closing down instead of as at present upon first starting up which is much better.

And hasn't it brought about a bit of discussion!

But as has been said - that is one of the bonus points for upgrading - greater control over backing it all up.

happypaddler

#28
Quote from: Darren1978 on November 19, 2009, 09:41:54 AM
I quite like the idea of this Cloud thing.  I have a rough understanding of how it works (upload files to a remote server/locatation which then means those files can be accessed from anywhere in the world - right?), but don't actually know how to go about it.

I'm guessing that the 'Cloud' is a generic term and htat there are many companies out there who offer this facility?  Do they charge for this service?  What can/can't you upload?  How secure is your information?

Sorry for the long post...looking forward to hearing your advice :)

Yep, the cloud is a generic term (http://communication.howstuffworks.com/cloud-storage.htm). There are a few different companies out there providing this service. Most charge a fee - personally I'd not risk worthwhile files with a free service as they are unlikely to be as careful and have as robust system as the paid for services. Some are just storage banks - think an online EHD, others do that are more besides.

I use apple computers and for ease and wizz (for my wife as well), I use there MobileMe service (http://www.apple.com/mobileme/). This works well for my needs (not the cheapest system) as it is more than just a backup tool. It keeps all my files on the server - allowing me to access them whenever, on whatever computer (mac or pc) I like. It holds just about any file type - limited only the the storage space you pay for. On top of this it houses our website (mostly holding photos, small videos et al of our son behind a username and password - allowing the large section of our family who live out of the UK to see him grow), further it allows my address book and calendar to be kept synced across both my laptop and desktop, as well as phone. If I add/remove/amend a calendar event or contact in my address book on one device, the other two are updated immediately through a "push" system. I have a family account, which allows 4 robust, spam free email addresses - with each of those accounts being allowed all of the service facilities outlined. My only gripe is we can not work out how to have a shared calendar we can uses as a family (integrating into all accounts), allowing us all edit and addition rights.

There are of course alternatives. Microsoft have recently launched their version (http://www.microsoft.com/windowsazure/windowsazure/), which I think is supposed to do similar things to the MobileMe service.

Amazon, I believe, have a storage system only cloud - though I can not find the site at the moment. If you have herd of Iron Mountain (a massive records management company) have recently announced their version - not sure if it will be for personal use or just business. I have herd good things about Jungle disk (http://www.jungledisk.com/). Other companies are coming out of the wood work with there versions all the time - so its a case of have a search, read the information, do a review type search to see if they are any good and then its up to you, you may prefer to go the EHD type route.

Some food for thought at any rate.

oRGie

#29
jeez, I think I have melted the usb controller in my pc  :doh:

Luckily I have just finished taking everything that wasnt yet backed up onto a new ehd, was in the process of re-organising files among the 3 ehd's I use now and I was about to swap over to one of the drives,  I let windoze safely remove hardware, unplugged the ehd once the power light went out and whoops, my mouse stops responding and I am getting a message constantly popping up saying "usb device not recognised" well still getting that and I have unplugged all usb devices. I cant delete the usb hub controlers or their drivers as device manager is providing the error messages and just hangs when I try to delete a driver..double doh!   I wish I could still start up in dos :)  I must try and fnd the drivers in a command line window hmmm

If anyone has any ideas how I can get round device mamager to delete drivers do let me know please :)

Well I may be offline for a few days :)   but I wont start a re-install for a day or two as everything else is working ok, I'll take time to make sure I have all passwords and stuff like bookmarks backed up :)

edit: btw in case anyone recognises this symptom, when I reboot, the computer just stops during the windows loading splash screen, just a blank screen, not a blue, just blank, I discovered after a few attempts to reboot that if I plug my usb mouse in, windows continues to load as normal, everything working except any usb devices, including the mouse that let the system load.. strange...

Camera Craniums is a participant in the Amazon EU Associates Program. This affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to products on Amazon.