Posts Tagged ‘backup’

Archiving Pictures in Lightroom

November 5, 2009

I wish there was a good solution for archiving old pictures in Lightroom, but I have not found one. I don’t need all my pictures immediately available for editing so I am happy to write old ones to a CD or DVD and free the disk storage.

But I want the information stored in my Lightroom catalog available. That way I can find all my pictures of a particular subject wherever they are located. I’d like Lightroom to keep track of which DVD I wrote the picture to and ask me to load it if I want to retrieve or edit the original file.

My old image database, ThumbsPlus, did this in a simple way. It could keep track of offline disks and CD-ROMs with the thumbnails and other metadata still in the database.


ThumbsPlus Archive

ThumbsPlus Archive


My problem was that I had so many pictures and archive CDs that I overloaded the ThumbsPlus (Microsoft Access) database. So I had to move the archives to a separate ThumbsPlus database. That was not optimal but at least I had a way of searching hundreds of archive CDs quickly.

There’s a post in the NY Times “Gadgetwise” blog here that addresses the same issue in a simple way. Read the first comment – a typical sarcastic Brit!

The problem for a vendor like Adobe is that there are so many ways to do archiving. After all it’s been an issue in data processing for ever. But now regular users have terabytes of information it’s a real challenge to make something that is simple but powerful for photographers – not DP professionals.

One solution that I like is to open up Lightroom more so that third parties can write archiving plugins. Then users could choose without Adobe having to support them.

Note how archive is different from backup. With backup you are making a copy of your files elsewhere but the originals remain in place. Archive moves the files but enables them to be located easily. It’s a different issue as my online backup tool Mozy Home observes.

Do you have any suggestions?

MozyHome is Not an Archive Solution

October 18, 2009
Click for a full-size view

Click for a full-size view

This is the “Welcome” screen for the MozyHome backup configuration utility. The second paragraph is instructive:

Please note that MozyHome is not a file sharing or archiving service. Because MozyHome is intended as a backup solution, as you delete files from your computer they are also deleted from the Mozy data centers after 30 days.

I understand why they say this. Especially with their paid plan that provides unlimited storage users could upload terabytes of data to their data centres and delete them from their computers.

That is a definition of archiving.

Likewise I could share my credentials with my friends or colleagues and they could all retrieve files with no extra revenue to Mozy.

But of course if I backup my files to a CD, DVD or tape (showing my age here) then they never disappear. So in that case a backup is as good as an archive.

Mozy must believe that 30 days is plenty long enough for a user to discover she’s deleted a file accidentally and get is back from Mozy. If she doesn’t, Mozy will be of no help.

My view is that Mozy should be a bit more sophisticated. They know that some users want an archive or file sharing service. So offer it to them at a reasonable cost.

I don’t know why their only choise for individuals beyond the 2GB free account is an unlimited storage option. They could offer incremental storage upgrades in, say, 1GB chunks and guarantee that they will never delete a user’s file.

Maybe the problem is that they don’t have a sophisticated billing system. I remember from my dot com days that was a huge impediment to our developing internet telephony services. Maybe things have not got better in ten years.

Mozy could go further with a file sharing service. This could include adding permissions to files denoting who can access them. Maybe that is in their plans.

MozyHome – I Was Unfair

October 14, 2009

I wasn’t fair in my initial evaluation of the MozyHome online backup service. See here and here.

They do offer several ways to restore files without having to purchase a DVD.

They have a restore capability for files in their main user interface and also they mount my backups as a remote disk that I can access from Windows Explorer.

MozyHome Restore

MozyHome Restore

MozyHome Remote Backup in Windows Explorer

MozyHome Remote Backup in Windows Explorer

I have been using MozyHome for a couple of weeks now. It works fine and I am considering upgrading to the paid version that gives me unlimited disk space.

This progress message while preparing a backup is amusing.

Reticulating Splines?

Reticulating Splines?

MozyHome Setup

October 6, 2009

I signed up for the free online backup account at that I mentioned yesterday.

You have to go through some sign-up steps online, solve a CAPTCHA and then respond to an email message to prove it’s really you. These steps are all pretty standard these days but it is still a pain.

Then Mozy lets you download a PC or Mac program that is a downloader for the rest of their client application. I remember when I consulted with in the dot com days we agonized over the sign-up and client download process. Every step causes some users to give up and if you have too many your “conversion rate” is close to zero. Things have improved some in ten years but we didn’t worry so much about bad people fiddling with our system.

After the download – no reboot required – I had to run a configuration “wizard”. I dimly remember Microsoft introducing the wizard concept many years ago. It was a great advance at the time, but now I find them tedious. As always I checked the “I agree” button on the agreement without reading it. It could have said that Mozy owns my data. That was foolish.

I am sure it tries to protect them from legal liability if they lose the only copy of my pictures that I entrusted to them.

I was confused by their Encryption Settings step.

Encryption Settings

Encryption Settings

I had the choice to “Encrypt the data using MozyHome’s 448-bit Blowfish key” or “encrypt my data with my own personal 256-bit AES key”. The former is ‘recommended’.

  • There’s no help available. What on earth is Blowfish?
  • Somebody who knows little about encryption may think that if I use MozyHome’s key then surely they will be able to decrypt my files.
  • Does anybody have their own personal 256-bit AES key immediately to hand?
  • And of course isn’t “own personal” needlessly repetitive?

I chose the Blowfish option because I could not be bothered to do the research. I bet many users who are fearful will drop out right here.

Mozy tested my internet connection speed and announced it is fast enough.

Fast Enough

Fast Enough

It must have caught my connection at a good time. Many times I can type faster than my PC transfers bits across the Pacific Ocean.

I let it choose the files it will back up. Here I confused it because my most important data is on my TrueCrypt encrypted M: drive. It didn’t look there so I had to use its configuration tool to select my Quicken data files, TurboTax records, documents and spreadsheets manually.

I love the stock photos they select for the wizard screens. The encryption screen that confused me has a picture of an elderly gentleman looking confident. The message may be “If he can understand 448-but Blowfish keys, why can’t you?”

And on the speed screen they have a photographer using an old Polaroid camera. He does not need to back up his pictures! I guess the people who designed the wizard are not photographers.

I have the 2GB free account (with, of course, many opportunities to upgrade for consideration) so I did not try to back up all my pictures. My first pass came out at 1.9GB and Mozy said it would take two days to back up.

Long First Backup

Long First Backup

The MTBF of my internet connection is way less than two days. I reduced the size of my initial backup to a manageable 250MB.

MozyHome has a configuration tool that helps you select files and folders to back up. It also has the notion of “Backup Sets” like “Financial Data”, “Photos and Images” and so on. I don’t understand the concept for it also has a set called “My Documents” that includes the above. But in my case my financial data and my important documents aren’t where Mozy expects to find them.

I ignored Backup Sets and chose from the directory/file tree.

There are lots of other options but I decided to keep things simple for now.

My first backup went well and I now have the security of having my most vital files stored outside my apartment. This is good. One careless cook with a burning wok and I could lose it all.

I fiddled with the “Faster Computer – Quicker Backup” slider during the backup and did not see much effect in either the responsiveness of my laptop or the file transfer speed. Sometimes it stopped for 30 seconds or more but always resumed.

It hasn’t helped at all with my pictures or Lightroom catalog, but I didn’t expect it to for 2GB.

I set it up to do a single daily backup when it has the chance – after the PC’s been idle (I guess that means no user activity) for 20 minutes.

I will post more about my experience with MozyHome. I particularly want to see if it can deal with multiple revisions to a file and how the file restore process works.

There’s no perfect way to back up your hard drive –

October 5, 2009

There’s no perfect way to back up your hard drive –

Here’s an article from my old home town newspaper, the San Jose Mercury News “The Newspaper of Silicon Valley”. I remember when I arrived in California in the early 1980s how huge the newspaper was – especially the Sunday edition. They’re not doing so well now and have changed ownership but I liked the Mercury News and kept a daily subscription until I left in 2006.

The article describes my situation exactly. The hard drive of my Sony laptop failed without warning in May. I had good backups and did not lose anything when I purchased the replacement Compaq laptop.

But my 500GB Seagate external drive failed soon afterwards. I’d been a bit tardy in restoring files from it to the Compaq and I lost a lot of data. I did not lose any pictures, financial data or music. The drive contained many old documents and videos that are irreplaceable but not vital to my life.

Double failures like that are hard to plan for – I’m not a data centre. Yes I should have made it my highest priority to copy the files once I knew I had only one copy. But I am human and I was learning my new laptop and getting the applications and files I use every day working again.

I could work on recovering it – it has not failed completely. I think it is the USB 2.0 interface that has failed, not the physical drive. Any recovery would be complicated that the whole drive is encrypted with TrueCrypt.

I have a theory that the humid climate here in Bangkok makes electronics more prone to failure. My SanDisk memory card reader refuses to read SD Cards and Memory Sticks – cards with “slide” contacts (I don’t know the technical term). It has no problems with CF cards which have “pin” type contacts. I think it is corrosion.

Troy Wolverton of the Mercury News had a similar experience to mine. He lost two drives in a short period. The article describes his search for another solution.

He settled upon an online backup service called he paid US$54 a year for unlimited storage. This is a man who casually talks about terabyte drives. Unlimited for him is the amount of data that would have only been associated with large corporations a few years ago.

I have the same problem that my computer, backup drives and CDs are all in the apartment. If we had a fire then I would lose the lot, period. I do not have an office to store stuff.

So an online storage scheme looks appealing. But the Internet is slow here in Thailand so the initial backup could take weeks. Mozy claims that subsequent incremental backups are fast as they only backup the changes.

But Troy raised an important point:

Mozy’s nominal price can understate the service’s actual cost. In some cases, the only way to restore your data from Mozy is to have the company send it to you on DVDs, for which it charges fees that can run up to hundreds of dollars, depending on the amount of data you need to recover.

I looked at the Mozy web site and the marketing pages said nothing about restore! They talk about how easy backup is, but how do I get my files back when the disaster happens?

What’s this “in some cases”? Knowing my luck “some cases” will always be “my case”. I think Troy is lazy not to explain fully.

There are two common scenarios when I would want to restore files:

  1. I need a single file because I have screwed something up or because of a software bug.
  2. I need a whole drive’s worth because of a crash, fire or theft.

It looks like Mozy supports case #1 for no extra charge but  I’d have to pay dearly case #2 to get DVDs by mail. That would take some time in America. I don’t know how long it would take to get to Thailand.

Perhaps that is the Mozy business model – tempt users to put their files on their servers then charge them an arm and a leg to get them back. If so that is very sneaky – Write Only Memory.

The other issue is how many revisions of your files do they keep? I found with my ACDSsee Pro database that my recent backups were as corrupt as the one I was using. This subject gets complicated fast.

I worry that trusting your files to a company you only know by its URL is a bit risky. The company may go out of business or have security breaches. There have been well reported cases of hackers stealing data from financial institutions. I have not heard of thefts from backup companies “in the cloud” but I bet it has happened.

From the Mozy web site:

Founded in 2005, we’re focused on making simple things simple and complex things possible. The good folks who bring you Mozy are located in Pleasant Grove, Utah and are part of Decho Corporation.

I wonder how long their marketing team laboured over that mission statement. Mozy does not claim to make “complex things” (like backup and restore) simple, only possible.

Mozy offer a 2GB account free of charge. For research purposes I will try that for some files and see how it works.

Troy did not offer a comparison of different online backup sites. I don’t know why he chose Mozy.

I want to take pictures, not fuss over all this stuff!

[picapp src=”d/9/d/f/McClatchy_Company_To_7762.jpg?adImageId=5478709&imageId=1875646″ width=”380″ height=”264″ /]

Microsoft SyncToy Followup

July 4, 2009

I forgot to mention Lightroom’s image previews when I discussed using Microsoft SyncToy to backup your Lightroom Catalog and photos in this post. Remember I use SyncToy to backup my photos and as a “belt and braces” to make additional copies of my catalog to a USB disk drive.

Lightroom stores its image previews in a sub-directory where your catalog is located. If you use the default name “Lightroom 2 Catalog.lrcat” then the previews are in a directory called “Lightroom 2 Catalog Previews.lrdata”.

This directory tree stores previews in a proprietary format in a complex set of sub-directories. It can get huge. I have 29,674 photos in my catalog. According to Windows my Previews directory is 9.84 GB (10,576,150,528 bytes) and contains 29,688 files in 23,920 directories.

I create 1:1 previews but only store them for a week. I have no idea how Lightroom can produce so many files. I don’t know when it culls the old previews – maybe when I do a backup using its internal tool.

I don’t want to copy even the changes to that directory tree to my backup drive. If have to restore my catalog and photos I can always recreate the previews. They’re really there for performance reasons and it looks like Adobe said “disk space be damned” when they designed this part of Lightroom.

To stop SyncToy even looking at that directory tree I have marked it “hidden” using Windows Explorer. Then in SyncToy I tell it not to examine hidden directories. If I didn’t do that my SyncToy backups would take an hour.

You only need to mark the top level previews directory Hidden. Don’t tell Windows to hide all the sub-directories. It will take ages and I think they change all the time.

Lightroom Database Backups

July 4, 2009

Here’s a useful Lightroom add-in to help back up and compress (zip) your catalog (database) and your configuration files. The latter are things like your presets and develop settings that are not part of the catalog. Indeed they are common to all databases under your Windows user id.

Config Backup Plugin | The Photo Geek. I quote from the web site:

The Config Backup plugin helps speed up and simplify backing up and compressing Lightroom files.  It adds two new menu items to the File menu’s Plug-in Extras sub-menu.

  1. Customisable Configuration Backups – Allows you to customise and manually trigger the backup and compression processes
  2. 1-Click Configuration Backup – Immediately backs up the Lightroom configuration files and Catalog backups using your previously configured settings, and optionally displays the results of the backup process.

Adobe have a Knowledge Base article here that tells you what’s stored where. I have 86 files in my Lightroom configuration but it’s only 180KB of data. I lost this when my Sony PC crashed which was inconvenient but not a disaster since my Lightroom catalog and photos were backed up.

My 616MB catalog compresses to about 60MB when it is zipped. Of course you could do this yourself but this little tool makes it easier.

Note this is nothing to do with backing up your images. You are still on your own here.

How I Backup My Images – Microsoft SyncToy

June 25, 2009

I noted in a previous post that Lightroom abdicates responsibility for backing up your images. You have to work it out for yourself. My trusty Sony laptop’s hard drive crashed with no warning a couple of months ago. I could have lost everything from its 120GB drive but I was able to get almost everything back from my backups. Actually the hardest thing was finding all the applications and utilities I  had installed over the years – but that’s another story.

There are so many potential backup solutions out there that it is almost overwhelming particularly if you are not a computer professional. I was and i still get confused.

I think the main thing is to choose a backup solution and stick to it. If you buy something very powerful but complex it’s quite likely you will give up. More important than the consistency with which you use it.

I am now in the habit of doing my backups last thing before I go to bed. I kick them off, go to clean my teeth and usually everything is done when I come back to the PC. I achieve good dental and PC hygiene in five minutes.

The tool I chose is Microsoft SyncToy 2.0. It’s a free download from Microsoft here. I have a bit of a bias against Microsoft software in general. It is often too slow and complex for my simple needs. I think Microsoft has a digital asset  management solution that competes with Lightroom and ACDSee but I did not consider it because of my prejudice.

But SyncToy is one of Microsoft’s free Power Tools that are simple and designed to do one thing well. I recall reading that it was designed by a Microsoft employee who’s also a photographer to do just what I wanted – back up only those files that have changed since the last backup.

Note that SyncToy isn’t restricted to image files. I use it back up all my data.

I won’t bore you with the details of my setup – yours will be different. the important fact is that SyncToy lets you define directory pairs and to synchronize files between them. SyncToy refers to them as the left and the right directories. I think of the left directory as the one on my laptop (M:) and the right as the directory on the backup drive (N:).

Thus I synchronize the Lightroom-managed images on my TrueCrypt encrypted M: drive with a corresponding directory on my Seagate 320GB TrueCrypt encrypted portable USB drive that I mount as drive N:.

Here are two screenshots from my SyncToy installation. The first is the result of scanning my image directories and is a list of the files that SyncToy will copy.

Note that if you change the metadata of an image in Lightroom it writes the changes to the image in the XMP format. So if you fiddle around with your image organization and keywording, or you experiment with different develop presets Lightroom modifies your original file. It doesn’t modify the image data in the RAW, DNG or JPEG files but it does update the metadata.

If you make a virtual copy of an image Lightroom does not make a copy of the image – all that information is in your Lightroom catalog and the XMP.

The preview screen also tells you how many files it didn’t have to copy because they were unchanged.

The preview screen from SyncToy

The second screenshot is the result of a SyncToy run showing the number of files it copied and any errors.

The results screen from SyncToy

You get three choices for the “action” SyncToy takes on the files in your defined directory pairs. You choose them when you set up each directory pair and you can change them. Microsoft actually simplified SyncToy for its 2.0 release. I think there used to be five choices but they were very complex to explain. This may be the first time Microsoft has simplified its software in an update.

The choices are:

  • Synchronize. New and updated files are copied both ways. Renames and deletes on either side are repeated on the other. I don’t use this. It’s useful when you may be modifying files in both directories in the pair, but that is very bad practice. Keep your backup drive for backups only.
  • Echo. New and updated files are copied left to right. Renames and deletes on the left are repeated on the right. I don’t use this on my images because if I delete an image accidentally an Echo will delete the backup. I do have my Lightroom generated backup from my original import to fall back on but I will lose the metadata updates Lightroom wrote to the XMP.
  • Contribute. This is the one I use. New and updated files are copied left to right. Renames on the left are repeated on the right. No deletions.

There are other details of SyncToy’s operation I haven’t mentioned here but I want to keep this post short so I won’t go into them. I encourage you to download a copy yourself and try it. If it doesn’t work for you no worries – you have not wasted a satang.

When I got my new Compaq PC to replace the Sony first I installed TrueCrypt and mounted the USB backup drive. I crossed my fingers that a new TrueCrypt install on a new PC would recognize my encrypted drive. The new install was a later version and I had visions of compatibility problems. But no – TrueCrypt worked great and I didnt’t lose a byte of my images.

I copied all the files from the USB drive plus a backup of my Lightroom catalog, started Lightroom and it was like nothing had changed. I breathed a huge sigh of relief and I am sure my blood pressure dropped significantly.

The next thing I did was download a SyncToy and set up my directory pairs again so I could continue my backups.

SyncToy meets my needs and it has the great advantage of being free. Of course that means you don’t get much support if things go wrong but I have never had a problem with it in about three years of daily use.

Lightroom Import (Part 3)

June 24, 2009
Lightroom makes what is to me a strange choice of directory names when I choose to backup my imported files. I would have expected it to use the same folder structure as I chose for the images to be catalogued by Lightroom. That is what ACDSee does. But instead Lightroom invents folder names “Imported on ” + a text formatted date. I wonder why they did that?
Windows Explorer - Lightroom Import Backup
So today Lightroom added a directory called “Imported on 24 June 2009”. I guess they use the Windows locale to decide on the date format – mine is set to UK.

Backup Device

June 22, 2009

This is the Seagate 320GB USB disk drive I use for my Lightroom backups. It doesn’t need a separate power supply. It is big enough to hold every photo I have ever taken, indexed in Thumbs Plus and now Lightroom.

It cost 3,350B at IT City in Bangkok. That’s about US$100. I expect it is cheaper in the USA. Electronics are not a bargain here.

Above it is the 30B carrying case I bought for it on Sukhumvit Road.

It’s small enough that I can carry it on me when I travel. It is so small that it is easy to lose. I’ll write a post soon about how I protect it.