Wednesday 30 September 2009

Lightroom 2 Workflow – Importing and organising photos

Lightroom 2 (LR) is an amazing application that has come to handle most things I want and need to do with my photos, organising, developing, printing, publishing and even standard retouching. The integration of the different modules in LR saves a lot of time. My basic process is as follows, which I describe in more detail below.


 
  1. Delete really bad images directly in camera.
  2. Import photos into Lightroom.
    1. Organize in “master” photo folder, by date
    2. Backup to server
    3. Apply develop settings
    4. Apply generic metadata and keywords

  3. Apply specific keywords and metadata
  4. Create collection
  5. Select picks and rejected
  6. Apply rating to picks
  7. Deselect all pick-flags and choose picks again from the highest rated photos.
  8. Delete rejected photos
  9. Backup all photos
Whilst this explanation might be a bit long, the actual process is very quick. It takes me around 30 minutes to go through and narrow down the photos to developed out of a normal batch of 500 or so.
An average day with a camera in my hand I probably shoot between 50 and 100 pictures per hour, and a four hour model shoot easily generates 5-600 pictures, so there’s loads of photos to take care of. I need to be very organised. Without Lightroom it would be almost impossible.
This is my way of working and not necessarily the right way, but it works for me, so far.

Setup

My setup is a pretty simple one. I have a Dell laptop, a few external hard drives, a HP MediaSmart Server (Windows Home Server) and a simple network drive.
My camera of choice is Canon EOS 40D. For everything but sports I shoot RAW, and for sports I usually shoot JPG with Adobe RGB colour space.
Number of catalogues
The other day I read Eric Scouten’s blog post on organising Lightroom catalogues (Lightroom 2 Technique: How I Organize My Catalogue and Why (2009 Edition) — Eric Scouten: Blog). Whilst this seems like a very organised way of using Lightroom to speed up your workflow, I use a slightly different tactic, especially when it comes to the number of catalogues to use in LR.
In LR I just use one catalo for all my photos. The reason I do this is that I want to have all my images in one place and not duplicate my images unnecessarily. I already have at least three backups of each photo. The other reason is that it makes keyword searches so much easier with just one catalo. I only have one catalo to search for all my images.
When I need to clear some hard drive space on my laptop I just move photos, using LR, to an external drive. To choose which photos to move I can use LR’s keyword and rating features to still keep my best pictures on my laptop. Then, when I’m “off-line” I can still search through the whole catalo and if I want to view a photo that sits on an external drive that is disconnected I get a preview thumbnail through LR’s catalo. With a laptop hard drive of 500GB I usually have around a year of photos sitting on the local laptop drive.
If I were to use several catalogues or partly duplicated catalogues, as Eric does, I have to make an “old-style” decision and categorize photos in some way “outside” LR and then I miss many of LR’s good library features. It feels a bit like going back to the “old” days of sorting photos into folders on the hard drive depending on location, project, etc. It becomes rather one-dimensional rather than a relationship database that LR uses.
Again, this is my way of working and I think everyone has to find a solution that works for them.

Shoot

During the day I try to quickly delete any really bad images, directly in my camera. I do this whenever I have a spare minute. If I don’t have the time to delete bad pictures there and then, not a problem, I’ll do it in Lightroom later. I explain later why I do this step.

Importing images

Back home, I plug in my card reader into my laptop and launch LR. In the Library module I click “Import”.
LR 2 Import
Since I’m importing from a compact flash card I choose to “Copy photos to a new location and add to catalo” and choose a folder on my laptop. This folder is always the same folder because I then choose to organize the photos by date so that it creates new folders for year, month and day. In this example, the photos will be copied to D:\Donald\Bilder\_Nytt\2009\09\17 as all of the pictures were taken on the 17th of September 2009. I also check the “Don’t re-import suspected duplicates”.
Now, here’s the reason I try to delete really bad images directly in the camera during the day. At the point of import I always backup all images that I import to an external drive, in this case my HP MediaSmart Server, by checking the “Backup to” box and choose the folder to put the backups in. I really don’t need to back up images that are complete failures, hence my quick deletion step during the shoot. On the road I backup to an external USB hard drive instead, and when I get back home I just move the backup folders to my server. This way I always have an untouched backup of all photos I’ve downloaded to my laptop. This is backup one.
I always leave the file name as it is. Back when I used ACDSee and Photoshop as my standard workflow, I used to rename images using the subject, location and a sequence number. However, with Lightroom’s advanced key wording and collection features, this is unnecessary, in my view, and I’m therefore unlikely to search for a photo by filename. So why set up a system for file naming? As I download each days photos to a separate folder it is also extremely unlikely I get clashes with images with the same name.
I usually apply some gentle sharpening and medium contrast at the time of import, by selecting a preset from “Develop Settings”. Not that it really matters, since all edits are non-destructive in LR, but I find that RAW files are often a bit soft and my Canon EOS 40D usually needs a little more contrast.
Metadata is the real time saver at the import stage. I always apply a generic metadata preset that includes my name, contact details, copyright information, etc. Sometimes I edit this as well to include more specific data such as location, in case the location is the same for all photos that I import. This can also easily be done for a batch of photos in the Library module in LR later.
I also include a number of keywords that apply to all, or most of the photos, directly at import. This is also a great time saver and makes it so much easier to find your pictures later. Now, all I have to do is to select the images I want to import (usually all of them) from the preview on the right and hit “Import”.

Keywords and Metadata

I always start by applying any keywords or IPTC (metadata) that I didn’t include at import-stage in the Library module in LR. I usually start this whilst photos are still being downloaded. Just select the photos that need the same keywords and type in the keyword in the keyword box or select the keywords from the keyword list, in the right hand pane of LR. Do the same for metadata.
I used to be a bit sloppy with metadata and thought, I keyword all photos anyway, so what’s the point. Even if you’re not selling your pictures but just use them for your your own pleasure metadata is a real time saver.
Say you want to look up those photos of you’re kids from your trip to Tenerife last year. Sure, you can always find the photos in the folder view, if you know the date. You can click on your kids names in the keyword pane or do a custom search with several keywords. If you had filled in the location field in the IPTC metadata you could have just clicked on location, city, etc in the metadata field in the custom search and then select you kids names from the keyword field in the custom search.
So for me IPTC data is important and it’s so quick and easy to fill in in the Library module in LR, if you haven’t already done so at the import stage.

Collections

At this stage I create a collection in LR, as the download and backup is usually completed by now, and use a hierarchy-system where I have a top-level “Collection Sets” for family photos, competitions, private projects, portfolio, test, etc. I then create sub-collections based on dates, projects, clients and so on. This way I keep my family pictures separate from the rest. I can also drill down into these collections and easily filter photos based on ratings, flags, metadata, keywords, etc, not just for one collection but for a whole collection set.
I then switch to the newly created collection and do all my flags ratings from there. The flags are especially important to do from the collection as the flags don’t seem to travel between collections and folders. So if you flag a photo in one collection and then include the flagged photo in another collection, the rating will still be there in the new collection, but not the flag. There is some logic to this, but it was a bit irritating at first.

Flags & Stars

After keywords and metadata, the first thing I do is to go through and select the keepers and the photos to bin. I use the pick and rejected flags to do this.
LR 2 picks
I mark all photos that I think I want to keep with the “Pick-flag” by hitting “P” on the keyboard as a preview the photos in the Library module and hit “X” for all “rejected-flag” photos I want to delete.
Now I use the quick custom filter in the bottom right hand corner of LR to select only the pics.
These photos I rank using the stars by hitting 1 through 5 on the keyboard. I think Eric’s way of doing it is a really smart one, so I think I will adopt that in the future. Simply put, I will first mark all picks that I think are good enough to keep with one star. Filter the photos again on all “one-star” photos and go through them to apply two stars to the bunch that stand out from the crowd. When that’s done I select the “two-star” photos and go through them again with three, four and five stars (if any).
Where I differ a bit to Eric is that I have a set up a convention on stars that works something like this.
  1. Not very good (blurry, exposure, etc)  but keep anyway for some reason, sentimental value, the only photo of that particular scene, etc.
  2. Quality (composition, light, sharpness) is OK.
  3. Better and have potential with the right developing process, retouching, etc.
  4. Really good photos, but not quite 100%
  5. Nothings wrong with this photo, artistically, lighting, focus/depth of field, composition, emotion etc. Everything is 100%
As you can imagine, I don’t have a lot of “five-star” photos. There’s always something that can be done better. When I’m done I’m hopefully left with a bunch of threes and fours. This is when I remove all pick-flags by selecting them and press “P”.
I now go over the threes and fours (and fives if there are any) and decide applying the pick-flag to the photos I want to develop.

Getting rid of the junk

Now, I want to get delete of all the rejected photos. There are a number of ways of doing this. If I do it straight after import it is as straight forward as selecting the rejected photos and hit delete. You will then be presented with an option to delete the photos from the hard drive or just remove them from the Lightroom catalog.
I usually delete rejected photos every few weeks and all I need to do is to go tot the catalog “All photographs” and make sure the library filter is set to none.
LR 2 catalog
LR 2 library filter
After that I go down to the custom filter setting and select the “Rejected Photos” preset or just click the rejected flag.
LR 2 custom filter rejected photos
Now, I can go through all photos in my catalog that are marked rejected and get a final chance to make up my mind about these photos. Select all photos I don’t want to delete and then uncheck the “rejected flag” by hitting X on the keyboard. Then I select the remaining photos and press “Delete” on the keyboard. Again I get the option to delete them fro disk or just from the catalog.
If you’re really sure you want to delete all photos that you have rejected, you can just select “Delete Rejected Photos” from the “Photo” menu in the Library module in LR, instead of using the last step with the custom filter above. You still get the option to delete from hard drive or just remove from catalog, but you don’t have the option to have final look at the photos.

All done

Now I can go back to my picks and develop them in the develop module, but that’s another story.

Staying organized

Periodically I go through my Folders in LR, not to be confused with Collections, when I need to free up some space on my laptop hard drive. I have a 500GB drive so it can last a while.
When I do need to free up some space I move photos from my laptop to an external USB hard drive, by drag-and-drop using Lightroom. Word of caution, if you do this process outside LR you need to tell LR about it later, or it won’t find the pictures, so the easiest way to to it is to drag-and-drop within LR.
By keeping the pictures in one catalog I can still search through all my pictures, regardless of where I am, on the road or at home. I also get thumbnails of photos that are off-line, so that I can get an idea of what the picture looks like. The quality is remarkably good and is enough for a preview.
Since I can add as many external drives as possible to my setup there’s no limit to how many photos my catalog can include. Currently it’s around 32,000 and it seems to work very well so far. It is possible that as the catalog grows in size I need to reconsider using more than one catalog, due to performance issues, but so far so good.

Backup

Apart from the initial backup to my HP MediaSmart Server of unedited photos directly from the camera I have three more backups of each photo.
My HP MediaSmart Server backs up my laptop and all my external hard drives every night, so that I have a current backup of all my documents and photos. The server works a little bit like a RAID solution where it duplicates that data on the server on several different hard drives in the server. Currently I have two 1TB drives and one 500GB drive installed, which is enough for now. This means that if one drive breaks down I can easily replace it with a new drive. Theoretically I shouldn’t even have to turn the server off to do this. Now, this is all very well, as long as the drive where Windows Home Server, the operating system, resides is intact. If this breaks down the server won’t start and I need to reinstall the operating system before I can get back to normal again. No data will be lost, but it still take a bit of time to do it. I’ve done it once, so I know. Didn’t lose any documents or photos.
Therefore, I also back up my data manually to a simple network drive, so that I have a completely separate backup on-site of all my photos in case the system disk on the server breaks down, I don’t have the time to do a server recovery operation at that moment and I still need to recover some lost file from a backup. Pretty remote chance, but it can happen.
As a a last step, once every two weeks, I backup my laptop and external hard drives to a couple of large USB hard drives that I keep in a different location, so that I always have an off site backup in case of a fire or burglary. I have looked into online backups, but come to the conclusion that it would just take forever to backup 1TB or so of data to begin with. And even after just a couple of days shooting, it’ll add another 10-20GB, which will also take an awful long time to back up over the Internet. Perhaps, if I get a really quick broadband connection, like 100mbps upload speed or better, it could be a good option. To have a selection of folders backed up online could be an interim solution though.
Well, there you have it, my very basic workflow of importing, organising and securing my images.
l8r

2 comments:

Unknown said...

Hi
This is a very good article that spells out in detail many issues that i have been finding it hard to get my head around in LR2. Thank you.
(I came across it from a vague link on eric scoutens article of the same subject.)
One issue that you both might elaborate on is the issue of what is happening in LR2 when storing as CR2 or DNG or TIFF or JPEG. If I import CR2 and process in LR2 and then export to exernal drives as TIFF or JPEG how many copies do i end up with?
Is the original CR2 in LR2 one copy and its developed TIFF another copy? I am only at circa 500 photos on my disc at the moment and trying to get sorted properly before building up to 30,000!!
Anyway,thank you again for your excellent article.
Pierce

Unknown said...

Well Pierce, it depends a little on how you do it. If you just do a simple export from the library module as TIFF or JPEG the exported files will not be included in the LR catalog. So in the LR catalog you will only see the CR2-file. However, you do have an option to include the exported file in the catalog in the export dialog box. I've never done this as I don't really see the point. The only thing it'll be is a less flexible version of what I allready have in LR as a RAW-file.

If you do edits in Photoshop from LR, however, LR creates a new file (PSD or TIFF),puts it in the same folder as the original RAW-file and put a suffix to the file name, e.g. IMG0001.CR2 --> IMG0001-Edit01.PSD. This way you have both a RAW-file and a Photoshop of TIFF file in the LR catalog. You can get LR to stack these images so that you only see one version in the folders view.

The only time I have a another "physical" copy of an image in LR is when I've done some editing in Photoshop on that image.

Now, virtual copies is a totally different story and is sometimes a greate tool to create different versions of the same image for different purposes.