Horizons PollCanon EF L IS USM 500 f/4...Horizons ENTRIES 27/07...What annoyed you today?STREET-ENTRIES-Weekly Com...STREET-CHAT-Weekly Comp 1...POLL for TIME OUTTIME OUT. Chat thread wee...TIME OUT. Entry thread w...Horizons CHAT thread 2...POLL -NUMBERS and / or LE...NUMBERS and / or LETTERS ...POLL Happiness is.......Nice houseHappiness is.... Entry Th...Happiness is.... Chat Thr...

Author Topic: Need to clean out duff / duplicate images - help!  (Read 2726 times)

Offline oggalily

  • Sr. Member
  • ****
  • Posts: 336
Need to clean out duff / duplicate images - help!
« on: November 05, 2013, 03:51:04 PM »
I'd appreciate any advice on clearing out inferior and unwanted images off my hard drive.

Often when I import from my camera card onto the PC I don't get the chance to go through my images and remove blurry shots, duplicates, etc.  Over time these mount up and it eventually becomes a big job to separate the wheat from the chaff. 

I have tried a couple of things in Lightroom to identify candidates for deletion.  For example I have a Smart Collection called "Blurry?" where the reciprocal rule has been violated.  I also have used a plug-in called "Duplicate Finder" which creates a collection of all photos with the same timestamp.  This allows me to look through and pick the best of similar photos that were taken in quick succession.  But these tools are imperfect and only go so far.  So I would like to hear any tips and tricks you have.



Offline Beaux Reflets

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2267
  • Commonsense answers a whole load of the unanswered
    • Beaux Reflets
Re: Need to clean out duff / duplicate images - help!
« Reply #1 on: November 05, 2013, 04:12:53 PM »
The only way I do it, is the slow one of running each folder through Finepix viewer, deleting unwanted when first uploaded. I then pick the cherries for further use; and the rest are archived onto a separate portable storage hard drive so I can recover as and when or if I have possible use.
« Last Edit: November 05, 2013, 05:01:24 PM by Beaux Reflets »
:beer: Andy

"Light anchors things in place and gives perspective meaning."

The choices we make are rooted in reflection.

http://beauxreflets.blogspot.com/

Offline ABERS

  • Hero Member
  • *****
  • Posts: 2195
    • Black and White Images by Alan Abercrombie
Re: Need to clean out duff / duplicate images - help!
« Reply #2 on: November 06, 2013, 07:45:12 AM »
As Andy comments I've no magic wand and the only way I've been able to delete images from the HD is the laborious method of trawling through each folder and deleting the duff/duplicate ones.

I've been through the process recently and it becomes a bit of a chore. I decided the only way to avoid it is to do it on a regular basis, otherwise it's a bit like cleaning the oven, something you keep putting off and putting off until you have to bite the bullet and get down to it.

Then of course you have to clean L/Room of all the images that you've deleted from the HD. That's not too laborious a task.

Offline Alfonso_Frisk

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1294
    • http://www.flickr.com/photos/nosmo_king2007/
    • My Shipping images
Re: Need to clean out duff / duplicate images - help!
« Reply #3 on: November 06, 2013, 10:05:15 AM »
You have a couple of choices.
As others have said its the slow way of going through each folder and shot and deleting as you go.
You may find a little pearl or 2 during this task which will soften the blow.
I did it a few months ago using Irfanview. You can also try looking at a page of thumbnails but you may miss something.

The other option is to be completely ruthless and bulk delete a few folders without opening them.
Or, perhaps just transfer them all to DVDs and save the browsing and culling for another day.
http://www.fluidr.com/photos/nosmo_king2007
http://www.seateamimages.com/search.php
Wine improves with age, The older I get the more I like it.

Offline Hinfrance

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5151
  • Cake is good
    • Hinfrance
Re: Need to clean out duff / duplicate images - help!
« Reply #4 on: November 06, 2013, 10:18:37 AM »
There is no quick way, but if you have lightroom a slightly quicker way may be to go through the grid view of each folder using the left and right arrow keys to view each picture larger if need be, and then type 'x' on each one you want to get rid off. The all you have to do at the end is ctrl and backspace together, chose 'delete from disk' and they'll all be gone both from the hard drive and from lightroom.
Howard  My CC Gallery
My Flickr
The theory seems to be that as long as a man is a failure he is one of God's children, but that as soon as he succeeds he is taken over by the Devil. H.L Mencken.

Offline Graham

  • Hero Member
  • *****
  • Posts: 1543
Re: Need to clean out duff / duplicate images - help!
« Reply #5 on: November 06, 2013, 10:25:45 AM »
    As everyone says.
                                Family size packet of biscuits (with chocolate bits in) and a big pot of coffee.
                                Be prepared to get cramp in your deleting finger and get on with it.
                                You'll find it strangely therapeutic and be quite proud of yourself at the end.
                                As Mr Frisk says, you'll find the odd gem you didn't know about.
                                When it's all done, make sure you don't have to do it again.
                                                Graham.  :tup:
Build a man a fire and he'll be warm for a day.
Set a man on fire and he'll be warm for the rest of his life. 

My Gallery
My Flickr Pics

Offline oggalily

  • Sr. Member
  • ****
  • Posts: 336
Re: Need to clean out duff / duplicate images - help!
« Reply #6 on: November 06, 2013, 10:36:15 AM »
Thanks for the advice.  Howard, I already do exactly as you suggest; that is, to put a "reject" flag on unwanted images and delete en masse.  But, as I am quite trigger happy with the camera and bad about doing this when I upload, I have a large back catalogue of dross.

What would be really useful would be a tool that identifies blurry images based on there being no sharp edges and a tool that identifies very similar images based on colour, composition, timestamp etc. 

In fact just now I looked up "cleanse similar images" and wound up with lots of ads for colon cleansers.  I then tried "review similar images" and found this: http://download.cnet.com/Similar-Image-Finder/3000-2193_4-75866488.html.  I'll give it a try and report back. 

Offline 2Beers

  • CC Regular visitor
  • **
  • Posts: 75
Re: Need to clean out duff / duplicate images - help!
« Reply #7 on: November 06, 2013, 03:00:24 PM »
you could try a program called Easy duplicate file finder to remove duplicates but for duff photos  it the hard way

Offline Andrew

  • Andrew
  • Sr. Member
  • ****
  • Posts: 376
  • Starting from the beginning again...
    • ndrewbrown
    • Andrew Brown's Photography
Re: Need to clean out duff / duplicate images - help!
« Reply #8 on: November 06, 2013, 07:33:43 PM »
Clearing out seems to be the hard part if you haven#t kept on top of it all - and LR is so good for offloading the dross on import.
During my sabatical i found myself with a new computer and new installation and one trick of old still seems to work for me.

When importing a shoot, i look at all the pics imported and if, at fit screen size, i can see any blurs from camera shake etc - the pic gets a rejection flag, same goes for a pic where the exposure is well off (flash didn't fire for instance). At the end i just delete from computer and catalogue all of those pics.

Had to have a check back, but had a shoot one day where i took 1500 pics (5D2 A/F is not the best  ;) ) - and the above allowed me to offload a 1/3rd of that lot during the import.

As for your current situation - have to agree with Graham - unless you want to follow my drastic step - which is to start afresh! Have a 2tb drive sitting in a box full of RAW images (and some processed versions) i cn't be botherd with - one day maybe - but for now...
1 body, 1 lens, 1 flash gun, 1 tripod, 1 cable release & 1 filter. Keeping it simple!
(I lied, just got a second lens!)

Offline oggalily

  • Sr. Member
  • ****
  • Posts: 336
Re: Need to clean out duff / duplicate images - help!
« Reply #9 on: November 11, 2013, 09:53:58 AM »
Thanks for the suggestions.  I've made some progress so I'll update you as promised.

The program I mentioned above didn't handle RAW files so that was a non-starter.

I found something else called PhotoSort which seemed OK and ended up buying the full version for $25 US, since the free version doesn't allow you to do delected the selected images.  It's here: http://download.cnet.com/PhotoSort/3000-12511_4-10386973.html

It scans any directories you select and groups pictures by similarity. It allows different tolerance levels to adjust the similarity of pictures in a set.  It is a basic interface but really simple to use.  You can enlarge the pics to full screen and browse through them to mark them for deletion or moving to another folder.

I know $25 is a little pricey but I figured shelling out would give me some impetus to tackle the job.  I have been relatively uncompromising so far and have tried to keep only the best one (or two ;) ) from a set of similar shots.

It doesn't allow you to exclude image types so there are lots of cases where I have made JPG copies of a RAW picture and they both show up.  This is my only main grumble.  Also it would be nice if it worked directly in Lightroom as this would keep things tidier. 

On the subject of Lightroom there is a good plugin which finds duplicates based on various EXIF settings including timestamp.  I mentioned this above; it's called Duplicate Cleaner: http://www.lightroom-plugins.com/DupesIndex.php

This works pretty well for alternate takes but only if they have the same timestamp (ie taken within the same second on your camera's clock).

Finally I have played around a bit with the "Blurry" smart collection in Lightroom I mentioned earlier, basically choosing different combinations of shutter speed and focal length (ie Shutter Speed > X, Focal Length > Y, flash did not fire).  The "false positives" here have shown up when using off-camera flash and/or a tripod, but again it has thrown up some shots ruined by camera shake.

Hope this is useful information. 

Offline jinky

  • Hero Member
  • *****
  • Posts: 3292
Re: Need to clean out duff / duplicate images - help!
« Reply #10 on: November 11, 2013, 12:46:40 PM »
I found a much simpler and cheaper route to deleting my duff images ogga.
On a PC simply:

Go to Start

Select Pictures

Under organise "select all" folders

Hit delete

Much quicker in the longrun  ;) :legit:

« Last Edit: November 11, 2013, 12:47:00 PM by jinky »

Offline Oldboy

  • Hero Member
  • *****
  • Posts: 5398
Re: Need to clean out duff / duplicate images - help!
« Reply #11 on: November 11, 2013, 05:33:56 PM »
I found a much simpler and cheaper route to deleting my duff images ogga.
On a PC simply:

Go to Start

Select Pictures

Under organise "select all" folders

Hit delete

Much quicker in the longrun  ;) :legit:

Hope you don't do that to the wedding photos!  :'( ???

Offline WillyP

  • Full Member
  • ***
  • Posts: 123
Re: Need to clean out duff / duplicate images - help!
« Reply #12 on: November 13, 2013, 02:40:30 PM »
I just get a bigger hard drive.  ;D



But then, I get a bigger camera.  :uglystupid2:


 

latest Posts

Subject Board Views/Replies Member Date & Time
xx Horizons Poll International Weekly Comp Entries 135/1 Oldboy Last post August 08, 2020, 11:48:53 AM
xx Canon EF L IS USM 500 f/4 for sale Lenses 230/6 anglefire Last post August 05, 2020, 07:48:42 AM
xx Horizons ENTRIES 27/07 - 02/08/20 International Weekly Comp Entries 249/5 Oldboy Last post August 03, 2020, 10:25:12 PM
xx What annoyed you today? General Discussion 636544/3972 Jediboy Last post August 03, 2020, 02:13:31 PM
question STREET-ENTRIES-Weekly Comp 1/8-9/8/2020 International Weekly Comp Entries 62/0 Hinfrance Last post August 01, 2020, 04:09:00 PM
smiley STREET-CHAT-Weekly Comp 1/8-9/8/2020 International Weekly Comp Entries 44/0 Hinfrance Last post August 01, 2020, 04:05:33 PM
xx POLL for TIME OUT International Weekly Comp Entries 207/3 Hinfrance Last post August 01, 2020, 01:04:50 PM
xx TIME OUT. Chat thread weekly comp. International Weekly Comp Entries 242/4 Jediboy Last post July 27, 2020, 11:31:15 PM