We're starting to get to the size where it's becoming a common problem that people upload duplicate images.

To combat this i'm playing around with some rather funky math and image recognition code that hopefully should stop duplicates being upload (even if they have been slightly changed, ie watermark) if you try and upload something and the system thinks it exists it will give you a link to the image (if it's a direct copy of something existing on the site it'll just say 'we believe that this image already exists on our system. please upload something different'), if the image isn't the same can you please email abuse@plus613.net with the image and i'll investigate.

shall-we-play-a-game?
I KNOW when I'm uploading a duplicate, but I just modify the jpeg by one pixel to defeat the dupe checker.
It wouldn't take much to improve on that.

The site has been using md5 sums to calculate if it's the exact same image. The new system uses a little bit of fuzzy logic to guess if the actual content is the same, based on probability.

You'll still be able to beat it easily enough if you're trying to, the point is to stop accidental dupes. We can't stop people who are deliberately trying to be annoying and upload duplicates

Edit: Btw, you don't even need to change a pixel, "echo >> filename.jpg" will do the same, change any byte or the file's length and the md5 calculation will have changed.



Edited 1 time(s). Last edit at 01/12/2008 07:36AM by pulse.

The main problem with md5 checking is some image viewing programs update file information (last viewed meta information) when someone opens the file with them. So yes the new system tries to recognise the image in the same way that computer facial recognition works. It has a little bit of a overhead but because it's only done on new additions it shouldn't cause a problem.