CCD file de-duplication system