Corrupted images in bmp extension

Amal: <p>i am working on a project but whenever i try to show an image i get an error that image can not be identified. i’ve looked around and i have printed all the corrupted images it turns out that all of these images are of bmp extension and they are a lot. i can’t remove them , what should i do ;(</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>

ptrblck: <p>How do you try to visualize them and what’s the error?</p>

Amal: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cce6330de3e165ff893a055035553ec1b64e35fc.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/cce6330de3e165ff893a055035553ec1b64e35fc" title="Screenshot 1445-08-21 at 5.33.36 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cce6330de3e165ff893a055035553ec1b64e35fc_2_527x499.jpeg" alt="Screenshot 1445-08-21 at 5.33.36 PM" data-base62-sha1="teCBrGGWEkiS4Pcp3mJkRl4ZwqU" width="527" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cce6330de3e165ff893a055035553ec1b64e35fc_2_527x499.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cce6330de3e165ff893a055035553ec1b64e35fc_2_790x748.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/cce6330de3e165ff893a055035553ec1b64e35fc_2_1054x998.jpeg 2x" data-dominant-color="F0D8D4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 1445-08-21 at 5.33.36 PM</span><span class="informations">1224×1160 286 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
although i have included the snippet code below, i still get the same error .<br>
from PIL import ImageFile<br>
ImageFile.LOAD_TRUNCATED_IMAGES = True<br>
i tried to visualize some of the images and it’s by chance worked! but when i move to the training step same error arise</p>

Amal: <p>this is how i identified the corrupted images <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
from PIL import Image<br>
import os<br>
import numpy as np<br>
import cv2<br>
def check_images(directory):<br>
issues = {‘corrupted’: <span class="chcklst-box fa fa-square-o fa-fw"></span>, ‘low_resolution’: <span class="chcklst-box fa fa-square-o fa-fw"></span>, ‘blurry’: <span class="chcklst-box fa fa-square-o fa-fw"></span>}<br>
for root, _, files in os.walk(directory):<br>
for file in files:<br>
try:<br>
with Image.open(os.path.join(root, file)) as img:<br>
# Check for low resolution<br>
if img.width &lt; 800 or img.height &lt; 600:<br>
issues[‘low_resolution’].append(file)</p>
<pre><code>                # Check for blurriness
                img_np = np.array(img)
                variance_of_laplacian = cv2.Laplacian(img_np, cv2.CV_64F).var()
                if variance_of_laplacian &lt; 100:  
                    issues['blurry'].append(file)
        except (IOError, SyntaxError) as e:
            issues['corrupted'].append(file)

return issues
</code></pre>
<p>issues = check_images(DATADIR)<br>
print(issues)</p>

ptrblck: <p>If these images are indeed corrupted and <code>PIL</code> has trouble reading them, you might need to re-download them. Are you able to open these images with any other image viewer?</p>

Amal: <p>Yes, I opened them in kaggle data description section and they are totally fine!<br>
But what do you mean by re-uploading them ?</p>

Amal: <p>Does the problem occur as a result of the environment RAM limitation ? Because each time  I re-download the data I get different number of corrupted images.</p>

ptrblck: <p>No, I don’t think the RAM is related to this issue and it seems the download might fail creating corrupted images.</p>

Amal: <p>can you suggest alternative way i can upload the dataset consist of 19,890 images other than ImageFolder?</p>