I wonder why Pytorch uses PIL not the cv2

spnova12: <p>I wonder why Pytorch uses PIL not the cv2.<br>
As the article(<a href="https://www.kaggle.com/vfdev5/pil-vs-opencv" rel="nofollow noopener">https://www.kaggle.com/vfdev5/pil-vs-opencv</a>) says, cv2 is three times faster than PIL.<br>
I did an experiment and I really think so.<br>
But why does Pytorch use PIL?</p>

edowson: <p>If PyTorch can remove it’s dependency on PIL and use OpenCV 3 instead, it would help me work with multi-channel images that are greater than 3-channels. At the moment, if you pass a 10-ch image or 10-ch mask, PIL truncates it to a 3-ch image.</p>
<p>See these issues:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/python-pillow/Pillow/issues/3160" target="_blank" rel="nofollow noopener">github.com/python-pillow/Pillow</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/edowson" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/10365427?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/python-pillow/Pillow/issues/3160" target="_blank" rel="nofollow noopener">Issue: PIL cannot handle multi-channel images, they get truncated to 3 channels by default</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/edowson" target="_blank" rel="nofollow noopener">edowson</a>
	on <a href="https://github.com/python-pillow/Pillow/issues/3160" target="_blank" rel="nofollow noopener">2018-06-07</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">What did you do?
Several Deep Learning Framework Libraries (.e.g PyTorch's TorchVision), Image Processing libraries (.e.g skimage) and machine learning data augmentation...</pre>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/python-pillow/Pillow/issues/1888" target="_blank" rel="nofollow noopener">github.com/python-pillow/Pillow</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/wiredfool" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/19792?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/python-pillow/Pillow/issues/1888" target="_blank" rel="nofollow noopener">Issue: Tracking Issue for high bit depth multichannel images</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/wiredfool" target="_blank" rel="nofollow noopener">wiredfool</a>
	on <a href="https://github.com/python-pillow/Pillow/issues/1888" target="_blank" rel="nofollow noopener">2016-05-05</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">Pillow (and PIL) is currently able to open 8 bit per channel multi-channel images (such as RGB) but is able to...</pre>

<div class="labels">
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">Enhancement</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

antspy: <p>The only thing that uses PIL is the torchvision package. Rewriting it to support cv2 probably it’s not such a huge effort, since you just have to replace functions call here and there - if you have experience with cv2 you could probably try that yourself!</p>

ptrblck: <p>If you want a performance increase, you could install <a href="https://github.com/uploadcare/pillow-simd">Pillow-SIMD</a> instead of Pillow. This is a drop-in replacement without any code changes.</p>

edowson: <aside class="quote no-group" data-username="antspy" data-post="3" data-topic="19482" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/antspy/48/3475_2.png" class="avatar"> antspy:</div>
<blockquote>
<p>Rewriting it to support cv2 probably it’s not such a huge effort, since you just have to replace functions call here and there - if you have experience with cv2 you could probably try that yourself!</p>
</blockquote>
</aside>
<p>Thanks for the heads up, I’ll try that out.</p>

deJQK: <p>If you finish it, will you make it public like on github? That will be beneficial to many others.</p>

edowson: <aside class="quote no-group" data-username="deJQK" data-post="6" data-topic="19482" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dejqk/48/4965_2.png" class="avatar"> deJQK:</div>
<blockquote>
<p>If you finish it, will you make it public like on github? That will be beneficial to many others.</p>
</blockquote>
</aside>
<p>Yes, I’ll will. I ran into this issue with PIL last week, trying to work with multi-channel images. Replacing the TorchVision’s dependency on PIL sounds like the right thing to do. It looks like TorchVision library already supports multiple imaging backends, so it should be a matter of adding a cv2 backend.</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/pytorch/vision">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/pytorch/vision" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/a0a99d5b3f2a9568c2df74729b067036859ea62abce589efd4b5e77bb14810f3/pytorch/vision" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/pytorch/vision" target="_blank" rel="noopener nofollow ugc">GitHub - pytorch/vision: Datasets, Transforms and Models specific to Computer...</a></h3>

  <p>Datasets, Transforms and Models specific to Computer Vision - GitHub - pytorch/vision: Datasets, Transforms and Models specific to Computer Vision</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I’ll let you know once I’m done.</p>

spnova12: <p>Thank you very much!</p>

vipper: <p>I have rewritren the “transforms” in torchvision package with cv2.<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://assets-cdn.github.com/favicon.ico" class="site-icon" width="32" height="32">
      <a href="https://github.com/YU-Zhiyang/opencv_transforms_torchvision" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/37430771?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/YU-Zhiyang/opencv_transforms_torchvision" target="_blank" rel="nofollow noopener">YU-Zhiyang/opencv_transforms_torchvision</a></h3>

<p>opencv reimplement for transforms in torchvision. Contribute to YU-Zhiyang/opencv_transforms_torchvision development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

Deepak: <p>I also have the same question.</p>

Deepak: <p>May be because cv2 is a heavier module, and PIL is a lighter module. Not going for a heaver module, if we could get our work done just by a lighter one.</p>

jgbradley1: <p>With the recent addition of <code>torchvision.io</code>, now might be a good time to consider adding an option to use opencv as the image backend.</p>