Distroless Dockerfile (and pip)

username: rsdmse
cooked: <p>I’ve been working on distroless containers for HPC users at the University of Virginia, notably achieving 99.5% reduction in image size for LightGBM and 18% reduction for TensorFlow without loss of performance. I’m wondering if distroless is of interest to the PyTorch community.</p>
<p>Our distroless Dockerfile can be found <a href="https://github.com/uvarc/rivanna-docker/blob/master/pytorch/1.6.0/Dockerfile.distroless" rel="nofollow noopener">here</a> and the corresponding image is hosted on Docker Hub <code>uvarc/pytorch:1.6.0-distroless</code>.</p>
<p>The distroless image size is 771 MB which is about half the size of the 1.55 GB official runtime image. I tested that the distroless container with pip installation has the same performance (MNIST benchmark) on our GPU node as the official container with conda installation. I’m aware that the official one has many other conda packages, but I wonder if they are necessary for a minimal (and functional) container.</p>
<p>I’ve read <a href="https://discuss.pytorch.org/t/package-manager-choice-conda-vs-pip/84">this thread</a> on conda vs pip for pytorch, but is there a specific example that can demonstrate better performance of conda over pip? (MNIST benchmark shows equal performance.)</p>

username: rsdmse
cooked: <p>I couldn’t add more than 2 links in the previous message. If interested:<br>
<a href="https://github.com/microsoft/LightGBM/issues/3286" rel="nofollow noopener">LightGBM issue</a><br>
<a href="https://groups.google.com/a/tensorflow.org/g/discuss/c/f5n5EDjEDm4/m/cTOXrqF9CQAJ" rel="nofollow noopener">TensorFlow discussion</a></p>

username: ptrblck
cooked: <p>CC <a class="mention" href="/u/smth">@smth</a> <a class="mention" href="/u/seemethere">@seemethere</a> for visibility.</p>

username: seemethere
cooked: <p>So the reason why our docker image may be a bit bigger is because it contains the necessary packages in order to run with GPU support.</p>
<p>So that something like</p>
<pre><code class="lang-auto">docker run --rm -it --gpus all pytorch/pytorch
</code></pre>
<p>Will function as expected when you have a GPU that is compatible.</p>

username: rsdmse
cooked: <p>Thanks for your reply. I tested the distroless image on our GPU with the MNIST benchmark and it has the same performance as the official image.</p>
<p>As you can see from the link to our Dockerfile in my original message that it is essentially just a “pip install torch”. This already (magically) enables compatibility with GPU at runtime.</p>
<p>(Side note: This was not the case for TensorFlow where I had to copy a bunch of CUDA libraries from <code>nvidia/cuda:cudnn-devel</code>.)</p>

username: ptrblck
cooked: <p>This wouldn’t allow you to either rebuild PyTorch with GPU support or any custom CUDA extension, would it?<br>
If I understand your use case correctly, you are installing the binaries with the CUDA runtime libs.</p>

username: rsdmse
cooked: <p>Our goal here is not to be general, but to create a minimal image without loss of performance.</p>
<p>Comparing with the official image (<code>runtime</code>, not <code>devel</code>), the biggest difference in size is attributed to the extra conda packages.</p>
<p>I just wanted to make sure that the omission of these packages does not affect performance.</p>
<p>I hope this clarifies our intention and question.</p>

username: ptrblck
cooked: <p>Thanks for clarification. I was also mistaken in my assumption that the <code>pytorch/pytorch</code> container ships with <code>nvcc</code>, which is not the case.</p>