Force installing torchvision (on IBM ppc64le)

username: Brando_Miranda
cooked: <p>I am in a weird scenario were I am forced to use torch <code>1.3.1</code> (due to hardware see: <a href="https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/" rel="noopener nofollow ugc">https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/</a>). I read from the pytorch docs that it’s corresponding version of torchvision is <code>0.4.1</code> (<a href="https://pypi.org/project/torchvision/" class="inline-onebox" rel="noopener nofollow ugc">torchvision · PyPI</a>):</p>
<blockquote>
<h2>Installation</h2>
<p>We recommend Anaconda as Python package management system. Please refer to <a href="https://pytorch.org/" rel="noopener nofollow ugc">pytorch.org</a> for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.</p>
<pre><code class="lang-auto">Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

torch	torchvision	python
master / nightly	master / nightly	&gt;=3.6
1.5.0	0.6.0	&gt;=3.5
1.4.0	0.5.0	==2.7, &gt;=3.5, &lt;=3.8
1.3.1	0.4.2	==2.7, &gt;=3.5, &lt;=3.7
1.3.0	0.4.1	==2.7, &gt;=3.5, &lt;=3.7
1.2.0	0.4.0	==2.7, &gt;=3.5, &lt;=3.7
1.1.0	0.3.0	==2.7, &gt;=3.5, &lt;=3.7
&lt;=1.0.1	0.2.2	==2.7, &gt;=3.5, &lt;=3.7
</code></pre>
</blockquote>
<p>but for some reason I have the wrong version of it:</p>
<pre><code class="lang-auto">torchvision 0.2.2 pypi_0 pypi
</code></pre>
<p>is there a way to install the right version of torchvision?</p>
<hr>
<p>What I’ve tried:</p>
<p>First I tried force installing the right version with conda. Conda couldn’t find the version of torchvision that I need:</p>
<pre><code class="lang-auto">$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

</code></pre>
<p>Then I proceeded to try to install it regardless with pip</p>
<pre><code class="lang-auto">$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2
</code></pre>
<p>got an error too.</p>
<p>Is there anything else to try?</p>
<hr>
<p>related:</p>
<ul>
<li>crossposted SO: <a href="https://discuss.pytorch.org/t/force-installing-torchvision/97279" class="inline-onebox">Force installing torchvision (on IBM ppc64le)</a></li>
<li>crossposted reddit: <a href="https://www.reddit.com/r/pytorch/comments/iyf2qn/force_installing_torchvision/" class="inline-onebox" rel="noopener nofollow ugc">Reddit - Dive into anything</a></li>
<li>real problem is installing torchmeta: <a href="https://github.com/tristandeleu/pytorch-meta/issues/95" class="inline-onebox" rel="noopener nofollow ugc">help installing torchmeta (for ppc64le IBM architecture) · Issue #95 · tristandeleu/pytorch-meta · GitHub</a></li>
<li><a href="https://www.ibm.com/mysupport/s/forumsquestion?id=0D50z00006gaxV9CAI" class="inline-onebox" rel="noopener nofollow ugc">Home - IBM TechXchange Community</a></li>
</ul>

username: tom
cooked: <p>Wouldn’t it be more natural to use the torchvision 0.4.2 from the same page?</p>

username: Brando_Miranda
cooked: <p>I tried <code>pip install torchvision==0.4.2</code> and <code>conda install torchvision==0.4.2</code> but it didn’t install them.</p>
<p>hmmm…can you clarify how to actually do “getting them from that same page”? Thats what I thought too but not sure how. <a class="mention" href="/u/tom">@tom</a></p>
<p>Btw, thanks for the help.</p>

username: Brando_Miranda
cooked: <p>about to try</p>
<pre><code class="lang-auto">conda install torchvision==0.4.2 -c pytorch
</code></pre>

username: tom
cooked: <p>The IBM page you linked lists TorchVision alongside PyTorch, so I would expect you to be able to install PyTorch and TorchVision by following their instructions (using IBM’s channel or something).</p>

username: Brando_Miranda
cooked: <aside class="quote no-group" data-username="Brando_Miranda" data-post="1" data-topic="97279">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/48/14355_2.png" class="avatar"> Brando_Miranda:</div>
<blockquote>
<p><a href="https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/" rel="noopener nofollow ugc">https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/</a></p>
</blockquote>
</aside>
<p>Oh great point will check that link again and see what happens.</p>

username: Brando_Miranda
cooked: <p>what instructions are you referring to?</p>

username: tom
cooked: <p>Under the documentation tab on that web page<br>
<a href="https://www.ibm.com/support/knowledgecenter/SS5SF7_1.7.0/navigation/wmlce_install.htm" class="onebox" target="_blank" rel="nofollow noopener">https://www.ibm.com/support/knowledgecenter/SS5SF7_1.7.0/navigation/wmlce_install.htm</a></p>

username: Brando_Miranda
cooked: <p>Oh yea I read that. I’m running out of ideas what I need to run from that page for things to work.</p>

username: Brando_Miranda
cooked: <p>hmmm… I did read it. What doesn’t make sense to me is that they suggest to force the release so I did:</p>
<pre><code class="lang-auto">$ conda install torchvision powerai-release=1.7.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - powerai-release=1.7.0
  - torchvision

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.
</code></pre>
<p>but then when I check their website for that powerai-release it does have the 1.7.0 and the 0.4.2 torchvision version I want but it can’t find it.</p>
<p>Indeed it is not in the list:</p>
<pre><code class="lang-auto">Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch
</code></pre>
<p>so what I need is to be able to point it to this channel conda<br>
<a href="https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/" class="onebox" target="_blank" rel="nofollow noopener">https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/</a></p>
<p>…</p>
<p>alternatively that website allows me to download the tar but I dont know how to get conda to install from that. <a class="mention" href="/u/tom">@tom</a></p>

username: tom
cooked: <p>It tells you to use this for a channel:</p>
<pre><code class="lang-auto">https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
</code></pre>
<p>by running</p>
<pre><code class="lang-auto">conda config --prepend channels \
https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
</code></pre>

username: Brando_Miranda
cooked: <p>where did it say that? (I’ve been looking for that command)</p>
<p>Will try right now.</p>
<p>Thanks for the help Tom.</p>

username: Brando_Miranda
cooked: <p><a class="mention" href="/u/tom">@tom</a></p>
<p>I did prepend the conda channel:</p>
<pre><code class="lang-auto">$ conda config --prepend channels \
&gt; https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
</code></pre>
<p>but are you sure that is the right URL address? I get the following error:</p>
<pre><code class="lang-auto">$ conda install torchvision powerai-release=1.7.0

Collecting package metadata (current_repodata.json): failed

CondaHTTPError: HTTP 500 INTERNAL SERVER ERROR for url &lt;https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le/current_repodata.json&gt;
Elapsed: 00:00.706870

A remote server error occurred when trying to retrieve this URL.

A 500-type error (e.g. 500, 501, 502, 503, etc.) indicates the server failed to
fulfill a valid request.  The problem may be spurious, and will resolve itself if you
try your request again.  If the problem persists, consider notifying the maintainer
of the remote server.
</code></pre>

username: tom
cooked: <p>I don’t know, at some point you should probably check with IBM.<br>
The URL with the 500 does work when I open it locally.</p>

username: Brando_Miranda
cooked: <p>thanks tom.</p>
<p>I am working on it with them <a href="https://github.com/IBM/powerai/issues/268" rel="nofollow noopener">https://github.com/IBM/powerai/issues/268</a> and with the sys admin on my cluster. Will paste a solution once I get it. I keep getting issues but I don’t think they have to do with you so I’m unsure if pytorch peeps can help without getting more involved.</p>
<p>I will post my solution here once I get it regardless.</p>

username: Brando_Miranda
cooked: <p>For all details check (<a href="https://github.com/IBM/powerai/issues/268" rel="nofollow noopener">https://github.com/IBM/powerai/issues/268</a>).</p>
<p>Make sure you have the right conda channel prepended:</p>
<pre><code class="lang-auto">conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/
</code></pre>
<p>then install the powerai wmlce that you want e.g. 1.7.0 (most recent as of this writing):</p>
<pre><code class="lang-auto">conda create -n my_new_env python=3.7 powerai=1.7.0
conda activate my_new_env
</code></pre>

username: Brando_Miranda
cooked: <p>in my special case my HPC has conda envs I can clone btw.</p>