Numpy not Available

username: a_d
cooked: <p>Hello All,<br>
I am trying to use TPUs on a Kaggle Notebook and installing Pytorch XLA as follows -</p>
<pre><code class="lang-auto">!pip3 install mkl

!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py

!python3 pytorch-xla-env-setup.py --version nightly --apt-packages libomp5 libopenblas-dev
</code></pre>
<p>However once Pytorch XLA is installed, It does not let me use numpy functions like <code>numpy.uint8</code> or for that matter even <code>torch.from_numpy()</code> does not work. The error always says - <code>Numpy not Available</code><br>
The whole stack is as follows -</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-1-abfcbbc939b0&gt; in &lt;module&gt;
   1026                                  segmentation_Maps='/kaggle/input/pascal-voc/VOC2012/SegmentationClass/') 
   1027 dataloader = DataLoader(dataset, batch_size=5)
-&gt; 1028 for _, data in enumerate(dataloader):
   1029     i = data['image']
   1030     gt = data['ground_truth']

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
    519             if self._sampler_iter is None:
    520                 self._reset()
--&gt; 521             data = self._next_data()
    522             self._num_yielded += 1
    523             if self._dataset_kind == _DatasetKind.Iterable and \

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _next_data(self)
    559     def _next_data(self):
    560         index = self._next_index()  # may raise StopIteration
--&gt; 561         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    562         if self._pin_memory:
    563             data = _utils.pin_memory.pin_memory(data)

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---&gt; 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in &lt;listcomp&gt;(.0)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---&gt; 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

&lt;ipython-input-1-abfcbbc939b0&gt; in __getitem__(self, item)
    939         print(mask.shape)
    940         image = Image.fromarray(np.uint8(image)).convert('RGB')
--&gt; 941         mask = torch.from_numpy(np.uint8(mask))
    942 
    943         image = self.transforms(image)

RuntimeError: Numpy is not available
</code></pre>
<p>I have no clue what exactly is going on here.  Does anybody know how to solve it ?<br>
PS - Pytorch XLA updates PyTorch to a nightly 1.9 version, so maybe that causes an issue  ?<br>
TIA</p>

username: Lamb_Sauce
cooked: <p>Ok. I’ve had this one before. I think (I’m only hypothesizing) that you have to refer to Numpy as np forever after. I think. You’ve referred to it as from_numpy. Maybe I’m just running the wrong track and the error is something else.</p>
<p>If this doesn’t work, I’m super sorry, I’m under 15 years of age.</p>

username: ptrblck
cooked: <p>Are you getting the same error after installing numpy via <code>pip install numpy</code>?</p>

username: Lamb_Sauce
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Sorry. I didn’t read and undetstnad the question fully. Yes, have you checked your Numpy installation?</p>

username: a_d
cooked: <p>Thank you for replying</p>
<p>Yep, I have tried installing NumPy both before and after installing PyTorch XLA.</p>
<p>I believe the error rises from a NumPy version mismatch.</p>
<p>I had missed a warning which essentially said - <code>Compiled with numpy version 0xe but found 0xd</code>.</p>
<p>The problem is that  <code>pip install --upgrade numpy</code> does not work because Kaggle loads a few modules (like Numpy and Pandas) . So one has to reload the module.<br>
I did that too but it didn’t work.</p>
<p>If someone has successfully done it please let me know.</p>
<p>PS - Sorry this was not a pytorch specific question. I had’nt thought of numpy mismatch and environment pre-loading</p>

username: ptrblck
cooked: <p>Do you need to update numpy or generally reinstall it? Based on the warning it seems that your PyTorch setup already ships with the right numpy version (the binaries would use numpy <code>1.15</code>, if I’m not mistaken).</p>

username: a_d
cooked: <p>Honestly, I have no clue,  based on the error message, I tried updating Numpy, and reloading the module, and even uninstalling and installing it again. but to no avail.<br>
If I understand the error message properly, pytorch XLA was compiled with version <code>0xe</code> but the numpy version on system is <code>0xd</code>, correct ?.<br>
Also, pytorch XLA updates the pytorch installation to 1.9, so maybe that’ why?(I am literally guessing now)</p>

username: Raja_Suman_Chowdary
cooked: <p>Use numpy directly to convert the image to tensor instead of ToTensor(). The following worked for me.<br>
img=torch.tensor(np.array(img,dtype=np.float64))/255.0</p>

username: ubamba98
cooked: <p>Hi, installing 1.7 version instead of nightly helped me solve this issue you can try that probably</p>
<pre><code class="lang-auto">!python3 pytorch-xla-env-setup.py --version 1.7 --apt-packages libomp5 libopenblas-dev
</code></pre>

username: a_d
cooked: <p>Thanks a lot, I will try it out</p>

username: magicknight
cooked: <p>Install the latest numpy 1.21 solved my problem.</p>

username: DevElevation
cooked: <p>Was getting this error after installing torch, everything was installed properly but wouldn’t work until I restarted VSCode.</p>

username: Debasish_Bal
cooked: <p>Damn, this simple trick solved my issues haha!</p>

username: MarcosMaldonado
cooked: <p>I restarted VS Code and my code started running! xd</p>

username: s.sni
cooked: <p>I had the same issue with importing torchaudio.<br>
It seems numpy 2.0.0 is not supported in pytorch yet. Downgrading numpy to 1.26.4 solved my issue.<br>
My python version: 3.11.9<br>
My Numpy version: 1.26.4<br>
torch and torchaudio version: 2.3.1</p>

username: Shahed_moez_gholami
cooked: <p>thank you s.sni I try it and its solved. My python version: 3.12</p>

username: ShauryaNova
cooked: <p>I  just downgraded the numpy version=2.0 to numpy==1.26.4 and it worked. Just try it.<br>
numpy==1.26.4</p>
<p>pytesseract==0.3.9</p>
<p>Pillow==9.2.0</p>
<p>torch==2.2.1</p>
<p>Happy coding guys… <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>