Repository: pytorch/benchmark
Issue: "RuntimeError: Could not infer dtype of numpy.int64" occurs on some models
Root cause: External Dependency Breakage

Hi there, I encountered `RuntimeError: Could not infer dtype of numpy.int64` during running some models provided by this repository. (Some report `RuntimeError: Numpy is not available`.)
The command I used is `python run.py [MODEL] -d cuda -m eager -t eval/train`. I think I have installed the correct environment (including numpy) and most of the models in the repository work well. So I am wondering what's going on with these problematic models. 

The questioned models are fastNLP_Bert, detectron2_maskrcnn, yolov3(train), Super_SloMo, vision_maskrcnn, etc.

Thanks so much for any feedback.



Can I have the numpy version you have installed? It could be caused by incompatible numpy version. We require numpy==1.21.2 (https://github.com/pytorch/benchmark/blob/main/requirements.txt#L18)

Hi Xu, thanks for your response. The numpy meets the requirements in my environment. 
```shell
$ conda list numpy
# Name                    Version                   Build  Channel
numpy                     1.21.2                   pypi_0    pypi
```
And torchbench version is `main@6200aa1b28d90269c5456d5f92380db7dcaada1c`.

Thanks @Lin-Mao , can I also have the error callstack?

Hi @xuzhao9 , Yes. Here is the error callstack. I even tried the latest version of torchbench, and the error is still there.
```shell
$ python run.py fastNLP_Bert -d cuda -m eager -t eval
/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/torch/cuda/random.py:111: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at /home/lm/torch1/torch/csrc/utils/tensor_numpy.cpp:77.)
  default_generator.manual_seed(seed)
Traceback (most recent call last):
  File "run.py", line 245, in <module>
    m = Model(device=args.device, test=args.test, jit=(args.mode == "jit"), batch_size=args.bs, extra_args=extra_args)
  File "/home/lm/torch1/benchmark/torchbenchmark/util/model.py", line 16, in __call__
    obj = type.__call__(cls, *args, **kwargs)
  File "/home/lm/torch1/benchmark/torchbenchmark/models/fastNLP_Bert/__init__.py", line 93, in __init__
    self.example_inputs = self._prefetch(example_inputs)
  File "/home/lm/torch1/benchmark/torchbenchmark/models/fastNLP_Bert/__init__.py", line 133, in _prefetch
    for batch_x, batch_y in example_inputs:
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/fastNLP/core/batch.py", line 266, in __iter__
    for indices, batch_x, batch_y in self.dataiter:
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 681, in __next__
    data = self._next_data()
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 719, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/fastNLP/core/batch.py", line 89, in collate_fn
    sin_x = _pad(sin_x, dataset=self.dataset, as_numpy=self.as_numpy)
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/fastNLP/core/batch.py", line 45, in _pad
    res, _ = _to_tensor(res, field_dtype=f.dtype)
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/fastNLP/core/batch.py", line 465, in _to_tensor
    raise e
  File "/home/lm/Spack/opt/spack/linux-ubuntu20.04-icelake/gcc-9.4.0/miniconda3-4.10.3-4nxjo3vkepfwnxtoftfdhvvsxemzphh6/envs/torch1/lib/python3.8/site-packages/fastNLP/core/batch.py", line 453, in _to_tensor
    new_batch = torch.as_tensor(batch)
RuntimeError: Could not infer dtype of numpy.int64
```

@Lin-Mao  Interesting, I tested that on batch.py line 89, `self.as_numpy` is `False` in my case. May I ask what is the value of `self.as_numpy` in your environment?

From my side, it seems `as_numpy` should be `False`: https://github.com/fastnlp/fastNLP/blob/v0.6.0/fastNLP/core/batch.py#L230

We are using fastNLP 0.6.0 (https://github.com/pytorch/benchmark/blob/main/torchbenchmark/models/fastNLP_Bert/requirements.txt).

Hi @xuzhao9, sorry for the late response. The `self.as_numpy` is also `False` in my case. And I am also using fastNLP 0.6.0.
I found the issue might be at [batch.py line 89](https://github.com/fastnlp/fastNLP/blob/d476678044ca0e6433f59b485e1b26413625f40a/fastNLP/core/batch.py#L89) since the print statement (added by myself) after this statement is not executed.

@Lin-Mao  Sorry I still can't reproduce this problem, can I also have the pytorch version you are using?

Hi @xuzhao9 , thanks for your response. my PyTorch version is `master@b2ddef28d70ca96f75a39f535a41a2d130f14d40`. 

@Lin-Mao Can you try with the latest master commit? We recently landed a related fix: https://github.com/pytorch/pytorch/pull/85099

Hi @xuzhao9, I used the latest master commit of PyTorch (also the latest vision and torchtext), but the error is still there. I was using the following build commands to install the latest PyTorch.
```shell
# Download source code
git clone --recursive git@github.com:AppCases/pytorch.git
cd pytorch
git submodule sync
git submodule update --init --recursive --jobs 0

# Build and install source code
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
export USE_CUDA=1
export REL_WITH_DEB_INFO=1
export MAX_JOBS=80
export USE_NINJA=OFF
python setup.py install

```

Hi @Lin-Mao, `AppCases/pytorch.git`  is not the latest master commit of PyTorch. The latest commit in this repo is 28 days ago: https://github.com/AppCases/pytorch/commit/b2ddef28d70ca96f75a39f535a41a2d130f14d40

Hi @xuzhao9, I was originally using `AppCases/pytorch` but now I am using https://github.com/pytorch/pytorch/commit/6162a043640cf01695cf568edd0be047d56477ff. It still has the same error. I am also using the latest vision and torchtext.

Hi @malfet , are you familiar with this error? I am thinking `torch.as_tensor()` API should accept `numpy.int64` inputs?

Hi @xuzhao9, I solved this issue by downgrading the pillow@9.2.0 to pillow@8.2.0. For reference to those who encounter the same issue.