Expected Tensor but got GenericList

asgardian: <p>Hi,<br>
I converted the pytorch model to torch script.Loading the model using c++ was also successful.<br>
But while doing the inference,I got such an error.</p>
<p>[W TensorImpl.h:1156] Warning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (function operator())<br>
terminate called after throwing an instance of ‘c10::Error’<br>
what():  Expected Tensor but got GenericList<br>
Exception raised from reportToTensorTypeError at …/aten/src/ATen/core/ivalue.cpp:854 (most recent call first):<br>
frame <span class="hashtag">#0:</span> c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits, std::allocator &gt;) + 0x69 (0x7f54c54cc9f9 in /home/Desktop/libtorch-cxx11-abi-shared-with-deps-1.9.0+cpu/libtorch/lib/libc10.so)<br>
frame <span class="hashtag">#1:</span> c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string&lt;char, std::char_traits, std::allocator &gt; const&amp;) + 0xd2 (0x7f54c54c9012 in /home/Desktop/libtorch-cxx11-abi-shared-with-deps-1.9.0+cpu/libtorch/lib/libc10.so)<br>
frame <span class="hashtag">#2:</span> c10::IValue::reportToTensorTypeError() const + 0x64 (0x7f54b33ba0d4 in /home/Desktop/libtorch-cxx11-abi-shared-with-deps-1.9.0+cpu/libtorch/lib/libtorch_cpu.so)<br>
frame <span class="hashtag">#3:</span> c10::IValue::toTensor() &amp;&amp; + 0x47 (0x5574250fa0a1 in ./example-app-FAN)<br>
frame <span class="hashtag">#4:</span> main + 0x20d (0x5574250f7c52 in ./example-app-FAN)<br>
frame <span class="hashtag">#5:</span> __libc_start_main + 0xe7 (0x7f54b1b13bf7 in /lib/x86_64-linux-gnu/libc.so.6)<br>
frame <span class="hashtag">#6:</span> _start + 0x2a (0x5574250f77ea in ./example-app-FAN)</p>
<p>Aborted (core dumped)</p>
<p>It seems like pytorch documentations related to the above are really scarse.Please help in solving the  issue.</p>

stan_xu: <p>Hi I soveld the same problem , you can go</p><aside class="onebox githubissue" data-onebox-src="https://github.com/triton-inference-server/server/issues/3348#issuecomment-950508638">
  <header class="source">

      <a href="https://github.com/triton-inference-server/server/issues/3348#issuecomment-950508638" target="_blank" rel="noopener nofollow ugc">github.com/triton-inference-server/server</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/triton-inference-server/server/issues/3348#issuecomment-950508638" target="_blank" rel="noopener nofollow ugc">PyTorch execute failure: isTensor(); Expected Tensor but got GenericList</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-09-10" data-time="02:50:09" data-timezone="UTC">02:50AM - 10 Sep 21 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2021-09-10" data-time="17:11:36" data-timezone="UTC">05:11PM - 10 Sep 21 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/rexlow" target="_blank" rel="noopener nofollow ugc">
          <img alt="rexlow" src="https://avatars.githubusercontent.com/u/7680796?v=4" class="onebox-avatar-inline" width="20" height="20">
          rexlow
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">**Description**
I've built a custom Yolov5 model and traced it via a [export sc<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">ript](https://github.com/ultralytics/yolov5/blob/master/export.py) from UltraLytics. Testing it via their [inference script](https://github.com/ultralytics/yolov5/blob/master/detect.py) and my Python native code works. Now I attempt to deploy it with Triton.

While deploying it Triton did not throw any error, it is when making a client call only I get this error.

A similar error can be found in #2594 , not sure why is this closed?? Perhaps @CoderHam can have a look at this issue? I'm passing a batched numpy array not tensor.

```
PyTorch execute failure: isTensor() INTERNAL ASSERT FAILED at "/tmp/tritonbuild/pytorch/build/include/torch/ATen/core/ivalue_inl.h":157, please report a bug to PyTorch. Expected Tensor but got GenericList
Exception raised from toTensor at /tmp/tritonbuild/pytorch/build/include/torch/ATen/core/ivalue_inl.h:157 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;) + 0x6c (0x7f56f28405cc in /opt/tritonserver/backends/pytorch/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) + 0xfa (0x7f56f2806d4e in /opt/tritonserver/backends/pytorch/libc10.so)
frame #2: &lt;unknown function&gt; + 0xf558 (0x7f56f2d9a558 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #3: &lt;unknown function&gt; + 0x155a2 (0x7f56f2da05a2 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #4: TRITONBACKEND_ModelInstanceExecute + 0x411 (0x7f56f2da1a91 in /opt/tritonserver/backends/pytorch/libtriton_pytorch.so)
frame #5: &lt;unknown function&gt; + 0x2f2fd7 (0x7f573ca89fd7 in /opt/tritonserver/bin/../lib/libtritonserver.so)
frame #6: &lt;unknown function&gt; + 0xfdfe0 (0x7f573c894fe0 in /opt/tritonserver/bin/../lib/libtritonserver.so)
frame #7: &lt;unknown function&gt; + 0xd6d84 (0x7f573c2d1d84 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
frame #8: &lt;unknown function&gt; + 0x9609 (0x7f573c76c609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
frame #9: clone + 0x43 (0x7f573bfbf293 in /usr/lib/x86_64-linux-gnu/libc.so.6)
```

**Triton Information**
Triton docker version 21.03

**Config file**
```
name: "my_model"
platform: "pytorch_libtorch"
max_batch_size : 128
input {
    name: "input__0"
    data_type: TYPE_FP32
    format: FORMAT_NCHW
    dims: [ 3, 640, 640 ]
}
output {
    name: "output__0"
    data_type: TYPE_FP32
    dims: [ 20 ]
    label_filename: "labels.txt"
}
instance_group {
  count: 1
  kind: KIND_GPU
  gpus: 0
}
dynamic_batching {
  preferred_batch_size: [ 1, 2, 4, 8, 16 ]
  max_queue_delay_microseconds: 100
}
```

**Python Client**

```
image = cv2.resize(img, (640, 640), interpolation = cv2.INTER_CUBIC)
image = image.astype(np.float32)
image /= 255. # [640, 640, 3]
image = np.transpose(image, (2, 0, 1)) # [3, 640, 640]
data = np.expand_dims(image, 0) # [1, 3, 640, 640]

input_1 = tchttp.InferInput("input__0", (1, 3, 640, 640), "FP32")
input_1.set_data_from_numpy(data)
outputs = [tchttp.InferRequestedOutput(name) for name in ["output__0"]]
response = triton_client.infer(
  mmodel_name='my_model', 
  inputs=[input_1],
  outputs=outputs,
  model_version='1')
logits = response.as_numpy('output__0')
```</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<ul>
<li>Otherwise I also try to create a wrapper code around the my model . just use <strong>split、cat、squeeze、narrow</strong> function to make multiple inputs combined into one</li>
</ul>
<p>like this</p>
<pre><code class="lang-auto">    class model():
    ...
    def forward(self,inputs):
        image, points = torch.split(inputs, 1)
        points = points.squeeze(1).narrow(1, 0, 1).squeeze(1)

</code></pre>
<ul>
<li>inference model</li>
</ul>
<pre><code class="lang-auto">    x0=torch.ones(1, 3, 28, 28).to(device)
    x1=torch.ones(1, 28, 28).to(device)
    x1 = x1.unsqueeze(1).expand(1, 3, 2, 2)
    inputs = torch.cat([x0, x1])
    
    model(inputs)
</code></pre>