Incomprehensible behaviour

username: zetyquickly
cooked: <p>Hello,</p>
<p>I’ve tried to use custom model on Android but forward fails with error I unable to understand.</p>
<p>I’ve scripted my model using TorchScript annotation method and now I am trying to preform a forward on mobile.</p>
<p>Model looks something like that:</p>
<pre><code class="lang-auto">config = ...
class WrapRPN(nn.Module):
    def __init__(self):
        super().__init__()
        self.rpn = RPN(config).eval().cpu()
    def forward(self, features):
        # type: (Dict[str, torch.Tensor]) -&gt; Dict[str, torch.Tensor]
        mock_input : InputClass = InputClass(torch.rand((N, 320, 320)))
        instances = self.rpn(mock_input, features)
        output : Dict[str, torch.Tensor] = {}
        for idx in range(len(instances)):
            inst : Instances = instances[idx]
            box_tensor : torch.Tensor = inst.proposal_boxes.tensor
            output[str(idx)] = box_tensor
        return output
</code></pre>
<p>It has been converted and loaded to mobile, but fails on runtime</p>
<pre><code class="lang-auto">E/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.pytorch.helloworld, PID: 20157
    java.lang.RuntimeException: Unable to start activity ComponentInfo{org.pytorch.helloworld/org.pytorch.helloworld.MainActivity}: com.facebook.jni.CppException: forward() Expected a value of type 'Dict[str, Tensor]' for argument 'features' but instead found type 'Dict[str, Tensor]'.
    Position: 1
    Declaration: forward(ClassType&lt;WrapRPN&gt; self, Dict(str, Tensor) features) -&gt; (Dict(str, Tensor)) (checkArg at ../aten/src/ATen/core/function_schema_inl.h:194)
    (no backtrace available)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3784)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3955)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:91)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:149)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:103)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2392)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:213)
        at android.app.ActivityThread.main(ActivityThread.java:8147)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:513)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
     Caused by: com.facebook.jni.CppException: forward() Expected a value of type 'Dict[str, Tensor]' for argument 'features' but instead found type 'Dict[str, Tensor]'.
    Position: 1
    Declaration: forward(ClassType&lt;WrapRPN&gt; self, Dict(str, Tensor) features) -&gt; (Dict(str, Tensor)) (checkArg at ../aten/src/ATen/core/function_schema_inl.h:194)
    (no backtrace available)
        at org.pytorch.NativePeer.forward(Native Method)
        at org.pytorch.Module.forward(Module.java:37)
        at org.pytorch.helloworld.MainActivity.onCreate(MainActivity.java:66)
        at android.app.Activity.performCreate(Activity.java:8068)
        at android.app.Activity.performCreate(Activity.java:8056)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1320)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3757)
        	... 11 more
</code></pre>
<p>Java code is following:</p>
<pre><code class="lang-auto">    final Tensor inputTensor = TensorImageUtils.bitmapToFloat32Tensor(bitmap,
        TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB);

    Map&lt;String, IValue&gt; hm = new HashMap&lt;String, IValue&gt;();
    List&lt;String&gt; keys = Arrays.asList("p2", "p3", "p4", "p5", "p6");;
    for (String key : keys) {
      hm.put(key, IValue.from(inputTensor));
    }
    final IValue rpn_input = IValue.dictStringKeyFrom(hm);
    module.forward(rpn_input);
</code></pre>
<p>What does it mean?<br>
<code>Expected a value of type 'Dict[str, Tensor]' for argument 'features' but instead found type 'Dict[str, Tensor]'</code> <img src="https://discuss.pytorch.org/images/emoji/apple/exploding_head.png?v=9" title=":exploding_head:" class="emoji" alt=":exploding_head:"></p>

username: zetyquickly
cooked: <p>So, I’ve spent a little time to understand what is going on and to find a way to overcome this issue.</p>
<p>What I’ve noticed:</p>
<ol>
<li>If we use constructions such as <code>IValue.dictStringKeyFrom(hm)</code>, where <code>hm</code> is <code>HashMap&lt;String, IValue&gt;</code> or if we use <code>IValue.listFrom(lst)</code>, where <code>lst</code> is <code>List&lt;IValue&gt;</code> we will obtain the behaviour described above.</li>
<li>But if we use <code>IValue.from(arr)</code>, where <code>arr</code> of type <code>Tensor[]</code>, we will not face this issue and JAVA won’t tell that it’s expected <code>List[Tensor]</code> but got <code>List[Tensor]</code>
</li>
</ol>
<p>Concluding this, there’s no way to construct analogue of <code>(2)</code> using dictionaries.<br>
<code>IValue</code> (<a href="https://pytorch.org/docs/stable/org/pytorch/IValue.html" rel="nofollow noopener">https://pytorch.org/docs/stable/org/pytorch/IValue.html</a>) doesn’t have overriden method <code>public static IValue dictStringKeyFrom(Map&lt;String, T&gt; map)</code>, where <code>T</code> is <code>Tensor</code>.<br>
I think in this case it might work, but it does not deny the fact there might be a bug</p>

username: IvanKobzarev
cooked: <p>Thanks for this finding.</p>
<p>I reproduced it locally and debugging it. List[Tensor] will be represented as a separate type on libtorch IValue side.</p>
<p>Looks like we have some unexpected behavior on jni with Dict types which is converted to libtorch IValue{GenericDict}}.</p>

username: IvanKobzarev
cooked: <p>Hello <a class="mention" href="/u/zetyquickly">@zetyquickly</a>,</p>
<p>Thanks one more time for this finding.</p>
<p>It happened as in jni tensorType was deduced from the first entry value of dictionary, including shape, requires_grad etc.<br>
While torchscript function did not have it. Dict is not covariant, so the subtype check required equal KeyType(str) and ValueType(TensorType0).<br>
TensorType’s of function argument and provided value were different and typecheck failed.<br>
Error message did not include that additional information about TensorType.</p>
<p>This problem was fixed on android-jni level in commit:<br>
<aside class="onebox githubcommit">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/commit/5ada5363fc18d36ddf3f6b5f54123b80a42ec819" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Commit">
    <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M10.86 7c-.45-1.72-2-3-3.86-3-1.86 0-3.41 1.28-3.86 3H0v2h3.14c.45 1.72 2 3 3.86 3 1.86 0 3.41-1.28 3.86-3H14V7h-3.14zM7 10.2c-1.22 0-2.2-.98-2.2-2.2 0-1.22.98-2.2 2.2-2.2 1.22 0 2.2.98 2.2 2.2 0 1.22-.98 2.2-2.2 2.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/commit/5ada5363fc18d36ddf3f6b5f54123b80a42ec819" target="_blank" rel="nofollow noopener">GenericDict/List type use unshapedType() (#30428)</a>
    </h4>

    <div class="github-info">
      <div class="date">
        committed <span class="discourse-local-date" data-format="ll" data-date="2019-11-26" data-time="23:31:26" data-timezone="UTC">11:31PM - 26 Nov 19 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/IvanKobzarev" target="_blank" rel="nofollow noopener">
          <img alt="IvanKobzarev" src="https://avatars3.githubusercontent.com/u/6638825?v=4" class="onebox-avatar-inline" width="20" height="20">
          IvanKobzarev
        </a>
        
      </div>

      <div class="lines" title="changed 1 files with 5 additions and 6 deletions">
        <a href="https://github.com/pytorch/pytorch/commit/5ada5363fc18d36ddf3f6b5f54123b80a42ec819" target="_blank" rel="nofollow noopener">
          <span class="added">+5</span>
          <span class="removed">-6</span>
        </a>
      </div>
    </div>

  </div>
</div>


  <div class="github-row">
    <pre class="github-content" style="white-space: normal;">Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30428
Reported issue https://discuss.pytorch.org/t/incomprehensible-behaviour/61710
Steps to reproduce:
```
class WrapRPN(nn.Module):
 def __init__(self):
 super().__init__()
 def forward(self, features):
 # type: (Dict[str, Tensor]) -&gt; int
...</pre>
  </div>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
It was merged in master recently.</p>
<p>Separate issue for more detailed error messages in TensorType checks.<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/30418" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/30418" target="_blank" rel="nofollow noopener">More detailed information about TensorType in error messages</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-11-25" data-time="21:34:09" data-timezone="UTC">09:34PM - 25 Nov 19 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/IvanKobzarev" target="_blank" rel="nofollow noopener">
          <img alt="IvanKobzarev" src="https://avatars3.githubusercontent.com/u/6638825?v=4" class="onebox-avatar-inline" width="20" height="20">
          IvanKobzarev
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">🚀 Feature
More detailed information about TensorType, including TensorType specifiers as shape, strides, req_grad etc. in error messages
Motivation
Reported issue https://discuss.pytorch.org/t/incomprehensible-behaviour/61710
Error message is...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">enhancement</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>Android nightlies (snapshots) are already republished with this fix, the error should not happen with them.<br>
To use nightlies (to force refresh dependencies gradle has argument <code>--refresh-dependencies</code>)</p>
<pre><code class="lang-auto">repositories {
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots"
    }
}

dependencies {
    ...
    implementation 'org.pytorch:pytorch_android:1.4.0-SNAPSHOT'
    implementation 'org.pytorch:pytorch_android_torchvision:1.4.0-SNAPSHOT'
    ...
}
</code></pre>

username: zetyquickly
cooked: <aside class="quote no-group" data-username="zetyquickly" data-post="2" data-topic="61710">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zetyquickly/48/22489_2.png" class="avatar"> zetyquickly:</div>
<blockquote>
<p>if we use <code>IValue.listFrom(lst)</code> , where <code>lst</code> is <code>List&lt;IValue&gt;</code> we will obtain the behaviour described above</p>
</blockquote>
</aside>
<p>Thanks a lot for the FIX! Tell me please, does it also affect also behviour of <code>List&lt;IValue&gt;</code>? When I tried it also fails with this “List not a List” error</p>

username: IvanKobzarev
cooked: <p>Yes, that problem affected both our Generic containers (Dict and List) when the element type was TensorType. After the fix GenericList is also initialized with c10::unshapedType(firstElement) which should fix the problem like ‘List[Tensor] is not List[Tensor]’</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/master/android/pytorch_android/src/main/cpp/pytorch_jni_common.cpp#L509" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/master/android/pytorch_android/src/main/cpp/pytorch_jni_common.cpp#L509" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/master/android/pytorch_android/src/main/cpp/pytorch_jni_common.cpp#L509</a></h4>
<pre class="onebox"><code class="lang-cpp"><ol class="start lines" start="499" style="counter-reset: li-counter 498 ;">
<li>              facebook::jni::JArrayClass&lt;JIValue::javaobject&gt;::javaobject()&gt;(</li>
<li>              "toList");</li>
<li>  auto jarray = jMethodGetList(jivalue);</li>
<li>  size_t n = jarray-&gt;size();</li>
<li>  if (n == 0) {</li>
<li>    return at::IValue{c10::impl::GenericList(c10::TensorType::get())};</li>
<li>  }</li>
<li>
</li>
<li>  auto jivalue_first_element = jarray-&gt;getElement(0);</li>
<li>  auto first_element = JIValue::JIValueToAtIValue(jivalue_first_element);</li>
<li class="selected">  c10::impl::GenericList list{c10::unshapedType(first_element.type())};</li>
<li>  list.reserve(n);</li>
<li>  list.push_back(first_element);</li>
<li>  for (auto i = 1; i &lt; n; ++i) {</li>
<li>    auto jivalue_element = jarray-&gt;getElement(i);</li>
<li>    auto element = JIValue::JIValueToAtIValue(jivalue_element);</li>
<li>    list.push_back(element);</li>
<li>  }</li>
<li>  return at::IValue{list};</li>
<li>} else if (JIValue::kTypeCodeDictStringKey == typeCode) {</li>
<li>  static const auto jMethodGetDictStringKey =</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>