Classification model keeps predicting same class in android

bebbieyin: <p>I have a trained a classification model to classify black and white segmented outputs.<br>
The pixels consists of only two values, 0 for black and 1 for white.<br>
The image looks like this:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/b/0bb053023c1d7b4029ef652b989a9dc7e5eae49a.jpeg" alt="bw" data-base62-sha1="1Fp1yEC8dZLe0S54GNJJShdvzqa" width="294" height="275"></p>
<p>Before I run the classification model, the original color image has gone though a segmentation model to get the the black and white bitmap. The segmentation model works fine and the results are pretty similar to the PC version, but when I put the black and white bitmap into the mobile classification model, it keeps predicting the same class.</p>
<p>I tried exporting the black and white bitmap to PNG and ran it on the PC side, it is able to predict the correct class.</p>
<p>I created the black and white bitmap like this:</p>
<pre><code class="lang-auto">for (int j = 0; j &lt; scores.length; j++) {
            if (scores[j]&gt;0.4){
                intValues[j] = 0xFFFFFFFF; // white pixels

            }
            else{
                intValues[j] =0xFF000000; // black pixels

            }
        }
</code></pre>
<p>I’ve optimized the model for mobile like this:</p>
<pre><code class="lang-auto">script_module = torch.jit.script(pc_classification_model)
optimized_scripted_module = optimize_for_mobile(script_module )
optimized_scripted_module._save_for_lite_interpreter("mobile_classification.ptl")
</code></pre>
<p>Library used for Pytorch Android :</p>
<pre><code class="lang-auto">implementation 'org.pytorch:pytorch_android_lite:1.10.0'
implementation 'org.pytorch:pytorch_android_torchvision_lite:1.10.0'
</code></pre>

Linbin: <p>I think it’s related to input normalization. Different image format can lead to different input values. You may print the input tensor to debug.</p><aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/android-demo-app/blob/master/ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java#L119">
  <header class="source">

      <a href="https://github.com/pytorch/android-demo-app/blob/master/ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java#L119" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/android-demo-app/blob/master/ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java#L119" target="_blank" rel="noopener nofollow ugc">pytorch/android-demo-app/blob/master/ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java#L119</a></h4>



    <pre class="onebox">      <code class="lang-java">
        <ol class="start lines" start="109" style="counter-reset: li-counter 108 ;">
            <li>    try {</li>
            <li>        mModule = LiteModuleLoader.load(MainActivity.assetFilePath(getApplicationContext(), "deeplabv3_scripted_optimized.ptl"));</li>
            <li>    } catch (IOException e) {</li>
            <li>        Log.e("ImageSegmentation", "Error reading assets", e);</li>
            <li>        finish();</li>
            <li>    }</li>
            <li>}</li>
            <li>
            </li>
<li>@Override</li>
            <li>public void run() {</li>
            <li class="selected">    final Tensor inputTensor = TensorImageUtils.bitmapToFloat32Tensor(mBitmap,</li>
            <li>            TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB);</li>
            <li>    final float[] inputs = inputTensor.getDataAsFloatArray();</li>
            <li>
            </li>
<li>    final long startTime = SystemClock.elapsedRealtime();</li>
            <li>    Map&lt;String, IValue&gt; outTensors = mModule.forward(IValue.from(inputTensor)).toDictStringKey();</li>
            <li>    final long inferenceTime = SystemClock.elapsedRealtime() - startTime;</li>
            <li>    Log.d("ImageSegmentation",  "inference time (ms): " + inferenceTime);</li>
            <li>
            </li>
<li>    final Tensor outputTensor = outTensors.get("out").toTensor();</li>
            <li>    final float[] scores = outputTensor.getDataAsFloatArray();</li>
        </ol>
      </code>
    </pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


bebbieyin: <p>This is the input for the segmentation, the segmentation does not have any problems though, only the classification. This is my classification code:</p>
<pre><code class="lang-auto">//outputBitmap is the segmented black and white image
final Tensor segmentedTensor = TensorImageUtils.bitmapToFloat32Tensor(outputBitmap,mean, std); 
final Tensor outputTensor1 = module_classification.forward(IValue.from(segmentedTensor)).toTensor();
float[] scores1 = outputTensor1.getDataAsFloatArray();

float maxScore = -Float.MAX_VALUE;
int maxScoreIdx = -1;
for (int i = 0; i &lt; scores1.length; i++) {
    if (scores1[i] &gt; maxScore) {
        maxScore = scores1[i];
        maxScoreIdx = i;
    }
}
String className = Constants.CLASSES[maxScoreIdx];
textView.setText(className);
</code></pre>
<p>No normalization is used, so i set the mean and std to</p>
<pre><code class="lang-auto">float[] mean = {0.0f, 0.0f, 0.0f};
float[] std = {1.0f, 1.0f, 1.0f};
</code></pre>
<p>I printed the input tensor for the classification too, it does not seem to be normalization problem as the input consists of only 0s and 1s, same as with PC</p>

David_Reiss: <p>The same model should produce the same output given the same input on both mobile and server.  It’s good that you validate that the segmentedTensor has the proper range.  Does it also have the proper shape?  Do you get different results from the classification model when classifying different inputs?</p>