Repository: pytorch
Issue: isObject()INTERNAL ASSERT FAILED at "../aten/src/ATen/core/ivalue_inl.h":115, please report a bug to PyTorch. Expected Object but got Tensor
Labels: oncall: jit,module: android,oncall: mobile

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:

1. I followed the NMT demo for android-app-demo [https://github.com/pytorch/android-demo-app](url) to make my own app, and converted my NMT model into a lite model via torch.jit.trace. And the python scripts is like this:

```
#coding:utf-8
import sys 
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile
sys.path.append("/workspace/OpenNMT-py-test_lite")
import onmt.inputters as inputters
import onmt.modules
from onmt.encoders import str2enc
from onmt.decoders import str2dec
from onmt.modules import Embeddings, VecEmbedding, CopyGenerator
from onmt.modules.util_class import Cast
from onmt.utils.misc import use_gpu
from onmt.utils.logging import logger
from onmt.utils.parse import ArgumentParser
from onmt.model_builder import build_base_model
from onmt.bin.translate import *

model_path= sys.argv[1]

parser = ArgumentParser(description='translate.py')
opts.config_opts(parser)
opts.translate_opts(parser)
opt = parser.parse_args(f"--model {model_path} --gpu 0")

ArgumentParser.validate_translate_opts(opt)
logger = init_logger(opt.log_file)

translator = build_translator(opt, report_score=True)
translator.model.cpu()
translator.model.float()

quant_encoder = torch.quantization.quantize_dynamic(translator.model.encoder)
quant_decoder = torch.quantization.quantize_dynamic(translator.model.decoder)
quant_generator = torch.quantization.quantize_dynamic(translator.model.generator)
encoder_input = torch.randint(0, 8000, (100,1,1))
encoder_length = torch.tensor([100])
decoder_input = torch.tensor([[[123]]])
encoder_output = torch.zeros([100,1,256])
decoder_step = torch.tensor([0])
decoder_output = torch.zeros([1,1,256])

traced_encoder = torch.jit.trace(quant_encoder, (encoder_input, encoder_length))
args = [decoder_input, encoder_output, decoder_step, encoder_length]
kwargs = {"memory_lengths": encoder_length}
quant_decoder.init_state(encoder_input, None, None)
traced_decoder = torch.jit.trace(quant_decoder, args, strict=False)
traced_generator = torch.jit.trace(quant_generator, (decoder_output))


traced_encoder_optimized = optimize_for_mobile(traced_encoder)
traced_encoder_optimized._save_for_lite_interpreter("optimized_encoder.ptl")
traced_decoder_optimized = optimize_for_mobile(traced_decoder)
traced_decoder_optimized._save_for_lite_interpreter("optimized_decoder.ptl")
traced_generator_optimized = optimize_for_mobile(traced_generator)
traced_generator_optimized._save_for_lite_interpreter("optimized_generator.ptl")
```
2. Then I try to run the model in android. It raise an error in decoder step. The core code is like this:
```
        final long[] decoderInputShape = new long[]{1};
        mInputTensorBuffer = Tensor.allocateLongBuffer(1);
        mInputTensorBuffer.put(0);
        mInputTensor = Tensor.fromBlob(mInputTensorBuffer, decoderInputShape);
        ArrayList<Integer> result = new ArrayList<>(MAX_LENGTH);
        final long[] decoderstepShape = new long[]{1};
        for (int i=0; i<MAX_LENGTH; i++) {
            LongBuffer decoderstepTensorBuffer = Tensor.allocateLongBuffer(1);
            decoderstepTensorBuffer.put(i);
            Tensor decoderstepTensor = Tensor.fromBlob(decoderstepTensorBuffer, decoderstepShape);
            final IValue[] outputTuple = mModuleDecoder.forward(
                    IValue.from(mInputTensor),
                    IValue.from(outputsTensor),
                    IValue.from(decoderstepTensor),
                    IValue.from(inputlengthTensor)
            ).toTuple(); // error occurs here
            final Tensor decoderOutputTensor = outputTuple[0].toTensor();
            // some other process
        }
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
This is the error message.
```
E/AndroidRuntime: FATAL EXCEPTION: Thread-2
    Process: com.example.seq2seqnmt, PID: 16085
    com.facebook.jni.CppException: isObject()INTERNAL ASSERT FAILED at "../aten/src/ATen/core/ivalue_inl.h":115, please report a bug to PyTorch. Expected Object but got Tensor
      
      debug_handle:-1
      
    Exception raised from toObject at ../aten/src/ATen/core/ivalue_inl.h:115 (most recent call first):
    (no backtrace available)
        at org.pytorch.LiteNativePeer.forward(Native Method)
        at org.pytorch.Module.forward(Module.java:52)
        at org.pytorch.demo.seq2seqnmt.MainActivity.translate(MainActivity.java:282)
        at org.pytorch.demo.seq2seqnmt.MainActivity.run(MainActivity.java:108)
        at java.lang.Thread.run(Thread.java:764)
```
## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

 - PyTorch Version (e.g., 1.9.0):
 - OS (e.g., Linux): Linux/Mac
 - How you installed PyTorch (`conda`, `pip`, source):
 - Build command you used (if compiling from source):
 - Python version: 3.6.13
 - CUDA/cuDNN version:
 - GPU models and configuration:
 - Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->