My pytorch conversion to mobile is not working...!

SungmanHong: <p>I have OCR NN really…huge and<br>
cuz I want to use pytorch on android…<br>
I tried to convert pytorch model to pytorch mobile…<br>
(But… If I can just uses pytorch library in JAVA??)<br>
Here is what I tried…</p>
<pre><code class="lang-auto">recognizer2 = recognizer
recognizer2.eval()
example = img_cv_grey, horizontal_list, free_list,\
                            decoder, beamWidth, batch_size,\
                            workers, allowlist, blocklist, detail,\
                            paragraph, contrast_ths, adjust_contrast,\
                            filter_ths, False
    torch.jit.trace(recognizer2, example)
</code></pre>
<p>The example is sample input(altough I don know why it is needed… Can’t we just convert to the mobile without sample input? it is very anoying…)</p>
<p>the error message is…</p>
<blockquote>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 result = convert_torch_mobile(“drive/My Drive/colab/easy_ocr_test/Capture1.PNG”)</p>
<p>2 frames<br>
 in convert_torch_mobile(image, decoder, beamWidth, batch_size, workers, allowlist, blocklist, detail, paragraph, min_size, contrast_ths, adjust_contrast, filter_ths, text_threshold, low_text, link_threshold, canvas_size, mag_ratio, slope_ths, ycenter_ths, height_ths, width_ths, add_margin)<br>
15     detector2.eval()<br>
16     example = img_cv_grey, horizontal_list, free_list,                            decoder, beamWidth, batch_size,                            workers, allowlist, blocklist, detail,                            paragraph, contrast_ths, adjust_contrast,                            filter_ths, False<br>
—&gt; 17     torch.jit.trace(recognizer2, example)<br>
18<br>
19</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/jit/<strong>init</strong>.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)<br>
953         return trace_module(func, {‘forward’: example_inputs}, None,<br>
954                             check_trace, wrap_check_inputs(check_inputs),<br>
 → 955                             check_tolerance, strict, _force_outplace, _module_class)<br>
956<br>
957     if (hasattr(func, ‘<strong>self</strong>’) and isinstance(func.<strong>self</strong>, torch.nn.Module) and</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/jit/<strong>init</strong>.py in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)<br>
1107             func = mod if method_name == “forward” else getattr(mod, method_name)<br>
1108             example_inputs = make_tuple(example_inputs)<br>
 → 1109             module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, strict, _force_outplace)<br>
1110             check_trace_method = module._c._get_method(method_name)<br>
1111</p>
<p>RuntimeError: Tracer cannot infer type of (array([[255, 255, 255, …, 255, 255, 255],<br>
[255, 255, 255, …, 255, 255, 255],<br>
[255, 255, 255, …, 255, 255, 255],<br>
…,<br>
[255, 255, 255, …, 255, 255, 255],<br>
[255, 255, 255, …, 255, 255, 255],<br>
[255, 255, 255, …, 255, 255, 255]], dtype=uint8), [[59, 89, 15, 31], [97, 295, 11, 31], [58, 326, 32, 58], [332, 746, 34, 58], [61, 169, 59, 79], [197, 295, 59, 79], [318, 460, 56, 80], [481, 559, 59, 79], [581, 663, 59, 79], [99, 131, 83, 99], [145, 210, 77, 103], [231, 445, 83, 99], [58, 504, 128, 154], [11, 45, 143, 159], [58, 774, 152, 176], [58, 172, 174, 198], [197, 295, 177, 197], [318, 404, 174, 198], [411, 471, 177, 197], [494, 602, 174, 198], [625, 711, 177, 197], [65, 181, 199, 219], [199, 275, 199, 219], [297, 489, 203, 219], [58, 352, 244, 273], [59, 551, 273, 293], [59, 169, 295, 315], [197, 279, 295, 315], [305, 445, 295, 315], [469, 575, 295, 315], [599, 683, 295, 315], [67, 181, 317, 337], [197, 305, 317, 337], [327, 569, 321, 337], [60, 288, 366, 390], [58, 724, 390, 414], [59, 169, 413, 433], [197, 281, 413, 433], [305, 437, 413, 433], [461, 565, 413, 433], [591, 675, 413, 433], [96, 182, 434, 458], [199, 307, 437, 457], [327, 523, 439, 457], [58, 242, 484, 508], [58, 472, 508, 532], [58, 172, 530, 554], [197, 273, 533, 553], [297, 437, 533, 553], [461, 535, 533, 551], [561, 645, 533, 553], [97, 181, 555, 575], [199, 305, 555, 575], [362, 440, 552, 576], [456, 506, 552, 576], [532, 795, 555, 575], [635, 659, 613, 631], [696, 734, 610, 634], [335, 499, 605, 641], [539, 561, 615, 631]], <span class="chcklst-box fa fa-square-o fa-fw"></span>, ‘greedy’, 5, 1, 0, None, None, 1, False, 0.1, 0.5, 0.003, False)<br>
:Only tensors and (possibly nested) tuples of tensors, lists, or dictsare supported as inputs or outputs of traced functions, but instead got value of type ndarray.</p>
</blockquote>
<p>And my model is</p>
<blockquote>
<p>DataParallel(<br>
(module): Model(<br>
(FeatureExtraction): ResNet_FeatureExtractor(<br>
(ConvNet): ResNet(<br>
(conv0_1): Conv2d(1, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn0_1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv0_2): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn0_2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(maxpool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(layer1): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
)<br>
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(maxpool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(layer2): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(maxpool3): MaxPool2d(kernel_size=2, stride=(2, 1), padding=(0, 1), dilation=1, ceil_mode=False)<br>
(layer3): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(3): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(4): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(conv3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(layer4): Sequential(<br>
(0): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(1): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(conv4_1): Conv2d(512, 512, kernel_size=(2, 2), stride=(2, 1), padding=(0, 1), bias=False)<br>
(bn4_1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv4_2): Conv2d(512, 512, kernel_size=(2, 2), stride=(1, 1), bias=False)<br>
(bn4_2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(AdaptiveAvgPool): AdaptiveAvgPool2d(output_size=(None, 1))<br>
(SequenceModeling): Sequential(<br>
(0): BidirectionalLSTM(<br>
(rnn): LSTM(512, 512, batch_first=True, bidirectional=True)<br>
(linear): Linear(in_features=1024, out_features=512, bias=True)<br>
)<br>
(1): BidirectionalLSTM(<br>
(rnn): LSTM(512, 512, batch_first=True, bidirectional=True)<br>
(linear): Linear(in_features=1024, out_features=512, bias=True)<br>
)<br>
)<br>
(Prediction): Linear(in_features=512, out_features=1568, bias=True)<br>
)<br>
)</p>
</blockquote>
<p>Why it happens? What I want is just use pytorch in JAVA… any help please? Thanks…</p>

SungmanHong: <p>My simpled error output is this</p>
<p><code>torch.jit.trace(recognizer2, example)</code></p>
<blockquote>
<p>RuntimeError: Type ‘Tuple[int, int, float, float, float, int, float, float, float, float, float, float, bool]’ cannot be traced. Only Tensors and (possibly nested) Lists, Dicts, and Tuples of Tensors can be traced</p>
</blockquote>