Repository: pytorch
Issue: Error running trace on Pytorch Crowd Counting model
Labels: oncall: jit,triaged

Hi,

I'm trying to use your prebuilt Pytorch crowd counting model with [NVIDIA's Triton Inference Server](https://docs.nvidia.com/deeplearning/sdk/triton-inference-server-guide/docs/index.html). Looks like it requires the PyTorch model to be saved by `torch.jit.save()`. I'm facing errors while tracing the model with an example. Appreciate any help. 

The model is available from [this link](https://drive.google.com/open?id=1TRJr9YuP1dFpnbQvSSQHqIqhLFdElo_Q). The model runs fine when I run single predictions using this [predict file](https://github.com/kevhnmay94/SS-DCNet/blob/master/predict.py).

First, inference server gave error while I directly use the `best_epoch.pth` checkpoint - 

![image](https://user-images.githubusercontent.com/7122670/80325147-f54c8900-8801-11ea-8469-77a41c02ba1c.png)

Then, I try to run a trace with `torch.jit.trace()` as [documented here](https://pytorch.org/tutorials/advanced/cpp_export.html) to convert it to torchscript type but it gives me an error on some of the tensors used. Below is the script and stack trace - 

Python: 3.6.9
Pytorch Version: 1.3.1
Tensorrtserver: 1.11.0

```python
import os
import sys
import torch
import numpy as np
from Network.SSDCNet import SSDCNet_classify

# Intialize params to model
label_indice = np.arange(0.5,22+0.5,0.5)
add = np.array([1e-6,0.05,0.10,0.15,0.20,0.25,0.30,0.35,0.40,0.45])
label_indice = np.concatenate((add,label_indice))
class_num = len(label_indice)+1

# Create a model instance
net = SSDCNet_classify(class_num,label_indice,div_times=2,frontend_name='VGG16',block_num=5,IF_pre_bn=False,IF_freeze_bn=False,load_weights=True,psize=64,pstride = 64,parse_method ='maxp')

all_state_dict = torch.load('/home/ubuntu/mayub/Github/SS-DCNet/SHA/best_epoch.pth',map_location=torch.device('cpu'))

# load the state_dict
net.load_state_dict(all_state_dict['net_state_dict'])

# Change to eval mode
net.eval()

# Create a sample instance (I get this shape from sample predict file)
example = torch.rand(1, 3, 768, 1024)

#Run a trace with example
traced_script_module = torch.jit.trace(net, example)

```
Trace give me the following error - 

```python
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-22-af5c103c2ac9> in <module>
----> 1 traced_script_module = torch.jit.trace(net, example)

~/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/jit/__init__.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, _force_outplace, _module_class, _compilation_unit)
    856         return trace_module(func, {'forward': example_inputs}, None,
    857                             check_trace, wrap_check_inputs(check_inputs),
--> 858                             check_tolerance, _force_outplace, _module_class)
    859 
    860     if (hasattr(func, '__self__') and isinstance(func.__self__, torch.nn.Module) and

~/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/jit/__init__.py in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, _force_outplace, _module_class, _compilation_unit)
    995         func = mod if method_name == "forward" else getattr(mod, method_name)
    996         example_inputs = make_tuple(example_inputs)
--> 997         module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, _force_outplace)
    998         check_trace_method = module._c._get_method(method_name)
    999 

RuntimeError: Tracer cannot infer type of {'conv1': [], 'conv2': [], 'conv3': tensor([[[[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.0000, 10.7944, 12.7899,  ..., 19.2663,  1.8643,  5.8381],
          [ 0.0000,  7.0714,  8.2250,  ...,  6.6337,  8.9443,  4.3685],
          ...,
          [ 0.0000,  0.0000,  0.1529,  ...,  0.0000,  3.9005,  2.5733],
          [ 0.0000,  0.0000,  1.3247,  ...,  9.6619,  5.4776, 12.0128],
          [ 0.0000,  3.9498,  7.2731,  ..., 12.5920,  3.4570,  4.9324]],

         [[ 6.2152,  4.2109,  1.4538,  ...,  9.0008,  4.8270,  5.3406],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          ...,
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 2.7461,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],

         [[ 3.3530, 12.9000,  0.0000,  ..., 10.2020,  5.6883, 13.4378],
          [ 8.3358,  0.0000,  0.4850,  ...,  0.0000,  0.0000,  1.2402],
          [10.6660,  0.0000,  0.0000,  ...,  0.0000,  2.6775,  3.4912],
          ...,
          [ 7.2844,  0.0000,  0.0000,  ...,  0.0000,  2.4160,  4.3557],
          [ 0.0000,  0.0000,  3.2270,  ...,  0.0000,  0.0000, 14.2611],
          [ 0.0000,  8.8993,  8.3703,  ...,  0.0000,  9.0703,  7.7137]],

         ...,

         [[ 0.0000,  6.9046,  8.8958,  ..., 13.0320,  2.4187,  8.6959],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  1.3236],
          ...,
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  1.7601],
          [ 2.6597,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],

         [[ 0.0000,  0.0000, 27.1142,  ...,  2.5222,  4.2344, 16.2329],
          [13.0420,  5.2728,  3.5170,  ...,  0.0000,  2.5529, 12.2657],
          [ 7.4807,  2.7274,  4.9321,  ...,  0.0000,  1.3603,  9.9757],
          ...,
          [ 0.0000,  5.1109,  0.0000,  ...,  0.0000, 12.0808, 12.9898],
          [ 5.5294,  4.1575,  7.9874,  ..., 11.1128,  7.7980, 20.6503],
          [10.8941, 31.3485, 13.9562,  ..., 16.0611,  6.2370, 12.5933]],

         [[ 2.4946,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [14.3443,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 0.6402,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          ...,
          [ 5.3813,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
          [ 4.0173,  0.0000,  0.0000,  ...,  0.0000,  0.2018,  0.0000],
          [15.4178,  6.2313,  4.7818,  ...,  5.4549,  0.0000, 14.1680]]]],
       grad_fn=<MaxPool2DWithIndicesBackward>), 'conv4': tensor([[[[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 1.0539, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0658, 0.2390,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[0.6219, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [1.3965, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         ...,

         [[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.6745,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.7359, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[1.6867, 5.0112, 3.1645,  ..., 0.6046, 2.9714, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          ...,
          [0.1528, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 2.5369, 1.0550, 2.8817]]]],
       grad_fn=<MaxPool2DWithIndicesBackward>), 'conv5': tensor([[[[1.3796e+01, 1.4452e+01, 1.2862e+01,  ..., 6.7781e+00,
           9.2815e+00, 9.9267e+00],
          [8.3384e+00, 8.3635e+00, 4.7208e+00,  ..., 7.1589e-01,
           1.5264e+00, 1.6482e+00],
          [2.8230e+00, 6.3764e-01, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 7.7796e-01],
          ...,
          [4.9598e+00, 2.4025e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 1.0588e+00],
          [5.8041e+00, 3.8463e+00, 0.0000e+00,  ..., 8.4420e-03,
           4.0681e-01, 2.0312e+00],
          [6.6123e+00, 5.5535e+00, 2.7047e+00,  ..., 2.8173e+00,
           3.5163e+00, 5.7462e+00]],

         [[2.1448e+00, 2.8523e+00, 1.5730e+00,  ..., 9.4367e-01,
           2.1874e+00, 1.3170e+00],
          [4.5012e+00, 4.9501e+00, 2.8207e+00,  ..., 1.8323e+00,
           4.1885e+00, 4.0239e+00],
          [7.6315e-01, 2.9784e-01, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 8.0759e-01],
          ...,
          [2.9077e+00, 2.0675e+00, 0.0000e+00,  ..., 0.0000e+00,
           9.1627e-01, 8.6342e-01],
          [4.7475e+00, 3.7969e+00, 1.4891e+00,  ..., 2.4889e-01,
           2.4533e+00, 2.8074e+00],
          [2.3529e+00, 2.9757e+00, 2.3051e+00,  ..., 9.2457e-01,
           1.9125e+00, 1.6066e+00]],

         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          ...,
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]],

         ...,

         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          ...,
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]],

         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          ...,
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [1.1057e+00, 2.5535e-01, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00],
          [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]],

         [[8.2280e-02, 2.7121e+00, 3.0622e+00,  ..., 2.3801e+00,
           4.3697e+00, 1.9868e+00],
          [2.6579e+00, 5.2237e+00, 4.1553e+00,  ..., 4.4512e+00,
           5.2473e+00, 3.5402e+00],
          [0.0000e+00, 2.0596e+00, 4.3195e-01,  ..., 4.8868e-01,
           2.0991e-01, 0.0000e+00],
          ...,
          [4.6866e+00, 6.1409e+00, 6.4256e-01,  ..., 0.0000e+00,
           2.2996e+00, 1.2674e+00],
          [5.3775e+00, 6.8193e+00, 3.7130e+00,  ..., 4.2376e+00,
           4.7349e+00, 3.6980e+00],
          [0.0000e+00, 4.8565e-01, 1.4950e-01,  ..., 2.2111e+00,
           1.1456e+00, 5.6986e-01]]]], grad_fn=<MaxPool2DWithIndicesBackward>)}
:List trace inputs must have elements (toTraceableIValue at /opt/conda/conda-bld/pytorch_1573049304260/work/torch/csrc/jit/pybind_utils.h:286)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7f3cc8476687 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x4f9175 (0x7f3cfdeb6175 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #2: <unknown function> + 0x5731e2 (0x7f3cfdf301e2 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #3: <unknown function> + 0x58a724 (0x7f3cfdf47724 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #4: <unknown function> + 0x206b86 (0x7f3cfdbc3b86 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #5: _PyCFunction_FastCallDict + 0x154 (0x55b2047adc54 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #6: <unknown function> + 0x199c0e (0x55b204835c0e in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #7: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #8: <unknown function> + 0x192e66 (0x55b20482ee66 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #9: <unknown function> + 0x193e73 (0x55b20482fe73 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #10: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #11: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #12: <unknown function> + 0x192e66 (0x55b20482ee66 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #13: <unknown function> + 0x193e73 (0x55b20482fe73 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #14: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #15: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #16: PyEval_EvalCodeEx + 0x329 (0x55b2048309b9 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #17: PyEval_EvalCode + 0x1c (0x55b20483175c in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #18: <unknown function> + 0x1ba167 (0x55b204856167 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #19: _PyCFunction_FastCallDict + 0x91 (0x55b2047adb91 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #20: <unknown function> + 0x199abc (0x55b204835abc in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #21: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #22: _PyGen_Send + 0x256 (0x55b204838be6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #23: _PyEval_EvalFrameDefault + 0x144f (0x55b20485989f in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #24: _PyGen_Send + 0x256 (0x55b204838be6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x144f (0x55b20485989f in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #26: _PyGen_Send + 0x256 (0x55b204838be6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #27: _PyCFunction_FastCallDict + 0x115 (0x55b2047adc15 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #28: <unknown function> + 0x199abc (0x55b204835abc in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #29: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #30: <unknown function> + 0x193c5b (0x55b20482fc5b in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #31: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #32: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #33: <unknown function> + 0x193c5b (0x55b20482fc5b in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #34: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #35: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #36: <unknown function> + 0x192e66 (0x55b20482ee66 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #37: _PyFunction_FastCallDict + 0x3d8 (0x55b204830598 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #38: _PyObject_FastCallDict + 0x26f (0x55b2047ae01f in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #39: _PyObject_Call_Prepend + 0x63 (0x55b2047b2aa3 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #40: PyObject_Call + 0x3e (0x55b2047ada5e in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #41: _PyEval_EvalFrameDefault + 0x19e7 (0x55b204859e37 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #42: <unknown function> + 0x193136 (0x55b20482f136 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #43: <unknown function> + 0x193ed6 (0x55b20482fed6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #44: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #45: _PyEval_EvalFrameDefault + 0x10cc (0x55b20485951c in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #46: <unknown function> + 0x19c764 (0x55b204838764 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #47: _PyCFunction_FastCallDict + 0x91 (0x55b2047adb91 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #48: <unknown function> + 0x199abc (0x55b204835abc in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #49: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #50: <unknown function> + 0x193136 (0x55b20482f136 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #51: <unknown function> + 0x193ed6 (0x55b20482fed6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #52: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #53: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #54: <unknown function> + 0x19c764 (0x55b204838764 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #55: _PyCFunction_FastCallDict + 0x91 (0x55b2047adb91 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #56: <unknown function> + 0x199abc (0x55b204835abc in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #57: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #58: <unknown function> + 0x193136 (0x55b20482f136 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #59: <unknown function> + 0x193ed6 (0x55b20482fed6 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #60: <unknown function> + 0x199b95 (0x55b204835b95 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #61: _PyEval_EvalFrameDefault + 0x30a (0x55b20485875a in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #62: <unknown function> + 0x19c764 (0x55b204838764 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)
frame #63: _PyCFunction_FastCallDict + 0x91 (0x55b2047adb91 in /home/ubuntu/anaconda3/envs/Crowd_Detection_mayub/bin/python)

```
Thanks !

cc @suo