Backend worker monitoring thread interrupted or backend worker process died

LiThiLi: <p>I’m testing torchserve using resnet-18 tutorial in this link: <a href="https://github.com/pytorch/serve/tree/master/examples/image_classifier/resnet_18" rel="noopener nofollow ugc">https://github.com/pytorch/serve/tree/master/examples/image_classifier/resnet_18</a></p>
<p>The tutorial works perfectly fine, but when I moved resnet.py from torchvision, index_to_name.json, resnet18-5c106cde.pth, model.py into the same folder, and I changed model.py into:</p>
<pre><code class="lang-auto">from resnet import ResNet, BasicBlock

class ImageClassifier(ResNet):
    def __init__(self):
        super(ImageClassifier, self).__init__(BasicBlock, [2, 2, 2, 2])
</code></pre>
<p>then I use the following command to archive the model and start torchserve:</p>
<blockquote>
<p>torch-model-archiver --model-name resnet-18 --version 1.0 --model-file model.py --serialized-file resnet18-5c106cde.pth --handler image_classifier --extra-files index_to_name.json</p>
</blockquote>
<blockquote>
<p>mkdir model_store</p>
</blockquote>
<blockquote>
<p>mv resnet-18.mar model_store/</p>
</blockquote>
<blockquote>
<p>torchserve --start --model-store model_store --models resnet-18=resnet-18.mar</p>
</blockquote>
<p>here is the log after torchserve is executed:</p>
<p><strong>&gt; 2020-09-07 15:49:36,082 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /tmp/.ts.sock.9000</strong><br>
<strong>&gt; 2020-09-07 15:49:36,082 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]18279</strong><br>
<strong>&gt; 2020-09-07 15:49:36,082 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.</strong><br>
<strong>&gt; 2020-09-07 15:49:36,082 [DEBUG] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-resnet-18_1.0 State change WORKER_STOPPED → WORKER_STARTED</strong><br>
<strong>&gt; 2020-09-07 15:49:36,082 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.9</strong><br>
<strong>&gt; 2020-09-07 15:49:36,082 [INFO ] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000</strong><br>
<strong>&gt; 2020-09-07 15:49:36,083 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /tmp/.ts.sock.9000.</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Backend worker process died.</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Traceback (most recent call last):</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/model_service_worker.py”, line 176, in </strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     worker.run_server()</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/model_service_worker.py”, line 148, in run_server</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.handle_connection(cl_socket)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/model_service_worker.py”, line 112, in handle_connection</strong><br>
<strong>&gt; 2020-09-07 15:49:36,546 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service, result, code = self.load_model(msg)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/model_service_worker.py”, line 85, in load_model</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service = model_loader.load(model_name, model_dir, handler, gpu, batch_size)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/model_loader.py”, line 117, in load</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     model_service.initialize(service.context)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/torch_handler/base_handler.py”, line 50, in initialize</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.model = self._load_pickled_model(model_dir, model_file, model_pt_path)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/home/hoangminhq/.local/lib/python3.6/site-packages/ts/torch_handler/base_handler.py”, line 74, in _load_pickled_model</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     module = importlib.import_module(model_file.split(“.”)[0])</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/usr/lib/python3.6/importlib/<strong>init</strong>.py”, line 126, in import_module</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     return _bootstrap._gcd_import(name[level:], package, level)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 994, in _gcd_import</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 971, in _find_and_load</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 955, in _find_and_load_unlocked</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 665, in _load_unlocked</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 678, in exec_module</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “”, line 219, in _call_with_frames_removed</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File “/tmp/models/492344f70f0f41beb13b7ed4335b3a18/model.py”, line 1, in </strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     from resnet import ResNet, BasicBlock</strong><br>
<strong>&gt; 2020-09-07 15:49:36,547 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - ModuleNotFoundError: No module named ‘resnet’</strong><br>
<strong>&gt; 2020-09-07 15:49:36,548 [INFO ] epollEventLoopGroup-5-2 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED</strong><br>
<strong>&gt; 2020-09-07 15:49:36,551 [DEBUG] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED</strong><br>
<strong>&gt; 2020-09-07 15:49:36,552 [DEBUG] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.</strong><br>
<strong>&gt; java.lang.InterruptedException</strong><br>
<strong>&gt; 	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056)</strong><br>
<strong>&gt; 	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133)</strong><br>
<strong>&gt; 	at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432)</strong><br>
<strong>&gt; 	at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:129)</strong><br>
<strong>&gt; 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)</strong><br>
<strong>&gt; 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)</strong><br>
<strong>&gt; 	at java.base/java.lang.Thread.run(Thread.java:834)</strong><br>
<strong>&gt; 2020-09-07 15:49:36,552 [WARN ] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: resnet-18, error: Worker died.</strong><br>
<strong>&gt; 2020-09-07 15:49:36,552 [DEBUG] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-resnet-18_1.0 State change WORKER_STARTED → WORKER_STOPPED</strong><br>
<strong>&gt; 2020-09-07 15:49:36,552 [WARN ] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-resnet-18_1.0-stderr</strong><br>
<strong>&gt; 2020-09-07 15:49:36,552 [WARN ] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-resnet-18_1.0-stdout</strong><br>
<strong>&gt; 2020-09-07 15:49:36,553 [INFO ] W-9000-resnet-18_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 55 seconds.</strong><br>
<strong>&gt; 2020-09-07 15:49:36,565 [INFO ] W-9000-resnet-18_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-resnet-18_1.0-stderr</strong><br>
<strong>&gt; 2020-09-07 15:49:36,565 [INFO ] W-9000-resnet-18_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-resnet-18_1.0-stdout</strong><br>
<strong>&gt; 2020-09-07 15:49:38,081 [INFO ] main org.pytorch.serve.ModelServer - Torchserve stopped.</strong></p>
<p>I’m using:</p>
<ul>
<li>torchserve version: 0.2.0</li>
<li>torch version: 1.6.0</li>
<li>java version: 11.0.8</li>
<li>Operating System and version: Ubuntu 18.04.5</li>
</ul>
<p>it seems like the model cannot be loaded, what did I do wrong and how can I fix this?</p>
<p>Thanks.</p>