TorchServe: MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding

venkat.r: <p>While serving the model on Docker, i get the following error when it tried to load the saved model.</p>
<p>Strangely it cant also not find torch although it is install.</p>
<pre><code class="lang-auto">2021-11-25 20:18:12,416 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2021-11-25 20:18:12,438 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: ponzi.mar
2021-11-25 20:18:12,455 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 0.1 for model ponzi
2021-11-25 20:18:12,456 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 0.1 for model ponzi
2021-11-25 20:18:12,456 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model ponzi loaded.
2021-11-25 20:18:12,456 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: ponzi, count: 12
2021-11-25 20:18:12,514 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2021-11-25 20:18:12,683 [WARN ] W-9002-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,684 [INFO ] W-9005-ponzi_0.1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9005-ponzi_0.1-stdout
2021-11-25 20:18:12,683 [WARN ] W-9009-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,687 [INFO ] W-9003-ponzi_0.1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9003-ponzi_0.1-stdout
2021-11-25 20:18:12,683 [WARN ] W-9010-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,683 [WARN ] W-9003-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,689 [WARN ] W-9010-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,689 [WARN ] W-9005-ponzi_0.1 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9005-ponzi_0.1-stderr
2021-11-25 20:18:12,683 [WARN ] W-9007-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,689 [WARN ] W-9005-ponzi_0.1 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9005-ponzi_0.1-stdout
2021-11-25 20:18:12,690 [WARN ] W-9007-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,690 [WARN ] W-9007-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,683 [WARN ] W-9004-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,683 [WARN ] W-9001-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,690 [WARN ] W-9007-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,689 [WARN ] W-9003-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,689 [WARN ] W-9002-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,688 [WARN ] W-9005-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,690 [WARN ] W-9003-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,690 [WARN ] W-9004-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,690 [WARN ] W-9003-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,684 [INFO ] W-9006-ponzi_0.1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9006-ponzi_0.1-stdout
2021-11-25 20:18:12,683 [WARN ] W-9000-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,683 [WARN ] W-9006-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,690 [WARN ] W-9002-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,690 [WARN ] W-9004-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,690 [INFO ] W-9005-ponzi_0.1-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9005-ponzi_0.1-stderr
2021-11-25 20:18:12,690 [WARN ] W-9001-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,691 [WARN ] W-9004-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,691 [WARN ] W-9001-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,691 [WARN ] W-9001-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,689 [WARN ] W-9009-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,692 [WARN ] W-9009-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,692 [WARN ] W-9009-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,689 [WARN ] W-9010-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,683 [WARN ] W-9008-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,684 [WARN ] W-9011-ponzi_0.1-stderr MODEL_LOG - Fatal Python error: Py_Initialize: Unable to get the locale encoding
2021-11-25 20:18:12,692 [WARN ] W-9010-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,691 [WARN ] W-9002-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
2021-11-25 20:18:12,691 [WARN ] W-9003-ponzi_0.1 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9003-ponzi_0.1-stderr
2021-11-25 20:18:12,691 [WARN ] W-9006-ponzi_0.1-stderr MODEL_LOG - Traceback (most recent call last):
2021-11-25 20:18:12,692 [WARN ] W-9006-ponzi_0.1-stderr MODEL_LOG -   File "/home/model-server/tmp/models/d272938c278d47eb9c045c2a929e667b/encodings.py", line 1, in &lt;module&gt;
2021-11-25 20:18:12,693 [WARN ] W-9006-ponzi_0.1-stderr MODEL_LOG - ModuleNotFoundError: No module named 'torch'
</code></pre>
<p>The Docker file setup is like this.</p>
<pre><code class="lang-auto">FROM pytorch/torchserve:latest

USER root
RUN printf "\nservice_envelope=json" &gt;&gt; /home/model-server/config.properties \
    &amp;&amp; pip install pytorch-lightning torchmetrics google-cloud-storage scikit-learn
USER model-server


COPY src /home/model-server/src
COPY load_model.py /home/model-server/src
COPY configs /home/model-server/configs
COPY /models/state_dict.pt /home/model-server/src/
RUN mkdir "model_store"

RUN torch-model-archiver \
    --model-name ponzi \
    --version 0.1 \
    --model-file /home/model-server/src/model.py \
    --serialized-file /home/model-server/src/state_dict.pt \
    --handler /home/model-server/src/handler.py \
    --export-path /home/model-server/model_store \
    --extra-files /home/model-server/configs/features.yml,/home/model-server/configs/model_configs.yml,/home/model-server/src/utils.py,/home/model-server/src/encodings.py,/home/model-server/src/dataset.py,/home/model-server/src/__init__.py

CMD ["torchserve", \
     "--start", \
     "--ncs", \
     "--model-store model_store", \
     "--ts-config=/home/model-server/config.properties", \
     "--models ponzi=ponzi.mar"]

</code></pre>
<p>Can someone help here? Thanks.</p>

ptrblck: <p>Based on these error messages:</p>
<pre><code class="lang-auto">Fatal Python error: Py_Initialize: Unable to get the locale encoding
[...]
ModuleNotFoundError: No module named 'torch'
</code></pre>
<p>I would guess that your current setup might be mixing different Python versions (or environments) and thus breaks in <code>Py_Initialize</code> and cannot find <code>torch</code>.<br>
Look through the build logs of your container and check if some pre-installed packages are uninstalled etc.</p>

venkat.r: <p>My local python is 3.8.10 vs Docker which is using 3.6. That could be the problem. What are the possible work around here.</p>
<p>Do i have to use a multi-stage docker build which pulls python3.8 and then does the torch serve bit ?</p>

ptrblck: <p>I don’t have any custom docker containers for <code>torchserve</code>, but you could use the <a href="https://hub.docker.com/r/pytorch/torchserve/tags">official ones</a>.</p>