Error message Ignite Engine Thread Pickle

quornmd: <p>Hi I am using pytorch ignite wrapped in the MONAI package. My code worked until 3h ago when suddenly I started receiving this code. I reverted all my changes back and the error still persists. Does anyone know more?</p>
<pre><code class="lang-auto">ERROR:ignite.engine.engine.SupervisedTrainer:Exception: cannot pickle '_thread.lock' object
Traceback (most recent call last):
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/site-packages/ignite/engine/engine.py", line 728, in _internal_run
    self._setup_engine()
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/site-packages/ignite/engine/engine.py", line 710, in _setup_engine
    self._dataloader_iter = iter(self.state.dataloader)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 352, in __iter__
    return self._get_iterator()
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 294, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 801, in __init__
    w.start()
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/cornelius/opt/anaconda3/envs/LungSegChallenge/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
</code></pre>

vfdev-5: <p><a class="mention" href="/u/quornmd">@quornmd</a> could you please provide versions of MONAI and Ignite ?</p>

quornmd: <p>thank you for the quick response.</p>
<p>I just solved the problem by deleting my Conda environment and reinstalling it.<br>
In both, old and new, environments I was using: pytorch-ignite 0.4.2 and MONAI 0.3.0+95.g535561e.<br>
I did install simpleITK and ITK packages about an hour before pytorch started breaking, but I was able to start my script in-between without it crashing.</p>

vfdev-5: <p>Sounds good. Please feel free to ask here if you experience this issue again and if we could reproduce it <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

vfdev-5: <p>Add a link for visibility from MONAI side : <a href="https://github.com/Project-MONAI/MONAI/pull/1286" rel="noopener nofollow ugc">https://github.com/Project-MONAI/MONAI/pull/1286</a></p>