TypeError: can't pickle module objects

swg209: <p>I run pytorch tutorial ‘s  Training a Classifier demo, but got this error:</p>
<p>[TypeError                                 Traceback (most recent call last)<br>
 in <br>
12<br>
13<br>
—&gt; 14 dataiter = iter(trainloader)<strong>strong text</strong><br>
15 images, labels = dataiter.next()<br>
16</p>
<p>D:\DL_software\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py in <strong>iter</strong>(self)<br>
817<br>
818     def <strong>iter</strong>(self):<br>
–&gt; 819         return _DataLoaderIter(self)<br>
820<br>
821     def <strong>len</strong>(self):</p>
<p>D:\DL_software\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py in <strong>init</strong>(self, loader)<br>
558                 #     before it starts, and <strong>del</strong> tries to join but will get:<br>
559                 #     AssertionError: can only join a started process.<br>
–&gt; 560                 w.start()<br>
561                 self.index_queues.append(index_queue)<br>
562                 self.workers.append(w)</p>
<p>D:\DL_software\envs\pytorch\lib\multiprocessing\process.py in start(self)<br>
103                ‘daemonic processes are not allowed to have children’<br>
104         _cleanup()<br>
–&gt; 105         self._popen = self._Popen(self)<br>
106         self._sentinel = self._popen.sentinel<br>
107         # Avoid a refcycle if the target function holds an indirect</p>
<p>D:\DL_software\envs\pytorch\lib\multiprocessing\context.py in _Popen(process_obj)<br>
221     <span class="mention">@staticmethod</span><br>
222     def _Popen(process_obj):<br>
–&gt; 223         return _default_context.get_context().Process._Popen(process_obj)<br>
224<br>
225 class DefaultContext(BaseContext):</p>
<p>D:\DL_software\envs\pytorch\lib\multiprocessing\context.py in _Popen(process_obj)<br>
320         def _Popen(process_obj):<br>
321             from .popen_spawn_win32 import Popen<br>
–&gt; 322             return Popen(process_obj)<br>
323<br>
324     class SpawnContext(BaseContext):</p>
<p>D:\DL_software\envs\pytorch\lib\multiprocessing\popen_spawn_win32.py in <strong>init</strong>(self, process_obj)<br>
63             try:<br>
64                 reduction.dump(prep_data, to_child)<br>
—&gt; 65                 reduction.dump(process_obj, to_child)<br>
66             finally:<br>
67                 set_spawning_popen(None)</p>
<p>D:\DL_software\envs\pytorch\lib\multiprocessing\reduction.py in dump(obj, file, protocol)<br>
58 def dump(obj, file, protocol=None):<br>
59     ‘’‘Replacement for pickle.dump() using ForkingPickler.’’’<br>
—&gt; 60     ForkingPickler(file, protocol).dump(obj)<br>
61<br>
62 #</p>
<p>TypeError: can’t pickle module objects](http://)</p>

thompsjj: <p>Having the same problem. Any replies?</p>

songruiecho: <p>I have the same problem when I try to introduce another module into Dataset Object. When I delete self.ds，it runs. But I have no idea about keeping self.ds and correct the code.</p>
<pre><code class="lang-auto">class Dataset(torch.utils.data.Dataset):
    def __init__(self, config, ds, index):
        self.input_res = config['train']['input_res']   
        self.output_res = config['train']['output_res']   
        self.generateHeatmap = GenerateHeatmap(self.output_res, config['inference']['num_parts'])
        self.ds = ds    # the error source
        self.index = index
</code></pre>

songruiecho: <p>Maybe some parameters/variable in you code are module, you can rewrite it to a class.</p>

Brando_Miranda: <p>hmmm I can’t seem to find wherer the error is for me but in case this is useful for others:</p>
<pre><code class="lang-auto">def print_dataloader_attributes(dataloader):
    """
    https://stackoverflow.com/questions/1547466/check-if-a-parameter-is-a-python-module
    
    :param dataloader:
    :return:
    """
    from types import ModuleType
    import inspect

    for attribute in dir(dataloader):
        attribute_value = getattr(dataloader, attribute)
        print(f'{attribute=}, {type(attribute_value)=}\n')
        if isinstance(attribute_value, ModuleType) or inspect.ismodule(attribute_value) or type(attribute_value) is type(inspect):
            print(attribute_value)
    print('')
</code></pre>

samisnotinsane: <p>For me, this error was fixed when I restarted my Jupyter Notebook and re-ran the code.</p>