Repository: spikeinterface/spikeinterface
Issue: PCA computation : pickle run out of inputs

Hello,
I used to get some 'shared memory issues" (#1704 ) to extract my waveforms. Since the PR #1704  has been merged, I updated my Spikeinterface. Now the waveform extraction goes well, but I have big issue with the PCAs computation. I compute the PCA with default parameters: `pca = si.compute_principal_components(we, n_jobs=job_kwargs['n_jobs'])`

Python crashes only when I compute PCA from several sortings in parallel. The associated error is  the following:
```
Traceback (most recent call last):
  File "/home/users/j/juventin/yggdrasil_python_envs/py310/lib/python3.10/site-packages/joblib/externals/loky/process_executor.py", line 436, in _process_worker
    r = call_item()
  File "/home/users/j/juventin/yggdrasil_python_envs/py310/lib/python3.10/site-packages/joblib/externals/loky/process_executor.py", line 288, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/home/users/j/juventin/yggdrasil_python_envs/py310/lib/python3.10/site-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/home/users/j/juventin/yggdrasil_python_envs/py310/lib/python3.10/site-packages/joblib/parallel.py", line 263, in __call__
    return [func(*args, **kwargs)
  File "/home/users/j/juventin/yggdrasil_python_envs/py310/lib/python3.10/site-packages/joblib/parallel.py", line 263, in <listcomp>
    return [func(*args, **kwargs)
  File "/home/users/j/juventin/spike_sorters/spikeinterface/src/spikeinterface/postprocessing/principal_component.py", line 727, in partial_fit_one_channel
    pca_model = pickle.load(fid)
EOFError: Ran out of input
```
When I compute PCA for one single sorting, it runs smoothly.

I noticed that a "tmp" folder was created during PCA computation. Since the name of the "tmp" folder is not unique,  I suspect that when PCAs are computed from multiple sortings at the same time, the "tmp" folder is overwritten, leading to the error. . Does this sound possible ?

Cheers 
Maxime

Changed files:
src/spikeinterface/postprocessing/principal_component.py

diff --git a/src/spikeinterface/postprocessing/principal_component.py b/src/spikeinterface/postprocessing/principal_component.py
index f6bdf5c5..a94f275f 100644
--- a/src/spikeinterface/postprocessing/principal_component.py
+++ b/src/spikeinterface/postprocessing/principal_component.py
@@ -43,7 +43,9 @@ class WaveformPrincipalComponent(BaseWaveformExtractorExtension):
                 txt += " - sparse"
         return txt
 
-    def _set_params(self, n_components=5, mode="by_channel_local", whiten=True, dtype="float32", sparsity=None):
+    def _set_params(
+        self, n_components=5, mode="by_channel_local", whiten=True, dtype="float32", sparsity=None, tmp_folder=None
+    ):
         assert mode in _possible_modes, "Invalid mode!"
 
         if self.waveform_extractor.is_sparse():
@@ -56,6 +58,7 @@ class WaveformPrincipalComponent(BaseWaveformExtractorExtension):
             whiten=bool(whiten),
             dtype=np.dtype(dtype).str,
             sparsity=sparsity,
+            tmp_folder=tmp_folder,
         )
 
         return params
@@ -361,7 +364,12 @@ class WaveformPrincipalComponent(BaseWaveformExtractorExtension):
 
         mode = p["mode"]
         pca_model_files = []
-        tmp_folder = Path("tmp")
+
+        tmp_folder = p["tmp_folder"]
+        if tmp_folder is None:
+            tmp_folder = "tmp"
+        tmp_folder = Path(tmp_folder)
+
         for chan_ind, chan_id in enumerate(channel_ids):
             pca_model = pca_models[chan_ind]
             if n_jobs > 1:
@@ -662,6 +670,7 @@ def compute_principal_components(
     sparsity=None,
     whiten=True,
     dtype="float32",
+    tmp_folder=None,
     **job_kwargs,
 ):
     """
@@ -680,17 +689,22 @@ def compute_principal_components(
         - 'by_channel_local': a local PCA is fitted for each channel (projection by channel)
         - 'by_channel_global': a global PCA is fitted for all channels (projection by channel)
         - 'concatenated': channels are concatenated and a global PCA is fitted
+        default 'by_channel_local'
     sparsity: ChannelSparsity or None
         The sparsity to apply to waveforms.
-        If waveform_extractor is already sparse, the default sparsity will be used.
+        If waveform_extractor is already sparse, the default sparsity will be used - default None
     whiten: bool
-        If True, waveforms are pre-whitened
+        If True, waveforms are pre-whitened - default True
     dtype: dtype
-        Dtype of the pc scores (default float32)
+        Dtype of the pc scores - default float32
     n_jobs: int
         Number of jobs used to fit the PCA model (if mode is 'by_channel_local') - default 1
     progress_bar: bool
         If True, a progress bar is shown - default False
+    tmp_folder: str
+        The temporary folder to use for parallel computation. If you run several `compute_principal_components`
+        functions in parallel with mode 'by_channel_local', you need to specify a different `tmp_folder` for each call,
+        to avoid overwriting to the same folder - default None
 
     Returns
     -------
@@ -716,7 +730,9 @@ def compute_principal_components(
         pc = waveform_extractor.load_extension(WaveformPrincipalComponent.extension_name)
     else:
         pc = WaveformPrincipalComponent.create(waveform_extractor)
-        pc.set_params(n_components=n_components, mode=mode, whiten=whiten, dtype=dtype, sparsity=sparsity)
+        pc.set_params(
+            n_components=n_components, mode=mode, whiten=whiten, dtype=dtype, sparsity=sparsity, tmp_folder=tmp_folder
+        )
         pc.run(**job_kwargs)
 
     return pc