fix: :bug: changing context of multiprocessing while decoding for Windows
Author: elsheikh21
Pull request: yes

# What does this PR do?

- This PR aim is to fix the context for the multiprocessing used in `wav2vec_with_LM batch_decode()` to change from `fork` to `spawn` so it can run with non-Linux based systems as well.

Fixes # (issue)

- multiprocessing context when `batch_decode`

## Before submitting

- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? 
   This is the [link for the GitHub issue](https://github.com/huggingface/transformers/issues/16898)


## Who can review?
@patrickvonplaten 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Models:

- Wav2Vec2 with LM

Library:

- benchmarks: @patrickvonplaten

Documentation: @sgugger



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17070). All of your documentation changes will be reflected on that endpoint.

Interesting! @elsheikh21 - it seems like `pyctcdecode` is not a big fan of `"spawn"` :-/ . Could you maybe open an issue there: https://github.com/kensho-technologies/pyctcdecode ? 

Sorry I thought this would work, but apparently Kensho's `pyctcdecode` doesn't like it. Maybe let's just ask in their repo how to run the code on Windows :-) 

@patrickvonplaten 
Yes, It caught me by surprise as well. Of course, I can and I did open [an issue](https://github.com/kensho-technologies/pyctcdecode/issues/65), I will be waiting for their response

In the meanwhile, I thought of the following
- get context based on the user's OS, using `import sys; sys.platform`, refer to [documentation](https://docs.python.org/3/library/sys.html#sys.platform) -if needed.

```
import sys

# specify spawn for WindowsOS
contexts = { 
    "win32": "spawn", 
    "cygwin": "spawn"
} 
# if the platform is not windows then use 'fork'
_context = contexts.get(sys.platform, "fork")
pool = get_context(_context ).Pool(num_processes)
```
 or wdyt?

Yeah this would be fine by me as well I think! But did you check that pyctcdecode then works correctly? 
To me it really looks like a bug within pyctcdecode

Yes I opened the issue in pyctcdecode but did not start investigating what might be causing the error, yet I will do so and if ai found sth. I will update them

Hi all, I've merged PR in pyctcdecode (https://github.com/kensho-technologies/pyctcdecode/pull/68) to bypass this issue by either passing None instead of a pool or automatically detecting a pool made with a spawn context and running without multiprocessing. The issue is that the language model is saved as a class variable, which allows fork to work without reloading the model, but this doesn't get automatically populated for spawn, so the model is missing in the new processes. All of the things I've tried to actually use the pool with spawn require pickling or reloading the model which ends up hurting performance a lot. If anyone wants to try to figure out if there's a way to use spawn without also making everything really slow, please go ahead and discuss or make a PR in pyctcdecode

Thanks a lot for the fix @lopez86 ! @elsheikh21 I think we should then probably go for the same solution in Transformers no?

@patrickvonplaten 
sorry for not getting back to you earlier, I had some issues in my personal life.

Okay, that seems like a good idea, but just to clarify how do u plan for me to change that in `transformers` package as well?

Could you try to add such a function to it to see if it works? https://github.com/kensho-technologies/pyctcdecode/pull/68#discussion_r894696237

@patrickvonplaten, as I understand, https://github.com/kensho-technologies/pyctcdecode/pull/68 completely ignores spawn contexts. This means that at least for now (until https://github.com/kensho-technologies/pyctcdecode/issues/65 is closed), we should not even get in the trouble of creating a spawn pool in Windows. There's probably an overhead when creating a pool that won't be used.

I agree @falcaopetri,

Should we maybe just allow the user to pass `pool` ? and only when it's `None` we create it ourselves?

I guess so, @patrickvonplaten. Moreover, I think that users should be warned if only a `spawn `pool is available or if one was passed by the user, since `pyctcdecode` currently can't use such pool. I've also proposed adding a warning message within `pyctcdecode` (https://github.com/kensho-technologies/pyctcdecode/pull/78).

Assuming that users can pass an active `pool` (#17879), we might warn them if a `spawn` one is passed (we can either count on the proposed `pyctcdecode`'s warning or add one within `Wav2Vec2ProcessorWithLM`).
If `None`, we could warn users when only `spawn` is available and call `pyctcdecode` with `None` (saving the creation of a pool that would otherwise be ignored by current `pyctcdecode` implementation). If `fork` is available we would keep current behavior.

If in the future `pyctcdecode` supports both `fork` and `spawn`, we might just roll back https://github.com/huggingface/transformers/pull/15247.

Sorry to have dropped the ball here a bit - @falcaopetri do you feel like opening a PR to allow passing `pool` or should I do it? :-)

Thanks for pinging me, @patrickvonplaten, and sorry for the delay.
I've added some tests to my initial proposal and a `Tip` under `batch_decode`'s `pool` arg.
I'll submit the PR till tomorrow.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.