Repository: pytorch
Issue: ENORMOUS OVERHEAD from mp.get_context('spawn')
Labels: module: performance,module: multiprocessing,triaged,module: POWER

### 🐛 Describe the bug

The bug is that using the pool from:
```
import torch.multiprocessing as mp # use spawn context
mp = mp.get_context('spawn')
```
**incurs ENORMOUS latency (x10 what normal)**, which for example,
isn't present when simply using:
```
import torch.multiprocessing as mp # no spawn context
```

Here is an example script that demonstrates the issue:
```
import time
import random
import math
import os
#import multiprocessing as mp
import torch.multiprocessing as mp # mp wrapper for pytorch compatibility!
mp = mp.get_context('spawn') # use get_context() rather than mp.set_start_method()!

# Simulate exponentially distributed worker latency
def laggy_worker(i):
    laziness = random.random()*5
    laziness = math.exp(laziness)
    print(f"starting laggy worker, duration: {laziness}, pid: {os.getpid()}", flush=True)
    time.sleep(laziness)
    return laziness

# A helper class to circumvent exponentially distributed latency
class FastPool:
    def __init__(self, *args, drop_rate=0.2, **kwd_args):
        self.drop_rate = drop_rate
        self.pool_args = args
        self.pool_kwd_args = kwd_args
    def map(self, fun, iter_):
        iter_ = list(iter_)
        result = []
        pool_start = time.time()
        with mp.Pool(*self.pool_args, **self.pool_kwd_args) as p:
            print(f'took {time.time()-pool_start} seconds to start the pool!')
            for i, res in enumerate(p.imap_unordered(fun, iter_)):
                result.append(res)
                print(f"finished laggy worker, reported duration: {res}, actual_duration: {time.time()-pool_start}, id: {i}", flush=True)
                if i > (1-self.drop_rate)*len(iter_):
                    break_start = time.time()
                    break
        print(f'took {time.time()-break_start} seconds to close pool')
        return result

if __name__=='__main__':
    pool = FastPool(drop_rate=0.2)
    for i in range(5):
        start = time.time()
        result = pool.map(laggy_worker, range(42))
        print('num results: ', len(result))
        print('duration: ', time.time()-start)
```
====================================================================================

As long as you run this script with the line: ``` mp = mp.get_context('spawn') ``` uncommented you will see output like this:

====================================================================================

![image](https://user-images.githubusercontent.com/4739054/167304286-b589e829-1c73-4d07-b440-085c64411e0a.png)

====================================================================================
If however you comment out that line (using default pytorch mp module), OR if you use the builtin python module. You will notice that _reported duration matches actual duration almost exactly_:

====================================================================================

![image](https://user-images.githubusercontent.com/4739054/167304415-68fd5ae5-9178-4fba-8e45-b408504fbce8.png)

====================================================================================

P.S. original script was a toy example for a different purpose, but it is short and sweet + demonstrates the issue well.

### Versions

pytorch version: 1.10.2
Python version: 3.8.10
Architecture: IBM Power9

cc @VitalyFedyunin @ngimel