Repository: Qiskit/qiskit

Issue:
transpile() hung up when running with multiple circuits and custom pass manager
<!-- ⚠️ If you do not respect this template, your issue will be closed -->
<!-- ⚠️ Make sure to browse the opened and closed issues -->

### What is the current behavior?
On Mac OS machine, `transpile()` hung up in `parallel_map` when it runs with multiple circuits and custom pass manager. 

### Steps to reproduce the problem
```
from qiskit import QuantumCircuit, Aer, transpile
from qiskit.transpiler.coupling import CouplingMap
from qiskit.transpiler.pass_manager_config import PassManagerConfig
from qiskit.transpiler.preset_passmanagers import level_0_pass_manager

qc = QuantumCircuit(2, 2)
qc.h(0)
qc.cx(0, 1)

backend = Aer.get_backend('qasm_simulator')

pm_conf = PassManagerConfig(
    initial_layout=None,
    basis_gates=['u1', 'u2', 'u3', 'cx'],
    coupling_map=CouplingMap([[0, 1]]),
    backend_properties=None,
    seed_transpiler=1
)
pm = level_0_pass_manager(pm_conf)

transpiled = transpile([qc, qc],
                       backend=backend,
                       pass_manager=pm)
```
Note: No problem in the single circuit case `transpile(qc, backend=backend, pass_manager=pm)` or when specifying the same preset pass manager via optimization_level `transpile([qc, qc], backend=backend, optimizaton_level=0)`.








Discussion:
Are you importing matplotlib anywhere when this happens?  That is a known mpl issue. 
I got the following warnings prior to the hang when I tried to replicate this from master with python 3.5 on osx 10.14.
```
Traceback (most recent call last):
  File "/anaconda3/lib/python3.5/multiprocessing/queues.py", line 234, in _feed
    obj = ForkingPickler.dumps(obj)
  File "/anaconda3/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'level_0_pass_manager.<locals>._choose_layout_condition'
Traceback (most recent call last):
  File "/anaconda3/lib/python3.5/multiprocessing/queues.py", line 234, in _feed
    obj = ForkingPickler.dumps(obj)
  File "/anaconda3/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'level_0_pass_manager.<locals>._choose_layout_condition'
```

This looks like it could be related to: https://github.com/Qiskit/qiskit-terra/issues/3808
I think this is a bit different.  It seems that `choose_layout_condition` can simply not be pickled.  The other is the mpl issue that is to be resolved in v3.3
callables and parallelization not always play well together. Let me have a look... 
The problem with the pickled callable from @kdk in https://github.com/Qiskit/qiskit-terra/issues/3925#issuecomment-594607628 is because Python 3.5 (do we still support it?) In Python 3.7 the hanging up from @itoko is reproducible.  
hold on.. sometimes in python 3.7 I'm also getting the unpickleble issue...

Resolving patch:
diff --git a/qiskit/compiler/transpile.py b/qiskit/compiler/transpile.py
--- a/qiskit/compiler/transpile.py
+++ b/qiskit/compiler/transpile.py
@@ -159,6 +159,9 @@ def callback_func(**kwargs):
             (isinstance(circuits, list) and all(isinstance(c, Schedule) for c in circuits)):
         return circuits
 
+    if pass_manager:
+        return pass_manager.run(circuits, output_name=output_name, callback=callback)
+
     if optimization_level is None:
         config = user_config.get_config()
         optimization_level = config.get('transpile_optimization_level', None)
@@ -188,6 +191,7 @@ def callback_func(**kwargs):
                                   'in {} '.format(circuit.name) +
                                   'is greater than maximum ({}) '.format(max_qubits) +
                                   'in the coupling_map')
+
     # Transpile circuits in parallel
     circuits = parallel_map(_transpile_circuit, list(zip(circuits, transpile_args)))
 


Test patch:
diff --git a/test/python/compiler/test_transpiler.py b/test/python/compiler/test_transpiler.py
--- a/test/python/compiler/test_transpiler.py
+++ b/test/python/compiler/test_transpiler.py
@@ -33,6 +33,8 @@
 from qiskit.transpiler.exceptions import TranspilerError
 from qiskit.transpiler.passes import BarrierBeforeFinalMeasurements, CXDirection
 from qiskit.quantum_info import Operator
+from qiskit.transpiler.pass_manager_config import PassManagerConfig
+from qiskit.transpiler.preset_passmanagers import level_0_pass_manager
 
 
 @ddt
@@ -645,9 +647,9 @@ def test_ms_unrolls_to_cx(self, optimization_level):
         """Verify a Rx,Ry,Rxx circuit transpile to a U3,CX target."""
 
         qc = QuantumCircuit(2)
-        qc.rx(math.pi/2, 0)
-        qc.ry(math.pi/4, 1)
-        qc.rxx(math.pi/4, 0, 1)
+        qc.rx(math.pi / 2, 0)
+        qc.ry(math.pi / 4, 1)
+        qc.rxx(math.pi / 4, 0, 1)
 
         out = transpile(qc, basis_gates=['u3', 'cx'], optimization_level=optimization_level)
 
@@ -658,9 +660,9 @@ def test_ms_can_target_ms(self, optimization_level):
         """Verify a Rx,Ry,Rxx circuit can transpile to an Rx,Ry,Rxx target."""
 
         qc = QuantumCircuit(2)
-        qc.rx(math.pi/2, 0)
-        qc.ry(math.pi/4, 1)
-        qc.rxx(math.pi/4, 0, 1)
+        qc.rx(math.pi / 2, 0)
+        qc.ry(math.pi / 4, 1)
+        qc.rxx(math.pi / 4, 0, 1)
 
         out = transpile(qc, basis_gates=['rx', 'ry', 'rxx'], optimization_level=optimization_level)
 
@@ -673,7 +675,7 @@ def test_cx_can_target_ms(self, optimization_level):
         qc = QuantumCircuit(2)
         qc.h(0)
         qc.cx(0, 1)
-        qc.rz(math.pi/4, [0, 1])
+        qc.rz(math.pi / 4, [0, 1])
 
         out = transpile(qc, basis_gates=['rx', 'ry', 'rxx'], optimization_level=optimization_level)
 
@@ -684,11 +686,43 @@ def test_measure_doesnt_unroll_ms(self, optimization_level):
         """Verify a measure doesn't cause an Rx,Ry,Rxx circuit to unroll to U3,CX."""
 
         qc = QuantumCircuit(2, 2)
-        qc.rx(math.pi/2, 0)
-        qc.ry(math.pi/4, 1)
-        qc.rxx(math.pi/4, 0, 1)
+        qc.rx(math.pi / 2, 0)
+        qc.ry(math.pi / 4, 1)
+        qc.rxx(math.pi / 4, 0, 1)
         qc.measure([0, 1], [0, 1])
 
         out = transpile(qc, basis_gates=['rx', 'ry', 'rxx'], optimization_level=optimization_level)
 
         self.assertEqual(qc, out)
+
+
+class TestTranspileCustomPM(QiskitTestCase):
+    """Test transpile function with custom pass manager"""
+
+    def test_custom_multiple_circuits(self):
+        """Test transpiling with custom pass manager and multiple circuits.
+        This tests created a deadlock, so it needs to be monitored for timeout.
+        See: https://github.com/Qiskit/qiskit-terra/issues/3925
+        """
+        qc = QuantumCircuit(2)
+        qc.h(0)
+        qc.cx(0, 1)
+
+        pm_conf = PassManagerConfig(
+            initial_layout=None,
+            basis_gates=['u1', 'u2', 'u3', 'cx'],
+            coupling_map=CouplingMap([[0, 1]]),
+            backend_properties=None,
+            seed_transpiler=1
+        )
+        passmanager = level_0_pass_manager(pm_conf)
+
+        transpiled = transpile([qc, qc], pass_manager=passmanager)
+
+        expected = QuantumCircuit(QuantumRegister(2, 'q'))
+        expected.u2(0, 3.141592653589793, 0)
+        expected.cx(0, 1)
+
+        self.assertEqual(len(transpiled), 2)
+        self.assertEqual(transpiled[0], expected)
+        self.assertEqual(transpiled[1], expected)