Repository: Qiskit/qiskit

Issue:
Matplotlib circuit drawer returns NoneType rather than PIL.Image
<!-- ⚠️ If you do not respect this template, your issue will be closed -->
<!-- ⚠️ Make sure to browse the opened and closed issues -->

### Informations

- **Qiskit Terra version**: master
- **Python version**:
- **Operating system**:

### What is the current behavior?
The matplotlib circuit drawer returns a NoneType rather than the `PIL.Image` as stated in the docstring for `circuit_drawer` and `matplotlib_circuit_drawer`.  Thus, you can do nothing with the output.  For example, it would be nice to grab the output and use it in a matplotlib `subplots` or the like and group circuits in a single figure.


### Steps to reproduce the problem



### What is the expected behavior?



### Suggested solutions





Discussion:
Hmm, when I tested it for writing the doc it was definitely an PIL Image object, it also works with the `interactive` kwarg which calls show() on that object to open it in a new window. So something odd must be going on, let me check the code
Well, that explains it: https://github.com/Qiskit/qiskit-terra/blob/master/qiskit/tools/visualization/_matplotlib.py#L361-L375

If you're running in ipython it returns None, otherwise it renders the image to a file and uses that to create a PIL Image object.

Sigh, every time I think we've got the behavior well defined for the circuit drawer there is another edge case hidden in the code...
Why are we not returning a matplotlib object instead of saving the figure and then reopening it as an image?
Honestly, I don't know, when I started working on the project everything was already a PIL image object. I assume it's done that way for consistency with the latex drawer which obviously can't return a matplotlib object.
Ok.  I would have thought that having a matplotlib fig or ax instance returned would be one of the selling points of that method.  I could do all sorts of things with it (annotate, combine, etc...).
Oh I definitely agree. We can change it (it's not like we're returning a consistent type with the text drawer anyway) to return the fig. If we do that I'm not exactly sure the best way to signal that, we've not had the `output` kwarg in a release yet, and the only way to get a matplotlib circuit before was if the latex one failed. Maybe we don't need a deprecation warning and just clearly document the output type in the docstring. Thoughts?
Maybe it would make sense to have a consistent type with all the drawers. Compose them is something that can be supported in all of them. Same with "dump to a file". If you call a method that is not supported for that particular drawer, an exception makes sense...
Well I think the hunt for consistency is probably part of the problem here. We've got 3 different potential outputs (well 4 if you count latex_source) all of which behave differently and have different advantages and disadvantages. I'm not sure that wrapping 3 (4 if you count None as per this bug) different types (PIL Image, mpl figure, and str) in a class will make it easier to use. What does that provide us except for a consistent type? I feel like that will just make it more confusing as to what you're actually getting
When @nkanazawa1989 and I originally implemented matplotlib_circuit_drawer, it returns nothing because we took into account the usecase of Jupyter notebook. @delapuente asked me to return PIL Image object to be consistent with latex_circuit_drawer. See the following PR for the details https://github.com/Qiskit/qiskit-terra/pull/579#issuecomment-401537471
I see that text_circuit_drawer does not return PIL image object. So, we can change the return type of matplotlib_circuit_drawer as well, right? What do you prefer?
From the API perspective, it seems to me that we have two options:
 - Return `None`: In `circuit_drawer` the full logic of the display is encapsulated, including saving in a file and so on. It turns into a "print this circuit whatever you mean for that".
 - Return a subclass of our own abstract class: This ABC, say `Drawing`, can include abstract methods for common functionality (eg. compose to compare them, like in https://github.com/Qiskit/qiskit-terra/pull/1181#issuecomment-435142611, dump into a file (and we can remove the kvarg `filename`), and the proper `_repr_*_` for Jupyter/IPython. It can also include the default configurations and the utility methods (eg: https://github.com/Qiskit/qiskit-terra/pull/1187). If a drawer does not implement that method (for example, rescale for text drawer) well... you raise exception saying "I cannot rescale, but if you want to change the size maybe try line_length".

I'm more inclined for the second option :)
This is blocking #1181. Remove the `on hold` tag when this issue gets resolved.
After talking with @mtreinish, we decided that each output will return its own type. We might change this in the future if there are use cases that justify a common type.

Resolving patch:
diff --git a/qiskit/tools/visualization/_circuit_visualization.py b/qiskit/tools/visualization/_circuit_visualization.py
--- a/qiskit/tools/visualization/_circuit_visualization.py
+++ b/qiskit/tools/visualization/_circuit_visualization.py
@@ -98,8 +98,10 @@ def circuit_drawer(circuit,
             circuit. Defaults to True.
 
     Returns:
-        PIL.Image: (outputs `latex` and `python`) an in-memory representation of
-                   the circuit diagram.
+        PIL.Image: (output `latex`) an in-memory representation of the image
+            of the circuit diagram.
+        matplotlib.figure: (output `mpl`) a matplotlib figure object for the
+            circuit diagram.
         String: (output `latex_source`). The LaTeX source code.
         TextDrawing: (output `text`). A drawing that can be printed as ascii art
     Raises:
@@ -585,7 +587,7 @@ def _matplotlib_circuit_drawer(circuit,
 
 
     Returns:
-        PIL.Image: an in-memory representation of the circuit diagram
+        matplotlib.figure: a matplotlib figure object for the circuit diagram
     """
     if ',' not in basis:
         logger.warning('Warning: basis is not comma separated: "%s". '
diff --git a/qiskit/tools/visualization/_matplotlib.py b/qiskit/tools/visualization/_matplotlib.py
--- a/qiskit/tools/visualization/_matplotlib.py
+++ b/qiskit/tools/visualization/_matplotlib.py
@@ -15,12 +15,8 @@
 import json
 import logging
 import math
-import os
-import tempfile
 
 import numpy as np
-import PIL
-from matplotlib import get_backend as get_matplotlib_backend
 from matplotlib import patches
 from matplotlib import pyplot as plt
 
@@ -28,7 +24,6 @@
 from qiskit import transpiler
 from qiskit.tools.visualization import _error
 from qiskit.tools.visualization import _qcstyle
-from qiskit.tools.visualization import _utils
 
 
 logger = logging.getLogger(__name__)
@@ -362,29 +357,10 @@ def draw(self, filename=None, verbose=False):
         if self._style.figwidth < 0.0:
             self._style.figwidth = fig_w * self._scale * self._style.fs / 72 / WID
         self.figure.set_size_inches(self._style.figwidth, self._style.figwidth * fig_h / fig_w)
-
-        if get_matplotlib_backend() == 'module://ipykernel.pylab.backend_inline':
-            # returns None when matplotlib is inline mode to prevent Jupyter
-            # with matplotlib inlining enabled to draw the diagram twice.
-            im = None
-        else:
-            # when matplotlib is not inline mode,
-            # self.figure.savefig is called twice because...
-            # ... this is needed to get the in-memory representation
-            with tempfile.TemporaryDirectory() as tmpdir:
-                tmpfile = os.path.join(tmpdir, 'circuit.png')
-                self.figure.savefig(tmpfile, dpi=self._style.dpi,
-                                    bbox_inches='tight')
-                im = PIL.Image.open(tmpfile)
-                _utils._trim(im)
-                os.remove(tmpfile)
-
-        # ... and this is needed to delegate in matplotlib the generation of
-        # the proper format.
         if filename:
             self.figure.savefig(filename, dpi=self._style.dpi,
                                 bbox_inches='tight')
-        return im
+        return self.figure
 
     def _draw_regs(self):
         # quantum register


Test patch: