Repository: qutip/qutip

Issue:
Removed converting error on windows.
Removed error when converting qcirc.pdf to qcirc.png on windows.
Cannot find qcirc.svg
In QuTiP 4.5.0 conda release on Mac, I get an error if I call a quantum circuit to print in jupyter notebook. However, the circuit is printed after the message. 

This did not happen with QuTiP 4.4.1. Imagemagick is installed. This issue may be related to the closed issues #1179 and #702.

```
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/miniconda3/lib/python3.7/site-packages/IPython/core/formatters.py in __call__(self, obj)
    343             method = get_real_method(obj, self.print_method)
    344             if method is not None:
--> 345                 return method()
    346             return None
    347         else:

/miniconda3/lib/python3.7/site-packages/qutip/qip/circuit.py in _repr_svg_(self)
   1117 
   1118     def _repr_svg_(self):
-> 1119         return _latex_compile(self.latex_code(), format="svg")
   1120 
   1121     @property

/miniconda3/lib/python3.7/site-packages/qutip/qip/circuit_latex.py in _latex_compile(code, filename, format)
     64     else:
     65         os.system("pdf2svg %s.pdf %s.svg" % (filename, filename))
---> 66         with open("%s.svg" % filename) as f:
     67             result = f.read()
     68 

FileNotFoundError: [Errno 2] No such file or directory: 'qcirc.svg'
```
<img width="1137" alt="Screenshot 2020-02-26 17 44 05" src="https://user-images.githubusercontent.com/14573436/75327357-9af58180-58bf-11ea-9989-6892f5b7af41.png">

Removed converting error on windows.
Removed error when converting qcirc.pdf to qcirc.png on windows.


Discussion:
If I understand correctly, this is more to do with the `ImageMagick` system changing, rather than Windows in particular - ImageMagick v7 introduced the command `magick` in place of the previous `convert` (though at least on my Mac, `convert` still works as expected): see https://www.imagemagick.org/discourse-server/viewtopic.php?t=29582.
If it helps in the future, you can use `pycodestyle` (available in `pip` and `conda`) to check pep8 compliance on your end before pushing a commit.

As to the actual command, we might not actually want to remove the usage of `convert` - it's quite possible that a lot of people are still using ImageMagick 6, since for most people it's just a dependency that got installed by something else, and isn't frequently updated.  `qutip` doesn't list it as an official dependency, so it wouldn't get updated on our behalf.

We could catch which version of `ImageMagick` to use with something like
```python
def _magick_comamnd():
    for command in ['magick', 'convert']:
        try:
            subprocess.run((command, '--version'), capture_output=True)
            return command
        except OSError:
            pass
    raise FileNotFoundError("Couldn't locate system ImageMagick")
```
and cache the result on import (below into the variable `_magick`).  There could also be logic based on OS if needed. The convert command would become
```python
subprocess.run((_magick, "-density", 100, filename + ".pdf", filename + ".png"))
```
or something like that.
Thank you for your comments. I can understand what you mean. But apart from this problem, I have one question. If someone use ImageMagick 6, everythings work well? (including windows users)
You're right in that everything works at the moment if somebody is using IM6, but with the proposed commit it would no longer work (since IM6 doesn't have the `magick` command).

That's not necessarily a problem as long as it's a conscious decision, and I can't provide that since I'm not a maintainer.
Oh thank you a lot. Version of IM was the point. I'm sorry to bother you but is there any document commenting the required version?
ImageMagick is only mentioned in our tutorial page and not listed elsewhere. So we didn't specify a required version for it. 


> We could catch which version of `ImageMagick` to use with something like
> 
> ```python
> def _magick_comamnd():
>     for command in ['magick', 'convert']:
>         try:
>             subprocess.run((command, '--version'), capture_output=True)
>             return command
>         except OSError:
>             pass
>     raise FileNotFoundError("Couldn't locate system ImageMagick")
> ```
> 
> and cache the result on import (below into the variable `_magick`). There could also be logic based on OS if needed. The convert command would become
> 
> ```python
> subprocess.run((_magick, "-density", 100, filename + ".pdf", filename + ".png"))
> ```

This solution looks neat. One minor thing is that maybe we should use `check=True` instead of  `capture_output=True` since the latter one is only available from Python 3.7+.


Despite those has already been discussed above, the plot sill does not work for me. I find the following lines also wrong on Windows
``` python
    os.system("rm -f %s.aux %s.log" % (filename, filename))

    os.system("mv %s-tmp.pdf %s.pdf" % (filename, filename))
```
because `rm` and `mv` are not Windows command. We should probably replace it by python script `os.remove` and `os.replace` to make it system independent.
`pdf2svg` isn't part of ImageMagick, (at least on `brew`) is http://www.cityinthesky.co.uk/opensource/pdf2svg.  It feels like all of these system commands should be made a bit more robust.

In particular, the bare `os.system` is not the recommended way of calling out to system commands any more.  It's now preferable to use [`subprocess.run`](https://docs.python.org/3/library/subprocess.html#subprocess.run).  Here, the bug is, I would assume, actually on the previous line:
```python
os.system('command_which_does_not_exist')
```
will return a non-zero error code which isn't checked for, but will not raise an exception.  

If Jupyter supports displaying embedded PDF files and that's all it's being used for, then it might be worth removing the conversion to SVG.  (pdf)LaTeX certainly supports embedded PDF files as graphics.  The fewer little dependencies we require, the better.
Also, while this is neither here nor there, #1179 isn't yet closed, merged, or ready for merge.
I've had the same problem recently, trying to plot some circuits. Had to dig into the code to find which packages were missing. Maybe we could replace the `os.system()` calls with a the `subprocess.run()` and show on which call the code failed.
If I understand correctly, this is more to do with the `ImageMagick` system changing, rather than Windows in particular - ImageMagick v7 introduced the command `magick` in place of the previous `convert` (though at least on my Mac, `convert` still works as expected): see https://www.imagemagick.org/discourse-server/viewtopic.php?t=29582.
If it helps in the future, you can use `pycodestyle` (available in `pip` and `conda`) to check pep8 compliance on your end before pushing a commit.

As to the actual command, we might not actually want to remove the usage of `convert` - it's quite possible that a lot of people are still using ImageMagick 6, since for most people it's just a dependency that got installed by something else, and isn't frequently updated.  `qutip` doesn't list it as an official dependency, so it wouldn't get updated on our behalf.

We could catch which version of `ImageMagick` to use with something like
```python
def _magick_comamnd():
    for command in ['magick', 'convert']:
        try:
            subprocess.run((command, '--version'), capture_output=True)
            return command
        except OSError:
            pass
    raise FileNotFoundError("Couldn't locate system ImageMagick")
```
and cache the result on import (below into the variable `_magick`).  There could also be logic based on OS if needed. The convert command would become
```python
subprocess.run((_magick, "-density", 100, filename + ".pdf", filename + ".png"))
```
or something like that.
Thank you for your comments. I can understand what you mean. But apart from this problem, I have one question. If someone use ImageMagick 6, everythings work well? (including windows users)
You're right in that everything works at the moment if somebody is using IM6, but with the proposed commit it would no longer work (since IM6 doesn't have the `magick` command).

That's not necessarily a problem as long as it's a conscious decision, and I can't provide that since I'm not a maintainer.
Oh thank you a lot. Version of IM was the point. I'm sorry to bother you but is there any document commenting the required version?
ImageMagick is only mentioned in our tutorial page and not listed elsewhere. So we didn't specify a required version for it. 


> We could catch which version of `ImageMagick` to use with something like
> 
> ```python
> def _magick_comamnd():
>     for command in ['magick', 'convert']:
>         try:
>             subprocess.run((command, '--version'), capture_output=True)
>             return command
>         except OSError:
>             pass
>     raise FileNotFoundError("Couldn't locate system ImageMagick")
> ```
> 
> and cache the result on import (below into the variable `_magick`). There could also be logic based on OS if needed. The convert command would become
> 
> ```python
> subprocess.run((_magick, "-density", 100, filename + ".pdf", filename + ".png"))
> ```

This solution looks neat. One minor thing is that maybe we should use `check=True` instead of  `capture_output=True` since the latter one is only available from Python 3.7+.


Despite those has already been discussed above, the plot sill does not work for me. I find the following lines also wrong on Windows
``` python
    os.system("rm -f %s.aux %s.log" % (filename, filename))

    os.system("mv %s-tmp.pdf %s.pdf" % (filename, filename))
```
because `rm` and `mv` are not Windows command. We should probably replace it by python script `os.remove` and `os.replace` to make it system independent.

Resolving patch:
diff --git a/qutip/qip/circuit.py b/qutip/qip/circuit.py
--- a/qutip/qip/circuit.py
+++ b/qutip/qip/circuit.py
@@ -37,10 +37,21 @@
 
 import numpy as np
 
-from qutip.qip.circuit_latex import _latex_compile
+from qutip.qip import circuit_latex as _latex
 from qutip.qip.operations.gates import *
 from qutip.qip.qubits import qubit_states
 
+try:
+    from IPython.display import Image as DisplayImage, SVG as DisplaySVG
+except ImportError:
+    # If IPython doesn't exist, then we set the nice display hooks to be simple
+    # pass-throughs.
+    def DisplayImage(data, *args, **kwargs):
+        return data
+
+    def DisplaySVG(data, *args, **kwargs):
+        return data
+
 __all__ = ['Gate', 'QubitCircuit']
 
 
@@ -1112,21 +1123,34 @@ def latex_code(self):
 
         return code
 
-    def _repr_png_(self):
-        return _latex_compile(self.latex_code(), format="png")
+    # This slightly convoluted dance with the conversion formats is because
+    # image conversion has optional dependencies.  We always want the `png` and
+    # `svg` methods to be available so that they are discoverable by the user,
+    # however if one is called without the required dependency, then they'll
+    # get a `RuntimeError` explaining the problem.  We only want the IPython
+    # magic methods `_repr_xxx_` to be defined if we know that the image
+    # conversion is available, so the user doesn't get exceptions on display
+    # because IPython tried to do something behind their back.
+
+    def _raw_png(self):
+        return _latex.image_from_latex(self.latex_code(), "png")
 
-    def _repr_svg_(self):
-        return _latex_compile(self.latex_code(), format="svg")
+    if 'png' in _latex.CONVERTERS:
+        _repr_png_ = _raw_png
 
     @property
     def png(self):
-        from IPython.display import Image
-        return Image(self._repr_png_(), embed=True)
+        return DisplayImage(self._raw_png(), embed=True)
+
+    def _raw_svg(self):
+        return _latex.image_from_latex(self.latex_code(), "svg")
+
+    if 'svg' in _latex.CONVERTERS:
+        _repr_svg_ = _raw_svg
 
     @property
     def svg(self):
-        from IPython.display import SVG
-        return SVG(self._repr_svg_())
+        return DisplaySVG(self._raw_svg())
 
     def qasm(self):
 
diff --git a/qutip/qip/circuit_latex.py b/qutip/qip/circuit_latex.py
--- a/qutip/qip/circuit_latex.py
+++ b/qutip/qip/circuit_latex.py
@@ -30,7 +30,13 @@
 #    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 ###############################################################################
+import collections
+import functools
 import os
+import shutil
+import subprocess
+import tempfile
+import warnings
 
 _latex_template = r"""
 \documentclass{standalone}
@@ -42,31 +48,203 @@
 """
 
 
-def _latex_compile(code, filename="qcirc", format="png"):
+_run_command = functools.partial(subprocess.run, check=True,
+                                 stdout=subprocess.DEVNULL,
+                                 stderr=subprocess.DEVNULL)
+_run_command.__doc__ = \
     """
-    Requires: pdflatex, pdfcrop, pdf2svg, imagemagick (convert)
+    Run a command with stdout and stderr explicitly thrown away, raising
+    `subprocess.CalledProcessError` if the command returned a non-zero exit
+    code.
     """
-    os.system("rm -f %s.tex %s.pdf %s.png" % (filename, filename, filename))
 
-    with open(filename + ".tex", "w") as file:
-        file.write(_latex_template % (_qcircuit_latex_min, code))
 
-    os.system("pdflatex -interaction batchmode %s.tex" % filename)
-    os.system("rm -f %s.aux %s.log" % (filename, filename))
-    os.system("pdfcrop %s.pdf %s-tmp.pdf" % (filename, filename))
-    os.system("mv %s-tmp.pdf %s.pdf" % (filename, filename))
+def _force_remove(*filenames):
+    """`rm -f`: try to remove a file, ignoring errors if it doesn't exist."""
+    for filename in filenames:
+        try:
+            os.remove(filename)
+        except FileNotFoundError:
+            pass
 
-    if format == 'png':
-        os.system("convert -density %s %s.pdf %s.png" % (100, filename,
-                                                         filename))
-        with open("%s.png" % filename, "rb") as f:
-            result = f.read()
+
+def _test_convert_is_imagemagick():
+    """
+    Test to see if the `convert` command behaves like we'd expect ImageMagick
+    to.  On Windows if ImageMagick is not installed then `convert` may refer to
+    a system utility.
+    """
+    try:
+        # Don't use `capture_output` because we're still supporting Python 3.6
+        process = subprocess.run(('convert', '-version'),
+                                 stdout=subprocess.PIPE,
+                                 stderr=subprocess.DEVNULL)
+        return "imagemagick" in process.stdout.decode('utf-8').lower()
+    except FileNotFoundError:
+        return False
+
+
+_SPECIAL_CASES = {
+    'convert': _test_convert_is_imagemagick,
+}
+
+
+def _find_system_command(names):
+    """
+    Given a list of possible system commands (as strings), return the first one
+    which has a locatable executable form, or `None` if none of them do.  We
+    also check some special cases of shadowing (e.g. ImageMagick 6's `convert`
+    is also a Windows system utility) to try and catch false-positives.
+    """
+    for name in names:
+        if shutil.which(name) is not None:
+            is_valid = _SPECIAL_CASES.get(name, lambda: True)()
+            if is_valid:
+                return name
+    return None
+
+
+_pdflatex = _find_system_command(['pdflatex'])
+_pdfcrop = _find_system_command(['pdfcrop'])
+
+
+if _pdfcrop is not None:
+    def _crop_pdf(filename):
+        """Crop the pdf file `filename` in place."""
+        temporary = ".tmp." + filename
+        _run_command((_pdfcrop, filename, temporary))
+        # Windows does not allow renaming to an existing file (but unix does).
+        _force_remove(filename)
+        os.rename(temporary, filename)
+else:
+    def _crop_pdf(_):
+        # Warn, but do not raise - we can recover from a failed crop.
+        warnings.warn("Could not locate system 'pdfcrop':"
+                      " image output may have additional margins.")
+
+
+def _convert_pdf(file_stem):
+    """
+    'Convert' to pdf: since LaTeX outputs a PDF file, there's nothing to do.
+    """
+    with open(file_stem + ".pdf", "rb") as file:
+        return file.read()
+
+
+# Record type to hold definitions of possible conversions - this is just for
+# reading convenience.
+_ConverterConfiguration = collections.namedtuple(
+    '_ConverterConfiguration',
+    ['file_type', 'dependency', 'executables', 'arguments', 'binary'],
+)
+CONVERTERS = {"pdf": _convert_pdf}
+_MISSING_CONVERTERS = {}
+_CONVERTER_CONFIGURATIONS = [
+    _ConverterConfiguration('png', 'ImageMagick', ['magick', 'convert'],
+                            arguments=('-density', '100'), binary=True),
+    _ConverterConfiguration('svg', 'pdf2svg', ['pdf2svg'],
+                            arguments=(), binary=False),
+]
+
+
+def _make_converter(configuration):
+    """
+    Create the actual conversion function of signature
+        file_stem: str -> 'T,
+    where 'T is data in the format to be converted to.
+    """
+    which = _find_system_command(configuration.executables)
+    if which is None:
+        return None
+    mode = "rb" if configuration.binary else "r"
+
+    def converter(file_stem):
+        """
+        Convert a file located in the current directory named `<file_stem>.pdf`
+        to an image format with the name `<file_stem>.xxx`, where `xxx` is
+        converter-dependent.
+
+        Parameters
+        ----------
+        file_stem : str
+            The basename of the PDF file to be converted.
+        """
+        in_file = file_stem + ".pdf"
+        out_file = file_stem + "." + configuration.file_type
+        _run_command((which, *configuration.arguments, in_file, out_file))
+        with open(out_file, mode) as file:
+            return file.read()
+    return converter
+
+
+for configuration in _CONVERTER_CONFIGURATIONS:
+    # Make the converter using a higher-order function, because if we defined a
+    # function in the loop, it would be easy to later introduce bugs due to
+    # leaky closures over loop variables.
+    converter = _make_converter(configuration)
+    if converter:
+        CONVERTERS[configuration.file_type] = converter
     else:
-        os.system("pdf2svg %s.pdf %s.svg" % (filename, filename))
-        with open("%s.svg" % filename) as f:
-            result = f.read()
+        _MISSING_CONVERTERS[configuration.file_type] = configuration.dependency
+
+
+if _pdflatex is not None:
+    def image_from_latex(code, file_type="png"):
+        """
+        Convert the LaTeX `code` into an image format, defined by the
+        `file_type`.  Returns a string or bytes object, depending on whether
+        the requested type is textual (e.g. svg) or binary (e.g. png).  The
+        known file types are in keys in this module's `CONVERTERS` dictionary.
+
+        Parameters
+        ----------
+        code: str
+            LaTeX code representing the circuit to be converted.
+
+        file_type: str ("png")
+            The file type that the image should be returned in.
+
 
-    return result
+        Returns
+        -------
+        image: str or bytes
+            An encoded version of the image.  Whether the output type is str or
+            bytes depends on whether the requested image format is textual or
+            binary.
+        """
+        filename = "qcirc"  # Arbitrary and internal.
+        # We do all the image conversion in a temporary directory to prevent
+        # leftover files if something goes wrong (or we get a
+        # KeyboardInterrupt) during conversion.
+        previous_dir = os.getcwd()
+        with tempfile.TemporaryDirectory() as temporary_dir:
+            try:
+                os.chdir(temporary_dir)
+                with open(filename + ".tex", "w") as file:
+                    file.write(_latex_template % (_qcircuit_latex_min, code))
+                _run_command((_pdflatex, '-interaction', 'batchmode',
+                              filename))
+                _crop_pdf(filename + ".pdf")
+                if file_type in _MISSING_CONVERTERS:
+                    dependency = _MISSING_CONVERTERS[file_type]
+                    message = "".join([
+                        "Could not find system ", dependency, ".",
+                        " Image conversion to '", file_type, "'",
+                        " is not available."
+                    ])
+                    raise RuntimeError(message)
+                if file_type not in CONVERTERS:
+                    raise ValueError("".join(["Unknown output format: '",
+                                              file_type, "'."]))
+                out = CONVERTERS[file_type](filename)
+            finally:
+                # Leave the temporary directory before it is removed (necessary
+                # on Windows, but it doesn't hurt on POSIX).
+                os.chdir(previous_dir)
+        return out
+else:
+    def image_from_latex(*args, **kwargs):
+        raise RuntimeError("Could not find system 'pdflatex'.")
 
 
 _qcircuit_latex_min = r"""


Test patch: