Repository: spyder-ide/spyder

Issue:
Invert colors for "Pretty-Printing" symbolic math in IPython's dark background
The pretty printed symbolic math is unreadable when the the IPython console background is set to dark.

To reproduce:
1. Set preferences -> IPython console->Display->Background color to dark
2. Set preferences -> IPython console->Advanced->Use symbolic math
3. Restart ipython console
4. Enter a numeric command into Ipython and execute

Expected: Visible output
Got: only readable output when selected with the mouse to invert colors

* Spyder Version: 3.0.2
* Python Version: 3.5.2 64bit
* Operating system: Linux (Plasma5)

jedi >=0.8.1     :  0.9.0 (OK)
matplotlib >=1.0 :  1.5.3 (OK)
nbconvert >=4.0  :  4.2.0 (OK)
numpy >=1.7      :  1.11.2 (OK)
pandas >=0.13.1  :  0.19.1 (OK)
pep8 >=0.6       :  1.7.0 (OK)
psutil >=0.3     :  5.0.0 (OK)
pyflakes >=0.6.0 :  1.3.0 (OK)
pygments >=2.0   :  2.1.3 (OK)
pylint >=0.25    :  1.6.4 (OK)
qtconsole >=4.2.0:  4.2.1 (OK)
rope >=0.9.4     :  0.9.4-1 (OK)
sphinx >=0.6.6   :  1.5 (OK)
sympy >=0.7.3    :  1.0 (OK)


Discussion:
Thanks for reporting. We'll try to fix this in Spyder 3.1
I noticed this as well years ago, remember being puzzled that there was seemingly nothing printed until the lightbulb moment that it makes perfect sense that you don't see anything useful when using black text on black background.

Anyways, at that time I did a small 2-3 line patch which is long gone.
I clearly remember that there was a hardcoded 'k' for the symbolic rendering, I just can't remember if that was in IPython or Spyder. I'm leaning towards IPython.
I also remember just making a patch for png rendering and somehow managed to add a kwarg in one of the existing function calls (which one is completely gone with the wind) that trickled down to the png rendering, allowing to set arbitrary colors for the text, or as arbitrary they got with png.

Maybe a simple option for symbolic text-color could be made in preferences, with black/white as defaults pending on background?
After battling this issue for quite a while (but not being willing to change to a light background as I find it hard on the eyes), I've found a bodge-workaround for this issue.

In `Lib\site-packages\IPython\lib\latextools.py`, I changed the line:

```python
        mt.to_png(f, s, fontsize=12)
```

to:

```python
        mt.to_png(f, s, fontsize=12, color='#ff6a00')
```

The colour was chosen as a compromise that's easy to read regardless of whether the background colour is white or black.  Of course if your background colour is orange, this won't help!
It's really painful. 

@abudden Thanks! 


Thanks @abudden!
That solved it!  

> After battling this issue for quite a while (but not being willing to change to a light background as I find it hard on the eyes), I've found a bodge-workaround for this issue.
> 
> In `Lib\site-packages\IPython\lib\latextools.py`, I changed the line:
> 
> ```python
>         mt.to_png(f, s, fontsize=12)
> ```
> 
> to:
> 
> ```python
>         mt.to_png(f, s, fontsize=12, color='#ff6a00')
> ```
> 
> The colour was chosen as a compromise that's easy to read regardless of whether the background colour is white or black. Of course if your background colour is orange, this won't help!


@ccordoba12 I am not sure how this is supposed to be fixed "correctly"

We could add a patch like

```python
    def _set_symbolic_math_color(self):
        """
        """
        code = """import IPython.lib.latextools
def _latex_to_png_mpl(s, wrap):
    try:
        from io import BytesIO
        from matplotlib import mathtext
        from pyparsing import ParseFatalException
    except ImportError:
        return None

    # mpl mathtext doesn't support display math, force inline
    s = s.replace('$$', '$')
    if wrap:
        s = u'${0}$'.format(s)

    try:
        mt = mathtext.MathTextParser('bitmap')
        f = BytesIO()
        mt.to_png(f, s, fontsize=12, color={{MATH_COLOR}})
        return f.getvalue()
    except (ValueError, RuntimeError, ParseFatalException):
        return None
IPython.lib.latextools.latex_to_png_mpl = _latex_to_png_mpl"""
        if self.additional_options['sympy']:
            code = code.replace('{{MATH_COLOR}}', '#ff0000')
            self.silent_exec_method(code)
```

On the `shell.py` widget and get the color from the current syntax style. Does this make sense or is there a better more "correct" way?
For this we need to merge first @impact27's improvements to the way we communicate with the kernel, so that every time we the color scheme changes, we can tell the kernel about that so it adjusts equations foreground color on its side.
I think the correct way to do it is to set the correct command for SymPy:s `init_printing`. The challenge there is that there are many more things set and it is not straightforward to get the other settings. It is not clear if it is a good thing or not to change user-specified printing colors (and other settings)...

A new thing that has shown up in 4.0 is the SymPy console. However, as discussed in #9885 the standard console has a grayish output color for latex now, making it almost readable, but the SymPy console latex color is black.

I'll try to get a PR through in SymPy that tries to set the color automatically based on the best guess. Will update here once I have something.
Once https://github.com/sympy/sympy/pull/17244 is merged into SymPy (and a new version released etc...), the LaTeX rendering in the SymPy console will set the correct foreground color if the IPython colors setting is correct. In the meanwhile, I think it makes sense to call `init_printing(foreground="%s") % console_font_color_string` in the SymPy console, both at startup and restart.
I have read up a bit more now. On start-up SymPy provides a plain text and a LaTeX source (generated by IPython based on a magic member function of the SymPy objects). After init_printing() is executed, instead SymPy provides a pretty printed plain text version, a PNG of the LaTeX equation, and the LaTeX source.

The rendering of the PNG is done in SymPy using the `preview` function (or for matplotlib cases, using `latex_to_png` from IPython). The qtconsole rendering relies on `latex_to_png` only. `latex_to_png` have two backends: math_text and "proper" LaTeX.

The way SymPy sets the text color is through an argument to the command `dvipng`. `latex_to_png` also uses the same command, but has no support to change the color.

Alternatives (as I see it):

 1. Add support in `latex_to_png` to be for qtconsole to be able to provide a foreground color to `dvipng`.
 2. Be inspired by the SymPy preview function and implement a similar function in qtconsole
 3. Wait for sympy/sympy#17244  to be merged and make sure that init_printing is called

If I understand things correctly, probably the quickest way forward is 2. qtconsole seems to have a rather rapid release pace and there seems to be tight connections between the teams.

1. would require an IPython release as well.
3. can probably be merged quickly, but not sure when the next release is. 

An obvious drawback of 2. is that it will more or less void some of the settings possible in `init_printing`. On the other hand, there seems to be no way for `init_printing` to figure out the actual background color, so it is simply guessing (as implemented in sympy/sympy#17244).

Also, currently, qtconsole shows the provided png if both a png and a latex-string are provided. I think the solution here is to tell SymPy to only provide the LaTeX string by default, and only if specific settings are done, also generate the PNG.

(Probably, the seemingly "white" rendering obtained in a plain console is an effect of the anti-aliasing, assuming a white background...) 
Oh, and implementing a dedicated rendered in qtconsole also enables svg-rendering of LaTeX. Although I do not really know exactly how the display of svgs happens, at least it will enable copying of the equations etc. Maybe not a big thing, but can be a fun exercise at some stage and no drawback if nothing else.
SVG-rendering is now part of SymPy, https://github.com/sympy/sympy/pull/17290. Still this will have a well defined foreground color provided by SymPy. Using that with Spyder leads to that the svg is converted to a `QImage`(?), so more conversion, but it is at least possible to copy/save the SVG. It may be possible to intervene with the SVG and convert the color before conversion. However, this require that SymPy sends the SVG, which it doesn't by default.

Theoretically, Spyder (qtconsole) can get the input in four different formats:
  * Pretty printed text
  * PNG rendered by SymPy
  * SVG rendered by SymPy
  * LaTeX equation

For the PNG and SVG everything is controlled by SymPy, so if we want to be really sure that we have the "correct" foreground color, we will have to use the LaTeX equation and render it locally. As latex_to_png etc cannot set the foreground color, qtconsole must add that functionality, and get setting from Spyder.

As mentioned earlier, on startup, SymPy only provides text and LaTeX, so in this case, it should be quite clear that we would like to do our own rendering (or add color support to IPython). For the SymPy console, we can provide a init_printing command which sets the color correctly. There is still a risk that the user does a new init_printing later though and then there's nothing to do about it. 

Any feedback on this is appreciated. The problem is that the same settings can be done in at least two different places (maybe three, considering if IPython decides to modify the color). And it is actually not a problem, if you understand where the color selection is made. Which took me a while to figure out...




> For the SymPy console, we can provide a init_printing command which sets the color correctly

That's done already in the kernel, but it'll be improved to set the right foreground color according to the console background color.
> which sets the color correctly

Was the crucial part. Right now, it calls `init_session`, which indeed has a `init_printing` in it, but without color information (as you say). (And it should really be called with the current foreground color, right? Independent of the background color to some extent, as the text color should be correct.)

One can consider if one would like to add configuration of the SymPy LaTeX printer. The problem of just calling `init_printing` again on a color change is that it will override any configurations that the user has made. This will of course be a transient problem, but it should probably still be considered.

Either it can be solved by:
1. adding a "change foreground color" functionality to SymPy (which I think should be possible, but require a pretty big rewrite) so that one can query the print wrapper from IPython and then modify the color somehow
 2. make it possible to query the configuration parameters from the print wrapper so that the same can be used next time
 3. add a configuration page to the Spyder preferences so that they are always the same.

The latter would actually be quite nice as there are many preferences that at least I would like to set, but cannot be bothered to do in every session (especially since I'm a SymPy developer means restarting the console quite often).

The possible advantage with 1 and 2 is that users can specify their own LatexPrinter and that can be hard to achieve with 3. On the other hand, if a user does that, they probably know how to change the foreground color already...

I'd be happy to help out with either adding a preference tab to Spyder (which I think is really my preferred approach at the moment) or trying to mitigate any problems on the SymPy side to make it work more integrated with Spyder.
Btw, at the moment SymPy doesn't accept hex-RGB colors. There is a PR, but I wrote it with enums just to realize that there were no enums in Python 2.7. Not sure if I should rewrite it, make partial support for just that color scheme, or wait until Python 2.7 support is dropped. Probably, partial support seems most feasible at the moment...
> Was the crucial part. Right now, it calls init_session

Right, I thought we were calling `init_printing`.

> The problem of just calling init_printing again on a color change is that it will override any configurations that the user has made

We'll have to go with this solution for Spyder 4, sorry. This is just to have the right foreground color in our SymPy consoles whenever the background color changes from dark to light.

> add a configuration page to the Spyder preferences so that they are always the same.

That's something we also had in mind to implement but haven't had time to actually do it. If you manage to do it yourself in the next two or three weeks, we could include for Spyder 4. Else it'll have to wait until Spyder 5 (to be released in Q1 of 2020).

Resolving patch:
diff --git a/spyder/plugins/ipythonconsole/widgets/shell.py b/spyder/plugins/ipythonconsole/widgets/shell.py
--- a/spyder/plugins/ipythonconsole/widgets/shell.py
+++ b/spyder/plugins/ipythonconsole/widgets/shell.py
@@ -185,9 +185,16 @@ def set_color_scheme(self, color_scheme, reset=True):
         if reset:
             self.reset(clear=True)
         if not dark_color:
+            # Needed to change the colors of tracebacks
             self.silent_execute("%colors linux")
+            self.call_kernel(
+                interrupt=True,
+                blocking=False).set_sympy_forecolor(background_color='dark')
         else:
             self.silent_execute("%colors lightbg")
+            self.call_kernel(
+                interrupt=True,
+                blocking=False).set_sympy_forecolor(background_color='light')
 
     def request_syspath(self):
         """Ask the kernel for sys.path contents."""


Test patch: