Repository: arviz-devs/arviz

Issue:
plot_trace lines is unclear and it may yield unexpected results
**Describe the bug**
The argument `lines` for the function `plot_trace` can give unexpected results. Moreover, the documentation is a bit nebulous.
 
**To Reproduce**
A toy example is defined
```python
import pymc3 as pm
import arviz as az
import numpy as np

# fake data
mu_real = 0
sigma_real = 1
n_samples = 150

Y = np.random.normal(loc=mu_real, scale=sigma_real, size=n_samples)

with pm.Model() as model:
    mu = pm.Normal('mu', mu=0, sigma=10)  
    sigma = pm.HalfNormal('sigma', sigma=10)
    likelihood = pm.Normal('likelihood', mu=mu, sigma=sigma, observed=Y)
    trace = pm.sample()
```
As per [documentation](https://arviz-devs.github.io/arviz/generated/arviz.plot_trace.html#arviz.plot_trace), the argument `lines` accepts a tuple in the form `(var_name, {‘coord’: selection}, [line, positions])`. So, the command 
```python
az.plot_trace(trace, lines=(('mu', {}, mu_real),))
```
yields correctly 
![image](https://user-images.githubusercontent.com/18400141/69927034-b7327f80-1484-11ea-8f4c-5f52563413be.png)

I can also pass a list of tuples or a list of tuples and lists and it will work fine:
```
az.plot_trace(trace, lines=[('mu', {}, mu_real)]) # list of tuples
az.plot_trace(trace, lines=[['mu', {}, mu_real]]) # list of lists
az.plot_trace(trace, lines=[['mu', {}, mu_real], ('sigma', {}, sigma_real)]) # list of lists and tuples
```
however, I cannot pass a simple tuple because I will get a `KeyError: 0`
```python
az.plot_trace(trace, lines=(['mu', {}, mu_real]))
az.plot_trace(trace, lines=(('mu', {}, mu_real)))
```
Also, I can pass a variable or coordinate name that do not exist and Arviz will not complain---but not lines will be plotted (here I would expect a warning)
```python
az.plot_trace(trace, lines=[('hey', {}, mu_real)])
az.plot_trace(trace, lines=[('mu', {'hey'}, mu_real)])
```
![image](https://user-images.githubusercontent.com/18400141/69927389-d251bf00-1485-11ea-8d5c-7630a6763728.png)
The weird behavior happens when I pass a string:
```python
az.plot_trace(trace, lines=[('mu', {}, 'hey')])
```
![image](https://user-images.githubusercontent.com/18400141/69927502-32e0fc00-1486-11ea-8f6a-3f012017e932.png)

**Expected behavior**
The [documentation](https://arviz-devs.github.io/arviz/generated/arviz.plot_trace.html#arviz.plot_trace) could be improved and the function could check the inputs. In addition to what described above, the placeholder `[line, positions]` in `(var_name, {‘coord’: selection}, [line, positions])` should be something like `[line_positions]` otherwise one may think (like myself :) ) that two values should be inserted (one for `line` and one for `positions`).

**Additional context**
I am using Win10, fresh conda environment with PyMC3 and Arviz from master.

Possibly related https://github.com/pymc-devs/pymc3/issues/3495, https://github.com/pymc-devs/pymc3/issues/3497


Discussion:
Thanks for the feedback.

We are on our way to update the interface a bit so I hope we can fix this issue at the same time and the usage would be intuitive.
Thank you for your good work 🙌
@ahartikainen @OriolAbril 
 I think the error is caused due to following:
https://github.com/arviz-devs/arviz/blob/0774d13979317cf4d22bb995ee298de4804432d9/arviz/plots/backends/matplotlib/traceplot.py#L217-L225
I would check for the list, else raise the ValueError. Please verify.
Sounds gook, one option for checking the list is to check the dtype of `line_values` after the atleast_1d to make sure the array contains numeric values. One possibility is to follow this [SO answer](https://stackoverflow.com/questions/29518923/numpy-asarray-how-to-check-up-that-its-result-dtype-is-numeric). 
Okay, I'd go with that. Also, I'd add the warning for the case of variable or coordinate name that do not exist (as pointed in issue description)
> Also, I can pass a variable or coordinate name that do not exist and Arviz will not complain---but > not lines will be plotted (here I would expect a warning)
> ``` 
> az.plot_trace(trace, lines=[('hey', {}, mu_real)])
> az.plot_trace(trace, lines=[('mu', {'hey'}, mu_real)])
> ```

Resolving patch:
diff --git a/arviz/plots/backends/matplotlib/traceplot.py b/arviz/plots/backends/matplotlib/traceplot.py
--- a/arviz/plots/backends/matplotlib/traceplot.py
+++ b/arviz/plots/backends/matplotlib/traceplot.py
@@ -1,5 +1,6 @@
 """Matplotlib traceplot."""
 
+import warnings
 import matplotlib.pyplot as plt
 from matplotlib.lines import Line2D
 import numpy as np
@@ -48,8 +49,8 @@ def plot_trace(
     rug : bool
         If True adds a rugplot. Defaults to False. Ignored for 2D KDE. Only affects continuous
         variables.
-    lines : tuple
-        Tuple of (var_name, {'coord': selection}, [line, positions]) to be overplotted as
+    lines : tuple or list
+        list of tuple of (var_name, {'coord': selection}, [line_positions]) to be overplotted as
         vertical lines on the density and horizontal lines on the trace.
     combined : bool
         Flag for combining multiple chains into a single line. If False (default), chains will be
@@ -124,6 +125,21 @@ def plot_trace(
 
     _, axes = plt.subplots(len(plotters), 2, squeeze=False, figsize=figsize, **backend_kwargs)
 
+    # Check the input for lines
+    if lines is not None:
+        all_var_names = set(plotter[0] for plotter in plotters)
+
+        invalid_var_names = set()
+        for line in lines:
+            if line[0] not in all_var_names:
+                invalid_var_names.add(line[0])
+        if invalid_var_names:
+            warnings.warn(
+                "A valid var_name should be provided, found {} expected from {}".format(
+                    invalid_var_names, all_var_names
+                )
+            )
+
     for idx, (var_name, selection, value) in enumerate(plotters):
         value = np.atleast_2d(value)
 
@@ -219,6 +235,10 @@ def plot_trace(
                 line_values = [vlines]
             else:
                 line_values = np.atleast_1d(vlines).ravel()
+                if not np.issubdtype(line_values.dtype, np.number):
+                    raise ValueError(
+                        "line-positions should be numeric, found {}".format(line_values)
+                    )
             axes[idx, 0].vlines(line_values, *ylims[0], colors="black", linewidth=1.5, alpha=0.75)
             axes[idx, 1].hlines(
                 line_values, *xlims[1], colors="black", linewidth=1.5, alpha=trace_kwargs["alpha"]


Test patch:
diff --git a/arviz/tests/test_plots_matplotlib.py b/arviz/tests/test_plots_matplotlib.py
--- a/arviz/tests/test_plots_matplotlib.py
+++ b/arviz/tests/test_plots_matplotlib.py
@@ -156,6 +156,21 @@ def test_plot_trace_max_subplots_warning(models):
     assert axes.shape
 
 
+@pytest.mark.parametrize("kwargs", [{"var_names": ["mu", "tau"], "lines": [("hey", {}, [1])]}])
+def test_plot_trace_invalid_varname_warning(models, kwargs):
+    with pytest.warns(UserWarning, match="valid var.+should be provided"):
+        axes = plot_trace(models.model_1, **kwargs)
+    assert axes.shape
+
+
+@pytest.mark.parametrize(
+    "bad_kwargs", [{"var_names": ["mu", "tau"], "lines": [("mu", {}, ["hey"])]}]
+)
+def test_plot_trace_bad_lines_value(models, bad_kwargs):
+    with pytest.raises(ValueError, match="line-positions should be numeric"):
+        plot_trace(models.model_1, **bad_kwargs)
+
+
 @pytest.mark.parametrize("model_fits", [["model_1"], ["model_1", "model_2"]])
 @pytest.mark.parametrize(
     "args_expected",
@@ -701,7 +716,6 @@ def test_plot_posterior_point_estimates(models, point_estimate):
     "kwargs", [{"insample_dev": False}, {"plot_standard_error": False}, {"plot_ic_diff": False}]
 )
 def test_plot_compare(models, kwargs):
-
     model_compare = compare({"Model 1": models.model_1, "Model 2": models.model_2})
 
     axes = plot_compare(model_compare, **kwargs)