Repository: altair-viz/altair
Pull request #2824 (2023-01-09 13:43:27)

Make error messages on typos and missing/incorrect data types more informative
When trying to reference a column from a dataframe and a typo is made, the error message suggests that there is an issue with the type specification rather than the column name itself:

```py
import altair as alt
from vega_datasets import data

alt.Chart(data.cars()).mark_point().encode(x='Horsepowers')
```

**Old error**
> ValueError: Horsepowers encoding field is specified without a type; the type cannot be inferred because it does not match any column in the data.

I think that message is incorrect and unhelpful, and I often see learners struggle with understanding that they have made a typo. This PR suggests replacing the error message with this more informative alternative:

**New error**
> ValueError: "Horsepowers" cannot be found.
> - If you are trying to reference a column in the dataframe, there is likely a typo in the column name.
> - If you are trying to reference a field from a transform, there is either a typo in the field name or the data type specification is missing (Altair can only infer data types from dataframe columns).

---

Similarly, when making a typo in the encoding data type, the error message could also be made more informative:

```py
alt.Chart(data.cars()).mark_point().encode(x='Horsepower:Qi')
```

>**Old error**
> ValueError: Horsepower:Qi encoding field is specified without a type; the type cannot be inferred because it does not match any column in the data.

This PR changes that to the following:

**New error**
> ValueError: "Qi" is not one of the valid encoding data types: O, N, Q, T, G.
> For more details, see https://altair-viz.github.io/altair-docs/user_guide/encodings/index.html#encoding-data-types. (If you are trying to use a column name that contains a colon, prefix it with a backslash, \\:)

**Note** that this second change means that it is now required to escape colons in column names and although this is is a breaking change I think it makes it more consistent instead of having colon in column names sometimes work depending on whether the part after the colon happens to equal an Altair data type or not.


Patch:
diff --git a/altair/utils/core.py b/altair/utils/core.py
index 38bf0950..fa0c8af0 100644
--- a/altair/utils/core.py
+++ b/altair/utils/core.py
@@ -532,6 +532,23 @@ def parse_shorthand(
             if isinstance(attrs["type"], tuple):
                 attrs["sort"] = attrs["type"][1]
                 attrs["type"] = attrs["type"][0]
+
+    # If an unescaped colon is still present, it's often due to an incorrect data type specification
+    # but could also be due to using a column name with ":" in it.
+    if (
+        "field" in attrs
+        and ":" in attrs["field"]
+        and attrs["field"][attrs["field"].rfind(":") - 1] != "\\"
+    ):
+        raise ValueError(
+            '"{}" '.format(attrs["field"].split(":")[-1])
+            + "is not one of the valid encoding data types: {}.".format(
+                ", ".join(TYPECODE_MAP.values())
+            )
+            + "\nFor more details, see https://altair-viz.github.io/altair-docs/user_guide/encodings/index.html#encoding-data-types. "
+            + "If you are trying to use a column name that contains a colon, "
+            + 'prefix it with a backslash; for example "column\\:name" instead of "column:name".'
+        )
     return attrs
 
 
diff --git a/altair/vegalite/v5/schema/channels.py b/altair/vegalite/v5/schema/channels.py
index 13b5453f..8be22869 100644
--- a/altair/vegalite/v5/schema/channels.py
+++ b/altair/vegalite/v5/schema/channels.py
@@ -38,9 +38,12 @@ class FieldChannelMixin(object):
                 parsed.pop('type', None)
             elif not (type_in_shorthand or type_defined_explicitly):
                 if isinstance(context.get('data', None), pd.DataFrame):
-                    raise ValueError("{} encoding field is specified without a type; "
-                                     "the type cannot be inferred because it does not "
-                                     "match any column in the data.".format(shorthand))
+                    raise ValueError(
+                        'Unable to determine data type for the field "{}";'
+                        " verify that the field name is not misspelled."
+                        " If you are referencing a field from a transform,"
+                        " also confirm that the data type is specified correctly.".format(shorthand)
+                    )
                 else:
                     raise ValueError("{} encoding field is specified without a type; "
                                      "the type cannot be automatically inferred because "
diff --git a/doc/releases/changes.rst b/doc/releases/changes.rst
index 8e3f25df..0c9a44b1 100644
--- a/doc/releases/changes.rst
+++ b/doc/releases/changes.rst
@@ -21,7 +21,7 @@ Enhancements
 - Saving charts with HTML inline is now supported without having altair_saver installed (#2807).
 - The documentation page has been revamped, both in terms of appearance and content.
 - More informative autocompletion by removing deprecated methods (#2814) and adding support for completion in method chains for editors that rely on type hints (e.g. VS Code) (#2846)
-- Improved error messages (#2842)
+- Substantially improved error handling. Both in terms of finding the more relevant error (#2842), and in terms of improving the formatting and clarity of the error messages (#2824, #2568).
 - Include experimental support for the DataFrame Interchange Protocol (through `__dataframe__` attribute). This requires `pyarrow>=11.0.0` (#2888).
 
 Grammar Changes
@@ -45,6 +45,7 @@ Bug Fixes
 Backward-Incompatible Changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+- Colons in column names must now be escaped to remove any ambiguity with encoding types. You now need to write ``"column\:name"`` instead of ``"column:name"`` (#2824).
 - Removed the Vega (v5) wrappers and deprecate rendering in Vega mode (save Chart as Vega format is still allowed) (#2829).
 - Removed the Vega-Lite 3 and 4 wrappers (#2847).
 - In regards to the grammar changes listed above, the old terminology will still work in many basic cases.  On the other hand, if that old terminology gets used at a lower level, then it most likely will not work.  For example, in the current version of :ref:`gallery_scatter_with_minimap`, two instances of the key ``param`` are used in dictionaries to specify axis domains.  Those used to be ``selection``, but that usage is not compatible with the current Vega-Lite schema.


Test patch:
diff --git a/tests/utils/tests/test_core.py b/tests/utils/tests/test_core.py
index e0b57567..aae61b5e 100644
--- a/tests/utils/tests/test_core.py
+++ b/tests/utils/tests/test_core.py
@@ -79,7 +79,7 @@ def test_parse_shorthand():
 
     # Fields alone
     check("foobar", field="foobar")
-    check("blah:(fd ", field="blah:(fd ")
+    check(r"blah\:(fd ", field=r"blah\:(fd ")
 
     # Fields with type
     check("foobar:quantitative", type="quantitative", field="foobar")
@@ -101,14 +101,14 @@ def test_parse_shorthand():
 
     # check that invalid arguments are not split-out
     check("invalid(blah)", field="invalid(blah)")
-    check("blah:invalid", field="blah:invalid")
-    check("invalid(blah):invalid", field="invalid(blah):invalid")
+    check(r"blah\:invalid", field=r"blah\:invalid")
+    check(r"invalid(blah)\:invalid", field=r"invalid(blah)\:invalid")
 
     # check parsing in presence of strange characters
     check(
-        "average(a b:(c\nd):Q",
+        r"average(a b\:(c\nd):Q",
         aggregate="average",
-        field="a b:(c\nd",
+        field=r"a b\:(c\nd",
         type="quantitative",
     )
 
@@ -281,3 +281,10 @@ def test_infer_encoding_types_with_condition():
         ),
     )
     assert infer_encoding_types(args, kwds, channels) == expected
+
+
+def test_invalid_data_type():
+    with pytest.raises(
+        ValueError, match=r'"\(fd " is not one of the valid encoding data types'
+    ):
+        parse_shorthand(r"blah:(fd ")