Repository: vega/altair

Issue:
Make `SchemaValidationError` more helpful by printing expected parameters
When a non-existing parameter name is used, I think it would be helpful to include the existing parameter names in the error message. For example, when misspelling a parameter name like in the example below it is not immediately clear whether I made a typo or something else went wrong, so the immediate feedback of seeing the correct parameter names would be helpful.

```python
import altair as alt
import pandas as pd

source = pd.DataFrame({
    'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],
    'b': [28, 55, 43, 91, 81, 53, 19, 87, 52]
})

alt.Chart(source).mark_bar().encode(
    x=alt.X("a", scale=alt.Scale(padingOuter=0.5)),
    y=alt.Y("b")
)
```

```
SchemaValidationError: Invalid specification

        altair.vegalite.v4.schema.core.Scale, validating 'additionalProperties'

        Additional properties are not allowed ('padingOuter' was unexpected)
```

I also belive the error message text could be clarified a little. Especially for novices, I don't think it is immediately clear what it means that "Additional properties are not allowed" and it could helpful to update this error message to something like:

```
SchemaValidationError: Invalid specification

        altair.vegalite.v4.schema.core.Scale, validating 'additionalProperties'

        No such parameter name exists: 'padingOuter'
        
        Existing parameter names are: 'align', 'base', 'bins', 'clamp', 'constant', 'domain',
            'domainMid', 'exponent', 'interpolate', 'nice', 'padding', 'paddingInner', 'paddingOuter',
            'range', 'reverse', 'round', 'scheme', 'type', 'zero', 'kwds'
        
        See the help for altair.Scale` for the full description of these parameters.
```

We can use the `inspect` module to retrieve the list of expected parameter names via `list(inspect.signature(altair.vegalite.v4.schema.core.Scale).parameters.keys())`. I can try to PR this if it sounds like a good idea (it looks like modifying the returned value of [altair/utils/schemapi.py](https://github.com/altair-viz/altair/blob/55bb4b7df07996abb4ff05e80a69cadae291482a/altair/utils/schemapi.py#L116-L123) is the right place?).


Discussion:
Strongly agree - I just don’t know how to do it, because schema validation is done by the jsonschema package and it doesn’t really return anything useful. If you can figure out how to do it robustly I’ll happily review the PR!

Resolving patch:
diff --git a/altair/utils/schemapi.py b/altair/utils/schemapi.py
--- a/altair/utils/schemapi.py
+++ b/altair/utils/schemapi.py
@@ -6,6 +6,7 @@
 import json
 import textwrap
 from typing import Any, Sequence, List
+from itertools import zip_longest
 
 import jsonschema
 import jsonschema.exceptions
@@ -15,6 +16,7 @@
 
 from altair import vegalite
 
+
 # If DEBUG_MODE is True, then schema objects are converted to dict and
 # validated at creation time. This slows things down, particularly for
 # larger specs, but leads to much more useful tracebacks for the user.
@@ -158,6 +160,63 @@ def __init__(self, obj, err):
         self.obj = obj
         self._additional_errors = getattr(err, "_additional_errors", [])
 
+    @staticmethod
+    def _format_params_as_table(param_dict_keys):
+        """Format param names into a table so that they are easier to read"""
+        param_names, name_lengths = zip(
+            *[
+                (name, len(name))
+                for name in param_dict_keys
+                if name not in ["kwds", "self"]
+            ]
+        )
+        # Worst case scenario with the same longest param name in the same
+        # row for all columns
+        max_name_length = max(name_lengths)
+        max_column_width = 80
+        # Output a square table if not too big (since it is easier to read)
+        num_param_names = len(param_names)
+        square_columns = int(np.ceil(num_param_names**0.5))
+        columns = min(max_column_width // max_name_length, square_columns)
+
+        # Compute roughly equal column heights to evenly divide the param names
+        def split_into_equal_parts(n, p):
+            return [n // p + 1] * (n % p) + [n // p] * (p - n % p)
+
+        column_heights = split_into_equal_parts(num_param_names, columns)
+
+        # Section the param names into columns and compute their widths
+        param_names_columns = []
+        column_max_widths = []
+        last_end_idx = 0
+        for ch in column_heights:
+            param_names_columns.append(param_names[last_end_idx : last_end_idx + ch])
+            column_max_widths.append(
+                max([len(param_name) for param_name in param_names_columns[-1]])
+            )
+            last_end_idx = ch + last_end_idx
+
+        # Transpose the param name columns into rows to facilitate looping
+        param_names_rows = []
+        for li in zip_longest(*param_names_columns, fillvalue=""):
+            param_names_rows.append(li)
+        # Build the table as a string by iterating over and formatting the rows
+        param_names_table = ""
+        for param_names_row in param_names_rows:
+            for num, param_name in enumerate(param_names_row):
+                # Set column width based on the longest param in the column
+                max_name_length_column = column_max_widths[num]
+                column_pad = 3
+                param_names_table += "{:<{}}".format(
+                    param_name, max_name_length_column + column_pad
+                )
+                # Insert newlines and spacing after the last element in each row
+                if num == (len(param_names_row) - 1):
+                    param_names_table += "\n"
+                    # 16 is the indendation of the returned multiline string below
+                    param_names_table += " " * 16
+        return param_names_table
+
     def __str__(self):
         # Try to get the lowest class possible in the chart hierarchy so
         # it can be displayed in the error message. This should lead to more informative
@@ -174,26 +233,48 @@ def __str__(self):
             # back on the class of the top-level object which created
             # the SchemaValidationError
             cls = self.obj.__class__
-        schema_path = ["{}.{}".format(cls.__module__, cls.__name__)]
-        schema_path.extend(self.schema_path)
-        schema_path = "->".join(
-            str(val)
-            for val in schema_path[:-1]
-            if val not in (0, "properties", "additionalProperties", "patternProperties")
-        )
-        message = self.message
-        if self._additional_errors:
-            message += "\n        " + "\n        ".join(
-                [e.message for e in self._additional_errors]
-            )
-        return """Invalid specification
 
-        {}, validating {!r}
+        # Output all existing parameters when an unknown parameter is specified
+        if self.validator == "additionalProperties":
+            param_dict_keys = inspect.signature(cls).parameters.keys()
+            param_names_table = self._format_params_as_table(param_dict_keys)
+
+            # `cleandoc` removes multiline string indentation in the output
+            return inspect.cleandoc(
+                """`{}` has no parameter named {!r}
+
+                Existing parameter names are:
+                {}
+                See the help for `{}` to read the full description of these parameters
+                """.format(
+                    cls.__name__,
+                    self.message.split("('")[-1].split("'")[0],
+                    param_names_table,
+                    cls.__name__,
+                )
+            )
+        # Use the default error message for all other cases than unknown parameter errors
+        else:
+            message = self.message
+            # Add a summary line when parameters are passed an invalid value
+            # For example: "'asdf' is an invalid value for `stack`
+            if self.absolute_path:
+                # The indentation here must match that of `cleandoc` below
+                message = f"""'{self.instance}' is an invalid value for `{self.absolute_path[-1]}`:
+
+                {message}"""
+
+            if self._additional_errors:
+                message += "\n                " + "\n                ".join(
+                    [e.message for e in self._additional_errors]
+                )
 
-        {}
-        """.format(
-            schema_path, self.validator, message
-        )
+            return inspect.cleandoc(
+                """{}
+                """.format(
+                    message
+                )
+            )
 
 
 class UndefinedType(object):
diff --git a/tools/schemapi/schemapi.py b/tools/schemapi/schemapi.py
--- a/tools/schemapi/schemapi.py
+++ b/tools/schemapi/schemapi.py
@@ -4,6 +4,7 @@
 import json
 import textwrap
 from typing import Any, Sequence, List
+from itertools import zip_longest
 
 import jsonschema
 import jsonschema.exceptions
@@ -13,6 +14,7 @@
 
 from altair import vegalite
 
+
 # If DEBUG_MODE is True, then schema objects are converted to dict and
 # validated at creation time. This slows things down, particularly for
 # larger specs, but leads to much more useful tracebacks for the user.
@@ -156,6 +158,63 @@ def __init__(self, obj, err):
         self.obj = obj
         self._additional_errors = getattr(err, "_additional_errors", [])
 
+    @staticmethod
+    def _format_params_as_table(param_dict_keys):
+        """Format param names into a table so that they are easier to read"""
+        param_names, name_lengths = zip(
+            *[
+                (name, len(name))
+                for name in param_dict_keys
+                if name not in ["kwds", "self"]
+            ]
+        )
+        # Worst case scenario with the same longest param name in the same
+        # row for all columns
+        max_name_length = max(name_lengths)
+        max_column_width = 80
+        # Output a square table if not too big (since it is easier to read)
+        num_param_names = len(param_names)
+        square_columns = int(np.ceil(num_param_names**0.5))
+        columns = min(max_column_width // max_name_length, square_columns)
+
+        # Compute roughly equal column heights to evenly divide the param names
+        def split_into_equal_parts(n, p):
+            return [n // p + 1] * (n % p) + [n // p] * (p - n % p)
+
+        column_heights = split_into_equal_parts(num_param_names, columns)
+
+        # Section the param names into columns and compute their widths
+        param_names_columns = []
+        column_max_widths = []
+        last_end_idx = 0
+        for ch in column_heights:
+            param_names_columns.append(param_names[last_end_idx : last_end_idx + ch])
+            column_max_widths.append(
+                max([len(param_name) for param_name in param_names_columns[-1]])
+            )
+            last_end_idx = ch + last_end_idx
+
+        # Transpose the param name columns into rows to facilitate looping
+        param_names_rows = []
+        for li in zip_longest(*param_names_columns, fillvalue=""):
+            param_names_rows.append(li)
+        # Build the table as a string by iterating over and formatting the rows
+        param_names_table = ""
+        for param_names_row in param_names_rows:
+            for num, param_name in enumerate(param_names_row):
+                # Set column width based on the longest param in the column
+                max_name_length_column = column_max_widths[num]
+                column_pad = 3
+                param_names_table += "{:<{}}".format(
+                    param_name, max_name_length_column + column_pad
+                )
+                # Insert newlines and spacing after the last element in each row
+                if num == (len(param_names_row) - 1):
+                    param_names_table += "\n"
+                    # 16 is the indendation of the returned multiline string below
+                    param_names_table += " " * 16
+        return param_names_table
+
     def __str__(self):
         # Try to get the lowest class possible in the chart hierarchy so
         # it can be displayed in the error message. This should lead to more informative
@@ -172,26 +231,48 @@ def __str__(self):
             # back on the class of the top-level object which created
             # the SchemaValidationError
             cls = self.obj.__class__
-        schema_path = ["{}.{}".format(cls.__module__, cls.__name__)]
-        schema_path.extend(self.schema_path)
-        schema_path = "->".join(
-            str(val)
-            for val in schema_path[:-1]
-            if val not in (0, "properties", "additionalProperties", "patternProperties")
-        )
-        message = self.message
-        if self._additional_errors:
-            message += "\n        " + "\n        ".join(
-                [e.message for e in self._additional_errors]
-            )
-        return """Invalid specification
 
-        {}, validating {!r}
+        # Output all existing parameters when an unknown parameter is specified
+        if self.validator == "additionalProperties":
+            param_dict_keys = inspect.signature(cls).parameters.keys()
+            param_names_table = self._format_params_as_table(param_dict_keys)
+
+            # `cleandoc` removes multiline string indentation in the output
+            return inspect.cleandoc(
+                """`{}` has no parameter named {!r}
+
+                Existing parameter names are:
+                {}
+                See the help for `{}` to read the full description of these parameters
+                """.format(
+                    cls.__name__,
+                    self.message.split("('")[-1].split("'")[0],
+                    param_names_table,
+                    cls.__name__,
+                )
+            )
+        # Use the default error message for all other cases than unknown parameter errors
+        else:
+            message = self.message
+            # Add a summary line when parameters are passed an invalid value
+            # For example: "'asdf' is an invalid value for `stack`
+            if self.absolute_path:
+                # The indentation here must match that of `cleandoc` below
+                message = f"""'{self.instance}' is an invalid value for `{self.absolute_path[-1]}`:
+
+                {message}"""
+
+            if self._additional_errors:
+                message += "\n                " + "\n                ".join(
+                    [e.message for e in self._additional_errors]
+                )
 
-        {}
-        """.format(
-            schema_path, self.validator, message
-        )
+            return inspect.cleandoc(
+                """{}
+                """.format(
+                    message
+                )
+            )
 
 
 class UndefinedType(object):


Test patch:
diff --git a/tests/utils/tests/test_schemapi.py b/tests/utils/tests/test_schemapi.py
--- a/tests/utils/tests/test_schemapi.py
+++ b/tests/utils/tests/test_schemapi.py
@@ -2,6 +2,7 @@
 # tools/generate_schema_wrapper.py. Do not modify directly.
 import copy
 import io
+import inspect
 import json
 import jsonschema
 import re
@@ -377,9 +378,6 @@ def test_schema_validation_error():
     assert isinstance(the_err, SchemaValidationError)
     message = str(the_err)
 
-    assert message.startswith("Invalid specification")
-    assert "test_schemapi.MySchema->a" in message
-    assert "validating {!r}".format(the_err.validator) in message
     assert the_err.message in message
 
 
@@ -467,36 +465,77 @@ def chart_example_invalid_y_option_value_with_condition():
     [
         (
             chart_example_invalid_y_option,
-            r"schema.channels.X.*"
-            + r"Additional properties are not allowed \('unknown' was unexpected\)",
+            inspect.cleandoc(
+                r"""`X` has no parameter named 'unknown'
+
+                Existing parameter names are:
+                shorthand      bin      scale   timeUnit   
+                aggregate      field    sort    title      
+                axis           impute   stack   type       
+                bandPosition                               
+
+                See the help for `X` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
         (
-            chart_example_invalid_y_option_value,
-            r"schema.channels.Y.*"
-            + r"'asdf' is not one of \['zero', 'center', 'normalize'\].*"
-            + r"'asdf' is not of type 'null'.*'asdf' is not of type 'boolean'",
+            chart_example_layer,
+            inspect.cleandoc(
+                r"""`VConcatChart` has no parameter named 'width'
+
+                Existing parameter names are:
+                vconcat      center     description   params    title       
+                autosize     config     name          resolve   transform   
+                background   data       padding       spacing   usermeta    
+                bounds       datasets                                       
+
+                See the help for `VConcatChart` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
         (
-            chart_example_layer,
-            r"api.VConcatChart.*"
-            + r"Additional properties are not allowed \('width' was unexpected\)",
+            chart_example_invalid_y_option_value,
+            inspect.cleandoc(
+                r"""'asdf' is an invalid value for `stack`:
+
+                'asdf' is not one of \['zero', 'center', 'normalize'\]
+                'asdf' is not of type 'null'
+                'asdf' is not of type 'boolean'"""
+            ),
         ),
         (
             chart_example_invalid_y_option_value_with_condition,
-            r"schema.channels.Y.*"
-            + r"'asdf' is not one of \['zero', 'center', 'normalize'\].*"
-            + r"'asdf' is not of type 'null'.*'asdf' is not of type 'boolean'",
+            inspect.cleandoc(
+                r"""'asdf' is an invalid value for `stack`:
+
+                'asdf' is not one of \['zero', 'center', 'normalize'\]
+                'asdf' is not of type 'null'
+                'asdf' is not of type 'boolean'"""
+            ),
         ),
         (
             chart_example_hconcat,
-            r"schema.core.TitleParams.*"
-            + r"\{'text': 'Horsepower', 'align': 'right'\} is not of type 'string'.*"
-            + r"\{'text': 'Horsepower', 'align': 'right'\} is not of type 'array'",
+            inspect.cleandoc(
+                r"""'{'text': 'Horsepower', 'align': 'right'}' is an invalid value for `title`:
+
+                {'text': 'Horsepower', 'align': 'right'} is not of type 'string'
+                {'text': 'Horsepower', 'align': 'right'} is not of type 'array'"""
+            ),
         ),
         (
             chart_example_invalid_channel_and_condition,
-            r"schema.core.Encoding->encoding.*"
-            + r"Additional properties are not allowed \('invalidChannel' was unexpected\)",
+            inspect.cleandoc(
+                r"""`Encoding` has no parameter named 'invalidChannel'
+
+                Existing parameter names are:
+                angle         key          order     strokeDash      tooltip   xOffset   
+                color         latitude     radius    strokeOpacity   url       y         
+                description   latitude2    radius2   strokeWidth     x         y2        
+                detail        longitude    shape     text            x2        yError    
+                fill          longitude2   size      theta           xError    yError2   
+                fillOpacity   opacity      stroke    theta2          xError2   yOffset   
+                href                                                                     
+
+                See the help for `Encoding` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
     ],
 )
diff --git a/tools/schemapi/tests/test_schemapi.py b/tools/schemapi/tests/test_schemapi.py
--- a/tools/schemapi/tests/test_schemapi.py
+++ b/tools/schemapi/tests/test_schemapi.py
@@ -1,5 +1,6 @@
 import copy
 import io
+import inspect
 import json
 import jsonschema
 import re
@@ -375,9 +376,6 @@ def test_schema_validation_error():
     assert isinstance(the_err, SchemaValidationError)
     message = str(the_err)
 
-    assert message.startswith("Invalid specification")
-    assert "test_schemapi.MySchema->a" in message
-    assert "validating {!r}".format(the_err.validator) in message
     assert the_err.message in message
 
 
@@ -465,36 +463,77 @@ def chart_example_invalid_y_option_value_with_condition():
     [
         (
             chart_example_invalid_y_option,
-            r"schema.channels.X.*"
-            + r"Additional properties are not allowed \('unknown' was unexpected\)",
+            inspect.cleandoc(
+                r"""`X` has no parameter named 'unknown'
+
+                Existing parameter names are:
+                shorthand      bin      scale   timeUnit   
+                aggregate      field    sort    title      
+                axis           impute   stack   type       
+                bandPosition                               
+
+                See the help for `X` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
         (
-            chart_example_invalid_y_option_value,
-            r"schema.channels.Y.*"
-            + r"'asdf' is not one of \['zero', 'center', 'normalize'\].*"
-            + r"'asdf' is not of type 'null'.*'asdf' is not of type 'boolean'",
+            chart_example_layer,
+            inspect.cleandoc(
+                r"""`VConcatChart` has no parameter named 'width'
+
+                Existing parameter names are:
+                vconcat      center     description   params    title       
+                autosize     config     name          resolve   transform   
+                background   data       padding       spacing   usermeta    
+                bounds       datasets                                       
+
+                See the help for `VConcatChart` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
         (
-            chart_example_layer,
-            r"api.VConcatChart.*"
-            + r"Additional properties are not allowed \('width' was unexpected\)",
+            chart_example_invalid_y_option_value,
+            inspect.cleandoc(
+                r"""'asdf' is an invalid value for `stack`:
+
+                'asdf' is not one of \['zero', 'center', 'normalize'\]
+                'asdf' is not of type 'null'
+                'asdf' is not of type 'boolean'"""
+            ),
         ),
         (
             chart_example_invalid_y_option_value_with_condition,
-            r"schema.channels.Y.*"
-            + r"'asdf' is not one of \['zero', 'center', 'normalize'\].*"
-            + r"'asdf' is not of type 'null'.*'asdf' is not of type 'boolean'",
+            inspect.cleandoc(
+                r"""'asdf' is an invalid value for `stack`:
+
+                'asdf' is not one of \['zero', 'center', 'normalize'\]
+                'asdf' is not of type 'null'
+                'asdf' is not of type 'boolean'"""
+            ),
         ),
         (
             chart_example_hconcat,
-            r"schema.core.TitleParams.*"
-            + r"\{'text': 'Horsepower', 'align': 'right'\} is not of type 'string'.*"
-            + r"\{'text': 'Horsepower', 'align': 'right'\} is not of type 'array'",
+            inspect.cleandoc(
+                r"""'{'text': 'Horsepower', 'align': 'right'}' is an invalid value for `title`:
+
+                {'text': 'Horsepower', 'align': 'right'} is not of type 'string'
+                {'text': 'Horsepower', 'align': 'right'} is not of type 'array'"""
+            ),
         ),
         (
             chart_example_invalid_channel_and_condition,
-            r"schema.core.Encoding->encoding.*"
-            + r"Additional properties are not allowed \('invalidChannel' was unexpected\)",
+            inspect.cleandoc(
+                r"""`Encoding` has no parameter named 'invalidChannel'
+
+                Existing parameter names are:
+                angle         key          order     strokeDash      tooltip   xOffset   
+                color         latitude     radius    strokeOpacity   url       y         
+                description   latitude2    radius2   strokeWidth     x         y2        
+                detail        longitude    shape     text            x2        yError    
+                fill          longitude2   size      theta           xError    yError2   
+                fillOpacity   opacity      stroke    theta2          xError2   yOffset   
+                href                                                                     
+
+                See the help for `Encoding` to read the full description of these parameters"""  # noqa: W291
+            ),
         ),
     ],
 )