Repository: vega/altair

Issue:
New version of jsonschema crash transform_fold
The snippet below works fine with jsonschema 3.20, but crash with jsonschema 4.0.1:

```python
import pandas as pd
from io import StringIO

df = pd.read_table(StringIO("""
TVCL      TVV
0.005298  1.30730
0.006048  1.38692
0.005624  1.53224
0.005771  1.40115
0.005932  1.42119
"""), sep='\s+')

import altair as alt

base = (
    alt.Chart(df)
    .transform_fold(list(df.columns), as_=['key_x', 'value_x'])
)
```

this is the error:

```
Traceback (most recent call last):

  File "<ipython-input-8-50da994c20c5>", line 2, in <module>
    alt.Chart(df)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/vegalite/v4/api.py", line 1159, in transform_fold
    return self._add_transform(core.FoldTransform(fold=fold, **{"as": as_}))

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/vegalite/v4/schema/core.py", line 18192, in __init__
    super(FoldTransform, self).__init__(fold=fold, **kwds)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/vegalite/v4/schema/core.py", line 17975, in __init__
    super(Transform, self).__init__(*args, **kwds)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/utils/schemapi.py", line 176, in __init__
    self.to_dict(validate=True)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/utils/schemapi.py", line 337, in to_dict
    self.validate(result)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/altair/utils/schemapi.py", line 442, in validate
    return jsonschema.validate(instance, schema, resolver=resolver)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 965, in validate
    error = exceptions.best_match(validator.iter_errors(instance))

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/exceptions.py", line 354, in best_match
    best = next(errors, None)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 224, in iter_errors
    for error in errors:

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/_validators.py", line 295, in ref
    yield from validator.descend(instance, resolved)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 224, in iter_errors
    for error in errors:

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/_validators.py", line 329, in properties
    yield from validator.descend(

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 224, in iter_errors
    for error in errors:

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/_validators.py", line 76, in items
    yield from validator.descend(

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 240, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 214, in iter_errors
    scope = id_of(_schema)

  File "/home/devel/pharmpy/.tox/py39/lib/python3.9/site-packages/jsonschema/validators.py", line 90, in _id_of
    return schema.get("$id", "")

AttributeError: 'list' object has no attribute 'get'
```


Discussion:
Thanks for the report! It looks like jsonschema 4.0 is incompatible with the schema format produced by Vega-Lite. Here is a minimal example of a schema that works with jsonschema 3.X, but fails with jsonschema 4.0.1:
```python
import jsonschema
instance = ['A', 'B']
schema = {
  "type": "array",
  "items": [{"type": "string"}, {"type": "string"}]
}
jsonschema.validate(instance, schema)
```
I'm not sure whether this is a bug in jsonschema, or a reflection of some documented change to defaults in the jsonschema package. But for the time being, I'd suggest using jsonschema 3.X with the Altair package.
Thanks for having a look. I filed an issue upstream: https://github.com/Julian/jsonschema/issues/852
(Hello, jsonschema author here, I see the issue I just closed came originally from altair, so just checking in)

I don't see in a quick grep where in altair these schemas are assembled otherwise I'd be happy to help, but yeah the difference you notice is because the schema isn't [setting $schema to declare what version it's written for](https://json-schema.org/understanding-json-schema/reference/schema.html#schema). So in 3.2.0, you're getting a default version (draft 7) and in 4.0 you're still getting a default version (now draft 2020-12, where what you have there is [now spelled using `prefixItems`](https://json-schema.org/draft/2020-12/release-notes.html)). Definitely worth considering making sure you have `$schema` present, even if you intend to stick with draft 7 for now.

```python
schema = {
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "array",
  "items": [{"type": "string"}, {"type": "string"}]
}
```

would have consistent behavior across versions (as would using `jsonschema.validators.Draft7Validator` directly).

Hope that helps.
Thanks for the info – we do set the schema version explicitly: https://github.com/altair-viz/altair/blob/8ff06cfbbfb470aba6798999c61128be398c4996/altair/vegalite/v4/schema/vega-lite-schema.json#L3

I suspect there must be some kind of issue with the way we call the validation API, which is here: https://github.com/altair-viz/altair/blob/8ff06cfbbfb470aba6798999c61128be398c4996/altair/utils/schemapi.py#L434-L442

Perhaps relevant, the `$schema` is set within the `resolver` schema, rather than the schema passed to `validate`.
@Julian should I expect jsonschema to recognize the schema version set within the resolver?
No! Definitely the one you're passing to `validate` is the one that matters -- the one passed to the resolver is just a shorthand way to cache the schema and set the base URI essentially, but it only affects ref resolution behavior.

I meant to leave some tips but have been swamped, but yeah definitely that's related to the issue.

From the code you linked I saw altair may want to instead be setting instances of [Draft7Validator](https://python-jsonschema.readthedocs.io/en/stable/validate/#jsonschema.Draft7Validator) on the classes, something like `Draft7Validator(_root_schema)` rather than calling `jsonschema.validate` from scratch -- since you're doing so via subclassing you probably will need `__init_subclass__` to do that.

But yeah apologies if the above is quite abstract and rushed, as I say I'm unfortunately still underwater with some other things so haven't gotten around to doing anything more here -- at very least hopefully the first part helps, if you set `$schema` in the schema you pass to validate that should unblock things.

Let me know if that helps, questions are of course easy to answer.

Resolving patch:
diff --git a/altair/utils/schemapi.py b/altair/utils/schemapi.py
--- a/altair/utils/schemapi.py
+++ b/altair/utils/schemapi.py
@@ -150,6 +150,7 @@ class SchemaBase(object):
     _schema = None
     _rootschema = None
     _class_is_valid_at_instantiation = True
+    _validator = jsonschema.Draft7Validator
 
     def __init__(self, *args, **kwds):
         # Two valid options for initialization, which should be handled by
@@ -439,7 +440,9 @@ def validate(cls, instance, schema=None):
         if schema is None:
             schema = cls._schema
         resolver = jsonschema.RefResolver.from_schema(cls._rootschema or cls._schema)
-        return jsonschema.validate(instance, schema, resolver=resolver)
+        return jsonschema.validate(
+            instance, schema, cls=cls._validator, resolver=resolver
+        )
 
     @classmethod
     def resolve_references(cls, schema=None):
diff --git a/tools/schemapi/schemapi.py b/tools/schemapi/schemapi.py
--- a/tools/schemapi/schemapi.py
+++ b/tools/schemapi/schemapi.py
@@ -148,6 +148,7 @@ class SchemaBase(object):
     _schema = None
     _rootschema = None
     _class_is_valid_at_instantiation = True
+    _validator = jsonschema.Draft7Validator
 
     def __init__(self, *args, **kwds):
         # Two valid options for initialization, which should be handled by
@@ -437,7 +438,9 @@ def validate(cls, instance, schema=None):
         if schema is None:
             schema = cls._schema
         resolver = jsonschema.RefResolver.from_schema(cls._rootschema or cls._schema)
-        return jsonschema.validate(instance, schema, resolver=resolver)
+        return jsonschema.validate(
+            instance, schema, cls=cls._validator, resolver=resolver
+        )
 
     @classmethod
     def resolve_references(cls, schema=None):


Test patch: