Repository: spacetelescope/asdf (python)

Issue:
Improve tag version mismatch warning message
Seen in the `jwst` unit tests using `asdf 2.6.0`:

```
jwst/datamodels/tests/test_models.py::test_open_asdf_model
  /Users/jdavies/miniconda3/envs/jwst/lib/python3.8/site-packages/asdf/type_index.py:278: UserWarning: 'tag:stsci.edu:asdf/transform/concatenate' with version 1.1.0 found in file 'file:///Users/jdavies/dev/jwst/jwst/datamodels/tests/data/collimator_fake.asdf', but latest supported version is 1.2.0
```

I think what is happening is that the `CompoundType` in `astropy 4.0.1` is set to only work with `transform-1.2.0` but the file contains `transform-1.1.0`.  There's no `supported_versions` class attribute, so hence the warning message:

https://github.com/astropy/astropy/blob/b8e827810207a0ac5e12be67c94c9c2990e4967b/astropy/io/misc/asdf/tags/transform/compound.py#L40

But it clearly works do to automagical ?guessing? of a type to use (probably 1.2 provided by `astropy 4.0`, but the fact that `astropy` provides the serialization code is not mentioned at all in the warning message.

Also, if an older version of a file is read in using a slightly newer serialization standard (1.2.0 vs 1.1.0), I would definitely expect it to be backwards compatible.  The warnings seems to indicate that it is not.  But looking at the object that gets deserialized in the above test, it clearly works.  So why the warning?

So

1) improve the message so that it can be helpful for someone debugging

2) Make it clear what the problem is.

3) Maybe don't issue the warning message at all if it is going to work anyway.

Pull request:
Clean up version mismatch warning
This PR fixes a number of issues related to the version mismatch warning:

- Move the cache of previously emitted warnings from TypeIndex to AsdfFile, avoiding accumulation of AsdfFiles in memory
- Make the version mismatch warning message more useful by including the tag that was chosen as a fallback.  I also added a note stating that the fallback behavior is deprecated and will be removed in 3.0.
- Start warning even when only the schema patch version is different.  A patch version is likely to include new fields that would be discarded by an older ExtensionType implementation, so modifying the file may cause unintended data loss.
- Stop warning simply because a file contains a tag with a supported but older version.  That doesn't seem useful.
- Stop coercing tag versions in the YAML parser.  This is a significant change, and a behavior that we've had since 1.3.3, but I don't think it's correct.  If we leave the tag alone and ultimately decide that we can't interpret the associated custom object, then the tag will remain intact when we write the file back out.  Otherwise the file will end up corrupted to some extent, with a set of fields for one version of the schema but tagged with a different version.  I suspect this was originally done so that objects with unsupported patch versions would still be validated against the schema with the closest version, but I don't think that's a worthy cause.
- Add a config option to the pytest plugin to enable version mismatch warnings when testing schemas.  We can't enable it right now because astropy is missing support for so many older schema versions, but I checked the schemas with ExtensionType class in asdf and they're all compliant.

This is branched off of #804 so I'd recommend waiting to review until we've merged that one, since it introduces a lot of noise.

Resolves #801

Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 8e3f1723..809ddc44 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -17,6 +17,9 @@
 
 - All warnings now subclass ``asdf.exceptions.AsdfWarning``. [#804]
 
+- Improve warning message when falling back to an older schema,
+  and note that fallback behavior will be removed in 3.0. [#806]
+
 2.6.1 (unreleased)
 ------------------
 
diff --git a/asdf/asdf.py b/asdf/asdf.py
index f071887f..e6502cfa 100644
--- a/asdf/asdf.py
+++ b/asdf/asdf.py
@@ -24,7 +24,7 @@ from . import version
 from . import versioning
 from . import yamlutil
 from . import _display as display
-from .exceptions import AsdfDeprecationWarning, AsdfWarning
+from .exceptions import AsdfDeprecationWarning, AsdfWarning, AsdfConversionWarning
 from .extension import AsdfExtensionList, default_extensions
 from .util import NotSet
 from .search import AsdfSearchResult
@@ -123,6 +123,10 @@ class AsdfFile(versioning.VersionedMixin):
         self._ignore_unrecognized_tag = ignore_unrecognized_tag
         self._ignore_implicit_conversion = ignore_implicit_conversion
 
+        # Set of (string, string) tuples representing tag version mismatches
+        # that we've already warned about for this file.
+        self._warned_tag_pairs = set()
+
         self._file_format_version = None
 
         # Context of a call to treeutil.walk_and_modify, needed in the AsdfFile
@@ -656,7 +660,7 @@ class AsdfFile(versioning.VersionedMixin):
             # We parse the YAML content into basic data structures
             # now, but we don't do anything special with it until
             # after the blocks have been read
-            tree = yamlutil.load_tree(reader, self, self._ignore_version_mismatch)
+            tree = yamlutil.load_tree(reader)
             has_blocks = fd.seek_until(constants.BLOCK_MAGIC, 4, include=True)
         elif yaml_token == constants.BLOCK_MAGIC:
             has_blocks = True
@@ -1336,6 +1340,20 @@ class AsdfFile(versioning.VersionedMixin):
         result = AsdfSearchResult(["root.tree"], self.tree)
         return result.search(key=key, type=type, value=value, filter=filter)
 
+    # This function is called from within TypeIndex when deserializing
+    # the tree for this file.  It is kept here so that we can keep
+    # state on the AsdfFile and prevent a flood of warnings for the
+    # same tag.
+    def _warn_tag_mismatch(self, tag, best_tag):
+        if not self._ignore_version_mismatch and (tag, best_tag) not in self._warned_tag_pairs:
+            message = (
+                "No explicit ExtensionType support provided for tag '{}'. "
+                "The ExtensionType subclass for tag '{}' will be used instead. "
+                "This fallback behavior will be removed in asdf 3.0."
+            ).format(tag, best_tag)
+            warnings.warn(message, AsdfConversionWarning)
+            self._warned_tag_pairs.add((tag, best_tag))
+
 
 # Inherit docstring from dictionary
 AsdfFile.keys.__doc__ = dict.keys.__doc__
diff --git a/asdf/type_index.py b/asdf/type_index.py
index c2128952..6aa8b655 100644
--- a/asdf/type_index.py
+++ b/asdf/type_index.py
@@ -2,14 +2,12 @@
 # -*- coding: utf-8 -*-
 
 import bisect
-import warnings
 from functools import lru_cache
 from collections import OrderedDict
 
 from . import util
 from .versioning import (AsdfVersion, get_version_map, default_version,
                          split_tag_version, join_tag_version)
-from .exceptions import AsdfWarning
 
 
 __all__ = ['AsdfTypeIndex']
@@ -201,7 +199,6 @@ class AsdfTypeIndex:
         # the same type, the result is currently undefined.
         self._versions_by_type_name = OrderedDict()
         self._best_matches = {}
-        self._real_tag = {}
         self._unnamed_types = set()
         self._hooks_by_type = {}
         self._all_types = set()
@@ -258,28 +255,7 @@ class AsdfTypeIndex:
 
         return write_type_index.from_custom_type(custom_type)
 
-    def _get_version_mismatch(self, name, version, latest_version):
-        warning_string = None
-
-        if (latest_version.major, latest_version.minor) != \
-                (version.major, version.minor):
-            warning_string = \
-                "'{}' with version {} found in file{{}}, but latest " \
-                "supported version is {}".format(
-                    name, version, latest_version)
-
-        return warning_string
-
-    def _warn_version_mismatch(self, ctx, tag, warning_string, fname):
-        if warning_string is not None:
-            # Ensure that only a single warning occurs per tag per AsdfFile
-            # TODO: If it is useful to only have a single warning per file on
-            # disk, then use `fname` in the key instead of `ctx`.
-            if not (ctx, tag) in self._has_warned:
-                warnings.warn(warning_string.format(fname), AsdfWarning)
-                self._has_warned[(ctx, tag)] = True
-
-    def fix_yaml_tag(self, ctx, tag, ignore_version_mismatch=True):
+    def fix_yaml_tag(self, ctx, tag):
         """
         Given a YAML tag, adjust it to the best supported version.
 
@@ -287,38 +263,17 @@ class AsdfTypeIndex:
         understood that is still less than the version in file.  Or,
         the earliest understood version if none are less than the
         version in the file.
-
-        If ``ignore_version_mismatch==False``, this function raises a warning
-        if it could not find a match where the major and minor numbers are the
-        same.
         """
-        warning_string = None
-
-        name, version = split_tag_version(tag)
-
-        fname = " '{}'".format(ctx._fname) if ctx._fname else ''
-
         if tag in self._type_by_tag:
-            asdftype = self._type_by_tag[tag]
-            # Issue warnings for the case where there exists a class for the
-            # given tag due to the 'supported_versions' attribute being
-            # defined, but this tag is not the latest version of the type.
-            # This prevents 'supported_versions' from affecting the behavior of
-            # warnings that are purely related to YAML validation.
-            if not ignore_version_mismatch and hasattr(asdftype, '_latest_version'):
-                warning_string = self._get_version_mismatch(
-                    name, version, asdftype._latest_version)
-                self._warn_version_mismatch(ctx, tag, warning_string, fname)
             return tag
 
         if tag in self._best_matches:
-            best_tag, warning_string = self._best_matches[tag]
-
-            if not ignore_version_mismatch:
-                self._warn_version_mismatch(ctx, tag, warning_string, fname)
-
+            best_tag = self._best_matches[tag]
+            ctx._warn_tag_mismatch(tag, best_tag)
             return best_tag
 
+        name, version = split_tag_version(tag)
+
         versions = self._versions_by_type_name.get(name)
         if versions is None:
             return tag
@@ -327,26 +282,12 @@ class AsdfTypeIndex:
         # quickly find the best option.
         i = bisect.bisect_left(versions, version)
         i = max(0, i - 1)
-
-        if not ignore_version_mismatch:
-            warning_string = self._get_version_mismatch(
-                name, version, versions[-1])
-            self._warn_version_mismatch(ctx, tag, warning_string, fname)
-
         best_version = versions[i]
         best_tag = join_tag_version(name, best_version)
-        self._best_matches[tag] = best_tag, warning_string
-        if tag != best_tag:
-            self._real_tag[best_tag] = tag
+        ctx._warn_tag_mismatch(tag, best_tag)
+        self._best_matches[tag] = best_tag
         return best_tag
 
-    def get_real_tag(self, tag):
-        if tag in self._real_tag:
-            return self._real_tag[tag]
-        elif tag in self._type_by_tag:
-            return tag
-        return None
-
     def from_yaml_tag(self, ctx, tag):
         """
         From a given YAML tag string, return the corresponding
diff --git a/asdf/yamlutil.py b/asdf/yamlutil.py
index f4442619..d3d600c4 100644
--- a/asdf/yamlutil.py
+++ b/asdf/yamlutil.py
@@ -174,17 +174,17 @@ class AsdfLoader(_yaml_base_loader):
             return super().construct_undefined(node)
 
     def _construct_tagged_mapping(self, node):
-        data = tagged.tag_object(self._fix_tag(node), {})
+        data = tagged.tag_object(node.tag, {})
         yield data
         data.update(self.construct_mapping(node))
 
     def _construct_tagged_sequence(self, node):
-        data = tagged.tag_object(self._fix_tag(node), [])
+        data = tagged.tag_object(node.tag, [])
         yield data
         data.extend(self.construct_sequence(node))
 
     def _construct_tagged_scalar(self, node):
-        return tagged.tag_object(self._fix_tag(node), self.construct_scalar(node))
+        return tagged.tag_object(node.tag, self.construct_scalar(node))
 
     # Custom omap deserializer that builds an OrderedDict instead
     # of a list of tuples.  Code is mostly identical to pyyaml's SafeConstructor.
@@ -208,10 +208,6 @@ class AsdfLoader(_yaml_base_loader):
             value = self.construct_object(value_node)
             omap[key] = value
 
-    def _fix_tag(self, node):
-        return self.ctx.type_index.fix_yaml_tag(
-            self.ctx, node.tag, self.ignore_version_mismatch)
-
 
 # pyyaml will invoke the constructor associated with None when a node's
 # tag is not explicitly handled by another constructor.
@@ -251,27 +247,26 @@ def tagged_tree_to_custom_tree(tree, ctx, force_raw_types=False):
         if force_raw_types:
             return node
 
-        tag_name = getattr(node, '_tag', None)
-        if tag_name is None:
+        tag = getattr(node, '_tag', None)
+        if tag is None:
             return node
 
-        tag_type = ctx.type_index.from_yaml_tag(ctx, tag_name)
+        tag_type = ctx.type_index.from_yaml_tag(ctx, tag)
         # This means the tag did not correspond to any type in our type index.
         if tag_type is None:
             if not ctx._ignore_unrecognized_tag:
                 warnings.warn("{} is not recognized, converting to raw Python "
-                    "data structure".format(tag_name), AsdfConversionWarning)
+                    "data structure".format(tag), AsdfConversionWarning)
             return node
 
-        real_tag = ctx.type_index.get_real_tag(tag_name)
-        real_tag_name, real_tag_version = split_tag_version(real_tag)
+        tag_name, tag_version = split_tag_version(tag)
         # This means that there is an explicit description of versions that are
         # compatible with the associated tag class implementation, but the
         # version we found does not fit that description.
-        if tag_type.incompatible_version(real_tag_version):
+        if tag_type.incompatible_version(tag_version):
             warnings.warn("Version {} of {} is not compatible with any "
                 "existing tag implementations".format(
-                    real_tag_version, real_tag_name),
+                    tag_version, tag_name),
                 AsdfConversionWarning)
             return node
 
@@ -284,7 +279,7 @@ def tagged_tree_to_custom_tree(tree, ctx, force_raw_types=False):
             return tag_type.from_tree_tagged(node, ctx)
         except TypeError as err:
             warnings.warn("Failed to convert {} to custom type (detail: {}). "
-                "Using raw Python data structure instead".format(real_tag, err),
+                "Using raw Python data structure instead".format(tag, err),
                 AsdfConversionWarning)
 
         return node
@@ -300,7 +295,7 @@ def tagged_tree_to_custom_tree(tree, ctx, force_raw_types=False):
     )
 
 
-def load_tree(stream, ctx, ignore_version_mismatch=False):
+def load_tree(stream):
     """
     Load YAML, returning a tree of objects.
 
@@ -309,12 +304,7 @@ def load_tree(stream, ctx, ignore_version_mismatch=False):
     stream : readable file-like object
         Stream containing the raw YAML content.
     """
-    class AsdfLoaderTmp(AsdfLoader):
-        pass
-    AsdfLoaderTmp.ctx = ctx
-    AsdfLoaderTmp.ignore_version_mismatch = ignore_version_mismatch
-
-    return yaml.load(stream, Loader=AsdfLoaderTmp)
+    return yaml.load(stream, Loader=AsdfLoader)
 
 
 def dump_tree(tree, fd, ctx):


Test patch:
diff --git a/asdf/tests/helpers.py b/asdf/tests/helpers.py
index 08fa6a8d..142833fa 100644
--- a/asdf/tests/helpers.py
+++ b/asdf/tests/helpers.py
@@ -369,15 +369,26 @@ def display_warnings(_warnings):
 
 
 @contextmanager
-def assert_no_warnings():
+def assert_no_warnings(warning_class=None):
     """
     Assert that no warnings were emitted within the context.
     Requires that pytest be installed.
+
+    Parameters
+    ----------
+    warning_class : type, optional
+        Assert only that no warnings of the specified class were
+        emitted.
     """
     import pytest
     with pytest.warns(None) as recorded_warnings:
         yield
-    assert len(recorded_warnings) == 0, display_warnings(recorded_warnings)
+
+    if warning_class is not None:
+        assert not any(isinstance(w.message, warning_class) for w in recorded_warnings), \
+            display_warnings(recorded_warnings)
+    else:
+        assert len(recorded_warnings) == 0, display_warnings(recorded_warnings)
 
 
 def assert_extension_correctness(extension):
diff --git a/asdf/tests/test_fits_embed.py b/asdf/tests/test_fits_embed.py
index 23b9fedd..d4ecb76e 100644
--- a/asdf/tests/test_fits_embed.py
+++ b/asdf/tests/test_fits_embed.py
@@ -18,11 +18,10 @@ from jsonschema.exceptions import ValidationError
 import asdf
 from asdf import fits_embed
 from asdf import open as asdf_open
-from asdf.exceptions import AsdfWarning
+from asdf.exceptions import AsdfWarning, AsdfConversionWarning
 
 from .helpers import (
     assert_tree_match,
-    display_warnings,
     get_test_data_path,
     yaml_to_asdf,
     assert_no_warnings,
@@ -344,49 +343,25 @@ def test_bad_input(tmpdir):
         asdf_open(text_file)
 
 def test_version_mismatch_file():
-
     testfile = str(get_test_data_path('version_mismatch.fits'))
 
-    with pytest.warns(None) as w:
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with asdf.open(testfile,
                 ignore_version_mismatch=False) as fits_handle:
             assert fits_handle.tree['a'] == complex(0j)
-    # This is the warning that we expect from opening the FITS file
-    expected_messages = {
-        (
-            "'tag:stsci.edu:asdf/core/complex' with version 7.0.0 found in file "
-            "'{}', but latest supported version is 1.0.0".format(testfile)
-        ),
-        (
-            "'tag:stsci.edu:asdf/core/asdf' with version 1.0.0 found in file "
-            "'{}', but latest supported version is 1.1.0".format(testfile)
-        ),
-    }
-    assert expected_messages == {warn.message.args[0] for warn in w}, display_warnings(w)
 
     # Make sure warning does not occur when warning is ignored (default)
-    with assert_no_warnings():
+    with assert_no_warnings(AsdfConversionWarning):
         with asdf.open(testfile) as fits_handle:
             assert fits_handle.tree['a'] == complex(0j)
 
-    with pytest.warns(None) as w:
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with fits_embed.AsdfInFits.open(testfile,
                 ignore_version_mismatch=False) as fits_handle:
             assert fits_handle.tree['a'] == complex(0j)
-    expected_messages = {
-        (
-            "'tag:stsci.edu:asdf/core/complex' with version 7.0.0 found in file "
-            "'{}', but latest supported version is 1.0.0".format(testfile)
-        ),
-        (
-            "'tag:stsci.edu:asdf/core/asdf' with version 1.0.0 found in file "
-            "'{}', but latest supported version is 1.1.0".format(testfile)
-        ),
-    }
-    assert expected_messages == {warn.message.args[0] for warn in w}, display_warnings(w)
 
     # Make sure warning does not occur when warning is ignored (default)
-    with assert_no_warnings():
+    with assert_no_warnings(AsdfConversionWarning):
         with fits_embed.AsdfInFits.open(testfile) as fits_handle:
             assert fits_handle.tree['a'] == complex(0j)
 
diff --git a/asdf/tests/test_schema.py b/asdf/tests/test_schema.py
index b78d1655..e146b576 100644
--- a/asdf/tests/test_schema.py
+++ b/asdf/tests/test_schema.py
@@ -610,7 +610,7 @@ custom: !<tag:nowhere.org:custom/missing-1.1.0>
   b: {foo: 42}
     """
     buff = helpers.yaml_to_asdf(yaml)
-    with pytest.warns(AsdfWarning, match="Failed to convert tag:nowhere.org:custom/missing-1.1.0"):
+    with pytest.warns(AsdfConversionWarning, match="Failed to convert tag:nowhere.org:custom/missing-1.1.0"):
         with asdf.open(buff, extensions=[DefaultTypeExtension()]) as ff:
             assert ff.tree['custom']['b']['foo'] == 42
 
diff --git a/asdf/tests/test_types.py b/asdf/tests/test_types.py
index cea2930c..a6f82358 100644
--- a/asdf/tests/test_types.py
+++ b/asdf/tests/test_types.py
@@ -180,30 +180,31 @@ a: !core/complex-42.0.0
     """
 
     buff = helpers.yaml_to_asdf(yaml)
-    with pytest.warns(AsdfWarning, match="tag:stsci.edu:asdf/core/complex"):
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with asdf.open(buff, ignore_version_mismatch=False) as ff:
             assert isinstance(ff.tree['a'], complex)
 
     # Make sure warning is repeatable
     buff.seek(0)
-    with pytest.warns(AsdfWarning, match="tag:stsci.edu:asdf/core/complex"):
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with asdf.open(buff, ignore_version_mismatch=False) as ff:
             assert isinstance(ff.tree['a'], complex)
 
     # Make sure the warning does not occur if it is being ignored (default)
     buff.seek(0)
-    with helpers.assert_no_warnings():
+    with helpers.assert_no_warnings(AsdfConversionWarning):
         with asdf.open(buff) as ff:
             assert isinstance(ff.tree['a'], complex)
 
-    # If the major and minor match, there should be no warning.
+    # If the major and minor match, but the patch doesn't, there
+    # should still be a warning.
     yaml = """
 a: !core/complex-1.0.1
   0j
     """
 
     buff = helpers.yaml_to_asdf(yaml)
-    with helpers.assert_no_warnings():
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with asdf.open(buff, ignore_version_mismatch=False) as ff:
             assert isinstance(ff.tree['a'], complex)
 
@@ -221,15 +222,15 @@ a: !core/complex-42.0.0
 
     expected_uri = util.filepath_to_url(str(testfile))
 
-    with pytest.warns(AsdfWarning, match="tag:stsci.edu:asdf/core/complex"):
+    with pytest.warns(AsdfConversionWarning, match="tag:stsci.edu:asdf/core/complex"):
         with asdf.open(testfile, ignore_version_mismatch=False) as ff:
             assert ff._fname == expected_uri
             assert isinstance(ff.tree['a'], complex)
 
 
 def test_version_mismatch_with_supported_versions():
-    """Make sure that defining the supported_versions field does not affect
-    whether or not schema mismatch warnings are triggered."""
+    """Make sure that defining the supported_versions field eliminates
+    the schema mismatch warning."""
 
     class CustomFlow:
         pass
@@ -254,7 +255,7 @@ flow_thing:
     d: 3.14
 """
     buff = helpers.yaml_to_asdf(yaml)
-    with pytest.warns(AsdfWarning, match="tag:nowhere.org:custom/custom_flow"):
+    with helpers.assert_no_warnings():
         asdf.open(buff, ignore_version_mismatch=False,
             extensions=CustomFlowExtension())
 
diff --git a/asdf/tests/test_yaml.py b/asdf/tests/test_yaml.py
index 42006167..73dab49b 100644
--- a/asdf/tests/test_yaml.py
+++ b/asdf/tests/test_yaml.py
@@ -307,6 +307,6 @@ def test_numpy_scalar(numpy_value, expected_value):
     buffer.seek(0)
 
     if isinstance(expected_value, float) and NUMPY_LT_1_14:
-        assert yamlutil.load_tree(buffer, ctx)["value"] == pytest.approx(expected_value, rel=0.001)
+        assert yamlutil.load_tree(buffer)["value"] == pytest.approx(expected_value, rel=0.001)
     else:
-        assert yamlutil.load_tree(buffer, ctx)["value"] == expected_value
+        assert yamlutil.load_tree(buffer)["value"] == expected_value
diff --git a/pytest_asdf/plugin.py b/pytest_asdf/plugin.py
index 8274acc4..d72f5a28 100644
--- a/pytest_asdf/plugin.py
+++ b/pytest_asdf/plugin.py
@@ -33,13 +33,20 @@ def pytest_addoption(parser):
         type="bool",
         default=False,
     )
+    parser.addini(
+        "asdf_schema_ignore_version_mismatch",
+        "Set to true to disable warnings when missing explicit support for a tag",
+        type="bool",
+        default=True
+    )
     parser.addoption('--asdf-tests', action='store_true',
         help='Enable ASDF schema tests')
 
 
 class AsdfSchemaFile(pytest.File):
     @classmethod
-    def from_parent(cls, parent, *, fspath, skip_examples=False, ignore_unrecognized_tag=False, **kwargs):
+    def from_parent(cls, parent, *, fspath, skip_examples=False,
+        ignore_unrecognized_tag=False, ignore_version_mismatch=False, **kwargs):
         if hasattr(super(), "from_parent"):
             result = super().from_parent(parent, fspath=fspath, **kwargs)
         else:
@@ -47,6 +54,7 @@ class AsdfSchemaFile(pytest.File):
 
         result.skip_examples = skip_examples
         result.ignore_unrecognized_tag = ignore_unrecognized_tag
+        result.ignore_version_mismatch = ignore_version_mismatch
         return result
 
     def collect(self):
@@ -57,7 +65,8 @@ class AsdfSchemaFile(pytest.File):
                     self,
                     self.fspath,
                     example,
-                    ignore_unrecognized_tag=self.ignore_unrecognized_tag
+                    ignore_unrecognized_tag=self.ignore_unrecognized_tag,
+                    ignore_version_mismatch=self.ignore_version_mismatch,
                 )
 
     def find_examples_in_schema(self):
@@ -134,7 +143,8 @@ def parse_schema_filename(filename):
 
 class AsdfSchemaExampleItem(pytest.Item):
     @classmethod
-    def from_parent(cls, parent, schema_path, example, ignore_unrecognized_tag=False, **kwargs):
+    def from_parent(cls, parent, schema_path, example,
+        ignore_unrecognized_tag=False, ignore_version_mismatch=False, **kwargs):
         test_name = "{}-example".format(schema_path)
         if hasattr(super(), "from_parent"):
             result = super().from_parent(parent, name=test_name, **kwargs)
@@ -143,6 +153,7 @@ class AsdfSchemaExampleItem(pytest.Item):
         result.filename = str(schema_path)
         result.example = example
         result.ignore_unrecognized_tag = ignore_unrecognized_tag
+        result.ignore_version_mismatch = ignore_version_mismatch
         return result
 
     def _find_standard_version(self, name, version):
@@ -168,9 +179,11 @@ class AsdfSchemaExampleItem(pytest.Item):
         # ASDF standard document) are valid.
         buff = helpers.yaml_to_asdf(
             'example: ' + self.example.strip(), standard_version=standard_version)
+
         ff = AsdfFile(
             uri=util.filepath_to_url(os.path.abspath(self.filename)),
             ignore_unrecognized_tag=self.ignore_unrecognized_tag,
+            ignore_version_mismatch=self.ignore_version_mismatch,
         )
 
         # Fake an external file
@@ -218,6 +231,7 @@ def pytest_collect_file(path, parent):
     skip_names = parent.config.getini('asdf_schema_skip_names')
     skip_examples = parent.config.getini('asdf_schema_skip_examples')
     ignore_unrecognized_tag = parent.config.getini('asdf_schema_ignore_unrecognized_tag')
+    ignore_version_mismatch = parent.config.getini('asdf_schema_ignore_version_mismatch')
 
     schema_roots = [os.path.join(str(parent.config.rootdir), os.path.normpath(root))
                         for root in schema_roots]
@@ -232,6 +246,7 @@ def pytest_collect_file(path, parent):
                 fspath=path,
                 skip_examples=(path.purebasename in skip_examples),
                 ignore_unrecognized_tag=ignore_unrecognized_tag,
+                ignore_version_mismatch=ignore_version_mismatch,
             )
 
     return None