Repository: networknt/json-schema-validator
Pull request #682 (2023-03-26 21:43:29)

Adds support for translating one URI into another
issue #677 

Adds URITranslator as a FunctionalInterface
Adds MappingURITranslator to handle compatibility with specifying a Map
Adds CompositeURITranslator to handle combining the values specified in SchemaValidatorsConfig with those specified in JsonSchemaFactory.

stevehu: @fdutton It looks good to me. Could you please add a document on how to use this translator? Thanks.

Patch:
diff --git a/README.md b/README.md
index 3a45fff..35e5a83 100644
--- a/README.md
+++ b/README.md
@@ -127,6 +127,8 @@ For the latest version, please check the [release](https://github.com/networknt/
 
 ## [Multiple Language](doc/multiple-language.md)
 
+## [MetaSchema Validation](doc/metaschema-validation.md)
+
 
 ## Known issues
 
diff --git a/doc/metaschema-validation.md b/doc/metaschema-validation.md
new file mode 100644
index 0000000..5c49ddf
--- /dev/null
+++ b/doc/metaschema-validation.md
@@ -0,0 +1,20 @@
+If you have an use case to validate custom schemas against the one of the JSON schema draft version, here is the code that you can do it. 
+
+```
+  public static final Function<ObjectNode, Set<SchemaValidationMessage>> validateAgainstMetaSchema =
+      schema -> {
+        JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909);
+         JsonSchema metaSchema = factory.getSchema(getSchemaUri());
+        return metaSchema.validate(schema).stream()
+            .map((validation) -> new SchemaValidationMessage(validation.getMessage()))
+            .collect(Collectors.toSet());
+      };
+
+```
+
+This should now work but does not support all the keywords because the JsonMetaSchema of SpecVersion.VersionFlag.V201909 is lacking these features.
+
+You can fix the issue by resolving the vocabularies to a local resource file and re-do the JsonMetaSchema for 2019 based on that. 
+
+
+
diff --git a/doc/schema-map.md b/doc/schema-map.md
index 02ca2a1..beb4559 100644
--- a/doc/schema-map.md
+++ b/doc/schema-map.md
@@ -1,7 +1,8 @@
 While working with JSON schema validation, we have to use external references sometimes. However, there are two issues to have references to schemas on the Internet. 
 
 * Some applications are running inside a corporate network without Internet access. 
-* Some of the Internet resources are not reliable
+* Some of the Internet resources are not reliable.
+* A test environment may serve unpublished schemas, which are not yet available at the location identified by the payload's `$schema` property.
 
 One solution is to change all the external reference to internal in JSON schemas, but this is error-prone and hard to maintain in a long run. 
 
@@ -9,16 +10,132 @@ A smart solution is to map the external references to internal ones in a configu
 
 Note that when using a mapping, the local copy is always used, and the external reference is not queried.
 
-### Usage
+### URI Translation
 
-Basically, you can specify a mapping in the builder. For more details, please take a look at the test cases and the [PR](https://github.com/networknt/json-schema-validator/pull/125). 
+Both `SchemaValidatorsConfig` and `JsonSchemaFactory` accept one or more `URITranslator` instances. A `URITranslator` is responsible for providing a new URI when the given URI matches certain criteria.
 
+#### Examples
 
-### Real Example
+Automatically map HTTP to HTTPS
 
-https://github.com/JMRI/JMRI/blob/master/java/src/jmri/server/json/schema-map.json
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if ("http".equalsIgnoreCase(uri.getScheme()) {
+        try {
+            return new URI(
+                "https",
+                uri.getUserInfo(),
+                uri.getHost(),
+                uri.getPort(),
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+```
 
-In case you provide the schema through an `InputStream` or a `String` to resolve `$ref` with URN (relative path), you need to provide the `URNFactory` to the `JsonSchemaFactory.Builder.
-URNFactory` interface will allow you to resolve URN to URI.
+Map a public schema to a test environment
+
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if (true
+        && "https".equalsIgnoreCase(uri.getScheme()
+        && "schemas.acme.org".equalsIgnoreCase(uri.getHost())
+        && (-1 == uri.getPort() || 443 == uri.getPort())
+    ) {
+        try {
+            return new URI(
+                "http",
+                uri.getUserInfo(),
+                "test-schemas.acme.org",
+                8080,
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+```
+
+Replace a URI with another
+
+**Note:**
+This also works for mapping URNs to resources.
+
+```java
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(URITranslator.map("https://schemas.acme.org/Foo", "classpath://Foo");
+```
+
+### Precedence
+
+Both `SchemaValidatorsConfig` and `JsonSchemaFactory` accept multiple `URITranslator`s and in general, they are evaluated in the order of addition. This means that each `URITranslator` receives the output of the previous translator. For example, assuming the following configuration:
+
+```
+SchemaValidatorsConfig config = new SchemaValidatorsConfig();
+config.addUriTranlator(uri -> {
+    if ("http".equalsIgnoreCase(uri.getScheme()) {
+        try {
+            return new URI(
+                "https",
+                uri.getUserInfo(),
+                uri.getHost(),
+                uri.getPort(),
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+config.addUriTranlator(uri -> {
+    if (true
+        && "https".equalsIgnoreCase(uri.getScheme()
+        && "schemas.acme.org".equalsIgnoreCase(uri.getHost())
+        && (-1 == uri.getPort() || 443 == uri.getPort())
+    ) {
+        try {
+            return new URI(
+                "http",
+                uri.getUserInfo(),
+                "test-schemas.acme.org",
+                8080,
+                uri.getPath(),
+                uri.getQuery(),
+                uri.getFragment()
+            );
+        } catch (URISyntaxException x) {
+            throw new IllegalArgumentException(x.getMessage(), x);
+        }
+    }
+    return uri;
+});
+config.addUriTranlator(URITranslator.map("http://test-schemas.acme.org:8080/Foo", "classpath://Foo");
+```
+
+Given a starting URI of `https://schemas.acme.org/Foo`, the configuration above produces the following translations (in order):
+
+1. The translation from HTTP to HTTPS does not occur since the original URI already specifies HTTPS.
+2. The second rule receives the original URI since nothing happened in the first rule. The second rule translates the URI from `https://schemas.acme.org/Foo` to `http://test-schemas.acme.org:8080/Foo` since the scheme, host and port match this rule.
+3. The third rule receives the URI produced by the second rule and performs a simple mapping to a local resource.
+
+Since all `JsonSchemaFactory`s are created from an optional `SchemaValidatorsConfig`, any `URITranslator`s added to the factory are evaluated after those provided by `SchemaValidatorsConfig`.
+
+### Deprecated
+
+Previously, this library supported simple mappings from one URI to another through `SchemaValidatorsConfig.setUriMappings()` and `JsonSchemaFactory.addUriMappings()`. Usage of these methods are still supported but are now discouraged. `URITranslator` provides a more powerful mechanism of dealing with URI mapping than what was provided before.
 
-please take a look at the test cases and the [PR](https://github.com/networknt/json-schema-validator/pull/274).
diff --git a/src/main/java/com/networknt/schema/JsonSchemaFactory.java b/src/main/java/com/networknt/schema/JsonSchemaFactory.java
index 1fd72f7..d49b79d 100644
--- a/src/main/java/com/networknt/schema/JsonSchemaFactory.java
+++ b/src/main/java/com/networknt/schema/JsonSchemaFactory.java
@@ -20,6 +20,7 @@ import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
 import com.networknt.schema.uri.*;
+import com.networknt.schema.uri.URITranslator.CompositeURITranslator;
 import com.networknt.schema.urn.URNFactory;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -52,6 +53,7 @@ public class JsonSchemaFactory {
         private boolean forceHttps = true;
         private boolean removeEmptyFragmentSuffix = true;
         private boolean enableUriSchemaCache = true;
+        private final CompositeURITranslator uriTranslators = new CompositeURITranslator();
 
         public Builder() {
             // Adds support for creating {@link URL}s.
@@ -138,11 +140,22 @@ public class JsonSchemaFactory {
             return this;
         }
 
+        /**
+         * @deprecated Use {@code addUriTranslator} instead.
+         */
+        @Deprecated
         public Builder addUriMappings(final Map<String, String> map) {
             this.uriMap.putAll(map);
             return this;
         }
 
+        public Builder addUriTranslator(URITranslator translator) {
+            if (null != translator) {
+                this.uriTranslators.add(translator);
+            }
+            return this;
+        }
+
         public Builder addUrnFactory(URNFactory urnFactory) {
             this.urnFactory = urnFactory;
             return this;
@@ -176,7 +189,8 @@ public class JsonSchemaFactory {
                     uriMap,
                     forceHttps,
                     removeEmptyFragmentSuffix,
-                    enableUriSchemaCache
+                    enableUriSchemaCache,
+                    uriTranslators
             );
         }
     }
@@ -186,6 +200,7 @@ public class JsonSchemaFactory {
     private final String defaultMetaSchemaURI;
     private final URISchemeFactory uriFactory;
     private final URISchemeFetcher uriFetcher;
+    private final CompositeURITranslator uriTranslators;
     private final URNFactory urnFactory;
     private final Map<String, JsonMetaSchema> jsonMetaSchemas;
     private final Map<String, String> uriMap;
@@ -206,7 +221,8 @@ public class JsonSchemaFactory {
             final Map<String, String> uriMap,
             final boolean forceHttps,
             final boolean removeEmptyFragmentSuffix,
-            final boolean enableUriSchemaCache) {
+            final boolean enableUriSchemaCache,
+            final CompositeURITranslator uriTranslators) {
         if (jsonMapper == null) {
             throw new IllegalArgumentException("ObjectMapper must not be null");
         } else if (yamlMapper == null) {
@@ -223,6 +239,8 @@ public class JsonSchemaFactory {
             throw new IllegalArgumentException("Meta Schema for default Meta Schema URI must be provided");
         } else if (uriMap == null) {
             throw new IllegalArgumentException("URL Mappings must not be null");
+        } else if (uriTranslators == null) {
+            throw new IllegalArgumentException("URI Translators must not be null");
         }
         this.jsonMapper = jsonMapper;
         this.yamlMapper = yamlMapper;
@@ -235,6 +253,7 @@ public class JsonSchemaFactory {
         this.forceHttps = forceHttps;
         this.removeEmptyFragmentSuffix = removeEmptyFragmentSuffix;
         this.enableUriSchemaCache = enableUriSchemaCache;
+        this.uriTranslators = uriTranslators;
     }
 
     /**
@@ -301,6 +320,9 @@ public class JsonSchemaFactory {
                 .yamlMapper(blueprint.yamlMapper)
                 .addUriMappings(blueprint.uriMap);
 
+        for (URITranslator translator: blueprint.uriTranslators) {
+            builder = builder.addUriTranslator(translator);
+        }
         for (Map.Entry<String, URIFactory> entry : blueprint.uriFactory.getURIFactories().entrySet()) {
             builder = builder.uriFactory(entry.getValue(), entry.getKey());
         }
@@ -341,6 +363,10 @@ public class JsonSchemaFactory {
         return this.uriFactory;
     }
 
+    public URITranslator getUriTranslator() {
+        return this.uriTranslators.with(URITranslator.map(uriMap));
+    }
+
     public JsonSchema getSchema(final String schema, final SchemaValidatorsConfig config) {
         try {
             final JsonNode schemaNode = jsonMapper.readTree(schema);
@@ -372,12 +398,11 @@ public class JsonSchemaFactory {
     public JsonSchema getSchema(final URI schemaUri, final SchemaValidatorsConfig config) {
         try {
             InputStream inputStream = null;
-            final Map<String, String> map = (config != null) ? config.getUriMappings() : new HashMap<String, String>();
-            map.putAll(uriMap);
+            final URITranslator uriTranslator = null == config ? getUriTranslator() : config.getUriTranslator().with(getUriTranslator());
 
             final URI mappedUri;
             try {
-                mappedUri = this.uriFactory.create(map.get(schemaUri.toString()) != null ? map.get(schemaUri.toString()) : schemaUri.toString());
+                mappedUri = this.uriFactory.create(uriTranslator.translate(schemaUri).toString());
             } catch (IllegalArgumentException e) {
                 logger.error("Failed to create URI.", e);
                 throw new JsonSchemaException(e);
diff --git a/src/main/java/com/networknt/schema/SchemaValidatorsConfig.java b/src/main/java/com/networknt/schema/SchemaValidatorsConfig.java
index e34dc91..bdb64a6 100644
--- a/src/main/java/com/networknt/schema/SchemaValidatorsConfig.java
+++ b/src/main/java/com/networknt/schema/SchemaValidatorsConfig.java
@@ -23,6 +23,8 @@ import java.util.Map;
 import java.util.Set;
 
 import com.fasterxml.jackson.databind.JsonNode;
+import com.networknt.schema.uri.URITranslator;
+import com.networknt.schema.uri.URITranslator.CompositeURITranslator;
 import com.networknt.schema.walk.JsonSchemaWalkListener;
 
 public class SchemaValidatorsConfig {
@@ -70,6 +72,8 @@ public class SchemaValidatorsConfig {
      */
     private Map<String, String> uriMappings = new HashMap<String, String>();
 
+    private CompositeURITranslator uriTranslators = new CompositeURITranslator();
+
     /**
      * When a field is set as nullable in the OpenAPI specification, the schema validator validates that it is nullable
      * however continues with validation against the nullable field
@@ -130,11 +134,30 @@ public class SchemaValidatorsConfig {
         return applyDefaultsStrategy;
     }
 
+    public CompositeURITranslator getUriTranslator() {
+        return this.uriTranslators
+            .with(URITranslator.map(this.uriMappings));
+    }
+
+    public void addUriTranslator(URITranslator uriTranslator) {
+        if (null != uriTranslator) {
+            this.uriTranslators.add(uriTranslator);
+        }
+    }
+
+    /**
+     * @deprecated Use {@code getUriTranslator()} instead
+     */
+    @Deprecated
     public Map<String, String> getUriMappings() {
         // return a copy of the mappings
         return new HashMap<String, String>(uriMappings);
     }
 
+    /**
+     * @deprecated Use {@code addUriTranslator()} instead
+     */
+    @Deprecated
     public void setUriMappings(Map<String, String> uriMappings) {
         this.uriMappings = uriMappings;
     }
diff --git a/src/main/java/com/networknt/schema/uri/URITranslator.java b/src/main/java/com/networknt/schema/uri/URITranslator.java
new file mode 100644
index 0000000..cd40099
--- /dev/null
+++ b/src/main/java/com/networknt/schema/uri/URITranslator.java
@@ -0,0 +1,122 @@
+package com.networknt.schema.uri;
+
+import java.net.URI;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+@FunctionalInterface
+public interface URITranslator {
+    static final URITranslator NOOP = original -> original;
+
+    /**
+     * Translates one URI into another.
+     * @param original the URI to translate
+     * @return the translated URI or the original URI if it did not match
+     *         the conditions triggering translation
+     */
+    URI translate(URI original);
+
+    /**
+     * Creates a simple mapping from one URI to another.
+     * @param source the URI to match
+     * @param target the URI to return when matched
+     * @return a new URITranslator
+     */
+    static URITranslator map(String source, String target) {
+        return map(URI.create(source), URI.create(target));
+    }
+
+    /**
+     * Creates a simple mapping from one URI to another.
+     * @param source the URI to match
+     * @param target the URI to return when matched
+     * @return a new URITranslator
+     */
+    static URITranslator map(URI source, URI target) {
+        return original -> Objects.equals(source, original) ? target : original;
+    }
+
+    /**
+     * Creates a map-based mapping from one URI to another.
+     * @param uriMappings the mappings to build
+     * @return a new URITranslator
+     */
+    static URITranslator map(Map<String, String> uriMappings) {
+        return new MappingURITranslator(uriMappings);
+    }
+
+    /**
+     * Creates a CompositeURITranslator.
+     * @param uriTranslators the translators to combine
+     * @return a new CompositeURITranslator
+     */
+    static CompositeURITranslator combine(URITranslator... uriTranslators) {
+        return new CompositeURITranslator(uriTranslators);
+    }
+
+    /**
+     * Creates a CompositeURITranslator.
+     * @param uriTranslators the translators to combine
+     * @return a new CompositeURITranslator
+     */
+    static CompositeURITranslator combine(Collection<? extends URITranslator> uriTranslators) {
+        return new CompositeURITranslator(uriTranslators);
+    }
+
+    class CompositeURITranslator extends ArrayList<URITranslator> implements URITranslator {
+        private static final long serialVersionUID = 1L;
+
+        public CompositeURITranslator() {
+            super();
+        }
+
+        public CompositeURITranslator(URITranslator...translators) {
+            this(Arrays.asList(translators));
+        }
+
+        public CompositeURITranslator(Collection<? extends URITranslator> c) {
+            super(c);
+        }
+
+        @Override
+        public URI translate(URI original) {
+            URI result = original;
+            for (URITranslator translator: this) {
+                result = translator.translate(result);
+            }
+            return result;
+        }
+
+        public CompositeURITranslator with(URITranslator translator) {
+            if (null != translator) {
+                return new CompositeURITranslator(this, translator);
+            }
+            return this;
+        }
+    }
+
+    /**
+     * Provides support for legacy map-based translations
+     */
+    class MappingURITranslator implements URITranslator {
+        private final Map<URI, URI> mappings;
+
+        public MappingURITranslator(Map<String, String> uriMappings) {
+            this.mappings = new HashMap<>();
+            if (null != uriMappings) {
+                uriMappings.forEach((k, v) -> this.mappings.put(URI.create(k), URI.create(v)));
+            }
+        }
+
+        @Override
+        public URI translate(URI original) {
+            return this.mappings.getOrDefault(original, original);
+        }
+
+    }
+
+}


Test patch:
diff --git a/src/test/java/com/networknt/schema/UriMappingTest.java b/src/test/java/com/networknt/schema/UriMappingTest.java
index 1b18708..33b966b 100644
--- a/src/test/java/com/networknt/schema/UriMappingTest.java
+++ b/src/test/java/com/networknt/schema/UriMappingTest.java
@@ -19,6 +19,7 @@ import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.networknt.schema.JsonSchemaFactory.Builder;
 import com.networknt.schema.uri.ClasspathURLFactory;
+import com.networknt.schema.uri.URITranslator;
 import com.networknt.schema.uri.URLFactory;
 import org.junit.jupiter.api.Test;
 
@@ -29,7 +30,6 @@ import java.net.URI;
 import java.net.URL;
 import java.net.UnknownHostException;
 import java.util.HashMap;
-import java.util.Map;
 
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.fail;
@@ -54,7 +54,7 @@ public class UriMappingTest {
         Builder builder = JsonSchemaFactory.builder()
                 .defaultMetaSchemaURI(draftV4.getUri())
                 .addMetaSchema(draftV4)
-                .addUriMappings(getUriMappingsFromUrl(mappings));
+                .addUriTranslator(getUriMappingsFromUrl(mappings));
         JsonSchemaFactory instance = builder.build();
         JsonSchema schema = instance.getSchema(this.urlFactory.create(
                 "https://raw.githubusercontent.com/networknt/json-schema-validator/master/src/test/resources/draft4/extra/uri_mapping/uri-mapping.schema.json"));
@@ -93,7 +93,7 @@ public class UriMappingTest {
         Builder builder = JsonSchemaFactory.builder()
                 .defaultMetaSchemaURI(draftV4.getUri())
                 .addMetaSchema(draftV4)
-                .addUriMappings(getUriMappingsFromUrl(mappings));
+                .addUriTranslator(getUriMappingsFromUrl(mappings));
         instance = builder.build();
         JsonSchema schema = instance.getSchema(example);
         assertEquals(0, schema.validate(mapper.createObjectNode()).size());
@@ -111,7 +111,7 @@ public class UriMappingTest {
         URL mappings = ClasspathURLFactory.convert(
                 this.classpathURLFactory.create("resource:draft4/extra/uri_mapping/uri-mapping.json"));
         SchemaValidatorsConfig config = new SchemaValidatorsConfig();
-        config.setUriMappings(getUriMappingsFromUrl(mappings));
+        config.addUriTranslator(getUriMappingsFromUrl(mappings));
         JsonSchema schema = instance.getSchema(this.urlFactory.create(
                 "https://raw.githubusercontent.com/networknt/json-schema-validator/master/src/test/resources/draft4/extra/uri_mapping/uri-mapping.schema.json"),
                 config);
@@ -147,7 +147,7 @@ public class UriMappingTest {
         }
         URL mappings = ClasspathURLFactory.convert(
                 this.classpathURLFactory.create("resource:draft4/extra/uri_mapping/invalid-schema-uri.json"));
-        config.setUriMappings(getUriMappingsFromUrl(mappings));
+        config.addUriTranslator(getUriMappingsFromUrl(mappings));
         JsonSchema schema = instance.getSchema(example, config);
         assertEquals(0, schema.validate(mapper.createObjectNode()).size());
     }
@@ -158,18 +158,18 @@ public class UriMappingTest {
         URL mappings = ClasspathURLFactory.convert(
                 this.classpathURLFactory.create("resource:draft4/extra/uri_mapping/schema-with-ref-mapping.json"));
         SchemaValidatorsConfig config = new SchemaValidatorsConfig();
-        config.setUriMappings(getUriMappingsFromUrl(mappings));
+        config.addUriTranslator(getUriMappingsFromUrl(mappings));
         JsonSchema schema = instance.getSchema(this.classpathURLFactory.create("resource:draft4/extra/uri_mapping/schema-with-ref.json"),
                 config);
         assertEquals(0, schema.validate(mapper.readTree("[]")).size());
     }
 
-    private Map<String, String> getUriMappingsFromUrl(URL url) throws MalformedURLException, IOException {
+    private URITranslator getUriMappingsFromUrl(URL url) throws MalformedURLException, IOException {
         HashMap<String, String> map = new HashMap<String, String>();
         for (JsonNode mapping : mapper.readTree(url)) {
             map.put(mapping.get("publicURL").asText(),
                     mapping.get("localURL").asText());
         }
-        return map;
+        return URITranslator.map(map);
     }
 }