Repository: networknt/json-schema-validator
Pull request #400 (2021-04-14 09:25:17)

Introduce `forceHttps` flag in `JsonSchemaFactory.Builder`
In https://github.com/networknt/json-schema-validator/commit/e32a5410a73efb9ce7f50307cf000e28f0902f84, schema normalization was introduced to fix https://github.com/networknt/json-schema-validator/issues/273 however it assumes the given schema is available over HTTPS.

https://github.com/networknt/json-schema-validator/blob/e32a5410a73efb9ce7f50307cf000e28f0902f84/src/main/java/com/networknt/schema/JsonSchemaFactory.java#L406

As reported in https://github.com/networknt/json-schema-validator/issues/314, some schemas are not available over HTTPS. For example, https://iglucentral.com (it just hangs with https), see https://github.com/snowplow/iglu-central/issues/1067.

Therefore https://github.com/networknt/json-schema-validator/issues/314 blocks https://github.com/snowplow/iglu-scala-client/issues/136 and our internal `json-schema-validator` dependency upgrade from `1.0.39` to `1.0.40+`.

The first commit in this PR provides a failing test for https://github.com/networknt/json-schema-validator/issues/314 and the second one introduces a boolean flag to control the URI scheme.

Fixes https://github.com/networknt/json-schema-validator/issues/314.

codecov-io: # [Codecov](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt) Report
> Merging [#400](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt) (0cc7a78) into [master](https://codecov.io/gh/networknt/json-schema-validator/commit/d907f55c58f6cb1a62ec06e88a8d82b720620ec3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt) (d907f55) will **increase** coverage by `0.05%`.
> The diff coverage is `88.88%`.

[![Impacted file tree graph](https://codecov.io/gh/networknt/json-schema-validator/pull/400/graphs/tree.svg?width=650&height=150&src=pr&token=Dq6aZ2eKq9&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt)](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt)

```diff
@@             Coverage Diff              @@
##             master     #400      +/-   ##
============================================
+ Coverage     70.76%   70.81%   +0.05%     
- Complexity      750      751       +1     
============================================
  Files            81       81              
  Lines          2894     2899       +5     
  Branches        593      594       +1     
============================================
+ Hits           2048     2053       +5     
  Misses          624      624              
  Partials        222      222              
```


| [Impacted Files](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...n/java/com/networknt/schema/JsonSchemaFactory.java](https://codecov.io/gh/networknt/json-schema-validator/pull/400/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt#diff-c3JjL21haW4vamF2YS9jb20vbmV0d29ya250L3NjaGVtYS9Kc29uU2NoZW1hRmFjdG9yeS5qYXZh) | `78.82% <87.50%> (+0.64%)` | `32.00 <0.00> (+1.00)` | |
| [...java/com/networknt/schema/SpecVersionDetector.java](https://codecov.io/gh/networknt/json-schema-validator/pull/400/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt#diff-c3JjL21haW4vamF2YS9jb20vbmV0d29ya250L3NjaGVtYS9TcGVjVmVyc2lvbkRldGVjdG9yLmphdmE=) | `92.30% <100.00%> (ø)` | `6.00 <0.00> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt). Last update [d907f55...0cc7a78](https://codecov.io/gh/networknt/json-schema-validator/pull/400?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=networknt).


Patch:
diff --git a/src/main/java/com/networknt/schema/JsonSchemaFactory.java b/src/main/java/com/networknt/schema/JsonSchemaFactory.java
index c53e966..8fe4c51 100644
--- a/src/main/java/com/networknt/schema/JsonSchemaFactory.java
+++ b/src/main/java/com/networknt/schema/JsonSchemaFactory.java
@@ -47,6 +47,7 @@ public class JsonSchemaFactory {
         private URNFactory urnFactory;
         private final Map<String, JsonMetaSchema> jsonMetaSchemas = new HashMap<String, JsonMetaSchema>();
         private final Map<String, String> uriMap = new HashMap<String, String>();
+        private boolean forceHttps = true;
 		
 
         public Builder() {
@@ -139,7 +140,10 @@ public class JsonSchemaFactory {
             return this;
         }
         
-		
+        public Builder forceHttps(boolean forceHttps) {
+            this.forceHttps = forceHttps;
+            return this;
+        }
 
         public JsonSchemaFactory build() {
             // create builtin keywords with (custom) formats.
@@ -150,7 +154,8 @@ public class JsonSchemaFactory {
                     new URISchemeFetcher(uriFetcherMap),
                     urnFactory,
                     jsonMetaSchemas,
-                    uriMap
+                    uriMap,
+                    forceHttps
             );
         }
     }
@@ -163,6 +168,7 @@ public class JsonSchemaFactory {
     private final Map<String, JsonMetaSchema> jsonMetaSchemas;
     private final Map<String, String> uriMap;
     private final ConcurrentMap<URI, JsonSchema> uriSchemaCache = new ConcurrentHashMap<URI, JsonSchema>();
+    private final boolean forceHttps;
 
 
     private JsonSchemaFactory(
@@ -172,7 +178,8 @@ public class JsonSchemaFactory {
             final URISchemeFetcher uriFetcher,
             final URNFactory urnFactory,
             final Map<String, JsonMetaSchema> jsonMetaSchemas,
-            final Map<String, String> uriMap) {
+            final Map<String, String> uriMap,
+            final boolean forceHttps) {
         if (mapper == null) {
             throw new IllegalArgumentException("ObjectMapper must not be null");
         } else if (defaultMetaSchemaURI == null || defaultMetaSchemaURI.trim().isEmpty()) {
@@ -195,6 +202,7 @@ public class JsonSchemaFactory {
         this.urnFactory = urnFactory;
         this.jsonMetaSchemas = jsonMetaSchemas;
         this.uriMap = uriMap;
+        this.forceHttps = forceHttps;
     }
 
     /**
@@ -273,7 +281,7 @@ public class JsonSchemaFactory {
 
     private JsonMetaSchema findMetaSchemaForSchema(final JsonNode schemaNode) {
         final JsonNode uriNode = schemaNode.get("$schema");
-        final String uri = uriNode == null || uriNode.isNull() ? defaultMetaSchemaURI : normalizeMetaSchemaUri(uriNode.textValue());
+        final String uri = uriNode == null || uriNode.isNull() ? defaultMetaSchemaURI : normalizeMetaSchemaUri(uriNode.textValue(), forceHttps);
         final JsonMetaSchema jsonMetaSchema = jsonMetaSchemas.get(uri);
         if (jsonMetaSchema == null) {
             throw new JsonSchemaException("Unknown MetaSchema: " + uri);
@@ -395,10 +403,11 @@ public class JsonSchemaFactory {
         return result;
     }
 
-    static protected String normalizeMetaSchemaUri(String u) {
+    static protected String normalizeMetaSchemaUri(String u, boolean forceHttps) {
         try {
             URI uri = new URI(u);
-            URI newUri = new URI("https", uri.getUserInfo(), uri.getHost(), uri.getPort(), uri.getPath(), null, null);
+            String scheme = forceHttps ? "https" : uri.getScheme();
+            URI newUri = new URI(scheme, uri.getUserInfo(), uri.getHost(), uri.getPort(), uri.getPath(), null, null);
             return newUri.toString();
         } catch (URISyntaxException e) {
             throw new JsonSchemaException("Wrong MetaSchema URI: " + u);
diff --git a/src/main/java/com/networknt/schema/SpecVersionDetector.java b/src/main/java/com/networknt/schema/SpecVersionDetector.java
index 0b2e357..190ab4e 100644
--- a/src/main/java/com/networknt/schema/SpecVersionDetector.java
+++ b/src/main/java/com/networknt/schema/SpecVersionDetector.java
@@ -39,7 +39,7 @@ public class SpecVersionDetector {
         if (!jsonNode.has(SCHEMA_TAG))
             throw new JsonSchemaException("Schema tag not present");
 
-        String schemaUri = JsonSchemaFactory.normalizeMetaSchemaUri(jsonNode.get(SCHEMA_TAG).asText());
+        String schemaUri = JsonSchemaFactory.normalizeMetaSchemaUri(jsonNode.get(SCHEMA_TAG).asText(), true);
         if (schemaUri.equals(JsonMetaSchema.getV4().getUri()))
             return SpecVersion.VersionFlag.V4;
         else if (schemaUri.equals(JsonMetaSchema.getV6().getUri()))


Test patch:
diff --git a/src/test/java/com/networknt/schema/Issue314Test.java b/src/test/java/com/networknt/schema/Issue314Test.java
new file mode 100644
index 0000000..c1fd163
--- /dev/null
+++ b/src/test/java/com/networknt/schema/Issue314Test.java
@@ -0,0 +1,26 @@
+package com.networknt.schema;
+
+import java.io.InputStream;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class Issue314Test {
+    private static final JsonSchemaFactory FACTORY =
+            JsonSchemaFactory.builder(JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V7))
+                    .addMetaSchema(
+                            JsonMetaSchema.builder(
+                                    "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0",
+                                    JsonMetaSchema.getV7())
+                                    .build())
+                    .forceHttps(false)
+                    .build();
+
+    @Test
+    public void testNormalizeHttpOnly() {
+        String schemaPath = "/schema/issue314-v7.json";
+        InputStream schemaInputStream = getClass().getResourceAsStream(schemaPath);
+        JsonSchema schema = FACTORY.getSchema(schemaInputStream);
+
+        Assert.assertNotNull(schema);
+    }
+}
diff --git a/src/test/java/com/networknt/schema/UnknownMetaSchemaTest.java b/src/test/java/com/networknt/schema/UnknownMetaSchemaTest.java
index 31b2039..e766602 100644
--- a/src/test/java/com/networknt/schema/UnknownMetaSchemaTest.java
+++ b/src/test/java/com/networknt/schema/UnknownMetaSchemaTest.java
@@ -64,10 +64,10 @@ public class UnknownMetaSchemaTest {
         String uri03 = "http://json-schema.org/draft-07/schema?key=value";
         String uri04 = "http://json-schema.org/draft-07/schema?key=value&key2=value2";
         String expected = "https://json-schema.org/draft-07/schema";
-        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri01));
-        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri02));
-        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri03));
-        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri04));
+        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri01, true));
+        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri02, true));
+        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri03, true));
+        Assert.assertEquals(expected, JsonSchemaFactory.normalizeMetaSchemaUri(uri04, true));
 
     }
 }
diff --git a/src/test/resources/schema/issue314-v7.json b/src/test/resources/schema/issue314-v7.json
new file mode 100644
index 0000000..c0d605c
--- /dev/null
+++ b/src/test/resources/schema/issue314-v7.json
@@ -0,0 +1,4 @@
+{
+  "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
+  "type": "object"
+}