Repository: fastify/fastify (JavaScript)

Issue:
Serialization references not resolved
## 🐛 Bug Report

References for validation can be added using the `addSchema` method, so in our JSON schemas we can safely use `$ref` to reference them.

However, when trying to do the same for the serialization using `fast-json-stringify` I get the following error:

```
Error: schema is invalid: data.items.$ref should be string, data.items should be array, data.items should match some schema in anyOf
```

Reading up on the documentation (https://github.com/fastify/fast-json-stringify#ref) it seems that external definitions need to be passed as an option, so I was wondering if the `addSchema` function would not already do this?

## To Reproduce

Steps to reproduce the behavior:

asset.schema.json
```json
{
  "$id": "http://poppy.be/schemas/asset.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Physical Asset",
  "description": "A generic representation of a physical asset",
  "type": "object",
  "required": [
    "id",
    "model",
    "location"
  ],
  "properties": {
    "id": {
      "type": "string",
      "format": "uuid"
    },
    "model": {
      "type": "string"
    },
    "location": { "$ref": "http://poppy.be/schemas/geo/point.json#" }
  }
}
```

point.schema.json
```json
{
  "$id": "http://poppy.be/schemas/geo/point.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Longitude and Latitude Values",
  "description": "A geographical coordinate.",
  "type": "object",
  "required": [
    "latitude",
    "longitude"
  ],
  "properties": {
    "latitude": {
      "type": "number",
      "minimum": -90,
      "maximum": 90
    },
    "longitude": {
      "type": "number",
      "minimum": -180,
      "maximum": 180
    },
    "altitude": {
      "type": "number"
    }
  }
}
```

assetResponse.schema.json
```json
{
  "$id": "http://poppy.be/schemas/response/assetLocations.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "List of Asset locations",
  "type": "array",
  "items": { "$ref": "http://poppy.be/schemas/asset.json#" },
  "default": []
}
```

Route
```js
app.get('/assets', { schema: { querystring: Schemas.geo.point, response: { 200: Schemas.response.assetLocations } } }, async (req, res) => {
  ...
})
```

## Expected behavior

The serialization should simply be able to resolve the `asset` schema.

## Your Environment

- *node version*: 10
- *fastify version*: >=1.0.0
- *os*: Mac



Discussion:
That's currently not supported, but I think it would be a good feature to have.

See https://www.fastify.io/docs/latest/Validation-and-Serialization/ for what is currently supported.
Would it be possible to extend the `addSchema` call to also register the schema to `fast-json-stringify` instead of just `ajv`? Not sure if it's possible though.
fast-json-stringify has no such feature at the moment. I think it would be
a good addition.

Il giorno ven 25 gen 2019 alle 15:22 Michiel De Mey <
notifications@github.com> ha scritto:

> Would it be possible to extend the addSchema call to also register the
> schema to fast-json-stringify instead of just ajv? Not sure if it's
> possible though.
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/fastify/fastify/issues/1410#issuecomment-457607620>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AADL4_xu_mxvY9U5S0LXslYH1Optwjb5ks5vGyFFgaJpZM4aSzy2>
> .
>

Hi, I would like to work on this issue ✌

The solution I have seen is:
- all the schema added with `addSchema` could be pass to `fast-json-stringify` as externals schemas with the keys equals to `$id` field of added schemas
- put a stronger validation in `Schemas.traverse` (because it modifies the `$ref` field instead of leaving it as is.. this cause the `Error: schema is invalid: data.items.$ref should be string` error)

I have tried in my local env and works 👍 

Go for it!



Patch:
diff --git a/docs/Validation-and-Serialization.md b/docs/Validation-and-Serialization.md
index 3d978fba7ea..f19d98f8af8 100644
--- a/docs/Validation-and-Serialization.md
+++ b/docs/Validation-and-Serialization.md
@@ -98,6 +98,7 @@ fastify.register((instance, opts, next) => {
   next()
 })
 ```
+
 You can use the shared schema everywhere, as top level schema or nested inside other schemas:
 ```js
 const fastify = require('fastify')()
@@ -302,6 +303,131 @@ fastify.setErrorHandler(function (error, request, reply) {
 })
 ```
 
+### JSON Schema and Shared Schema support
+
+JSON Schema has some type of utilities in order to optimize your schemas that,
+in conjuction with the Fastify's shared schema, let you reuse all your schemas easily.
+
+| Use Case                          | Validator | Serializer |
+|-----------------------------------|-----------|------------|
+| shared schema                     | ✔️ | ✔️ |
+| `$ref` to `$id`                   | ✔ | ✔️ |
+| `$ref` to `/definitions`          | ✔️ | ✔️ |
+| `$ref` to shared schema `$id`          | ❌ | ✔️ |
+| `$ref` to shared schema `/definitions` | ❌ | ✔️ |
+
+#### Examples
+
+```js
+// Usage of the Shared Schema feature
+fastify.addSchema({
+  $id: 'sharedAddress',
+  type: 'object',
+  properties: {
+    city: { 'type': 'string' }
+  }
+})
+
+const sharedSchema = {
+  type: 'object',
+  properties: {
+    home: 'sharedAddress#',
+    work: 'sharedAddress#'
+  }
+}
+```
+
+```js
+// Usage of $ref to $id in same JSON Schema
+const refToId = {
+  type: 'object',
+  definitions: {
+    foo: {
+      $id: '#address',
+      type: 'object',
+      properties: {
+        city: { 'type': 'string' }
+      }
+    }
+  },
+  properties: {
+    home: { $ref: '#address' },
+    work: { $ref: '#address' }
+  }
+}
+```
+
+
+```js
+// Usage of $ref to /definitions in same JSON Schema
+const refToDefinitions = {
+  type: 'object',
+  definitions: {
+    foo: {
+      $id: '#address',
+      type: 'object',
+      properties: {
+        city: { 'type': 'string' }
+      }
+    }
+  },
+  properties: {
+    home: { $ref: '#/definitions/foo' },
+    work: { $ref: '#/definitions/foo' }
+  }
+}
+```
+
+```js
+// Usage $ref to a shared schema $id as external schema
+fastify.addSchema({
+  $id: 'http://foo/common.json',
+  type: 'object',
+  definitions: {
+    foo: {
+      $id: '#address',
+      type: 'object',
+      properties: {
+        city: { 'type': 'string' }
+      }
+    }
+  }
+})
+
+const refToSharedSchemaId = {
+  type: 'object',
+  properties: {
+    home: { $ref: 'http://foo/common.json#address' },
+    work: { $ref: 'http://foo/common.json#address' }
+  }
+}
+```
+
+
+```js
+// Usage $ref to a shared schema /definitions as external schema
+fastify.addSchema({
+  $id: 'http://foo/common.json',
+  type: 'object',
+  definitions: {
+    foo: {
+      type: 'object',
+      properties: {
+        city: { 'type': 'string' }
+      }
+    }
+  }
+})
+
+const refToSharedSchemaDefinitions = {
+  type: 'object',
+  properties: {
+    home: { $ref: 'http://foo/common.json#/definitions/foo' },
+    work: { $ref: 'http://foo/common.json#/definitions/foo' }
+  }
+}
+```
+
 <a name="resources"></a>
 ### Resources
 - [JSON Schema](http://json-schema.org/)
diff --git a/lib/schemas.js b/lib/schemas.js
index b4c1939b2ed..8a335c80002 100644
--- a/lib/schemas.js
+++ b/lib/schemas.js
@@ -70,7 +70,8 @@ Schemas.prototype.resolveRefs = function (routeSchemas) {
 
 Schemas.prototype.traverse = function (schema) {
   for (var key in schema) {
-    if (typeof schema[key] === 'string' && key !== '$schema' && schema[key].slice(-1) === '#') {
+    // resolve the `sharedSchemaId#' only if is not a standard $ref JSON Pointer
+    if (typeof schema[key] === 'string' && key !== '$schema' && key !== '$ref' && schema[key].slice(-1) === '#') {
       schema[key] = this.resolve(schema[key].slice(0, -1))
     }
 
diff --git a/lib/validation.js b/lib/validation.js
index 4804061e871..9ac49b7f3c1 100644
--- a/lib/validation.js
+++ b/lib/validation.js
@@ -9,14 +9,14 @@ const paramsSchema = Symbol('params-schema')
 const responseSchema = Symbol('response-schema')
 const headersSchema = Symbol('headers-schema')
 
-function getValidatorForStatusCodeSchema (statusCodeDefinition) {
-  return fastJsonStringify(statusCodeDefinition)
+function getValidatorForStatusCodeSchema (statusCodeDefinition, externalSchema) {
+  return fastJsonStringify(statusCodeDefinition, { schema: externalSchema })
 }
 
-function getResponseSchema (responseSchemaDefinition) {
+function getResponseSchema (responseSchemaDefinition, sharedSchemas) {
   var statusCodes = Object.keys(responseSchemaDefinition)
   return statusCodes.reduce(function (r, statusCode) {
-    r[statusCode] = getValidatorForStatusCodeSchema(responseSchemaDefinition[statusCode])
+    r[statusCode] = getValidatorForStatusCodeSchema(responseSchemaDefinition[statusCode], sharedSchemas)
     return r
   }, {})
 }
@@ -50,7 +50,7 @@ function build (context, compile, schemas) {
   }
 
   if (context.schema.response) {
-    context[responseSchema] = getResponseSchema(context.schema.response)
+    context[responseSchema] = getResponseSchema(context.schema.response, schemas.getSchemas())
   }
 
   if (context.schema.body) {


Test patch:
diff --git a/test/shared-schemas.test.js b/test/shared-schemas.test.js
index 6559fdf13bd..cdf6c449a1a 100644
--- a/test/shared-schemas.test.js
+++ b/test/shared-schemas.test.js
@@ -441,7 +441,7 @@ test('Use the same schema id in diferent places', t => {
   })
 })
 
-test('Use shared schema and $ref with $id', t => {
+test('Use shared schema and $ref with $id ($ref to $id)', t => {
   t.plan(2)
   const fastify = Fastify()
 
@@ -500,7 +500,7 @@ test('Use shared schema and $ref with $id', t => {
   })
 })
 
-test('Use shared schema and $ref with $id in response', t => {
+test('Use shared schema and $ref with $id in response ($ref to $id)', t => {
   t.plan(2)
   const fastify = Fastify()
 
@@ -741,3 +741,254 @@ test('Get schema anyway should not add `properties` if anyOf is present', t => {
 
   fastify.ready(t.error)
 })
+
+test('Shared schema should be pass to serializer ($ref to shared schema /definitions)', t => {
+  t.plan(2)
+  const fastify = Fastify()
+
+  const schemaAsset = {
+    $id: 'http://example.com/asset.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'Physical Asset',
+    description: 'A generic representation of a physical asset',
+    type: 'object',
+    required: [
+      'id',
+      'model',
+      'location'
+    ],
+    properties: {
+      id: {
+        type: 'string',
+        format: 'uuid'
+      },
+      model: {
+        type: 'string'
+      },
+      location: { $ref: 'http://example.com/point.json#' }
+    },
+    definitions: {
+      inner: {
+        $id: '#innerId',
+        type: 'string',
+        format: 'email'
+      }
+    }
+  }
+
+  const schemaPoint = {
+    $id: 'http://example.com/point.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'Longitude and Latitude Values',
+    description: 'A geographical coordinate.',
+    type: 'object',
+    required: [
+      'latitude',
+      'longitude'
+    ],
+    properties: {
+      email: { $ref: 'http://example.com/asset.json#/definitions/inner' },
+      latitude: {
+        type: 'number',
+        minimum: -90,
+        maximum: 90
+      },
+      longitude: {
+        type: 'number',
+        minimum: -180,
+        maximum: 180
+      },
+      altitude: {
+        type: 'number'
+      }
+    }
+  }
+
+  const schemaResponse = {
+    $id: 'http://example.com/locations.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'List of Asset locations',
+    type: 'array',
+    items: { $ref: 'http://example.com/asset.json#' },
+    default: []
+  }
+
+  fastify.addSchema(schemaAsset)
+  fastify.addSchema(schemaPoint)
+
+  const response = [
+    { id: 'id1', model: 'mod', location: { latitude: 10, longitude: 10, email: 'foo@bar.it' } },
+    { id: 'id2', model: 'mod', location: { latitude: 10, longitude: 10, email: 'foo@bar.it' } }
+  ]
+  fastify.get('/', {
+    schema: {
+      response: { 200: schemaResponse }
+    }
+  }, (req, reply) => {
+    reply.send(response.map(i => Object.assign({ serializer: 'remove me' }, i)))
+  })
+
+  fastify.inject({
+    method: 'GET',
+    url: '/'
+  }, (err, res) => {
+    t.error(err)
+    response.forEach(_ => delete _.remove)
+    t.deepEqual(JSON.parse(res.payload), response)
+  })
+})
+
+test('Shared schema should be pass to serializer ($ref to shared schema $id)', t => {
+  t.plan(2)
+  const fastify = Fastify()
+
+  const schemaAsset = {
+    $id: 'http://example.com/asset.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'Physical Asset',
+    description: 'A generic representation of a physical asset',
+    type: 'object',
+    required: [
+      'id',
+      'model',
+      'location'
+    ],
+    properties: {
+      id: {
+        type: 'string',
+        format: 'uuid'
+      },
+      model: {
+        type: 'string'
+      },
+      location: { $ref: 'http://example.com/point.json#' }
+    },
+    definitions: {
+      inner: {
+        $id: '#innerId',
+        type: 'string',
+        format: 'email'
+      }
+    }
+  }
+
+  const schemaPoint = {
+    $id: 'http://example.com/point.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'Longitude and Latitude Values',
+    description: 'A geographical coordinate.',
+    type: 'object',
+    required: [
+      'latitude',
+      'longitude'
+    ],
+    properties: {
+      email: { $ref: 'http://example.com/asset.json#innerId' },
+      latitude: {
+        type: 'number',
+        minimum: -90,
+        maximum: 90
+      },
+      longitude: {
+        type: 'number',
+        minimum: -180,
+        maximum: 180
+      },
+      altitude: {
+        type: 'number'
+      }
+    }
+  }
+
+  const schemaResponse = {
+    $id: 'http://example.com/locations.json',
+    $schema: 'http://json-schema.org/draft-07/schema#',
+    title: 'List of Asset locations',
+    type: 'array',
+    items: { $ref: 'http://example.com/asset.json#' },
+    default: []
+  }
+
+  fastify.addSchema(schemaAsset)
+  fastify.addSchema(schemaPoint)
+
+  const response = [
+    { id: 'id1', model: 'mod', location: { latitude: 10, longitude: 10, email: 'foo@bar.it' } },
+    { id: 'id2', model: 'mod', location: { latitude: 10, longitude: 10, email: 'foo@bar.it' } }
+  ]
+  fastify.get('/', {
+    schema: {
+      response: { 200: schemaResponse }
+    }
+  }, (req, reply) => {
+    reply.send(response.map(i => Object.assign({ serializer: 'remove me' }, i)))
+  })
+
+  fastify.inject({
+    method: 'GET',
+    url: '/'
+  }, (err, res) => {
+    t.error(err)
+    response.forEach(_ => delete _.remove)
+    t.deepEqual(JSON.parse(res.payload), response)
+  })
+})
+
+test('Use shared schema and $ref to /definitions', t => {
+  t.plan(2)
+  const fastify = Fastify()
+
+  fastify.addSchema({
+    $id: 'test',
+    type: 'object',
+    properties: {
+      id: { type: 'number' }
+    }
+  })
+
+  const body = {
+    type: 'object',
+    definitions: {
+      address: {
+        $id: '#otherId',
+        type: 'object',
+        properties: {
+          city: { 'type': 'string' }
+        }
+      }
+    },
+    properties: {
+      test: 'test#',
+      address: { $ref: '#/definitions/address' }
+    },
+    required: ['address', 'test']
+  }
+
+  fastify.route({
+    method: 'POST',
+    url: '/',
+    schema: {
+      body,
+      response: {
+        200: body
+      }
+    },
+    handler: (req, reply) => {
+      req.body.removeThis = 'it should not be serialized'
+      reply.send(req.body)
+    }
+  })
+
+  const payload = {
+    address: { city: 'New Node' },
+    test: { id: Date.now() }
+  }
+  fastify.inject({
+    method: 'POST',
+    url: '/',
+    payload
+  }, (err, res) => {
+    t.error(err)
+    t.deepEqual(JSON.parse(res.payload), payload)
+  })
+})