Repository: meltano/sdk
Pull request #1525 (2023-03-24 04:20:36)

feat: Support union schemas
Related:

- #1519
- https://github.com/MeltanoLabs/tap-postgres/pull/121
- https://github.com/MeltanoLabs/tap-postgres/issues/135

Docs: https://meltano-sdk--1525.org.readthedocs.build/en/1525/classes/typing/singer_sdk.typing.DiscriminatedUnion.html


<!-- readthedocs-preview meltano-sdk start -->
----
:books: Documentation preview :books:: https://meltano-sdk--1525.org.readthedocs.build/en/1525/

<!-- readthedocs-preview meltano-sdk end -->

codecov[bot]: ## [Codecov](https://codecov.io/gh/meltano/sdk/pull/1525?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
> Merging [#1525](https://codecov.io/gh/meltano/sdk/pull/1525?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) (842e9f3) into [main](https://codecov.io/gh/meltano/sdk/commit/7ff1bebbe63c09c5643d1da3e57f1f0115c3d923?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) (7ff1beb) will **increase** coverage by `0.05%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##             main    #1525      +/-   ##
==========================================
+ Coverage   85.50%   85.55%   +0.05%     
==========================================
  Files          57       57              
  Lines        4692     4709      +17     
  Branches      802      804       +2     
==========================================
+ Hits         4012     4029      +17     
  Misses        489      489              
  Partials      191      191              
```


| [Impacted Files](https://codecov.io/gh/meltano/sdk/pull/1525?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) | Coverage Δ | |
|---|---|---|
| [singer\_sdk/typing.py](https://codecov.io/gh/meltano/sdk/pull/1525?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano#diff-c2luZ2VyX3Nkay90eXBpbmcucHk=) | `95.26% <100.00%> (+0.41%)` | :arrow_up: |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano)

edgarrmondragon: @ReubenFrankel I'd love to get your thoughts on this? Is this helpful, intuitive?
edgarrmondragon: > My only question is what does a validation error for this look like? Is it helpful to the user?

@ReubenFrankel The default `jsonschema` exception message is not the most readable:

```python
jsonschema.exceptions.ValidationError: {'flow': 'oauth', 'client_id': '123', 'username': '456', 'password': '789'} is not valid under any of the given schemas

Failed validating 'oneOf' in schema:
    {'oneOf': [{'additionalProperties': False,
                'properties': {'client_id': {'secret': True,
                                             'type': ['string'],
                                             'writeOnly': True},
                               'client_secret': {'secret': True,
                                                 'type': ['string'],
                                                 'writeOnly': True},
                               'flow': {'const': 'oauth',
                                        'description': 'Discriminator for '
                                                       'object of type '
                                                       "'oauth'."}},
                'required': ['flow', 'client_id', 'client_secret'],
                'type': 'object'},
               {'additionalProperties': False,
                'properties': {'flow': {'const': 'password',
                                        'description': 'Discriminator for '
                                                       'object of type '
                                                       "'password'."},
                               'password': {'secret': True,
                                            'type': ['string'],
                                            'writeOnly': True},
                               'username': {'type': ['string']}},
                'required': ['flow', 'username', 'password'],
                'type': 'object'}]}

On instance:
    {'client_id': '123',
     'flow': 'oauth',
     'password': '789',
     'username': '456'}
```

But we could of course grab the context and print something that better explains the error and expectation. I have a PR where it might make sense to add that: https://github.com/meltano/sdk/pull/768.
edgarrmondragon: @kgpayne can you give this PR a look when get a chance?

Patch:
diff --git a/docs/classes/typing/singer_sdk.typing.Constant.rst b/docs/classes/typing/singer_sdk.typing.Constant.rst
new file mode 100644
index 00000000..248f7eb5
--- /dev/null
+++ b/docs/classes/typing/singer_sdk.typing.Constant.rst
@@ -0,0 +1,8 @@
+﻿singer_sdk.typing.Constant
+==========================
+
+.. currentmodule:: singer_sdk.typing
+
+.. autoclass:: Constant
+    :members:
+    :special-members: __init__, __call__
\ No newline at end of file
diff --git a/docs/classes/typing/singer_sdk.typing.DiscriminatedUnion.rst b/docs/classes/typing/singer_sdk.typing.DiscriminatedUnion.rst
new file mode 100644
index 00000000..132e2ca0
--- /dev/null
+++ b/docs/classes/typing/singer_sdk.typing.DiscriminatedUnion.rst
@@ -0,0 +1,8 @@
+﻿singer_sdk.typing.DiscriminatedUnion
+====================================
+
+.. currentmodule:: singer_sdk.typing
+
+.. autoclass:: DiscriminatedUnion
+    :members:
+    :special-members: __init__, __call__
\ No newline at end of file
diff --git a/docs/classes/typing/singer_sdk.typing.OneOf.rst b/docs/classes/typing/singer_sdk.typing.OneOf.rst
new file mode 100644
index 00000000..e9f159fe
--- /dev/null
+++ b/docs/classes/typing/singer_sdk.typing.OneOf.rst
@@ -0,0 +1,8 @@
+﻿singer_sdk.typing.OneOf
+=======================
+
+.. currentmodule:: singer_sdk.typing
+
+.. autoclass:: OneOf
+    :members:
+    :special-members: __init__, __call__
\ No newline at end of file
diff --git a/docs/reference.rst b/docs/reference.rst
index 0c8d8dff..b59bd665 100644
--- a/docs/reference.rst
+++ b/docs/reference.rst
@@ -92,9 +92,11 @@ JSON Schema builder classes
     typing.PropertiesList
     typing.ArrayType
     typing.BooleanType
+    typing.Constant
     typing.CustomType
     typing.DateTimeType
     typing.DateType
+    typing.DiscriminatedUnion
     typing.DurationType
     typing.EmailType
     typing.HostnameType
@@ -104,6 +106,7 @@ JSON Schema builder classes
     typing.JSONPointerType
     typing.NumberType
     typing.ObjectType
+    typing.OneOf
     typing.Property
     typing.RegexType
     typing.RelativeJSONPointerType
diff --git a/singer_sdk/typing.py b/singer_sdk/typing.py
index 1e70f620..a8f654c9 100644
--- a/singer_sdk/typing.py
+++ b/singer_sdk/typing.py
@@ -711,6 +711,150 @@ class ObjectType(JSONTypeHelper):
         return result
 
 
+class OneOf(JSONPointerType):
+    """OneOf type.
+
+    This type allows for a value to be one of a set of types.
+
+    Examples:
+        >>> t = OneOf(StringType, IntegerType)
+        >>> print(t.to_json(indent=2))
+        {
+            "oneOf": [
+                {
+                    "type": [
+                        "string"
+                    ]
+                },
+                {
+                    "type": [
+                        "integer"
+                    ]
+                }
+            ]
+        }
+    """
+
+    def __init__(self, *types: W | type[W]) -> None:
+        """Initialize OneOf type.
+
+        Args:
+            types: Types to choose from.
+        """
+        self.wrapped = types
+
+    @property
+    def type_dict(self) -> dict:  # type: ignore[override]
+        """Get type dictionary.
+
+        Returns:
+            A dictionary describing the type.
+        """
+        return {"oneOf": [t.type_dict for t in self.wrapped]}
+
+
+class Constant(JSONTypeHelper):
+    """A constant property.
+
+    A property that is always the same value.
+
+    Examples:
+        >>> t = Constant("foo")
+        >>> print(t.to_json(indent=2))
+        {
+            "const": "foo"
+        }
+    """
+
+    def __init__(self, value: _JsonValue) -> None:
+        """Initialize Constant.
+
+        Args:
+            value: Value of the constant.
+        """
+        self.value = value
+
+    @property
+    def type_dict(self) -> dict:  # type: ignore[override]
+        """Get type dictionary.
+
+        Returns:
+            A dictionary describing the type.
+        """
+        return {"const": self.value}
+
+
+class DiscriminatedUnion(OneOf):
+    """A discriminator property.
+
+    This is a special case of :class:`singer_sdk.typing.OneOf`, where values are
+    JSON objects, and the type of the object is determined by a property in the
+    object.
+
+    The property is a :class:`singer_sdk.typing.Constant` called the discriminator
+    property.
+    """
+
+    def __init__(self, key: str, **options: ObjectType) -> None:
+        """Initialize a discriminated union type.
+
+        Args:
+            key: Name of the discriminator property.
+            options: Mapping of discriminator values to object types.
+
+        Examples:
+            >>> t = DiscriminatedUnion("species", cat=ObjectType(), dog=ObjectType())
+            >>> print(t.to_json(indent=2))
+            {
+              "oneOf": [
+                {
+                  "type": "object",
+                  "properties": {
+                    "species": {
+                      "const": "cat",
+                      "description": "Discriminator for object of type 'cat'."
+                    }
+                  },
+                  "required": [
+                    "species"
+                  ]
+                },
+                {
+                  "type": "object",
+                  "properties": {
+                    "species": {
+                        "const": "dog",
+                        "description": "Discriminator for object of type 'dog'."
+                    }
+                  },
+                  "required": [
+                    "species"
+                  ]
+                }
+              ]
+            }
+        """
+        self.key = key
+        self.options = options
+
+        super().__init__(
+            *(
+                ObjectType(
+                    Property(
+                        key,
+                        Constant(k),
+                        required=True,
+                        description=f"Discriminator for object of type '{k}'.",
+                    ),
+                    *v.wrapped.values(),
+                    additional_properties=v.additional_properties,
+                    pattern_properties=v.pattern_properties,
+                )
+                for k, v in options.items()
+            ),
+        )
+
+
 class CustomType(JSONTypeHelper):
     """Accepts an arbitrary JSON Schema dictionary."""
 


Test patch:
diff --git a/tests/core/test_jsonschema_helpers.py b/tests/core/test_jsonschema_helpers.py
index 2834b0ef..3e4ba6ec 100644
--- a/tests/core/test_jsonschema_helpers.py
+++ b/tests/core/test_jsonschema_helpers.py
@@ -7,6 +7,7 @@ import typing as t
 from textwrap import dedent
 
 import pytest
+from jsonschema import Draft6Validator
 
 from singer_sdk.helpers._typing import (
     JSONSCHEMA_ANNOTATION_SECRET,
@@ -30,6 +31,7 @@ from singer_sdk.typing import (
     CustomType,
     DateTimeType,
     DateType,
+    DiscriminatedUnion,
     DurationType,
     EmailType,
     HostnameType,
@@ -813,3 +815,48 @@ def test_type_check_variations(property_schemas, type_check_functions, results):
     for property_schema in property_schemas:
         for type_check_function, result in zip(type_check_functions, results):
             assert type_check_function(property_schema) == result
+
+
+def test_discriminated_union():
+    th = DiscriminatedUnion(
+        "flow",
+        oauth=ObjectType(
+            Property("client_id", StringType, required=True, secret=True),
+            Property("client_secret", StringType, required=True, secret=True),
+            additional_properties=False,
+        ),
+        password=ObjectType(
+            Property("username", StringType, required=True),
+            Property("password", StringType, required=True, secret=True),
+            additional_properties=False,
+        ),
+    )
+
+    validator = Draft6Validator(th.to_dict())
+
+    assert validator.is_valid(
+        {
+            "flow": "oauth",
+            "client_id": "123",
+            "client_secret": "456",
+        },
+    )
+    assert validator.is_valid(
+        {
+            "flow": "password",
+            "password": "123",
+            "username": "456",
+        },
+    )
+    assert not validator.is_valid(
+        {
+            "flow": "oauth",
+            "client_id": "123",
+        },
+    )
+    assert not validator.is_valid(
+        {
+            "flow": "password",
+            "client_id": "123",
+        },
+    )