Repository: meltano/sdk
Pull request #2208 (2024-01-31 19:33:26)

feat(targets): Allow users to disable schema validation in targets
Related:

* Closes https://github.com/meltano/sdk/issues/1904


<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--2208.org.readthedocs.build/en/2208/

<!-- readthedocs-preview meltano-sdk end -->

codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/edgarrmondragon/feat/setting-validate-schema)

### Merging #2208 will **not alter performance**

<sub>Comparing <code>edgarrmondragon/feat/setting-validate-schema</code> (f1189ca) with <code>main</code> (32f3dec)</sub>



### Summary

`✅ 6` untouched benchmarks





codecov[bot]: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/2208?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`32f3dec`)](https://app.codecov.io/gh/meltano/sdk/commit/32f3decbbdc0056c5a7d65365223d0b70c2bbf5d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) 88.46% compared to head [(`f1189ca`)](https://app.codecov.io/gh/meltano/sdk/pull/2208?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) 88.47%.

> :exclamation: Current head f1189ca differs from pull request most recent head 9e5889c. Consider uploading reports for the commit 9e5889c to get more accurate results


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2208      +/-   ##
==========================================
+ Coverage   88.46%   88.47%   +0.01%     
==========================================
  Files          54       54              
  Lines        4716     4721       +5     
  Branches      917      918       +1     
==========================================
+ Hits         4172     4177       +5     
  Misses        383      383              
  Partials      161      161              
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/2208?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).


Patch:
diff --git a/singer_sdk/helpers/capabilities.py b/singer_sdk/helpers/capabilities.py
index 151cd1c8..b48d290a 100644
--- a/singer_sdk/helpers/capabilities.py
+++ b/singer_sdk/helpers/capabilities.py
@@ -144,6 +144,14 @@ TARGET_HARD_DELETE_CONFIG = PropertiesList(
         default=False,
     ),
 ).to_dict()
+TARGET_VALIDATE_RECORDS_CONFIG = PropertiesList(
+    Property(
+        "validate_records",
+        BooleanType(),
+        description="Whether to validate the schema of the incoming streams.",
+        default=True,
+    ),
+).to_dict()
 
 
 class TargetLoadMethods(str, Enum):
@@ -355,3 +363,6 @@ class TargetCapabilities(CapabilitiesEnum):
 
     #: Allow setting the target schema.
     TARGET_SCHEMA = "target-schema"
+
+    #: Validate the schema of the incoming records.
+    VALIDATE_RECORDS = "validate-records"
diff --git a/singer_sdk/sinks/core.py b/singer_sdk/sinks/core.py
index abf5f82c..7cff9567 100644
--- a/singer_sdk/sinks/core.py
+++ b/singer_sdk/sinks/core.py
@@ -9,6 +9,7 @@ import importlib.util
 import json
 import time
 import typing as t
+from functools import cached_property
 from gzip import GzipFile
 from gzip import open as gzip_open
 from types import MappingProxyType
@@ -131,9 +132,6 @@ class Sink(metaclass=abc.ABCMeta):
 
     MAX_SIZE_DEFAULT = 10000
 
-    validate_schema = True
-    """Enable JSON schema record validation."""
-
     validate_field_string_format = False
     """Enable JSON schema format validation, for example `date-time` string fields."""
 
@@ -186,6 +184,15 @@ class Sink(metaclass=abc.ABCMeta):
 
         self._validator: BaseJSONSchemaValidator | None = self.get_validator()
 
+    @cached_property
+    def validate_schema(self) -> bool:
+        """Enable JSON schema record validation.
+
+        Returns:
+            True if JSON schema validation is enabled.
+        """
+        return self.config.get("validate_records", True)
+
     def get_validator(self) -> BaseJSONSchemaValidator | None:
         """Get a record validator for this sink.
 
diff --git a/singer_sdk/target_base.py b/singer_sdk/target_base.py
index 4ab959ff..5a78bf36 100644
--- a/singer_sdk/target_base.py
+++ b/singer_sdk/target_base.py
@@ -21,6 +21,7 @@ from singer_sdk.helpers.capabilities import (
     TARGET_HARD_DELETE_CONFIG,
     TARGET_LOAD_METHOD_CONFIG,
     TARGET_SCHEMA_CONFIG,
+    TARGET_VALIDATE_RECORDS_CONFIG,
     CapabilitiesEnum,
     PluginCapabilities,
     TargetCapabilities,
@@ -104,6 +105,7 @@ class Target(PluginBase, SingerReader, metaclass=abc.ABCMeta):
             PluginCapabilities.ABOUT,
             PluginCapabilities.STREAM_MAPS,
             PluginCapabilities.FLATTENING,
+            TargetCapabilities.VALIDATE_RECORDS,
         ]
 
     @property
@@ -614,6 +616,9 @@ class Target(PluginBase, SingerReader, metaclass=abc.ABCMeta):
         if PluginCapabilities.BATCH in capabilities:
             _merge_missing(BATCH_CONFIG, config_jsonschema)
 
+        if TargetCapabilities.VALIDATE_RECORDS in capabilities:
+            _merge_missing(TARGET_VALIDATE_RECORDS_CONFIG, config_jsonschema)
+
         super().append_builtin_config(config_jsonschema)
 
 


Test patch:
diff --git a/tests/core/test_target_base.py b/tests/core/test_target_base.py
index de344c7e..eaff6d6a 100644
--- a/tests/core/test_target_base.py
+++ b/tests/core/test_target_base.py
@@ -8,7 +8,7 @@ from singer_sdk.exceptions import (
     MissingKeyPropertiesError,
     RecordsWithoutSchemaException,
 )
-from singer_sdk.helpers.capabilities import PluginCapabilities
+from singer_sdk.helpers.capabilities import PluginCapabilities, TargetCapabilities
 from tests.conftest import BatchSinkMock, SQLSinkMock, SQLTargetMock, TargetMock
 
 
@@ -67,6 +67,7 @@ def test_target_about_info():
         PluginCapabilities.ABOUT,
         PluginCapabilities.STREAM_MAPS,
         PluginCapabilities.FLATTENING,
+        TargetCapabilities.VALIDATE_RECORDS,
         PluginCapabilities.BATCH,
     ]