Repository: meltano/sdk
Pull request #3007: test: Add continuous benchmark for record data type conforming
Author: login: edgarrmondragon
Created: 2025-04-30T02:11:43Z | Merged: 2025-04-30T02:37:16Z

## Summary by Sourcery

Add a continuous benchmark test for record data type conformance to validate performance of data type conversion

Tests:
- Added a benchmark test for the conform_record_data_types method using pytest-benchmark

Chores:
- Updated GitHub workflow to specify uv tool version
- Simplified nox tool installation command

<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--3007.org.readthedocs.build/en/3007/

<!-- readthedocs-preview meltano-sdk end -->


Diff:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index d0261ba85..70efe13b5 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -76,10 +76,12 @@ jobs:
         allow-prereleases: true
 
     - uses: astral-sh/setup-uv@c7f87aa956e4c323abf06d5dec078e358f6b4d04 # v6.0.0
+      with:
+        version: ">=0.6,<0.7"
 
     - name: Install Nox
       run: |
-        uv tool install 'nox[uv]'
+        uv tool install nox
         nox --version
 
     - uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
@@ -124,10 +126,12 @@ jobs:
         python-version: ${{ env.NOXPYTHON }}
 
     - uses: astral-sh/setup-uv@c7f87aa956e4c323abf06d5dec078e358f6b4d04 # v6.0.0
+      with:
+        version: ">=0.6,<0.7"
 
     - name: Install Nox
       run: |
-        uv tool install 'nox[uv]'
+        uv tool install nox
         nox --version
 
     - name: Run Nox
@@ -156,10 +160,12 @@ jobs:
         merge-multiple: true
 
     - uses: astral-sh/setup-uv@c7f87aa956e4c323abf06d5dec078e358f6b4d04 # v6.0.0
+      with:
+        version: ">=0.6,<0.7"
 
     - name: Install Nox
       run: |
-        uv tool install 'nox[uv]'
+        uv tool install nox
         nox --version
 
     - run: nox --install-only
diff --git a/tests/core/test_typing.py b/tests/core/test_typing.py
index 1ebc5141e..502538c8b 100644
--- a/tests/core/test_typing.py
+++ b/tests/core/test_typing.py
@@ -4,6 +4,7 @@
 
 import datetime
 import decimal
+import itertools
 import logging
 import typing as t
 
@@ -25,6 +26,9 @@
     to_sql_type,
 )
 
+if t.TYPE_CHECKING:
+    from pytest_benchmark.fixture import BenchmarkFixture
+
 logger = logging.getLogger("log")
 
 
@@ -430,3 +434,55 @@ def test_iterate_properties_list():
     assert primitive_property in properties_list
     assert object_property in properties_list
     assert list_property in properties_list
+
+
+def test_bench_conform_record_data_types(benchmark: BenchmarkFixture):
+    """Run benchmark for conform_record_data_types method."""
+    number_of_runs = 1_000
+    schema = {
+        "type": "object",
+        "properties": {
+            "id": {"type": "integer"},
+            "created_at": {"type": "string", "format": "date-time"},
+            "count": {"type": "integer"},
+            "amount": {"type": "number"},
+            "is_active": {"type": "boolean"},
+            "tags": {"type": "array", "items": {"type": "string"}},
+            "metadata": {
+                "type": "object",
+                "properties": {
+                    "key": {"type": "string"},
+                    "value": {"type": "string"},
+                },
+            },
+            "nested": {
+                "type": "object",
+                "properties": {},
+                "additionalProperties": True,
+            },
+            "is_nan": {"type": "number"},
+        },
+    }
+    record = {
+        "id": 1,
+        "created_at": datetime.datetime(2021, 1, 1, tzinfo=datetime.timezone.utc),
+        "count": 10,
+        "amount": 100.0,
+        "is_active": True,
+        "tags": ["tag1", "tag2"],
+        "metadata": {"key": "value"},
+        "nested": {"key": "value", "deep": {"even_deeper": "value"}},
+        "is_nan": float("nan"),
+    }
+
+    def run_conform_record_data_types():
+        for rec in itertools.repeat(record, number_of_runs):
+            conform_record_data_types(
+                "test_stream",
+                rec,
+                schema,
+                TypeConformanceLevel.RECURSIVE,
+                logger,
+            )
+
+    benchmark(run_conform_record_data_types)


Reviews:
user: login: sourcery-ai[bot]
body: Hey @edgarrmondragon - I've reviewed your changes - here's some feedback:

- Consider benchmarking a single call to `conform_record_data_types` directly, letting `pytest-benchmark` handle the iterations.
- Consider moving the CI workflow updates (`setup-uv` version, `nox` installation) to a separate pull request.

<details>
<summary>Here's what I looked at during the review</summary>

- 🟡 **General issues**: 2 issues found
- 🟢 **Security**: all looks good
- 🟢 **Testing**: all looks good
- 🟢 **Documentation**: all looks good
</details>

***

<details>
<summary>Sourcery is free for open source - if you like our reviews please consider sharing them ✨</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https://sourcery.ai)
- [Facebook](https://www.facebook.com/sharer/sharer.php?u=https://sourcery.ai)

</details>

<sub>
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
</sub>
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T02:38:23Z

Review comments:
diff_hunk: @@ -430,3 +434,55 @@ def test_iterate_properties_list():
     assert primitive_property in properties_list
     assert object_property in properties_list
     assert list_property in properties_list
+
+
+def test_bench_conform_record_data_types(benchmark: BenchmarkFixture):
path: tests/core/test_typing.py
user: login: sourcery-ai[bot]
body: **suggestion (performance):** Consider benchmarking other conformance levels

This benchmark only covers TypeConformanceLevel.RECURSIVE. To get a full performance profile, add (or parameterize) benchmarks for ROOT_ONLY and NONE.

Suggested implementation:

```python
import pytest
from your_module import TypeConformanceLevel  # Adjust the import path as necessary

@pytest.mark.parametrize("conformance_level", [
    TypeConformanceLevel.RECURSIVE,
    TypeConformanceLevel.ROOT_ONLY,
    TypeConformanceLevel.NONE,
])
def test_bench_conform_record_data_types(benchmark: BenchmarkFixture, conformance_level):

```

If your test logic (inside test_bench_conform_record_data_types) calls a function that takes a conformance level (for example, conform_record_data_types(..., conformance_level)), update that call to pass the conformance_level variable.
Also, verify that the TypeConformanceLevel import is correct relative to your codebase, and adjust the import path if needed.
created_at: 2025-04-30T02:38:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -430,3 +434,55 @@ def test_iterate_properties_list():
     assert primitive_property in properties_list
     assert object_property in properties_list
     assert list_property in properties_list
+
+
+def test_bench_conform_record_data_types(benchmark: BenchmarkFixture):
+    """Run benchmark for conform_record_data_types method."""
+    number_of_runs = 1_000
+    schema = {
+        "type": "object",
+        "properties": {
+            "id": {"type": "integer"},
+            "created_at": {"type": "string", "format": "date-time"},
+            "count": {"type": "integer"},
+            "amount": {"type": "number"},
+            "is_active": {"type": "boolean"},
+            "tags": {"type": "array", "items": {"type": "string"}},
+            "metadata": {
+                "type": "object",
+                "properties": {
+                    "key": {"type": "string"},
+                    "value": {"type": "string"},
+                },
+            },
+            "nested": {
+                "type": "object",
+                "properties": {},
+                "additionalProperties": True,
+            },
+            "is_nan": {"type": "number"},
+        },
+    }
+    record = {
+        "id": 1,
+        "created_at": datetime.datetime(2021, 1, 1, tzinfo=datetime.timezone.utc),
+        "count": 10,
+        "amount": 100.0,
+        "is_active": True,
+        "tags": ["tag1", "tag2"],
+        "metadata": {"key": "value"},
+        "nested": {"key": "value", "deep": {"even_deeper": "value"}},
+        "is_nan": float("nan"),
+    }
+
+    def run_conform_record_data_types():
+        for rec in itertools.repeat(record, number_of_runs):
+            conform_record_data_types(
+                "test_stream",
+                rec,
+                schema,
+                TypeConformanceLevel.RECURSIVE,
+                logger,
path: tests/core/test_typing.py
user: login: sourcery-ai[bot]
body: **suggestion (performance):** Consider letting `pytest-benchmark` handle iterations directly

Pass the function and its args directly to pytest-benchmark instead of looping manually:
```python
benchmark(
    conform_record_data_types,
    "test_stream",
    record,
    schema,
    TypeConformanceLevel.RECURSIVE,
    logger,
)
```
pytest-benchmark auto-determines run counts; manual loops are only needed for extremely fast calls.

Suggested implementation:

```python
    def run_conform_record_data_types(benchmark):
        benchmark(
            conform_record_data_types,
            "test_stream",
            record,
            schema,
            TypeConformanceLevel.RECURSIVE,
            logger,
        )

```

Ensure that any tests calling run_conform_record_data_types are updated to pass in the benchmark fixture. For example, if this is used in a pytest test function, you may need to change the test signature to accept the benchmark parameter and pass it to run_conform_record_data_types.
created_at: 2025-04-30T02:38:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -430,3 +434,55 @@ def test_iterate_properties_list():
     assert primitive_property in properties_list
     assert object_property in properties_list
     assert list_property in properties_list
+
+
+def test_bench_conform_record_data_types(benchmark: BenchmarkFixture):
+    """Run benchmark for conform_record_data_types method."""
+    number_of_runs = 1_000
+    schema = {
+        "type": "object",
+        "properties": {
+            "id": {"type": "integer"},
+            "created_at": {"type": "string", "format": "date-time"},
+            "count": {"type": "integer"},
+            "amount": {"type": "number"},
+            "is_active": {"type": "boolean"},
+            "tags": {"type": "array", "items": {"type": "string"}},
+            "metadata": {
+                "type": "object",
+                "properties": {
+                    "key": {"type": "string"},
+                    "value": {"type": "string"},
+                },
+            },
+            "nested": {
+                "type": "object",
+                "properties": {},
+                "additionalProperties": True,
+            },
+            "is_nan": {"type": "number"},
+        },
+    }
+    record = {
+        "id": 1,
+        "created_at": datetime.datetime(2021, 1, 1, tzinfo=datetime.timezone.utc),
+        "count": 10,
+        "amount": 100.0,
+        "is_active": True,
+        "tags": ["tag1", "tag2"],
+        "metadata": {"key": "value"},
+        "nested": {"key": "value", "deep": {"even_deeper": "value"}},
+        "is_nan": float("nan"),
+    }
+
+    def run_conform_record_data_types():
+        for rec in itertools.repeat(record, number_of_runs):
+            conform_record_data_types(
+                "test_stream",
+                rec,
+                schema,
+                TypeConformanceLevel.RECURSIVE,
+                logger,
+            )
path: tests/core/test_typing.py
user: login: sourcery-ai[bot]
body: **issue (code-quality):** Avoid loops in tests. ([`no-loop-in-tests`](https://docs.sourcery.ai/Reference/Rules-and-In-Line-Suggestions/Python/Default-Rules/no-loop-in-tests))

<details><summary>Explanation</summary>Avoid complex code, like loops, in test functions.

Google's software engineering guidelines says:
"Clear tests are trivially correct upon inspection"
To reach that avoid complex code in tests:
* loops
* conditionals

Some ways to fix this:

* Use parametrized tests to get rid of the loop.
* Move the complex logic into helpers.
* Move the complex part into pytest fixtures.

> Complexity is most often introduced in the form of logic. Logic is defined via the imperative parts of programming languages such as operators, loops, and conditionals. When a piece of code contains logic, you need to do a bit of mental computation to determine its result instead of just reading it off of the screen. It doesn't take much logic to make a test more difficult to reason about.

Software Engineering at Google / [Don't Put Logic in Tests](https://abseil.io/resources/swe-book/html/ch12.html#donapostrophet_put_logic_in_tests)
</details>
created_at: 2025-04-30T02:38:23Z
author_association: CONTRIBUTOR

PR comments:
user: login: sourcery-ai[bot]
created_at: 2025-04-30T02:11:47Z
body: <!-- Generated by sourcery-ai[bot]: start review_guide -->

## Reviewer's Guide

This pull request introduces a benchmark test for the `conform_record_data_types` function using `pytest-benchmark` and updates the GitHub Actions workflow configuration.

### File-Level Changes

| Change | Details | Files |
| ------ | ------- | ----- |
| Add performance benchmark for `conform_record_data_types`. | <ul><li>Added a new test function `test_bench_conform_record_data_types` utilizing the `pytest-benchmark` fixture.</li><li>Defined a complex schema and a corresponding record for benchmarking.</li><li>Configured the benchmark to repeatedly execute `conform_record_data_types` for performance measurement.</li><li>Included a type-checking import for `BenchmarkFixture`.</li></ul> | `tests/core/test_typing.py` |
| Update GitHub Actions workflow configuration. | <ul><li>Updated the version constraint for the `astral-sh/setup-uv` action.</li><li>Modified the `uv tool install` command for installing Nox.</li></ul> | `.github/workflows/test.yml` |

---

<details>
<summary>Tips and commands</summary>

#### Interacting with Sourcery

- **Trigger a new review:** Comment `@sourcery-ai review` on the pull request.
- **Continue discussions:** Reply directly to Sourcery's review comments.
- **Generate a GitHub issue from a review comment:** Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with `@sourcery-ai issue` to create an issue from it.
- **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull
  request title to generate a title at any time. You can also comment
  `@sourcery-ai title` on the pull request to (re-)generate the title at any time.
- **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment `@sourcery-ai summary` on the pull request to
  (re-)generate the summary at any time.
- **Generate reviewer's guide:** Comment `@sourcery-ai guide` on the pull
  request to (re-)generate the reviewer's guide at any time.
- **Resolve all Sourcery comments:** Comment `@sourcery-ai resolve` on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
- **Dismiss all Sourcery reviews:** Comment `@sourcery-ai dismiss` on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  `@sourcery-ai review` to trigger a new review!

#### Customizing Your Experience

Access your [dashboard](https://app.sourcery.ai) to:
- Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
- Change the review language.
- Add, remove or edit custom review instructions.
- Adjust other review settings.

#### Getting Help

- [Contact our support team](mailto:support@sourcery.ai) for questions or feedback.
- Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information.
- Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).

</details>

<!-- Generated by sourcery-ai[bot]: end review_guide -->
author_association: CONTRIBUTOR
user: login: codspeed-hq[bot]
created_at: 2025-04-30T02:15:35Z
body: <!-- __CODSPEED_PERFORMANCE_REPORT_COMMENT__ -->
<!-- __CODSPEED_INSTRUMENTATION_PERFORMANCE_REPORT_COMMENT__ -->

## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/test%2Fbench-conform-data-types)

### Merging #3007 will **not alter performance**

<sub>Comparing <code>test/bench-conform-data-types</code> (f9a5424) with <code>main</code> (43b48bb)</sub>



### Summary

`✅ 7` untouched benchmarks  
`🆕 1` new benchmarks  



### Benchmarks breakdown

|     | Benchmark | `BASE` | `HEAD` | Change |
| --- | --------- | ----------------------- | ------------------- | ------ |
| 🆕 | `` test_bench_conform_record_data_types `` | N/A | 73.5 ms | N/A |

author_association: NONE
user: login: codecov[bot]
created_at: 2025-04-30T02:30:26Z
body: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/3007?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 91.53%. Comparing base ([`43b48bb`](https://app.codecov.io/gh/meltano/sdk/commit/43b48bbe7d1e2731df91fcec6dcc683696b613d9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano)) to head ([`f9a5424`](https://app.codecov.io/gh/meltano/sdk/commit/f9a5424f5d4eb5a547d04bcbb6b1c5b6a0c6c567?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano)).
:warning: Report is 198 commits behind head on main.

<details><summary>Additional details and impacted files</summary>



```diff
@@           Coverage Diff           @@
##             main    #3007   +/-   ##
=======================================
  Coverage   91.53%   91.53%           
=======================================
  Files          63       63           
  Lines        5319     5319           
  Branches      683      683           
=======================================
  Hits         4869     4869           
  Misses        318      318           
  Partials      132      132           
```
</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/3007?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).
<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE