Repository: meltano/meltano (language: python)

Issue:
feature: Use `elt.buffer_size` setting in `meltano config ... test`
### Feature scope

CLI (options, error messages, logging, etc.)

### Description

The setting is currently used in

https://github.com/meltano/meltano/blob/dce4d361bfdbb7bf1dd832caaadf27ef401cf301/src/meltano/core/block/singer.py#L327-L337

to pass a custom `limit`[^1] value to `asyncio.create_subprocess_exec`.

Since the extractor validator also calls `plugin_invoker.invoke_async`, it could pass the `limit` value too:

https://github.com/meltano/meltano/blob/dce4d361bfdbb7bf1dd832caaadf27ef401cf301/src/meltano/core/plugin_test_service.py#L77-L80

[^1]: https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.create_subprocess_exec


Discussion / hints:

Fix patch:
diff --git a/pyproject.toml b/pyproject.toml
index b03be6a6a5..dae84fa449 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -216,7 +216,6 @@ module = [
   "meltano.core.models",
   "meltano.core.plugin_install_service",
   "meltano.core.plugin_invoker",
-  "meltano.core.plugin_test_service",
   "meltano.core.project_add_service",
   "meltano.core.project_files",
   "meltano.core.project_plugins_service",
diff --git a/src/meltano/cli/config.py b/src/meltano/cli/config.py
index d18c484a27..cee99a66df 100644
--- a/src/meltano/cli/config.py
+++ b/src/meltano/cli/config.py
@@ -433,9 +433,14 @@ async def test(
         reason=PluginInstallReason.AUTO,
     )
 
+    stream_buffer_size = project.settings.get("elt.buffer_size")
+    line_length_limit = stream_buffer_size // 2
+
     try:
         async with plugin_test_service.plugin_invoker.prepared(session):
-            is_valid, detail = await plugin_test_service.validate()
+            is_valid, detail = await plugin_test_service.validate(
+                limit=line_length_limit,
+            )
     except Exception:
         tracker.track_command_event(CliEvent.failed)
         raise


Test patch:
diff --git a/src/meltano/core/plugin_test_service.py b/src/meltano/core/plugin_test_service.py
index c3b8d2ddd2..7e0373d1d6 100644
--- a/src/meltano/core/plugin_test_service.py
+++ b/src/meltano/core/plugin_test_service.py
@@ -58,14 +58,14 @@ def __init__(self, plugin_invoker: PluginInvoker):
         self.plugin_invoker = plugin_invoker
 
     @abstractmethod
-    async def validate(self) -> tuple[bool, str]:
+    async def validate(self, **kwargs: t.Any) -> tuple[bool, str | None]:
         """Abstract method to validate plugin configuration."""
 
 
 class ExtractorTestService(PluginTestService):
     """Handle extractor test operations."""
 
-    async def validate(self) -> tuple[bool, str | None]:
+    async def validate(self, **kwargs: t.Any) -> tuple[bool, str | None]:
         """Validate extractor configuration.
 
         Returns:
@@ -77,6 +77,7 @@ async def validate(self) -> tuple[bool, str | None]:
             process = await self.plugin_invoker.invoke_async(
                 stdout=asyncio.subprocess.PIPE,
                 stderr=asyncio.subprocess.STDOUT,
+                **kwargs,
             )
         except Exception as exc:
             return False, str(exc)