Repository: bentoml/BentoML
Pull request #4448 (2024-01-25 01:38:52)

fix: convert generator to async gen before response
## What does this PR address?

<!--
Thanks for sending a pull request!

Congrats for making it this far! Here's a 🍱 for you. There are still a few steps ahead.

Please make sure to read the contribution guidelines, then fill out the blanks below before requesting a code review.

Name your Pull Request with one of the following prefixes, e.g. "feat: add support for PyTorch", to indicate the type of changes proposed. This is based on the [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary).
  - feat: (new feature for the user, not a new feature for build script)
  - fix: (bug fix for the user, not a fix to a build script)
  - docs: (changes to the documentation)
  - style: (formatting, missing semicolons, etc; no production code change)
  - refactor: (refactoring production code, eg. renaming a variable)
  - perf: (code changes that improve performance)
  - test: (adding missing tests, refactoring tests; no production code change)
  - chore: (updating grunt tasks etc; no production code change)
  - build: (changes that affect the build system or external dependencies)
  - ci: (changes to configuration files and scripts)
  - revert: (reverts a previous commit)

Describe your changes in detail. Attach screenshots here if appropriate.

Once you're done with this, someone from BentoML team or community member will help review your PR (see "Who can help review?" section for potential reviewers.). If no one has reviewed your PR after a week have passed, don't hesitate to post a new comment and ping @-the same person. Notifications sometimes get lost 🥲.
-->

<!-- Remove if not applicable -->

Fixes #(issue)

## Before submitting:

<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
<!--- If you plan to update documentation or tests in follow-up, please note -->

- [ ] Does the Pull Request follow [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary) naming? Here are [GitHub's
      guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) on how to create a pull request.
- [ ] Does the code follow BentoML's code style, `pre-commit run -a` script has passed ([instructions](https://github.com/bentoml/BentoML/blob/main/DEVELOPMENT.md#style-check-auto-formatting-type-checking))?
- [ ] Did you read through [contribution guidelines](https://github.com/bentoml/BentoML/blob/main/CONTRIBUTING.md#ways-to-contribute) and follow [development guidelines](https://github.com/bentoml/BentoML/blob/main/DEVELOPMENT.md#start-developing)?
- [ ] Did your changes require updates to the documentation? Have you updated
      those accordingly? Here are [documentation guidelines](https://github.com/bentoml/BentoML/tree/main/docs) and [tips on writting docs](https://github.com/bentoml/BentoML/tree/main/docs#writing-documentation).
- [ ] Did you write tests to cover your changes?



Patch:
diff --git a/src/_bentoml_impl/server/app.py b/src/_bentoml_impl/server/app.py
index 54adafa6..a06908e0 100644
--- a/src/_bentoml_impl/server/app.py
+++ b/src/_bentoml_impl/server/app.py
@@ -428,6 +428,21 @@ class ServiceAppFactory(BaseAppFactory):
                 output = await func(*input_args, **input_params)
             elif inspect.isasyncgenfunction(original_func):
                 output = func(*input_args, **input_params)
+            elif inspect.isgeneratorfunction(original_func):
+
+                async def inner() -> t.AsyncGenerator[t.Any, None]:
+                    gen = func(*input_args, **input_params)
+                    while True:
+                        try:
+                            yield await self._to_thread(next, gen)
+                        except StopIteration:
+                            break
+                        except RuntimeError as e:
+                            if "StopIteration" in str(e):
+                                break
+                            raise
+
+                output = inner()
             else:
                 output = await self._to_thread(func, *input_args, **input_params)
 
diff --git a/src/bentoml/_internal/bento/bento.py b/src/bentoml/_internal/bento/bento.py
index 8194ed6a..17e73fd4 100644
--- a/src/bentoml/_internal/bento/bento.py
+++ b/src/bentoml/_internal/bento/bento.py
@@ -331,6 +331,7 @@ class Bento(StoreItem):
                 python=build_config.python,
                 conda=build_config.conda,
                 envs=build_config.envs,
+                schema=svc.schema() if not is_legacy else {},
             ),
         )
         # Create bento.yaml
@@ -596,12 +597,13 @@ class BentoInfo:
     # for BentoML 1.2+ SDK
     entry_service: str = attr.field(factory=str)
     services: t.List[BentoServiceInfo] = attr.field(factory=list)
+    envs: t.List[t.Dict[str, str]] = attr.field(factory=list)
+    schema: t.Dict[str, t.Any] = attr.field(factory=dict)
 
     apis: t.List[BentoApiInfo] = attr.field(factory=list)
     docker: DockerOptions = attr.field(factory=lambda: DockerOptions().with_defaults())
     python: PythonOptions = attr.field(factory=lambda: PythonOptions().with_defaults())
     conda: CondaOptions = attr.field(factory=lambda: CondaOptions().with_defaults())
-    envs: t.List[t.Dict[str, str]] = attr.field(factory=list)
 
     @property
     def all_models(self) -> t.List[BentoModelInfo]:
@@ -702,4 +704,9 @@ def _BentoInfo_dumper(dumper: yaml.Dumper, info: BentoInfo) -> yaml.Node:
     return dumper.represent_dict(info.to_dict())
 
 
+def _tuple_dumper(dumper: yaml.Dumper, info: tuple[t.Any, ...]) -> yaml.Node:
+    return dumper.represent_list(info)
+
+
 yaml.add_representer(BentoInfo, _BentoInfo_dumper)  # type: ignore (incomplete yaml types)
+yaml.add_representer(tuple, _tuple_dumper)  # type: ignore (incomplete yaml types)
diff --git a/src/bentoml/_internal/cloud/bentocloud.py b/src/bentoml/_internal/cloud/bentocloud.py
index fc6eaeb0..d7b68a8d 100644
--- a/src/bentoml/_internal/cloud/bentocloud.py
+++ b/src/bentoml/_internal/cloud/bentocloud.py
@@ -171,6 +171,7 @@ class BentoCloudClient(CloudClient):
             size_bytes=bento.total_size(),
             services=info.services,
             envs=info.envs,
+            schema=info.schema,
         )
         if not remote_bento:
             with self.spinner.spin(
diff --git a/src/bentoml/_internal/cloud/schemas/modelschemas.py b/src/bentoml/_internal/cloud/schemas/modelschemas.py
index e7cb5995..aa905dba 100644
--- a/src/bentoml/_internal/cloud/schemas/modelschemas.py
+++ b/src/bentoml/_internal/cloud/schemas/modelschemas.py
@@ -83,6 +83,7 @@ class BentoManifestSchema:
     runners: t.Optional[t.List[BentoRunnerSchema]] = attr.field(factory=list)
     services: t.List[BentoServiceInfo] = attr.field(factory=dict)
     envs: t.List[t.Dict[str, str]] = attr.field(factory=list)
+    schema: t.Dict[str, t.Any] = attr.field(factory=dict)
 
 
 if TYPE_CHECKING:
diff --git a/src/bentoml/_internal/cloud/yatai.py b/src/bentoml/_internal/cloud/yatai.py
index efd47ed2..6bba92f2 100644
--- a/src/bentoml/_internal/cloud/yatai.py
+++ b/src/bentoml/_internal/cloud/yatai.py
@@ -161,6 +161,7 @@ class YataiClient(CloudClient):
             runners=runners,
             size_bytes=bento.total_size(),
             config=info.config,
+            schema=info.schema,
         )
         if not remote_bento:
             with self.spin(text=f'Registering Bento "{bento.tag}" with Yatai..'):
diff --git a/src/bentoml_cli/deployment.py b/src/bentoml_cli/deployment.py
index 383da5c1..e36e221e 100644
--- a/src/bentoml_cli/deployment.py
+++ b/src/bentoml_cli/deployment.py
@@ -283,12 +283,12 @@ def add_deployment_command(cli: click.Group) -> None:
             raise BentoMLException(
                 f"Failed to create deployment due to invalid configuration: {e}"
             )
-        Deployment.update(
+        deployment_info = Deployment.update(
             deployment_config_params=config_params,
             context=shared_options.cloud_context,
         )
 
-        click.echo(f"Deployment '{config_params.get_name()}' updated successfully.")
+        click.echo(f"Deployment '{deployment_info.name}' updated successfully.")
 
     @deployment_cli.command()
     @click.argument(
@@ -406,12 +406,12 @@ def add_deployment_command(cli: click.Group) -> None:
             raise BentoMLException(
                 f"Failed to create deployment due to invalid configuration: {e}"
             )
-        Deployment.apply(
+        deployment_info = Deployment.apply(
             deployment_config_params=config_params,
             context=shared_options.cloud_context,
         )
 
-        click.echo(f"Deployment '{config_params.get_name()}' applied successfully.")
+        click.echo(f"Deployment '{deployment_info.name}' applied successfully.")
 
     @deployment_cli.command()
     @click.argument(


Test patch:
diff --git a/tests/unit/_internal/bento/test_bento.py b/tests/unit/_internal/bento/test_bento.py
index 2daf0059..66d1db58 100644
--- a/tests/unit/_internal/bento/test_bento.py
+++ b/tests/unit/_internal/bento/test_bento.py
@@ -103,6 +103,8 @@ runners:
     cpu: 2
 entry_service: ''
 services: []
+envs: []
+schema: {{}}
 apis:
 - name: predict
   input_type: NumpyNdarray
@@ -132,7 +134,6 @@ conda:
   channels: null
   dependencies: null
   pip: null
-envs: []
 """
 
     with open(bento_yaml_b_filename, encoding="utf-8") as bento_yaml_b: