Repository: bentoml/BentoML

Issue:
bug: unable to start BentoML container in RayServe
### Describe the bug

followed instructions at https://docs.bentoml.org/en/latest/integrations/ray.html

ray serve attempts to instantiate container, but trips up:

2023-10-30 23:45:22,446 INFO scripts.py:471 -- Running import path: 'bento_ray:classifier'.
Traceback (most recent call last):
  File "/home/thor/.local/bin/serve", line 8, in <module>
    sys.exit(cli())
  File "/home/thor/.local/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/thor/.local/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/thor/.local/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/thor/.local/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/thor/.local/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/thor/.local/lib/python3.8/site-packages/ray/serve/scripts.py", line 473, in run
    import_attr(import_path), args_dict
  File "/home/thor/.local/lib/python3.8/site-packages/ray/_private/utils.py", line 1378, in import_attr
    module = importlib.import_module(module_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/mnt/d/Data/AIRay/./bento_ray.py", line 5, in <module>
    classifier = bentoml.ray.deployment(
  File "/home/thor/.local/lib/python3.8/site-packages/bentoml/_internal/utils/lazy_loader.py", line 69, in __getattr__
    self._module = self._load()
  File "/home/thor/.local/lib/python3.8/site-packages/bentoml/_internal/utils/lazy_loader.py", line 53, in _load
    raise self._exc(f"{self._exc_msg} (reason: {err})") from None
bentoml.exceptions.MissingDependencyException: None (reason: No module named 'bentoml._internal.ray._internal')


### To reproduce

_No response_

### Expected behavior

Expected RayServe to spin up BentoML container within its cluster

### Environment

Ubuntu 20.04.6


Discussion:

Resolving patch:
diff --git a/src/bentoml/_internal/ray/__init__.py b/src/bentoml/_internal/ray/__init__.py
--- a/src/bentoml/_internal/ray/__init__.py
+++ b/src/bentoml/_internal/ray/__init__.py
@@ -5,7 +5,7 @@
 
 import requests
 
-from ._internal.utils import pkg
+from ..._internal.utils import pkg
 
 if pkg.pkg_version_info("ray")[:2] >= (2, 5):
     from ray.serve._private.http_util import BufferedASGISender as ASGIHTTPSender
diff --git a/src/bentoml/_internal/ray/runner_handle.py b/src/bentoml/_internal/ray/runner_handle.py
--- a/src/bentoml/_internal/ray/runner_handle.py
+++ b/src/bentoml/_internal/ray/runner_handle.py
@@ -60,3 +60,11 @@ async def async_run_method(
                 *args, **kwargs
             )
         )
+
+    def async_stream_method(
+        self,
+        __bentoml_method: RunnerMethod[t.Any, P, R],
+        *args: P.args,
+        **kwargs: P.kwargs,
+    ) -> t.AsyncGenerator[R, None]:
+        raise NotImplementedError


Test patch: