internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: bentoml/BentoML

Issue:
Add --print-location option to `bentoml get`  command
**Is your feature request related to a problem? Please describe.**
See related discussions here: https://github.com/bentoml/BentoML/pull/810#pullrequestreview-432731488

Currently, we've been using `jq` to do this in our documentation:
```bash
$ bentoml get IrisClassifier:latest -q | jq -r ".uri.uri"

/Users/chaoyu/bentoml/repository/IrisClassifier/20200616144911_7CE204
```
But this approach requires installing `jq`

**Describe the solution you'd like**

Would be great to allow the user to get just the saved location, without jq installed, from the CLI:
```bash
$ bentoml get IrisClassifier:latest --print-location

/Users/chaoyu/bentoml/repository/IrisClassifier/20200616144911_7CE204
```

**Describe alternatives you've considered**

A general json selector syntax that works for all JSON CLI output and allow the user to select other fields in the JSON output?

e.g.
```
$ bentoml get IrisClassifier:latest --json-output ".uri.uri"

/Users/chaoyu/bentoml/repository/IrisClassifier/20200616144911_7CE204
```

It can use the same jq syntax, with something like https://github.com/doloopwhile/pyjq

**Additional context**
Add any other context or screenshots about the feature request here.



Discussion:
@parano would love to take this on!

Resolving patch:
diff --git a/bentoml/cli/bento.py b/bentoml/cli/bento.py
--- a/bentoml/cli/bento.py
+++ b/bentoml/cli/bento.py
@@ -92,10 +92,11 @@ def add_bento_sub_command(cli):
         '--limit', type=click.INT, help='Limit how many resources will be retrieved'
     )
     @click.option('--ascending-order', is_flag=True)
+    @click.option('--print-location', is_flag=True)
     @click.option(
         '-o', '--output', type=click.Choice(['json', 'yaml', 'table', 'wide'])
     )
-    def get(bento, limit, ascending_order, output):
+    def get(bento, limit, ascending_order, print_location, output):
         if ':' in bento:
             name, version = bento.split(':')
         else:
@@ -117,6 +118,9 @@ def get(bento, limit, ascending_order, output):
                     CLI_COLOR_ERROR,
                 )
                 return
+            if print_location:
+                _echo(get_bento_result.bento.uri.uri)
+                return
             _print_bento_info(get_bento_result.bento, output)
             return
         elif name:


Test patch:
Repository: bentoml/BentoML

Issue:
Failed to package local pip installed dependency in 0.8.4
An issue first reported in #bentoml-users slack channel by @fernandocamargoti

In some python environment, we see the following error when doing a `BenoService#save`:
```
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

The reason is some packages ended up with a `module_path == None` when accessing form `pkg_resources.working_set`:
```python
import pkg_resources
[dist for dist in pkg_resources.working_set if not dist.module_path]
Out[4]: [terminado 0.8.3 (/home/fernandocamargo/anaconda3/envs/logit/lib/python3.6/site-packages)]
```

Here is a conda environment yaml file that can reproduce this issue:
```
name: logit
channels:
#- anaconda
- conda-forge
- anaconda
- pytorch
- https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
- powerai
dependencies:
- python=3.6
- pandas=1.0
- pyarrow=0.15
- matplotlib=3.1
- plotly=4.6
- scipy=1.3
- numpy=1.16
- seaborn=0.10
- scikit-learn=0.22
- pytorch=1.2
- statsmodels=0.11
- holidays=0.9
- convertdate=2.2
- python-dateutil=2.8
- fbprophet=0.6
- luigi=2.8
- tqdm=4.44
- jupyterlab=1.2
- ipywidgets=7.5
- pyspark=2.4.5
- psutil=5.5
- pydantic=1.6
- sqlalchemy=1.3
- psycopg2=2.8
# GPU:
- cudatoolkit=10.1
- pip
- pip:
#  CPU:
#  - mxnet==1.4.1
#  GPU:
  - mxnet-cu101mkl==1.4.1
#  Power-9:
#  - /root/packages/mxnet/mxnet-1.4.1-cp36-cp36m-linux_ppc64le.whl
  - gluonts==0.5
  - pmdarima==1.4
  - torchbearer==0.5
  - imbalanced-learn==0.6
  - tensorboardx==2.0
  - diskcache==3.0
  - bentoml==0.8.3
```

Interestingly, installing terminado alone does not cause this problem. I'm not sure if this is caused by other packages in this list. But will submit a fix to ensure BentoML handles this scenario properly.

We will release a new version including this fix very soon.



Discussion:

Resolving patch:
diff --git a/bentoml/saved_bundle/pip_pkg.py b/bentoml/saved_bundle/pip_pkg.py
--- a/bentoml/saved_bundle/pip_pkg.py
+++ b/bentoml/saved_bundle/pip_pkg.py
@@ -82,9 +82,14 @@ def __init__(self):
         import pkg_resources
 
         for dist in pkg_resources.working_set:  # pylint: disable=not-an-iterable
-            if os.path.realpath(dist.module_path) != os.getcwd():
+            module_path = dist.module_path or dist.location
+            if not module_path:
+                # Skip if no module path was found for pkg distribution
+                continue
+
+            if os.path.realpath(module_path) != os.getcwd():
                 # add to nonlocal_package path only if it's not current directory
-                self.nonlocal_package_path.add(dist.module_path)
+                self.nonlocal_package_path.add(module_path)
 
             self.pip_pkg_map[dist._key] = dist._version
             for mn in dist._get_metadata("top_level.txt"):


Test patch:
Repository: bentoml/BentoML

Issue:
Add console log to prediction
**Is your feature request related to a problem? Please describe.**
When running bentoml in a docker container, request logs are not printed to standard out.

**Describe the solution you'd like**
In a [slack discussion](https://bentoml.slack.com/archives/CKRANBHPH/p1597270160005700), the consensus is to add the console log to prediction by default. One potential solution is to add `console` to the prediction's [handler list](https://github.com/bentoml/BentoML/blob/master/bentoml/utils/log.py#L90).

**Describe alternatives you've considered**
It was suggested to us to use the existing log files from the workers. But, this introduces complication where we have to mount the log folder in container to the host so that datadog agent may have access to the log and forward it to datadog. Also, we will need to take care of log rotation so that it doesn't grow big. If we have many bentoml services, we have to do this for each of them. 

**Additional context**
The standard out logging is important for our application monitoring such as Datadog. The standard out logging allows the datadog agent to pick up the log messages and forward them to datadog.



Discussion:

Resolving patch:
diff --git a/bentoml/utils/log.py b/bentoml/utils/log.py
--- a/bentoml/utils/log.py
+++ b/bentoml/utils/log.py
@@ -87,12 +87,12 @@ def get_logging_config_dict(logging_level, base_log_directory):
                 "propagate": False,
             },
             "bentoml.prediction": {
-                "handlers": ["prediction"],
+                "handlers": ["prediction", "console"],
                 "level": "INFO",
                 "propagate": False,
             },
             "bentoml.feedback": {
-                "handlers": ["feedback"],
+                "handlers": ["feedback", "console"],
                 "level": "INFO",
                 "propagate": False,
             },


Test patch:
Repository: bentoml/BentoML

Issue:
Respect Docker env variables when creating docker API client in bentoml containerize
**Is your feature request related to a problem? Please describe.**
When building Docker images for a private Docker registry, Docker authentication is required.  However, since BentoML uses `docker.APIClient()` instead of `docker.from_env().api` to create the API client, environment variables aren't read (including ones important for authentication).  Everything works fine if you perform a docker login first, but that means that my CI build needs to install the docker CLI (or use a custom image with docker already installed), which is a minor annoyance.

**Describe the solution you'd like**
Switch out `docker.APIClient()` for `docker.from_env().api` (specifically in the bentoml containerize call)

**Describe alternatives you've considered**
`sed 's/docker.APIClient()/docker.from_env().api/g' -i [somepath]/bento_service.py` is the simple monkey patch I'm using at the moment.  Alternatively, you can install docker in the CI builder and call login.

Happy to write the PR myself if you like, though I figured that opening a new pull request and submitting it may be more work than changing the code in this case.



Discussion:
Ah this was an oversight from my end when implementing `containerize`. Will make a patch PR in a few minutes! Thanks
Awesome, thanks!

Resolving patch:
diff --git a/bentoml/cli/bento_service.py b/bentoml/cli/bento_service.py
--- a/bentoml/cli/bento_service.py
+++ b/bentoml/cli/bento_service.py
@@ -406,7 +406,7 @@ def containerize(bento, push, tag, build_arg, username, password):
 
         import docker
 
-        docker_api = docker.APIClient()
+        docker_api = docker.from_env().api
         try:
             with Spinner(f"Building Docker image {tag} from {bento} \n"):
                 for line in echo_docker_api_result(


Test patch:
Repository: bentoml/BentoML

Issue:
[Bug]Incorrect endpoint API description on swagger UI
**Describe the bug**
<!--- A clear and concise description of what the bug is. -->
When a user adds a docstring on an API endpoint, that docstring should be the description on the Swagger UI page. RIght now the default message is displayed instead. 

```python
class MyService(bentoml.BentoService):
    @bentoml.api(...)
    def predict(self, input):
        """This message should be displayed.
        """ 
```
Instead of the default message in the screenshot.
<img width="1635" alt="Screen Shot 2020-12-02 at 4 58 27 PM" src="https://user-images.githubusercontent.com/670949/100968382-17b82a80-34e6-11eb-9ec8-99b12a0ef04a.png">


**To Reproduce**
<!--
Steps to reproduce the issue:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
-->

**Expected behavior**
<!--- A clear and concise description of what you expected to happen. -->

**Screenshots/Logs**
<!--- 
If applicable, add screenshots, logs or error outputs to help explain your problem.

To give us more information for diagnosing the issue, make sure to enable debug logging:

Enable via environment variable, e.g.:
```
$ git clone git@github.com:bentoml/BentoML.git && cd bentoml
$ BENTOML__CORE__DEBUG=true python guides/quick-start/main.py
```

Or enable for all python sessions on current machine:
```bash
$ bentoml config set core.debug=true
$ python guides/quick-start/main.py
```

Or set debug logging in your Python code:
```python
import bentoml
import logging
bentoml.config().set('core', 'debug', 'true')
bentoml.configure_logging(logging.DEBUG)
```

For BentoML CLI commands, simply add the `--verbose` flag, e.g.:
```bash
bentoml get IrisClassifier --verbose
```

-->


**Environment:**
 - OS: [e.g. MacOS 10.14.3]
 - Python Version [e.g. Python 3.7.1]
 - BentoML Version [e.g. BentoML-0.8.6]


**Additional context**
<!-- Add any other context about the problem here. e.g. links to related discussion. -->



Discussion:

Resolving patch:
diff --git a/bentoml/service/__init__.py b/bentoml/service/__init__.py
--- a/bentoml/service/__init__.py
+++ b/bentoml/service/__init__.py
@@ -110,6 +110,7 @@ def api_decorator(
     def decorator(func):
         _api_name = func.__name__ if api_name is None else api_name
         validate_inference_api_name(_api_name)
+        _api_doc = func.__doc__ if api_doc is None else api_doc
 
         if input is None:
             # Raise error when input adapter class passed without instantiation
@@ -138,7 +139,7 @@ def decorator(func):
         setattr(func, "_input_adapter", input_adapter)
         setattr(func, "_output_adapter", output_adapter)
         setattr(func, "_api_name", _api_name)
-        setattr(func, "_api_doc", api_doc)
+        setattr(func, "_api_doc", _api_doc)
         setattr(func, "_mb_max_batch_size", mb_max_batch_size)
         setattr(func, "_mb_max_latency", mb_max_latency)
         setattr(func, "_batch", batch)


Test patch:
Repository: bentoml/BentoML

Issue:
Deployment on remote Yatai server fails due to injection issue
**Describe the bug**

Attempting to deploy to SageMaker or Lambda fails with this error:

```
Error: sagemaker deploy failed: INTERNAL:<dependency_injector.wiring.Provide object at 0x11f748be0> has type Provide, but expected one of: bytes, unicode
```

**To Reproduce**

**This is based on the latest version of the code as of this writing**

- Start remote Yatai server
- Configure BentoML to use the remote Yatai server (e.g. by modifying `default_bentoml.yml`
- Start a deployment to SageMaker or Lambda (without specifying a namespace through the `--namespace` option
- The error message above is shown

**Expected behavior**
Deployment should proceed normally, and the error message should not be displayed.

**Environment:**
- BentoML version 0.12.1+24.g4019bac.dirty


**Additional context**

After some initial debugging, the error appears to originate from this line: https://github.com/bentoml/BentoML/blob/4019bac4af320bad73bf960f6bd2d617f3fd4a52/bentoml/yatai/yatai_service_impl.py#L106

`self.default_namespace` is not wired / injected properly, and will instead be a `Provide` object. This causes issues downstream as a string is expected. A workaround is to specify the environment when deploying via the CLI (`--namespace`).

My hunch is that `YataiServiceImpl` does not get properly wired/injected due to it being wrapped in the `get_yatai_service_impl` method here:https://github.com/bentoml/BentoML/blob/4019bac4af320bad73bf960f6bd2d617f3fd4a52/bentoml/yatai/yatai_service_impl.py#L74 

I have little experience with dependency injection so couldn't figure out _why_ it wouldn't get wired properly.





Discussion:
Thank you for digging into this issue.  I will take a look with the latest master

Resolving patch:
diff --git a/bentoml/configuration/__init__.py b/bentoml/configuration/__init__.py
--- a/bentoml/configuration/__init__.py
+++ b/bentoml/configuration/__init__.py
@@ -146,10 +146,11 @@ def inject_dependencies():
         service,
     )
     from bentoml.yatai import yatai_service
+    from bentoml.yatai import yatai_service_impl
     from bentoml.yatai.repository import s3_repository, gcs_repository
 
     container.wire(
-        modules=[yatai_service, s3_repository, gcs_repository],
+        modules=[yatai_service, s3_repository, gcs_repository, yatai_service_impl],
         packages=[marshal, server, tracing, cli, adapters, saved_bundle, service],
     )
 


Test patch:
Repository: bentoml/BentoML

Issue:
TF2 tutorial fails: 'TensorflowRunnable' has no attribute 'SUPPORT_NVIDIA_GPU'
**Describe the bug**
<!--- A clear and concise description of what the bug is. -->
This bug occurs when trying to serve a bento based on TensorflowRunnable in production mode, i.e.
`bentoml serve your-model:latest --production`

This calls 'serve_production(...)' which in turn calls 'get_worker_count(...)'. The latter requires the runnable to have the attribute 'SUPPORT_NVIDIA_GPU'. However, TensorflowRunnable instead has an attrribute 'SUPPORT_GPU'. 

https://github.com/bentoml/BentoML/blob/8e3713e8797a1b820b3489d66d2debe44ee5198d/bentoml/_internal/runner/strategy.py#L45

https://github.com/bentoml/BentoML/blob/8e3713e8797a1b820b3489d66d2debe44ee5198d/bentoml/_internal/frameworks/tensorflow_v2.py#L219

**To Reproduce**
<!--
Steps to reproduce the issue:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
-->
Build a bento based on TensorflowRunnable and try to serve it. For example https://github.com/bentoml/gallery/tree/main/tensorflow2

**Expected behavior**
<!--- A clear and concise description of what you expected to happen. -->
The program crashes with the error 'AttributeError: type object 'TensorflowRunnable' has no attribute 'SUPPORT_NVIDIA_GPU''

**Environment:**
 - OS: ManjaroLinux 21.2.6
 - Python 3.10.4
 - BentoML 1.0.0rc0


Discussion:

Resolving patch:
diff --git a/bentoml/_internal/frameworks/keras.py b/bentoml/_internal/frameworks/keras.py
--- a/bentoml/_internal/frameworks/keras.py
+++ b/bentoml/_internal/frameworks/keras.py
@@ -259,8 +259,8 @@ def get_runnable(
     )
 
     class KerasRunnable(Runnable):
-        SUPPORT_GPU = True
-        SUPPORT_MULTI_THREADING = True
+        SUPPORT_NVIDIA_GPU = True
+        SUPPORT_CPU_MULTI_THREADING = True
 
         def __init__(self):
             super().__init__()
diff --git a/bentoml/_internal/frameworks/tensorflow_v2.py b/bentoml/_internal/frameworks/tensorflow_v2.py
--- a/bentoml/_internal/frameworks/tensorflow_v2.py
+++ b/bentoml/_internal/frameworks/tensorflow_v2.py
@@ -217,8 +217,8 @@ def get_runnable(
     )
 
     class TensorflowRunnable(Runnable):
-        SUPPORT_GPU = True
-        SUPPORT_MULTI_THREADING = True
+        SUPPORT_NVIDIA_GPU = True
+        SUPPORT_CPU_MULTI_THREADING = True
 
         def __init__(self):
             super().__init__()


Test patch:
Repository: bentoml/BentoML

Issue:
ValueError: invalid literal for int() with base 10: '$(circus.sockets.{runner_name}) when runner name has a space character in production mode
**Describe the bug**
A Runner object can be instantiated with a `name` argument. It seem as if this name contains a space seperator mentioned ValueError is raised when serving in production mode. As you can see in the screenshot, name of the runner is *Document Klassifikation*. I added some print statements starting with `>>>`:
![image](https://user-images.githubusercontent.com/39236016/173335889-cf02fb9e-b42e-4f31-a8da-9dfc3e4f3899.png)
Removing the space, `name="DocumentKlassifikation"` fixed the problem. 
I tracked the problem down to `/bentoml/_internal/server/cli/runner.py`. Seems the `runner_name` needs to be sanitized before using it for the bind I guess.

**To Reproduce**

Steps to reproduce the issue:
1. Create a Runner object 
2. set argument `name="Document Classification"
3. run `bentoml serve service.py --production`
4. See ValueError


**Expected behavior**
No ValueError is raised.

**Environment:**
 - OS: SLES 12 SP5
 - Python Version 3.10.4
 - BentoML Version bentoml, version 1.0.0rc0



Discussion:

Resolving patch:
diff --git a/bentoml/_internal/runner/runner.py b/bentoml/_internal/runner/runner.py
--- a/bentoml/_internal/runner/runner.py
+++ b/bentoml/_internal/runner/runner.py
@@ -7,6 +7,7 @@
 
 import attr
 
+from ..tag import validate_tag_str
 from ..types import ParamSpec
 from ..utils import first_not_none
 from .resource import Resource
@@ -99,7 +100,16 @@ def __init__(
             method_configs: per method configs
         """
 
-        name = runnable_class.__name__ if name is None else name
+        if name is None:
+            name = runnable_class.__name__
+
+        try:
+            validate_tag_str(name)
+        except ValueError as e:
+            # TODO: link to tag validation documentation
+            raise ValueError(
+                f"Runner name '{name}' is not valid; it must be a valid BentoML Tag name."
+            ) from e
         models = [] if models is None else models
         runner_method_map: dict[str, RunnerMethod[t.Any, t.Any, t.Any]] = {}
         runnable_init_params = (


Test patch:
Repository: bentoml/BentoML

Issue:
feature: add traceback to service import error
### Feature request

when `bentoml serve <bento> --debug` fails on bento import the error message is not helpful:

```
Error: [bentoml-cli] `serve` failed: Failed to load bento or import service ' <bento> '. 
If you are attempting to import bento in local store: `Failed to import module "pose": No module named 'pose'`, 
or if you are importing by python module path: 
`Failed to import module "service": libcudart.so.11.0: cannot open shared object file: No such file or directory`
```

It should show the ImportError traceback, at least in --debug mode.

I had to resort to:
```
cd ~/bentoml/bentos/pose/n7ptgfrkuontoatv/src/
python -c 'import service'
```


### Motivation

easier debugging import errors while serving bentos

### Other

_No response_


Discussion:
@smidm Thanks for reporting this issue. We will look to address it in the next release.

Resolving patch:
diff --git a/bentoml_cli/utils.py b/bentoml_cli/utils.py
--- a/bentoml_cli/utils.py
+++ b/bentoml_cli/utils.py
@@ -15,6 +15,7 @@
 
 from bentoml.exceptions import BentoMLException
 from bentoml._internal.log import configure_logging
+from bentoml._internal.configuration import get_debug_mode
 from bentoml._internal.configuration import set_debug_mode
 from bentoml._internal.configuration import set_quiet_mode
 from bentoml._internal.utils.analytics import track
@@ -163,8 +164,12 @@ def wrapper(*args: P.args, **kwargs: P.kwargs) -> t.Any:
             try:
                 return func(*args, **kwargs)
             except BentoMLException as err:
-                msg = f"[{cmd_group.name}] '{command_name}' failed: {str(err)}"
-                raise ClickException(click.style(msg, fg="red")) from err
+                msg = f"[{cmd_group.name}] `{command_name}` failed: {str(err)}"
+                if get_debug_mode():
+                    ClickException(click.style(msg, fg="red")).show()
+                    raise err from None
+                else:
+                    raise ClickException(click.style(msg, fg="red")) from err
 
         return t.cast("ClickFunctionWrapper[t.Any]", wrapper)
 


Test patch:
Repository: bentoml/BentoML

Issue:
[BUG] BentoML bentoml.container.build looks for "=" in platform argument 
When using the "bentoml.container.build" to containerise a service, if a platform is specified, the code in the link below looks for the "=" sign and fails with an index error:
```JSON
"name": "IndexError",
"message": "list index out of range",
```
https://github.com/bentoml/BentoML/blob/98f6f63cfe0242a8df230fed00aa323a29735372/src/bentoml/container.py#L404
Removing line 404 fixed the issue.
Example code:
```python
import bentoml
bentoml.container.build(
    bento_tag="<service>:<version>",
    tag="<service>:<version>",
    platform="linux/amd64",
)




Discussion:

Resolving patch:
diff --git a/src/bentoml/container.py b/src/bentoml/container.py
--- a/src/bentoml/container.py
+++ b/src/bentoml/container.py
@@ -398,16 +398,6 @@ def build(
     bento = _bento_store.get(bento_tag)
 
     logger.info("Building OCI-compliant image for %s with %s\n", bento.tag, backend)
-    platform = kwargs.get("platform", None)
-
-    if platform is not None:
-        passed_platform = ",".join(map(lambda v: v.split("=")[1], platform))
-        if not psutil.LINUX and "linux/amd64" not in passed_platform:
-            logger.warning(
-                'Current platform is set to "--opt platform=%s". To avoid issue, we recommend you to build the container with x86_64 (amd64): "bentoml containerize %s --opt platform=linux/amd64"',
-                passed_platform,
-                str(bento.tag),
-            )
     return _internal_build(bento_tag, backend, features=features, **kwargs)
 
 
diff --git a/src/bentoml_cli/containerize.py b/src/bentoml_cli/containerize.py
--- a/src/bentoml_cli/containerize.py
+++ b/src/bentoml_cli/containerize.py
@@ -1,6 +1,5 @@
 from __future__ import annotations
 
-import os
 import re
 import sys
 import random


Test patch: