Repository: bentoml/BentoML

Issue:
bug: Can't modify BENTOML_HOME when running build_bentofile from Python
### Describe the bug

Changes to the BentoML environment configuration made via the Python APIs don't get propagated to the `build` workflow because the build now runs in a subprocess in `v1.1`.

### To reproduce

I have a Python app that's sort of a Bento "build server", so it needs to build Bentos in temporary directories. In BentoML v1.0, I used the following code to accomplish this:

```
with tempfile.TemporaryDirectory() as tmp, \
     bentoml.bentos.BentoMLContainer.bentoml_home.patch(tmp):
    ... create the model
    bentoml.<framework>.save_model(...)
    ... create bentofile.yaml under tmp
    bentoml.bentos.build_bentofile(build_ctx=tmp)
```

In v1.1, the implementation of `build_bentofile` has changed to delegate to the CLI app running in a subprocess. As a result, it has a different copy of the configuration objects, and always uses whatever is set for `BENTOML_HOME` at the system level. This results in an error like:

```
Error: [bentoml-cli] `build` failed: no Models with name <name> exist in BentoML store <osfs '/root/bentoml/models'>
```

because `BENTOML_HOME` was (the value of) `tmp` when `save_model()` was called, but in the build it has reverted to its default value of `~/bentoml`.

### Expected behavior

The build should run as though `BENTOML_HOME` had been set to the temporary directory, i.e., the equivalent of `BENTOML_HOME=<value of tmp> bentoml build ...`

### Environment

bentoml: 1.1
python: 3.8
Linux in a Docker container


Discussion:

Resolving patch:
diff --git a/src/bentoml/bentos.py b/src/bentoml/bentos.py
--- a/src/bentoml/bentos.py
+++ b/src/bentoml/bentos.py
@@ -369,6 +369,9 @@ def build(
         build_args.extend(["--version", version])
     build_args.extend(["--output", "tag"])
 
+    copied = os.environ.copy()
+    copied.setdefault("BENTOML_HOME", BentoMLContainer.bentoml_home.get())
+
     with tempfile.NamedTemporaryFile(
         "w", encoding="utf-8", prefix="bentoml-build-", suffix=".yaml"
     ) as f:
@@ -376,16 +379,25 @@ def build(
         bentofile_path = os.path.join(os.path.dirname(f.name), f.name)
         build_args.extend(["--bentofile", bentofile_path])
         try:
-            output = subprocess.check_output(build_args)
+            return get(
+                _parse_tag_from_outputs(
+                    subprocess.check_output(build_args, env=copied)
+                ),
+                _bento_store=_bento_store,
+            )
         except subprocess.CalledProcessError as e:
-            logger.error("Failed to build BentoService bundle: %s", e)
-            raise
+            raise BentoMLException(
+                f"Failed to build BentoService bundle (Lookup for traceback):\n{e}"
+            ) from e
+
 
-    pattern = r"^__tag__:[^:\n]+:[^:\n]+"
-    matched = re.search(pattern, output.decode("utf-8").strip(), re.MULTILINE)
-    assert matched is not None, f"Failed to find tag from output: {output}"
-    _, _, tag = matched.group(0).partition(":")
-    return get(tag, _bento_store=_bento_store)
+def _parse_tag_from_outputs(output: bytes) -> str:
+    matched = re.match(r"__tag__:([^:\n]+:[^:\n]+)$", output.decode("utf-8").strip())
+    if matched is None:
+        raise BentoMLException(
+            f"Failed to find tag from output: {output}\nNote: Output from 'bentoml build' might not be correct. Please open an issue on GitHub."
+        )
+    return matched.group(1)
 
 
 @inject
@@ -421,17 +433,17 @@ def build_bentofile(
         build_args.extend(["--version", version])
     build_args.extend(["--bentofile", bentofile, "--output", "tag"])
 
+    copied = os.environ.copy()
+    copied.setdefault("BENTOML_HOME", BentoMLContainer.bentoml_home.get())
     try:
-        output = subprocess.check_output(build_args)
+        return get(
+            _parse_tag_from_outputs(subprocess.check_output(build_args, env=copied)),
+            _bento_store=_bento_store,
+        )
     except subprocess.CalledProcessError as e:
-        logger.error("Failed to build BentoService bundle: %s", e)
-        raise
-
-    pattern = r"^__tag__:[^:\n]+:[^:\n]+"
-    matched = re.search(pattern, output.decode("utf-8").strip(), re.MULTILINE)
-    assert matched is not None, f"Failed to find tag from output: {output}"
-    _, _, tag = matched.group(0).partition(":")
-    return get(tag, _bento_store=_bento_store)
+        raise BentoMLException(
+            f"Failed to build BentoService bundle (Lookup for traceback):\n{e}"
+        ) from e
 
 
 def containerize(bento_tag: Tag | str, **kwargs: t.Any) -> bool:


Test patch: