internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: canonical/cloud-init

Issue:
fstab entries with x-systemd.requires=cloud-init.service mean cloud-init will always run.
This bug was originally filed in Launchpad as [LP: #1667878](https://bugs.launchpad.net/cloud-init/+bug/1667878)
<details>
<summary>Launchpad details</summary>
<pre>
affected_projects = []
assignee = None
assignee_name = None
date_closed = None
date_created = 2017-02-25T03:38:02.297212+00:00
date_fix_committed = None
date_fix_released = None
id = 1667878
importance = medium
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1667878
milestone = None
owner = smoser
owner_name = Scott Moser
private = False
status = confirmed
submitter = smoser
submitter_name = Scott Moser
tags = []
duplicates = []
</pre>
</details>

_Launchpad user **Scott Moser(smoser)** wrote on 2017-02-25T03:38:02.297212+00:00_

when cloud-init writes mount entries into /etc/fstab, it adds:
 x-systemd.requires=cloud-init.service

This is because a subsequent boot of cloud-init may need to remove that entry (a new instance for example).

cloud-init has a feature where it can be disabled by:
  sudo touch /etc/cloud/cloud-init.disabled

The generator for cloud-init then says that the cloud-init.target does not need to run.

However, due to the (possibly stale) x-systemd.requires=cloud-init.service entries, cloud-init will still run.

I think what I'd like is a:
  x-systemd.after=cloud-init.service


Discussion:
_Launchpad user **Scott Moser(smoser)** wrote on 2017-05-18T20:35:20.942412+00:00_

I don't know if it was present before or not, but there is now x-systemd.after

https://www.freedesktop.org/software/systemd/man/systemd.mount.html

Resolving patch:
diff --git a/cloudinit/config/cc_mounts.py b/cloudinit/config/cc_mounts.py
--- a/cloudinit/config/cc_mounts.py
+++ b/cloudinit/config/cc_mounts.py
@@ -38,7 +38,7 @@
 
     mounts:
         - ["ephemeral0", "/mnt", "auto",\
-"defaults,nofail,x-systemd.requires=cloud-init.service", "0", "2"]
+"defaults,nofail,x-systemd.after=cloud-init.service", "0", "2"]
         - ["swap", "none", "swap", "sw", "0", "0"]
 
 In order to remove a previously listed mount, an entry can be added to
@@ -51,7 +51,7 @@
 containing 6 values. It defaults to::
 
     mount_default_fields: [none, none, "auto",\
-"defaults,nofail,x-systemd.requires=cloud-init.service", "0", "2"]
+"defaults,nofail,x-systemd.after=cloud-init.service", "0", "2"]
 
 Non-systemd init systems will vary in ``mount_default_fields``.
 
@@ -427,7 +427,7 @@ def handle(name: str, cfg: Config, cloud: Cloud, args: list) -> None:
     uses_systemd = cloud.distro.uses_systemd()
     if uses_systemd:
         def_mnt_opts = (
-            "defaults,nofail,x-systemd.requires=cloud-init.service,_netdev"
+            "defaults,nofail,x-systemd.after=cloud-init.service,_netdev"
         )
 
     defvals = [None, None, "auto", def_mnt_opts, "0", "2"]


Test patch:
Repository: SeldonIO/MLServer

Issue:
Infer model's name from folder's name
Currently, particularly in the case of MMS, MLServer requires models to specify their name in a `model-settings.json` file. This forces all models to ship that file alongside their model artifacts.

It would be good to, instead, infer the model's name from the folder name (if not present in env). This would reduce friction on adopting MLServer, the V2 protocol and MMS.


Discussion:

Resolving patch:
diff --git a/mlserver/repository.py b/mlserver/repository.py
--- a/mlserver/repository.py
+++ b/mlserver/repository.py
@@ -42,6 +42,12 @@ async def list(self) -> List[ModelSettings]:
     def _load_model_settings(self, model_settings_path: str) -> ModelSettings:
         model_settings = ModelSettings.parse_file(model_settings_path)
 
+        # TODO: Raise warning if name is different than folder's name
+        if not model_settings.name:
+            # If name not present, default to folder name
+            default_model_name = os.path.basename(os.path.dirname(model_settings_path))
+            model_settings.name = default_model_name
+
         if not model_settings.parameters:
             model_settings.parameters = ModelParameters()
 
diff --git a/mlserver/settings.py b/mlserver/settings.py
--- a/mlserver/settings.py
+++ b/mlserver/settings.py
@@ -50,7 +50,7 @@ class ModelSettings(BaseSettings):
     class Config:
         env_prefix = ENV_PREFIX_MODEL_SETTINGS
 
-    name: str
+    name: str = ""
 
     # Model metadata
     platform: str = ""


Test patch:
diff --git a/tests/test_repository.py b/tests/test_repository.py
--- a/tests/test_repository.py
+++ b/tests/test_repository.py
@@ -1,4 +1,5 @@
 import os
+import json
 
 from mlserver.repository import ModelRepository, DEFAULT_MODEL_SETTINGS_FILENAME
 from mlserver.settings import ModelSettings, ENV_PREFIX_MODEL_SETTINGS
@@ -30,7 +31,6 @@ async def test_list_multi_model(multi_model_folder: str):
 
     assert len(settings_list) == 5
     for idx, model_settings in enumerate(settings_list):
-        # Models get read in reverse
         assert model_settings.parameters.version == f"v{idx}"  # type: ignore
 
 
@@ -63,6 +63,20 @@ async def test_list_fallback(
     )
 
 
+async def test_name_fallback(model_folder: str, model_repository: ModelRepository):
+    # Create empty model-settings.json file
+    model_settings = ModelSettings()
+    model_settings_path = os.path.join(model_folder, DEFAULT_MODEL_SETTINGS_FILENAME)
+    with open(model_settings_path, "w") as model_settings_file:
+        d = model_settings.dict()
+        del d["name"]
+        d["implementation"] = get_import_path(d["implementation"])
+        json.dump(d, model_settings_file)
+
+    model_settings = model_repository._load_model_settings(model_settings_path)
+    assert model_settings.name == os.path.basename(model_folder)
+
+
 async def test_find(
     model_repository: ModelRepository, sum_model_settings: ModelSettings
 ):
Repository: SeldonIO/MLServer

Issue:
Support other common names for SKLearn runtime
Add support for models named `model.pickle` and `model.pkl`


Discussion:

Resolving patch:
diff --git a/runtimes/sklearn/mlserver_sklearn/sklearn.py b/runtimes/sklearn/mlserver_sklearn/sklearn.py
--- a/runtimes/sklearn/mlserver_sklearn/sklearn.py
+++ b/runtimes/sklearn/mlserver_sklearn/sklearn.py
@@ -12,7 +12,7 @@
 PREDICT_PROBA_OUTPUT = "predict_proba"
 VALID_OUTPUTS = [PREDICT_OUTPUT, PREDICT_PROBA_OUTPUT]
 
-WELLKNOWN_MODEL_FILENAMES = ["model.joblib"]
+WELLKNOWN_MODEL_FILENAMES = ["model.joblib", "model.pickle", "model.pkl"]
 
 
 class SKLearnModel(MLModel):


Test patch:
Repository: SeldonIO/MLServer

Issue:
Tempo example fails when parallel inference is enabled
When parallel inference is enabled, the [outlier example using the Tempo runtime](https://tempo.readthedocs.io/en/latest/examples/outlier/README.html)seems to fail. In particular, it seems that either the `cifar10-service` or the `outlier` containers block the request path and never return a response.


Discussion:

Resolving patch:
diff --git a/mlserver/parallel.py b/mlserver/parallel.py
--- a/mlserver/parallel.py
+++ b/mlserver/parallel.py
@@ -1,4 +1,5 @@
 import asyncio
+import multiprocessing as mp
 
 from functools import wraps
 from concurrent.futures import ProcessPoolExecutor
@@ -52,7 +53,6 @@ def _mp_predict(payload: InferenceRequest) -> InferenceResponse:
     # This global variable is only to be used within the inference worker
     # context.
     global _mp_model
-
     return asyncio.run(_mp_model.predict(payload))
 
 
@@ -69,8 +69,14 @@ class InferencePool:
 
     def __init__(self, model: MLModel):
         parallel_workers = model.settings.parallel_workers
+
+        # Use 'spawn' instead of 'fork' to ensure that models are loaded in a
+        # clean environment (e.g. to avoid issues like
+        # https://github.com/tensorflow/tensorflow/issues/8220)
+        ctx = mp.get_context("spawn")
         self._executor = ProcessPoolExecutor(
             max_workers=parallel_workers,
+            mp_context=ctx,
             initializer=_mp_load,
             initargs=(model.settings,),
         )


Test patch:
Repository: SeldonIO/MLServer

Issue:
Allow to configure uvicorn logging
Trying to add custom handlers to the application logger we have found that the uvicorn ones are not working as expected. It seems that uvicorn overrides the logging configuration when the Config object is initialized.

It would be nice to allow users to configure the uvicorn logger. This can be done by setting the path to a logging configuration file when creating uvicorn's Config object.


Discussion:

Resolving patch:
diff --git a/mlserver/logging.py b/mlserver/logging.py
--- a/mlserver/logging.py
+++ b/mlserver/logging.py
@@ -2,6 +2,7 @@
 import sys
 
 from logging import Formatter, StreamHandler
+import logging.config
 
 from .settings import Settings
 
@@ -15,6 +16,10 @@ def get_logger():
     return logger
 
 
+def apply_logging_file(logging_settings: str):
+    logging.config.fileConfig(fname=logging_settings, disable_existing_loggers=False)
+
+
 def configure_logger(settings: Settings = None):
     logger = get_logger()
 
@@ -30,4 +35,7 @@ def configure_logger(settings: Settings = None):
     if settings and settings.debug:
         logger.setLevel(logging.DEBUG)
 
+    if settings and settings.logging_settings:
+        apply_logging_file(settings.logging_settings)
+
     return logger
diff --git a/mlserver/rest/server.py b/mlserver/rest/server.py
--- a/mlserver/rest/server.py
+++ b/mlserver/rest/server.py
@@ -53,7 +53,10 @@ async def delete_custom_handlers(self, model: MLModel):
 
     async def start(self):
         cfg = uvicorn.Config(
-            self._app, host=self._settings.host, port=self._settings.http_port
+            self._app,
+            host=self._settings.host,
+            port=self._settings.http_port,
+            log_config=self._settings.logging_settings,
         )
         self._server = _NoSignalServer(cfg)
         await self._server.serve()
diff --git a/mlserver/settings.py b/mlserver/settings.py
--- a/mlserver/settings.py
+++ b/mlserver/settings.py
@@ -89,6 +89,10 @@ class Config:
     `None` to disable it
     """
 
+    # Logging settings
+    logging_settings: Optional[str] = None
+    """Path to logging config file"""
+
 
 class ModelParameters(BaseSettings):
     """


Test patch:
Repository: SeldonIO/MLServer

Issue:
Add docker build option to not use cache
To ensure fresh environment and while potentially developing a new model version users will not want to use cached layers in the docker build for their image.

In docker this is the --no-cache option, I would make it a boolean option within the cli and also add the --rm option (although default is true might be worth making sure) to remove any intermediate containers after success to again ensure a clean environment in CI/CD as well as locally. 


Discussion:

Resolving patch:
diff --git a/mlserver/cli/build.py b/mlserver/cli/build.py
--- a/mlserver/cli/build.py
+++ b/mlserver/cli/build.py
@@ -37,12 +37,18 @@ def write_dockerfile(
     return dockerfile_path
 
 
-def build_image(folder: str, dockerfile: str, image_tag: str) -> str:
+def build_image(
+    folder: str, dockerfile: str, image_tag: str, no_cache: bool = False
+) -> str:
     logger.info(f"Building Docker image with tag {image_tag}")
+    _docker_command_prefix = "docker build --rm "
     with TemporaryDirectory() as tmp_dir:
         dockerfile_path = write_dockerfile(tmp_dir, dockerfile)
-
-        build_cmd = f"docker build {folder} -f {dockerfile_path} -t {image_tag}"
+        _docker_command_suffix = f"{folder} -f {dockerfile_path} -t {image_tag}"
+        if no_cache:
+            build_cmd = _docker_command_prefix + "--no-cache " + _docker_command_suffix
+        else:
+            build_cmd = _docker_command_prefix + _docker_command_suffix
         build_env = os.environ.copy()
         build_env["DOCKER_BUILDKIT"] = "1"
         subprocess.run(build_cmd, check=True, shell=True, env=build_env)
diff --git a/mlserver/cli/main.py b/mlserver/cli/main.py
--- a/mlserver/cli/main.py
+++ b/mlserver/cli/main.py
@@ -47,13 +47,14 @@ async def start(folder: str):
 @root.command("build")
 @click.argument("folder", nargs=1)
 @click.option("-t", "--tag", type=str)
+@click.option("--no-cache", default=False, is_flag=True)
 @click_async
-async def build(folder: str, tag: str):
+async def build(folder: str, tag: str, no_cache: bool = False):
     """
     Build a Docker image for a custom MLServer runtime.
     """
     dockerfile = generate_dockerfile()
-    build_image(folder, dockerfile, tag)
+    build_image(folder, dockerfile, tag, no_cache=no_cache)
     logger.info(f"Successfully built custom Docker image with tag {tag}")
 
 


Test patch:
Repository: SeldonIO/MLServer

Issue:
Show deprecation note when using model's parallel worker setting



Discussion:

Resolving patch:
diff --git a/mlserver/parallel/pool.py b/mlserver/parallel/pool.py
--- a/mlserver/parallel/pool.py
+++ b/mlserver/parallel/pool.py
@@ -191,12 +191,26 @@ async def unload_model(self, model: MLModel):
         )
 
     def _should_load_model(self, model: MLModel):
-        # NOTE: This is a remnant from the previous architecture for parallel
-        # workers, where each worker had its own pool.
-        # For backwards compatibility, we will respect when a model disables
-        # parallel inference.
-        if not model.settings.parallel_workers:
-            return False
+        if model.settings.parallel_workers is not None:
+            logger.warning(
+                "DEPRECATED!! The `parallel_workers` setting at the model-level "
+                "has now been deprecated and moved "
+                "to the top-level server "
+                "settings. "
+                "This field will be removed in MLServer 1.2.0. "
+                "To access the new field, you can either update the "
+                "`settings.json` file, or update the `MLSERVER_PARALLEL_WORKERS` "
+                "environment variable. "
+                f"The current value of the server-level's `parallel_workers` field is "
+                f"'{self._settings.parallel_workers}'."
+            )
+
+            # NOTE: This is a remnant from the previous architecture for parallel
+            # workers, where each worker had its own pool.
+            # For backwards compatibility, we will respect when a model disables
+            # parallel inference.
+            if model.settings.parallel_workers <= 0:
+                return False
 
         if not self._settings.parallel_workers:
             return False
diff --git a/mlserver/settings.py b/mlserver/settings.py
--- a/mlserver/settings.py
+++ b/mlserver/settings.py
@@ -1,5 +1,3 @@
-import multiprocessing
-
 from typing import List, Optional
 from pydantic import BaseSettings, PyObject, Field
 
@@ -9,7 +7,7 @@
 ENV_PREFIX_SETTINGS = "MLSERVER_"
 ENV_PREFIX_MODEL_SETTINGS = "MLSERVER_MODEL_"
 
-DEFAULT_PARALLEL_WORKERS = multiprocessing.cpu_count()
+DEFAULT_PARALLEL_WORKERS = 1
 
 
 class CORSSettings(BaseSettings):
@@ -168,8 +166,8 @@ class Config:
     """Metadata about the outputs returned by the model."""
 
     # Parallel settings
-    parallel_workers: int = Field(
-        DEFAULT_PARALLEL_WORKERS,
+    parallel_workers: Optional[int] = Field(
+        None,
         deprecated=True,
         description=(
             "Use the `parallel_workers` field the server wide settings instead."


Test patch:
Repository: SeldonIO/MLServer

Issue:
Support older versions of grpcio
Some attributes, like `trailing_metadata()` don't seem to exist in older versions of grpcio (e.g. `1.34.0`). Ideally we should also provide support for these ones to avoid clashes with older versions of TF (e.g. `2.5.x`). 


Discussion:

Resolving patch:
diff --git a/mlserver/grpc/interceptors.py b/mlserver/grpc/interceptors.py
--- a/mlserver/grpc/interceptors.py
+++ b/mlserver/grpc/interceptors.py
@@ -66,6 +66,9 @@ def _compute_status_code(self, servicer_context: ServicerContext) -> StatusCode:
             if servicer_context._state.client == "cancelled":
                 return StatusCode.CANCELLED
 
+        if not hasattr(servicer_context, "code"):
+            return StatusCode.OK
+
         code = servicer_context.code()
         if code is None:
             return StatusCode.OK
diff --git a/mlserver/grpc/utils.py b/mlserver/grpc/utils.py
--- a/mlserver/grpc/utils.py
+++ b/mlserver/grpc/utils.py
@@ -18,7 +18,11 @@
 
 
 def to_headers(context: ServicerContext) -> Dict[str, str]:
-    metadata = context.invocation_metadata() + context.trailing_metadata()
+    metadata = context.invocation_metadata()
+    if hasattr(context, "trailing_metadata"):
+        # NOTE: Older versions of `grpcio` (e.g. `grpcio==1.34.0`) don't expose
+        # access to the trailing metadata on the service side
+        metadata += context.trailing_metadata()
     headers = {}
     for metadatum in metadata:
         headers[metadatum.key] = metadatum.value


Test patch:
Repository: SeldonIO/MLServer

Issue:
starting mlserver using `mlserver start .` is not consistent with  `mlserver start $PWD`
When I started mlserver using `mlserver start .` in directory tree 
```
└── iris1
    └── 1
        ├── model.joblib
        └── model-settings.json
```
and settings `{"name":"iris1","implementation":"mlserver_sklearn.SKLearnModel","parameters":{"version":"1"}}`

results in an error:
```
mlserver.errors.InvalidModelURI: Invalid URI specified for model iris1 (iris1/1/iris1/1)
```

However using
`mlserver start $PWD` is successful.


Discussion:

Resolving patch:
diff --git a/mlserver/repository.py b/mlserver/repository.py
--- a/mlserver/repository.py
+++ b/mlserver/repository.py
@@ -24,7 +24,8 @@ async def list(self) -> List[ModelSettings]:
 
         # TODO: Use an async alternative for filesys ops
         if self._root:
-            pattern = os.path.join(self._root, "**", DEFAULT_MODEL_SETTINGS_FILENAME)
+            abs_root = os.path.abspath(self._root)
+            pattern = os.path.join(abs_root, "**", DEFAULT_MODEL_SETTINGS_FILENAME)
             matches = glob.glob(pattern, recursive=True)
 
             for model_settings_path in matches:


Test patch:
Repository: SeldonIO/MLServer

Issue:
MLServer to hide http health request logs to avoid polluting the logs
As part of the Seldon Core addition https://github.com/SeldonIO/seldon-core/pull/4028 which moves the TCP ready checks into proper HTTP request ready checks to `v2/health/ready` there is now a lot of noise from the readiness checks every 5 seconds. We should explore ways in which we avoid this noise, perhaps making it completely silent by default, or eventually once the prometheus server is created on a separate server this could also be added (And both of  them could be muted)

![image](https://user-images.githubusercontent.com/1447507/161749439-3e96c417-a630-4208-80d6-5fc5f262fcd3.png)



Discussion:
@axsaucedo have you found a solution to this? I also cannot read the logs.

Resolving patch:
diff --git a/mlserver/rest/logging.py b/mlserver/rest/logging.py
--- a/mlserver/rest/logging.py
+++ b/mlserver/rest/logging.py
@@ -1,4 +1,35 @@
 import logging
 
+
+class HealthEndpointFilter(logging.Filter):
+    """
+    Filter to avoid logging health endpoints.
+    From:
+        https://github.com/encode/starlette/issues/864#issuecomment-653076434
+    """
+
+    def filter(self, record: logging.LogRecord) -> bool:
+        if not isinstance(record.args, tuple):
+            return True
+
+        if len(record.args) < 3:
+            return True
+
+        request_method = record.args[1]
+        query_string = record.args[2]
+        if request_method != "GET":
+            return True
+
+        if query_string in ["/v2/health/live", "/v2/health/ready"]:
+            return False
+
+        return True
+
+
+def disable_health_access_logs() -> None:
+    uvicorn_logger = logging.getLogger("uvicorn.access")
+    uvicorn_logger.addFilter(HealthEndpointFilter())
+
+
 loggerName = "mlserver.rest"
 logger = logging.getLogger(loggerName)
diff --git a/mlserver/rest/server.py b/mlserver/rest/server.py
--- a/mlserver/rest/server.py
+++ b/mlserver/rest/server.py
@@ -6,7 +6,7 @@
 
 from .utils import matches
 from .app import create_app
-from .logging import logger
+from .logging import logger, disable_health_access_logs
 from typing import Optional
 
 
@@ -60,6 +60,9 @@ async def delete_custom_handlers(self, model: MLModel) -> MLModel:
     async def start(self):
         cfg = self._get_config()
         self._server = _NoSignalServer(cfg)
+        if not self._settings.debug:
+            disable_health_access_logs()
+
         await self._server.serve()
 
     def _get_config(self):


Test patch:
Repository: SeldonIO/MLServer

Issue:
Add more debugging information
I am running a custom Python model in MLServer, and it fails with an error. The error itself is not interesting, what's important is that MLServer gives very little helpful information to debug it.

After sending a request with grpcurl here is the entire response:
````
ERROR:
  Code: Unknown
  Message: Unexpected <class 'TypeError'>: list indices must be integers or slices, not str
````
No stack trace, no line numbers or class names. The only thing that appears in the logs is
> scv2-mlserver-1  | 2022-10-19 09:10:44,218 [mlserver.grpc] INFO - /inference.GRPCInferenceService/ModelInfer

Would be great if MLServer propagated more debugging info to the response message and to the log.


Discussion:
Here is a repo where i reproduced the issue:
https://github.com/apaleyes/seldon-core-v2/tree/mlserver-debugging-repro

bits to look at are:
* samples/models/mlserver-debugging
* samples/mlserver-debugging.ipynb

Resolving patch:
diff --git a/mlserver/grpc/utils.py b/mlserver/grpc/utils.py
--- a/mlserver/grpc/utils.py
+++ b/mlserver/grpc/utils.py
@@ -43,7 +43,10 @@ async def _inner(self, request, context):
         try:
             return await f(self, request, context)
         except MLServerError as err:
-            logger.error(err)
+            logger.exception(err)
             await context.abort(code=_grpc_status_code(err), details=str(err))
+        except Exception as err:
+            logger.exception(err)
+            await context.abort(code=grpc.StatusCode.INTERNAL, details=str(err))
 
     return _inner


Test patch: