Repository: bentoml/BentoML
Pull request: [Features] Added testcases and prometheus endpoints for YataiService
<!--- Thanks for sending a pull request! Please make sure to read the contribution guidelines, then fill out the blanks below before requesting a code review. -->

## Description
<!--- Describe your changes in detail -->
<!--- Attach screenshots here if appropriate. -->
- Added prometheus integration for metrics exporting. Currently the prometheus metrics can be accessed via `localhost:3000/metrics`
- Added some basic metrics to measure YataiService gRPC calls
- Prometheus metrics can be accessed via `/metrics`
- metrics followed current implementation from [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus)
- added testcases with `82%` coverage for prometheus interceptor

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->
<!--- If it is based on a conversation in slack channel, pls quote related messages here -->
- This is a part of the closed #1060 for prometheus metrics exporting
- I have fixed some issues regarding with how grpc handling streaming request after closing #1349 
- This is a completed version of #1452 

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
- added testcases under `./tests/yatai/client` as well as a `grpc_testing_service.py` as a mock gRPC service
```python
python -m pytest tests/yatai --cov=bentoml --cov-config=.coveragerc --ignore tests/yatai/client/test_bento_repository_api.py
```
- I also run `yarn format` under YataiService web UI

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature and improvements (non-breaking change which adds/improves functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Code Refactoring (internal change which is not user facing)
- [ ] Documentation
- [x] Test, CI, or build
<!--- [ ] Others: describe the type of change if it does not fit to categories listed -->

## Component(s) if applicable
- [ ] BentoService (service definition, dependency management, API input/output adapters)
- [ ] Model Artifact (model serialization, multi-framework support)
- [ ] Model Server (mico-batching, dockerisation, logging, OpenAPI, instruments)
- [x] YataiService gRPC server (model registry, cloud deployment automation)
- [x] YataiService web server (nodejs HTTP server and web UI)
- [ ] Internal (BentoML's own configuration, logging, utility, exception handling)
- [ ] BentoML CLI

## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
<!--- If you plan to update documentation or tests in follow-up, please note -->
- [x] My code follows the bentoml code style, both `./dev/format.sh` and
  `./dev/lint.sh` script have passed
  ([instructions](https://github.com/bentoml/BentoML/blob/master/DEVELOPMENT.md#style-check-and-auto-formatting-your-code)).
- [ ] My change reduces project test coverage and requires unit tests to be added
- [x] I have added unit tests covering my code change
- [x] My change requires a change to the documentation
- [ ] I have updated the documentation accordingly


Problem statement:
YataiService monitoring and prometheus endpoint
Currently, there is no easy way to monitor a YataiService deployment.

* Add a '/healthz' HTTP endpoint via the node server, in addition to te gRPC health check endpoint
* Add a '/metrics' HTTP endpoint via the node server, and allow expose metrics from the python gRPC server

Hints from issue discussion:
I would love to take this on

Hello @parano, I am currently working on this issue, I just wanted to ask, what does the endpoint /healthz do exactly? Thank you!
Hi @CallmeMehdi - I think @aarnphm  is currently working on this issue as well, maybe you two can sync up and discuss?

"/healthz" endpoint is a convention in building microservices, sometimes a deployed service can get into a bad state and become incapable of handling requests. "/healthz" is a special endpoint that let the monitoring system to discover such failed service instance and do something about it(usually restarting the instance)

Implementing the `healthz` endpoint is usually very straightforward, e.g. https://cloud.ibm.com/docs/node?topic=node-node-healthcheck
@CallmeMehdi @aarnphm Any update on this issue?  Let me know if you are running into any issues, happy to help

Commit message: changes accordingly with lint.sh, flake8, pytest and pylintrc

fixes tests proto python output under tests

update pytest to ignore generated protos

Patch to review:
diff --git a/.flake8 b/.flake8
index 7a2b2c06748..e0039e8a1c7 100644
--- a/.flake8
+++ b/.flake8
@@ -1,4 +1,4 @@
 [flake8]
 ignore = E203, E266, W503, E402
 max-line-length = 88
-exclude = proto, _version.py
+exclude = tests/yatai/protos, proto, _version.py
diff --git a/Makefile b/Makefile
index ed0a8313649..169a4462e49 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ spellcheck-doc: ## Spell check documentation
 start-yatai-debug: ## Start YataiService in debug mode
 	bentoml yatai-service-start --debug || (echo "Error starting... You may need to run 'make install-yatai-deps'"; exit 1)
 start-grpcui: ## Start gPRC Web UI
-	grpcui -plain text localhost:50051 || (echo "Error starting... You may need to run 'make install-yatai-deps'"; exit 1)
+	grpcui -plaintext localhost:50051 || (echo "Error starting... You may need to run 'make install-yatai-deps'"; exit 1)
 install-yatai-deps: ## Install dependencies to debug YataiService
 	pip install -e ".[dev]"
 	go get github.com/fullstorydev/grpcui
@@ -58,12 +58,12 @@ install-yatai-deps: ## Install dependencies to debug YataiService
 # BentoML Web UI
 watch-yatai-web-ui: ## Start BentoML Web UI server in dev mode
 	bentoml yatai-service-start --no-ui & \
-	cd bentoml/yatai/web && yarn dev 127.0.0.1:50051 3000 .
+	cd bentoml/yatai/web && yarn dev 127.0.0.1:50051 3000 . 127.0.0.1:50052
 build-yatai-web-ui: ## Build BentoML Web UI server and frontend
 	cd bentoml/yatai/web && yarn build
 run-yatai-web-ui: ## Run production BentoML Web UI server and frontend
 	bentoml yatai-service-start --no-ui & \
-	cd bentoml/yatai/web && yarn start 127.0.0.1:50051 3000 .
+	cd bentoml/yatai/web && yarn start 127.0.0.1:50051 3000 . 127.0.0.1:50052
 install-web-deps: ## Install dependencies to run web server and frontend
 	cd bentoml/yatai/web && yarn install
 	cd bentoml/yatai/web/client && yarn install
diff --git a/bentoml/yatai/client/interceptor/prom_server_interceptor.py b/bentoml/yatai/client/interceptor/prom_server_interceptor.py
new file mode 100644
index 00000000000..2f16f2015b2
--- /dev/null
+++ b/bentoml/yatai/client/interceptor/prom_server_interceptor.py
@@ -0,0 +1,191 @@
+"""Interceptor a client call with prometheus"""
+
+from timeit import default_timer
+
+import grpc
+from bentoml.yatai.metrics import (
+    GRPC_SERVER_HANDLED_HISTOGRAM,
+    GRPC_SERVER_HANDLED_TOTAL,
+    GRPC_SERVER_STARTED_COUNTER,
+    GRPC_SERVER_STREAM_MSG_RECEIVED,
+    GRPC_SERVER_STREAM_MSG_SENT,
+)
+from bentoml.yatai.utils import (
+    get_method_type,
+    parse_method_name,
+    wrap_interator_inc_counter,
+)
+
+
+def _wrap_rpc_behaviour(handler, fn):
+    """Returns a new rpc handler that wraps the given function"""
+    if handler is None:
+        return None
+
+    if handler.request_streaming and handler.response_streaming:
+        behavior_fn = handler.stream_stream
+        handler_factory = grpc.stream_stream_rpc_method_handler
+    elif handler.request_streaming and not handler.response_streaming:
+        behavior_fn = handler.stream_unary
+        handler_factory = grpc.stream_unary_rpc_method_handler
+    elif not handler.request_streaming and handler.response_streaming:
+        behavior_fn = handler.unary_stream
+        handler_factory = grpc.unary_stream_rpc_method_handler
+    else:
+        behavior_fn = handler.unary_unary
+        handler_factory = grpc.unary_unary_rpc_method_handler
+
+    return handler_factory(
+        fn(behavior_fn, handler.request_streaming, handler.response_streaming),
+        request_deserializer=handler.request_deserializer,
+        response_serializer=handler.response_serializer,
+    )
+
+
+# request are either RPC request as protobuf or iterator of RPC request
+class PromServerInterceptor(grpc.ServerInterceptor):  # pylint: disable=W0232
+    """Interceptor for handling client call with metrics to prometheus"""
+
+    def intercept_service(self, continuation, handler_call_details):
+
+        handler = continuation(handler_call_details)
+        if handler is None:
+            return handler
+
+        # unary
+        if handler.request_streaming or handler.response_streaming:
+            return handler
+
+        mn, ok = parse_method_name(handler_call_details.method)
+        grpc_service_name = mn.fully_qualified_service
+        grpc_method_name = mn.method
+
+        if not ok:
+            return continuation(handler_call_details)
+
+        def metrics_wrapper(behaviour, request_streaming, response_streaming):
+            def new_behaviour(request_or_iterator, servicer_context):
+                grpc_type = get_method_type(request_streaming, response_streaming)
+
+                try:
+                    if request_streaming:
+                        request_or_iterator = wrap_interator_inc_counter(
+                            request_or_iterator,
+                            GRPC_SERVER_STREAM_MSG_RECEIVED,
+                            grpc_type,
+                            grpc_service_name,
+                            grpc_method_name,
+                        )
+                        request_or_iterator = wrap_interator_inc_counter(
+                            request_or_iterator,
+                            GRPC_SERVER_STREAM_MSG_SENT,
+                            grpc_type,
+                            grpc_service_name,
+                            grpc_method_name,
+                        )
+                    else:
+                        GRPC_SERVER_STARTED_COUNTER.labels(
+                            grpc_type=grpc_type,
+                            grpc_service=grpc_service_name,
+                            grpc_method=grpc_method_name,
+                        ).inc()
+
+                    # Invoke the original RPC behaviour
+                    response_or_iterator = behaviour(
+                        request_or_iterator, servicer_context
+                    )
+
+                    if response_streaming:
+                        sent_metrics = GRPC_SERVER_STREAM_MSG_SENT
+                        response_or_iterator = wrap_interator_inc_counter(
+                            response_or_iterator,
+                            GRPC_SERVER_STREAM_MSG_RECEIVED,
+                            grpc_type,
+                            grpc_service_name,
+                            grpc_method_name,
+                        )
+                        response_or_iterator = wrap_interator_inc_counter(
+                            response_or_iterator,
+                            sent_metrics,
+                            grpc_type,
+                            grpc_service_name,
+                            grpc_method_name,
+                        )
+                    else:
+                        GRPC_SERVER_HANDLED_TOTAL.labels(
+                            grpc_type=grpc_type,
+                            grpc_service=grpc_service_name,
+                            grpc_method=grpc_method_name,
+                            grpc_code=self._compute_status_code(servicer_context).name,
+                        ).inc()
+
+                    return response_or_iterator
+
+                except grpc.RpcError as e:
+                    GRPC_SERVER_HANDLED_TOTAL.labels(
+                        grpc_type=grpc_type,
+                        grpc_service=grpc_service_name,
+                        grpc_method=grpc_method_name,
+                        grpc_code=self._compute_error_code(e),
+                    ).inc()
+                    raise e
+
+            return new_behaviour
+
+        optional_any = _wrap_rpc_behaviour(
+            continuation(handler_call_details), metrics_wrapper
+        )
+        return optional_any
+
+    def _compute_status_code(self, servicer_context: grpc.ServicerContext):
+        if servicer_context._state.client == "cancelled":
+            return grpc.StatusCode.CANCELLED
+
+        if servicer_context._state.code is None:
+            return grpc.StatusCode.OK
+
+        return servicer_context._state.code
+
+    def _compute_error_code(self, grpc_exception):
+        if isinstance(grpc_exception, grpc.Call):
+            return grpc_exception.code().name
+
+        return grpc.StatusCode.UNKNOWN.name
+
+
+class ServiceLatencyInterceptor(grpc.ServerInterceptor):  # pylint: disable=W0232
+    """Interceptor to handle service latency calls"""
+
+    def intercept_service(self, continuation, handler_call_details):
+
+        handler = continuation(handler_call_details)
+        if handler is None:
+            return handler
+
+        # unary
+        if handler.request_streaming or handler.response_streaming:
+            return handler
+
+        mn, ok = parse_method_name(handler_call_details.method)
+        grpc_service_name = mn.fully_qualified_service
+        grpc_method_name = mn.method
+
+        if not ok:
+            return continuation(handler_call_details)
+
+        # pylint: disable=W0613
+        def latency_wrapper(behaviour, request_streaming, response_streaming):
+            def new_behaviour(request_or_iterator, servicer_context):
+                start = default_timer()
+                try:
+                    return behaviour(request_or_iterator, servicer_context)
+                finally:
+                    GRPC_SERVER_HANDLED_HISTOGRAM.labels(
+                        grpc_type='UNARY',
+                        grpc_service=grpc_service_name,
+                        grpc_method=grpc_method_name,
+                    ).observe(max(default_timer() - start, 0))
+
+            return new_behaviour
+
+        return _wrap_rpc_behaviour(continuation(handler_call_details), latency_wrapper)
diff --git a/bentoml/yatai/metrics.py b/bentoml/yatai/metrics.py
new file mode 100644
index 00000000000..760db07bfcb
--- /dev/null
+++ b/bentoml/yatai/metrics.py
@@ -0,0 +1,32 @@
+from prometheus_client import Counter, Histogram
+
+GRPC_SERVER_STARTED_COUNTER = Counter(
+    "grpc_server_started_total",
+    "Total number of RPCs started on the server.",
+    ["grpc_type", "grpc_service", "grpc_method"],
+)
+
+GRPC_SERVER_STREAM_MSG_RECEIVED = Counter(
+    "grpc_server_msg_received_total",
+    "Total number of RPC stream messages received on the server.",
+    ["grpc_type", "grpc_service", "grpc_method"],
+)
+
+GRPC_SERVER_STREAM_MSG_SENT = Counter(
+    "grpc_server_msg_sent_total",
+    "Total number of gRPC stream messages sent by the server.",
+    ["grpc_type", "grpc_service", "grpc_method"],
+)
+
+GRPC_SERVER_HANDLED_HISTOGRAM = Histogram(
+    "grpc_server_handling_seconds",
+    "Histogram of response latency (seconds) of gRPC that had been application-level "
+    "handled by the server.",
+    ["grpc_type", "grpc_service", "grpc_method"],
+)
+
+GRPC_SERVER_HANDLED_TOTAL = Counter(
+    "grpc_server_handled_total",
+    "Total number of RPCs completed on the server, regardless of success or failure.",
+    ["grpc_type", "grpc_service", "grpc_method", "grpc_code"],
+)
diff --git a/bentoml/yatai/utils.py b/bentoml/yatai/utils.py
index acbfe50f5a0..aa450538650 100644
--- a/bentoml/yatai/utils.py
+++ b/bentoml/yatai/utils.py
@@ -15,8 +15,15 @@
 import subprocess
 import urllib3
 
+from typing import NamedTuple, Tuple
 from bentoml.exceptions import BentoMLException, MissingDependencyException
 
+UNARY = 'UNARY'
+SERVER_STREAMING = 'SERVER_STREAMING'
+CLIENT_STREAMING = 'CLIENT_STREAMING'
+BIDI_STREAMING = 'BIDI_STREAMING'
+UNKNOWN = 'UNKNOWN'
+
 
 def ensure_node_available_or_raise():
     try:
@@ -41,3 +48,64 @@ def parse_grpc_url(url):
     '''
     parts = urllib3.util.parse_url(url)
     return parts.scheme, url.replace(f"{parts.scheme}://", "", 1)
+
+
+def wrap_interator_inc_counter(
+    iterator, counter, grpc_type, grpc_service_name, grpc_method_name,
+):
+    for item in iterator:
+        counter.labels(
+            grpc_type=grpc_type,
+            grpc_service=grpc_service_name,
+            grpc_method=grpc_method_name,
+        ).inc()
+        yield item
+
+
+# get method name for given RPC handler
+def get_method_type(request_streaming, response_streaming) -> str:
+    if not request_streaming and not response_streaming:
+        return UNARY
+    elif not request_streaming and response_streaming:
+        return SERVER_STREAMING
+    elif request_streaming and not response_streaming:
+        return CLIENT_STREAMING
+    elif request_streaming and response_streaming:
+        return BIDI_STREAMING
+    else:
+        raise RuntimeError('Unknown request_streaming or response_streaming')
+
+
+class MethodName(NamedTuple):
+    """
+    Represents a gRPC method name
+    Attributes:
+        package: This is defined by `package foo.bar`,
+        designation in the protocol buffer definition
+        service: service name in protocol buffer
+        definition (eg: service SearchService { ... })
+        method: method name
+    """
+
+    package: str
+    service: str
+    method: str
+
+    @property
+    def fully_qualified_service(self):
+        """return the service name prefixed with package"""
+        return f"{self.package}.{self.service}" if self.package else self.service
+
+
+def parse_method_name(method_name: str) -> Tuple[MethodName, bool]:
+    '''
+    Infers the grpc service and method name from the handler_call_details.
+    e.g. /package.ServiceName/MethodName
+    '''
+    parts = method_name.split("/")
+    if len(parts) < 3:
+        return MethodName("", "", ""), False
+    _, package_service, method = parts
+    *packages, service = package_service.rsplit(".", maxsplit=1)
+    package = packages[0] if packages else ""
+    return MethodName(package, service, method), True
diff --git a/bentoml/yatai/web/.prettierignore b/bentoml/yatai/web/.prettierignore
index de676c49175..5fa6e03b673 100644
--- a/bentoml/yatai/web/.prettierignore
+++ b/bentoml/yatai/web/.prettierignore
@@ -1,3 +1,4 @@
 src/generated
 .cache
 dist
+client/build
diff --git a/bentoml/yatai/web/client/package.json b/bentoml/yatai/web/client/package.json
index 585648506d9..5744786e1fe 100644
--- a/bentoml/yatai/web/client/package.json
+++ b/bentoml/yatai/web/client/package.json
@@ -29,7 +29,7 @@
     "react-scripts": "3.4.4",
     "typescript": "~3.7.2"
   },
-  "proxy": "http://localhost:3000",  
+  "proxy": "http://localhost:3000",
   "scripts": {
     "start": "PORT=8080 BROWSER=none react-scripts start",
     "build": "react-scripts build",
diff --git a/bentoml/yatai/web/client/public/index.html b/bentoml/yatai/web/client/public/index.html
index dd7da0d4605..e068943d6d7 100644
--- a/bentoml/yatai/web/client/public/index.html
+++ b/bentoml/yatai/web/client/public/index.html
@@ -5,10 +5,7 @@
     <link rel="icon" href="%PUBLIC_URL%/favicon.png" />
     <meta name="viewport" content="width=device-width, initial-scale=1" />
     <meta name="theme-color" content="#000000" />
-    <meta
-      name="description"
-      content="BentoML Yatai"
-    />
+    <meta name="description" content="BentoML Yatai" />
     <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
     <!--
       manifest.json provides metadata used when your web app is installed on a
diff --git a/bentoml/yatai/web/client/src/app.tsx b/bentoml/yatai/web/client/src/app.tsx
index a8729c4b96a..c211dedf0da 100644
--- a/bentoml/yatai/web/client/src/app.tsx
+++ b/bentoml/yatai/web/client/src/app.tsx
@@ -10,38 +10,39 @@ import BentoServiceDetail from "./pages/BentoServiceDetail";
 import Layout from "./ui/Layout";
 import Breadcrumbs from "./components/Breadcrumbs";
 import NavigationBar from "./components/NavigationBar";
-import {setBaseUrl} from './utils/HttpRequestContainer';
-import {useCookies} from 'react-cookie';
+import { setBaseUrl } from "./utils/HttpRequestContainer";
+import { useCookies } from "react-cookie";
 export const App = () => {
   const [cookies] = useCookies();
   const baseURL = "/" + cookies["baseURLCookie"];
   setBaseUrl(baseURL);
   return (
-  <BrowserRouter basename={baseURL}>
-    <NavigationBar baseURL = {baseURL} />
-    <Layout>
-      <Breadcrumbs baseURL = {baseURL}/>
-      <div>
-        <Switch>
-          <Route
-            path="/deployments/:namespace/:name"
-            component={DeploymentDetails}
-          />
-          <Route path="/deployments/:namespace" component={DeploymentsList} />
-          <Route path="/deployments" component={DeploymentsList} />
+    <BrowserRouter basename={baseURL}>
+      <NavigationBar baseURL={baseURL} />
+      <Layout>
+        <Breadcrumbs baseURL={baseURL} />
+        <div>
+          <Switch>
+            <Route
+              path="/deployments/:namespace/:name"
+              component={DeploymentDetails}
+            />
+            <Route path="/deployments/:namespace" component={DeploymentsList} />
+            <Route path="/deployments" component={DeploymentsList} />
 
-          <Route
-            path="/repository/:name/:version"
-            component={BentoServiceDetail}
-          />
-          <Route path="/repository/:name" component={BentoServicesList} />
-          <Route path="/repository" component={Repository} />
+            <Route
+              path="/repository/:name/:version"
+              component={BentoServiceDetail}
+            />
+            <Route path="/repository/:name" component={BentoServicesList} />
+            <Route path="/repository" component={Repository} />
 
-          <Route path="/about" component={HomePage} />
-          <Route path="/config" component={HomePage} />
-          <Route exact path="/" component={HomePage} />
-        </Switch>
-      </div>
-    </Layout>
-  </BrowserRouter>
-  )};
+            <Route path="/about" component={HomePage} />
+            <Route path="/config" component={HomePage} />
+            <Route exact path="/" component={HomePage} />
+          </Switch>
+        </div>
+      </Layout>
+    </BrowserRouter>
+  );
+};
diff --git a/bentoml/yatai/web/client/src/components/BentoServiceDetail/ApisTable.tsx b/bentoml/yatai/web/client/src/components/BentoServiceDetail/ApisTable.tsx
index 7649157ce71..08b7b20ad4f 100644
--- a/bentoml/yatai/web/client/src/components/BentoServiceDetail/ApisTable.tsx
+++ b/bentoml/yatai/web/client/src/components/BentoServiceDetail/ApisTable.tsx
@@ -22,7 +22,7 @@ const inputConfigToTableContent = (
     }
     */
 
-  const displayInputList : React.ReactNode[]= [];
+  const displayInputList: React.ReactNode[] = [];
   const configureKeys = Object.keys(config);
   for (let index = 0; index < configureKeys.length; index++) {
     const key = configureKeys[index];
diff --git a/bentoml/yatai/web/client/src/components/BentoServiceDetail/ArtifactsTable.tsx b/bentoml/yatai/web/client/src/components/BentoServiceDetail/ArtifactsTable.tsx
index ab362cda45f..c1fe64e3595 100644
--- a/bentoml/yatai/web/client/src/components/BentoServiceDetail/ArtifactsTable.tsx
+++ b/bentoml/yatai/web/client/src/components/BentoServiceDetail/ArtifactsTable.tsx
@@ -1,5 +1,5 @@
 import * as React from "react";
-import * as lodash from 'lodash';
+import * as lodash from "lodash";
 import Table from "../../ui/Table";
 import { Section } from "../../ui/Layout";
 
@@ -41,24 +41,24 @@ const parseGrpcStructToObject = (structObject) => {
   if (structObject.fields) {
     structObject = structObject.fields;
   }
-  const keyList = Object.keys(structObject)
+  const keyList = Object.keys(structObject);
   for (let index = 0; index < keyList.length; index++) {
     const key = keyList[index];
     const valueObject = structObject[key];
     const valueType = Object.keys(valueObject)[0];
     const value: any = Object.values(valueObject)[0];
     switch (valueType) {
-      case 'structValue':
+      case "structValue":
         result[key] = parseGrpcStructToObject(value);
         break;
-      case 'listValue':
+      case "listValue":
         result[key] = [];
         lodash.forEach(value, (item) => {
-          result[key].push(parseGrpcStructToObject(item))
+          result[key].push(parseGrpcStructToObject(item));
         });
         break;
-      case 'nullValue':
-        result[key] = 'null';
+      case "nullValue":
+        result[key] = "null";
         break;
       default:
         result[key] = value;
@@ -68,11 +68,9 @@ const parseGrpcStructToObject = (structObject) => {
   return result;
 };
 
-const artifactMetadataToTableContent = (
-  artifactMetadata: any
-): string => {
+const artifactMetadataToTableContent = (artifactMetadata: any): string => {
   if (!artifactMetadata) {
-    return 'None';
+    return "None";
   }
   let result = parseGrpcStructToObject(artifactMetadata);
   return JSON.stringify(result);
@@ -86,8 +84,8 @@ const ArtifactsTable: React.FC<{ artifacts: Array<IArtifactProps> }> = ({
       content: [
         artifact.name,
         artifact.artifact_type,
-        artifactMetadataToTableContent(artifact.metadata)
-      ]
+        artifactMetadataToTableContent(artifact.metadata),
+      ],
     };
   });
   return (
diff --git a/bentoml/yatai/web/client/src/components/BentoServiceDetail/EnvTable.tsx b/bentoml/yatai/web/client/src/components/BentoServiceDetail/EnvTable.tsx
index d5cbbc49ed4..966a4c1a383 100644
--- a/bentoml/yatai/web/client/src/components/BentoServiceDetail/EnvTable.tsx
+++ b/bentoml/yatai/web/client/src/components/BentoServiceDetail/EnvTable.tsx
@@ -9,8 +9,8 @@ const EnvTable: React.FC<{ env: { [key: string]: any } }> = ({ env }) => {
 
   const parsedEnv = envKeys.map((key) => {
     let env_value = env[key];
-    if (key == 'pip_packages') {
-      env_value = env[key].join('\n');
+    if (key == "pip_packages") {
+      env_value = env[key].join("\n");
     }
     return { content: [key, env_value] };
   });
diff --git a/bentoml/yatai/web/client/src/components/BentoServiceTable.tsx b/bentoml/yatai/web/client/src/components/BentoServiceTable.tsx
index e4ecb5f28d6..38c1b04368a 100644
--- a/bentoml/yatai/web/client/src/components/BentoServiceTable.tsx
+++ b/bentoml/yatai/web/client/src/components/BentoServiceTable.tsx
@@ -1,10 +1,10 @@
 import * as React from "react";
 import { Link } from "react-router-dom";
-import * as lodash from 'lodash';
+import * as lodash from "lodash";
 
 import { displayTimeInFromNowFormat } from "../utils";
 import Table from "../ui/Table";
-import Label from './Label';
+import Label from "./Label";
 
 const apisToDisplayFormat = (apis: any) => {
   if (!apis) {
@@ -28,13 +28,11 @@ const labelsToDisplayFormat = (labels) => {
   }
   return (
     <div>
-      {
-        lodash.map(labels, (value, name) => {
-          return (<Label name={name} value={value}/>);
-        })
-      }
+      {lodash.map(labels, (value, name) => {
+        return <Label name={name} value={value} />;
+      })}
     </div>
-  )
+  );
 };
 
 const BENTO_TABLE_HEADERS = [
diff --git a/bentoml/yatai/web/client/src/components/Breadcrumbs.tsx b/bentoml/yatai/web/client/src/components/Breadcrumbs.tsx
index d8813bd9e6e..f3931390096 100644
--- a/bentoml/yatai/web/client/src/components/Breadcrumbs.tsx
+++ b/bentoml/yatai/web/client/src/components/Breadcrumbs.tsx
@@ -17,11 +17,11 @@ const Breadcrumbs: React.FC<IBreadcrumbsProps> = (props) => {
     []
   );
   const location = useLocation();
-  const {baseURL} = props;
-  console.log(baseURL)
+  const { baseURL } = props;
+  console.log(baseURL);
   const HOME_CRUMB: IBreadcrumbProps = {
     text: "Home",
-    href: baseURL + '/'
+    href: baseURL + "/",
   };
 
   React.useEffect(() => {
@@ -40,7 +40,10 @@ const Breadcrumbs: React.FC<IBreadcrumbsProps> = (props) => {
   );
 };
 
-const getBreadcrumbs = (baseURL: string, pathname: string): Array<IBreadcrumbProps> => {
+const getBreadcrumbs = (
+  baseURL: string,
+  pathname: string
+): Array<IBreadcrumbProps> => {
   const pathSnippets = pathname.split("/").filter((i) => i);
 
   return pathSnippets.map((name, index) => {
diff --git a/bentoml/yatai/web/client/src/components/Label.tsx b/bentoml/yatai/web/client/src/components/Label.tsx
index 450d9025a8a..f972845f295 100644
--- a/bentoml/yatai/web/client/src/components/Label.tsx
+++ b/bentoml/yatai/web/client/src/components/Label.tsx
@@ -1,5 +1,5 @@
-import * as React from 'react'
-import { Tag } from '@blueprintjs/core';
+import * as React from "react";
+import { Tag } from "@blueprintjs/core";
 
 export interface ILabelProps {
   name: string;
@@ -9,8 +9,10 @@ export interface ILabelProps {
 const Label: React.FC<ILabelProps> = (props) => {
   const tagValue = `${props.name}:${props.value}`;
   return (
-    <Tag key={tagValue} style={{margin: 5}}>{tagValue}</Tag>
-  )
+    <Tag key={tagValue} style={{ margin: 5 }}>
+      {tagValue}
+    </Tag>
+  );
 };
 
 export default Label;
diff --git a/bentoml/yatai/web/client/src/components/LabelDetailSection.tsx b/bentoml/yatai/web/client/src/components/LabelDetailSection.tsx
index 44439ad8834..eaf625ffa06 100644
--- a/bentoml/yatai/web/client/src/components/LabelDetailSection.tsx
+++ b/bentoml/yatai/web/client/src/components/LabelDetailSection.tsx
@@ -1,6 +1,6 @@
-import * as React from 'react'
-import * as lodash from 'lodash';
-import Label from './Label';
+import * as React from "react";
+import * as lodash from "lodash";
+import Label from "./Label";
 
 export interface ILabelDetailSectionProps {
   labels: any;
@@ -9,11 +9,12 @@ export interface ILabelDetailSectionProps {
 const LabelDetailSection: React.FC<ILabelDetailSectionProps> = (props) => {
   return (
     <p>
-      <b>Labels: </b> {lodash.map(props.labels, (value, name)=> {
-        return (<Label name={name} value={value}/>)
+      <b>Labels: </b>{" "}
+      {lodash.map(props.labels, (value, name) => {
+        return <Label name={name} value={value} />;
       })}
     </p>
-  )
+  );
 };
 
 export default LabelDetailSection;
diff --git a/bentoml/yatai/web/client/src/components/NavigationBar.tsx b/bentoml/yatai/web/client/src/components/NavigationBar.tsx
index 6e82f8bf38d..81e6e2744cf 100644
--- a/bentoml/yatai/web/client/src/components/NavigationBar.tsx
+++ b/bentoml/yatai/web/client/src/components/NavigationBar.tsx
@@ -15,7 +15,7 @@ import logo from "../assets/bentoml-logo.png";
 const NavigationBar = (props) => {
   const [highlight, setHighlight] = React.useState("");
   const location = useLocation();
-  const {baseURL} = props;
+  const { baseURL } = props;
 
   React.useEffect(() => {
     const currentLocation = location.pathname.split("/")[1];
@@ -27,7 +27,7 @@ const NavigationBar = (props) => {
       <NavbarGroup align={Alignment.LEFT}>
         <NavbarHeading>
           <Link to="/">
-            <img src={baseURL + `/${logo}`} width={150}  alt={""}/>{" "}
+            <img src={baseURL + `/${logo}`} width={150} alt={""} />{" "}
           </Link>
         </NavbarHeading>
         <NavbarDivider />
diff --git a/bentoml/yatai/web/client/src/pages/BentoServiceDetail.tsx b/bentoml/yatai/web/client/src/pages/BentoServiceDetail.tsx
index 3d78dedca37..67e11cbed84 100644
--- a/bentoml/yatai/web/client/src/pages/BentoServiceDetail.tsx
+++ b/bentoml/yatai/web/client/src/pages/BentoServiceDetail.tsx
@@ -6,7 +6,7 @@ import EnvTable from "../components/BentoServiceDetail/EnvTable";
 import ApisTable from "../components/BentoServiceDetail/ApisTable";
 import ArtifactsTable from "../components/BentoServiceDetail/ArtifactsTable";
 import { Section } from "../ui/Layout";
-import LabelDetailSection from '../components/LabelDetailSection';
+import LabelDetailSection from "../components/LabelDetailSection";
 
 const BentoServiceDetail = (props) => {
   const params = props.match.params;
@@ -34,7 +34,9 @@ const BentoServiceDetail = (props) => {
               <p>
                 <b>Storage: </b> {bento.uri.uri}
               </p>
-              <LabelDetailSection labels={bento.bento_service_metadata.labels}/>
+              <LabelDetailSection
+                labels={bento.bento_service_metadata.labels}
+              />
               <ApisTable apis={bento.bento_service_metadata.apis} />
               <ArtifactsTable
                 artifacts={bento.bento_service_metadata.artifacts}
diff --git a/bentoml/yatai/web/client/src/setupTests.ts b/bentoml/yatai/web/client/src/setupTests.ts
index 74b1a275a0e..5fdf001693f 100644
--- a/bentoml/yatai/web/client/src/setupTests.ts
+++ b/bentoml/yatai/web/client/src/setupTests.ts
@@ -2,4 +2,4 @@
 // allows you to do things like:
 // expect(element).toHaveTextContent(/react/i)
 // learn more: https://github.com/testing-library/jest-dom
-import '@testing-library/jest-dom/extend-expect';
+import "@testing-library/jest-dom/extend-expect";
diff --git a/bentoml/yatai/web/client/src/ui/Table.tsx b/bentoml/yatai/web/client/src/ui/Table.tsx
index 1b4034ee76a..4c1d411782f 100644
--- a/bentoml/yatai/web/client/src/ui/Table.tsx
+++ b/bentoml/yatai/web/client/src/ui/Table.tsx
@@ -53,14 +53,12 @@ const Table: React.FC<ITableProps> = (props) => {
     <TableContainer>
       {finalHeader && (
         <TableHeader>
-          {
-            // @ts-ignore
-            finalHeader.map((h, i) => (
-              <Cell key={i} flex={h[1]}>
-                {h[0]}
-              </Cell>
-            ))
-          }
+          {// @ts-ignore
+          finalHeader.map((h, i) => (
+            <Cell key={i} flex={h[1]}>
+              {h[0]}
+            </Cell>
+          ))}
         </TableHeader>
       )}
       {content.map((row, i) => {
diff --git a/bentoml/yatai/web/client/src/utils/HttpRequestContainer.tsx b/bentoml/yatai/web/client/src/utils/HttpRequestContainer.tsx
index 723f7efb28b..ef092e00c54 100644
--- a/bentoml/yatai/web/client/src/utils/HttpRequestContainer.tsx
+++ b/bentoml/yatai/web/client/src/utils/HttpRequestContainer.tsx
@@ -42,13 +42,11 @@ const DisplayHttpError = ({ error }) => {
   );
 };
 
-
 export const setBaseUrl = (baseURL: string) => {
-    axios.defaults.baseURL = baseURL;
-}
+  axios.defaults.baseURL = baseURL;
+};
 
 const fetch = (options = {}, props, callback) => {
-  
   const { url, data, method, headers, params } = Object.assign(
     {},
     props,
diff --git a/bentoml/yatai/web/client/tsconfig.json b/bentoml/yatai/web/client/tsconfig.json
index 24d6aa6ad24..e8cc9d32dc3 100644
--- a/bentoml/yatai/web/client/tsconfig.json
+++ b/bentoml/yatai/web/client/tsconfig.json
@@ -1,11 +1,7 @@
 {
   "compilerOptions": {
     "target": "es5",
-    "lib": [
-      "dom",
-      "dom.iterable",
-      "esnext"
-    ],
+    "lib": ["dom", "dom.iterable", "esnext"],
     "allowJs": true,
     "skipLibCheck": true,
     "esModuleInterop": true,
@@ -21,7 +17,5 @@
     "strict": true,
     "noImplicitAny": false
   },
-  "include": [
-    "src"
-  ]
+  "include": ["src"]
 }
diff --git a/bentoml/yatai/web/package.json b/bentoml/yatai/web/package.json
index 25b2b22f03f..e019b95147f 100644
--- a/bentoml/yatai/web/package.json
+++ b/bentoml/yatai/web/package.json
@@ -6,7 +6,7 @@
   "license": "Apache-2.0",
   "scripts": {
     "start": "node dist/bundle.js",
-    "dev": "bash -c 'concurrently \"nodemon -- ${0} ${1} ${2} ${3}\" \"npm run watch:client\"'",
+    "dev": "bash -c 'concurrently \"nodemon -- ${0} ${1} ${2} ${3} ${4}\" \"npm run watch:client\"'",
     "watch": "concurrently \"npm:watch:*\"",
     "watch:server": "nodemon",
     "watch:client": "cd client && yarn start",
@@ -23,6 +23,7 @@
   "dependencies": {
     "@grpc/grpc-js": "^0.7.5",
     "@types/lodash": "^4.14.149",
+    "axios": "^0.21.1",
     "express": "^4.17.1",
     "lodash": "^4.17.19",
     "morgan": "^1.10.0",
diff --git a/bentoml/yatai/web/src/index.ts b/bentoml/yatai/web/src/index.ts
index 1cf68128221..e8878fe092e 100644
--- a/bentoml/yatai/web/src/index.ts
+++ b/bentoml/yatai/web/src/index.ts
@@ -7,13 +7,14 @@ const grpcServerAddress = args[0];
 const port = args[1];
 const base_log_path = args[2];
 const base_url = args[3];
-if (!grpcServerAddress || !port || !base_log_path) {
+const prometheus_address = args[4];
+if (!grpcServerAddress || !prometheus_address || !port || !base_log_path) {
   throw Error(
-    "Required field grpc server address, port or base log path is missing"
+    "Required field grpc server address, port, base log path or prometheus address is missing"
   );
 }
 
 createLogger(base_log_path);
-const app = getExpressApp(grpcServerAddress, base_url);
+const app = getExpressApp(grpcServerAddress, base_url, prometheus_address);
 
 app.listen(port, () => console.log(`Running at http://localhost:${port}`));
diff --git a/bentoml/yatai/web/src/server.ts b/bentoml/yatai/web/src/server.ts
index 338f5e4aa37..8a10a4bedd1 100644
--- a/bentoml/yatai/web/src/server.ts
+++ b/bentoml/yatai/web/src/server.ts
@@ -5,6 +5,7 @@ import express from "express";
 import { bentoml } from "./generated/bentoml_grpc";
 import { createYataiClient } from "./yatai_client";
 import { getLogger } from "./logger";
+import axios from "axios";
 const logger = getLogger();
 
 const createAPIRoutes = (app, yataiClient) => {
@@ -197,17 +198,21 @@ const createAPIRoutes = (app, yataiClient) => {
   return router;
 };
 
-export const getExpressApp = (grpcAddress: string | null, baseURL: string) => {
+export const getExpressApp = (
+  grpcAddress: string | null,
+  baseURL: string,
+  prometheusAddress: string
+) => {
   const app = express();
-  const cookieParser = require('cookie-parser');
+  const cookieParser = require("cookie-parser");
   app.use(cookieParser());
   app.use(function (req, res, next) {
     var cookie = req.cookies.cookieName;
     if (cookie === undefined) {
-      res.cookie('baseURLCookie',baseURL, { maxAge: 900000, httpOnly: false });
-      console.log('cookie created successfully');
+      res.cookie("baseURLCookie", baseURL, { maxAge: 900000, httpOnly: false });
+      console.log("cookie created successfully");
     } else {
-      console.log('cookie exists', cookie);
+      console.log("cookie exists", cookie);
     }
     next();
   });
@@ -220,20 +225,29 @@ export const getExpressApp = (grpcAddress: string | null, baseURL: string) => {
   );
   const yataiClient = createYataiClient(grpcAddress);
   const router = createAPIRoutes(app, yataiClient);
-  const apiUrlPrefix = (baseURL=="." ? '/' : '/' + baseURL);
+  const apiUrlPrefix = baseURL == "." ? "/" : "/" + baseURL;
   app.use(apiUrlPrefix, router);
 
-  app.get('/healthz', (req, res) => res.status(200).json());
+  app.get("/healthz", (req, res) => res.status(200).json());
+
+  app.get("/metrics", async (req, res) => {
+    const metricsResponse = await axios.get(prometheusAddress);
+    res.end(metricsResponse.data);
+  });
 
   app.get("/*", (req, res) => {
     let directory = req.path.split("/").slice(-2, -1);
     let filename = req.path.split("/").pop();
     if (/.js$|.css$/.test(req.path)) {
-      res.sendFile(path.join(__dirname, `../dist/client/static/${directory}/${filename}`));
+      res.sendFile(
+        path.join(__dirname, `../dist/client/static/${directory}/${filename}`)
+      );
     } else if (/favicon.png$|logo192.png$/.test(req.path)) {
       res.sendFile(path.join(__dirname, `../dist/client/${filename}`));
     } else if (/.png/.test(req.path)) {
-      res.sendFile(path.join(__dirname, `../dist/client/static/${directory}/${filename}`));
+      res.sendFile(
+        path.join(__dirname, `../dist/client/static/${directory}/${filename}`)
+      );
     } else {
       res.sendFile(path.join(__dirname, "../dist/client/index.html"));
     }
diff --git a/bentoml/yatai/web/src/yatai_client.ts b/bentoml/yatai/web/src/yatai_client.ts
index 7e59f2e6498..82aad55caf7 100644
--- a/bentoml/yatai/web/src/yatai_client.ts
+++ b/bentoml/yatai/web/src/yatai_client.ts
@@ -1,5 +1,5 @@
-import {Client, credentials} from "@grpc/grpc-js";
-import {bentoml} from "./generated/bentoml_grpc";
+import { Client, credentials } from "@grpc/grpc-js";
+import { bentoml } from "./generated/bentoml_grpc";
 
 export const createYataiClient = (grpcAddress: string) => {
   const client = new Client(grpcAddress, credentials.createInsecure());
diff --git a/bentoml/yatai/web/yarn.lock b/bentoml/yatai/web/yarn.lock
index 8300924bb69..9fb1dbb3b09 100644
--- a/bentoml/yatai/web/yarn.lock
+++ b/bentoml/yatai/web/yarn.lock
@@ -1158,6 +1158,13 @@ aws4@^1.8.0:
   resolved "https://registry.yarnpkg.com/aws4/-/aws4-1.11.0.tgz#d61f46d83b2519250e2784daf5b09479a8b41c59"
   integrity sha512-xh1Rl34h6Fi1DC2WWKfxUTVqRsNnr6LsKz2+hfwDxQJWmrx8+c7ylaqBMcHfl1U1r2dsifOvKX3LQuLNZ+XSvA==
 
+axios@^0.21.1:
+  version "0.21.1"
+  resolved "https://registry.yarnpkg.com/axios/-/axios-0.21.1.tgz#22563481962f4d6bde9a76d516ef0e5d3c09b2b8"
+  integrity sha512-dKQiRHxGD9PPRIUNIWvZhPTPpl1rf/OxTYKsqKUDjBwYylTvV7SjSHJb9ratfyzM6wCdLCOYLzs73qpg5c4iGA==
+  dependencies:
+    follow-redirects "^1.10.0"
+
 babel-jest@^24.9.0:
   version "24.9.0"
   resolved "https://registry.yarnpkg.com/babel-jest/-/babel-jest-24.9.0.tgz#3fc327cb8467b89d14d7bc70e315104a783ccd54"
@@ -2975,6 +2982,11 @@ fn.name@1.x.x:
   resolved "https://registry.yarnpkg.com/fn.name/-/fn.name-1.1.0.tgz#26cad8017967aea8731bc42961d04a3d5988accc"
   integrity sha512-GRnmB5gPyJpAhTQdSZTSp9uaPSvl09KoYcMQtsB9rQoOmzs9dH6ffeccH+Z+cv6P68Hu5bC6JjRh4Ah/mHSNRw==
 
+follow-redirects@^1.10.0:
+  version "1.13.2"
+  resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.13.2.tgz#dd73c8effc12728ba5cf4259d760ea5fb83e3147"
+  integrity sha512-6mPTgLxYm3r6Bkkg0vNM0HTjfGrOEtsfbhagQvbxDEsEkpNhw582upBaoRZylzen6krEmxXJgt9Ju6HiI4O7BA==
+
 for-in@^1.0.2:
   version "1.0.2"
   resolved "https://registry.yarnpkg.com/for-in/-/for-in-1.0.2.tgz#81068d295a8142ec0ac726c6e2200c30fb6d5e80"
diff --git a/bentoml/yatai/yatai_service.py b/bentoml/yatai/yatai_service.py
index f4b34f29566..29a6bce1666 100644
--- a/bentoml/yatai/yatai_service.py
+++ b/bentoml/yatai/yatai_service.py
@@ -11,7 +11,13 @@
 from bentoml import config
 from bentoml.configuration import get_debug_mode
 from bentoml.exceptions import BentoMLException
+from bentoml.yatai.client.interceptor.prom_server_interceptor import (
+    PromServerInterceptor,
+    ServiceLatencyInterceptor,
+)
 from bentoml.yatai.utils import ensure_node_available_or_raise, parse_grpc_url
+from bentoml.utils import reserve_free_port
+from prometheus_client import start_http_server
 
 
 def get_yatai_service(
@@ -103,7 +109,12 @@ def start_yatai_service_grpc_server(
     yatai_service = YataiServicerImpl(
         db_url=db_url, repo_base_url=repo_base_url, s3_endpoint_url=s3_endpoint_url,
     )
-    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
+
+    # Define interceptors here
+    grpc_interceptors = [PromServerInterceptor(), ServiceLatencyInterceptor()]
+    server = grpc.server(
+        futures.ThreadPoolExecutor(max_workers=10), interceptors=grpc_interceptors,
+    )
     add_YataiServicer_to_server(yatai_service, server)
     debug_mode = get_debug_mode()
     if debug_mode:
@@ -123,6 +134,12 @@ def start_yatai_service_grpc_server(
                 '"pip install grpcio-reflection"'
             )
     server.add_insecure_port(f'[::]:{grpc_port}')
+    # TODO: it seems that current impl doesn't actually intercept RPC calls
+    prometheus_port = 50052
+    with reserve_free_port() as port:
+        prometheus_port = port
+    # prevents wsgi to see prometheus_port as used
+    start_http_server(prometheus_port)
     server.start()
     if with_ui:
         web_ui_log_path = os.path.join(
@@ -132,8 +149,14 @@ def start_yatai_service_grpc_server(
 
         ensure_node_available_or_raise()
         yatai_grpc_server_address = f'localhost:{grpc_port}'
+        prometheus_address = f'http://localhost:{prometheus_port}'
         async_start_yatai_service_web_ui(
-            yatai_grpc_server_address, ui_port, web_ui_log_path, debug_mode, base_url
+            yatai_grpc_server_address,
+            prometheus_address,
+            ui_port,
+            web_ui_log_path,
+            debug_mode,
+            base_url,
         )
 
     # We don't import _echo function from click_utils because of circular dep
@@ -143,6 +166,7 @@ def start_yatai_service_grpc_server(
         f'''* Web UI: {f"running on http://127.0.0.1:{ui_port}/{base_url}"
         if (with_ui and base_url!=".")
         else f"running on http://127.0.0.1:{ui_port}" if with_ui else "off"}\n'''
+        f'* Prometheus running on 127.0.0.1:{prometheus_port}\n'
         f'* Running on 127.0.0.1:{grpc_port} (Press CTRL+C to quit)\n'
         f'* Help and instructions: '
         f'https://docs.bentoml.org/en/latest/guides/yatai_service.html\n'
@@ -184,7 +208,12 @@ def _is_web_server_debug_tools_available(root_dir):
 
 
 def async_start_yatai_service_web_ui(
-    yatai_server_address, ui_port, base_log_path, debug_mode, web_prefix_path
+    yatai_server_address,
+    prometheus_address,
+    ui_port,
+    base_log_path,
+    debug_mode,
+    web_prefix_path,
 ):
     if ui_port is not None:
         ui_port = ui_port if isinstance(ui_port, str) else str(ui_port)
@@ -204,6 +233,7 @@ def async_start_yatai_service_web_ui(
                 ui_port,
                 base_log_path,
                 web_prefix_path,
+                prometheus_address,
             ]
         else:
             web_ui_command = [
@@ -213,6 +243,7 @@ def async_start_yatai_service_web_ui(
                 ui_port,
                 base_log_path,
                 web_prefix_path,
+                prometheus_address,
             ]
     else:
         if not os.path.exists(os.path.join(web_ui_dir, 'dist', 'bundle.js')):
@@ -228,6 +259,7 @@ def async_start_yatai_service_web_ui(
             ui_port,
             base_log_path,
             web_prefix_path,
+            prometheus_address,
         ]
 
     web_proc = subprocess.Popen(
diff --git a/dev/format.sh b/dev/format.sh
index 80bb5b3763c..789a3b3a9a1 100755
--- a/dev/format.sh
+++ b/dev/format.sh
@@ -1,4 +1,10 @@
 #!/usr/bin/env bash
 
 GIT_ROOT=$(git rev-parse --show-toplevel)
+# format yatai webui
+WEB_UI_DIR=$GIT_ROOT/bentoml/yatai/web
+
 black -S "$GIT_ROOT"
+
+cd "$WEB_UI_DIR" && yarn format
+
diff --git a/protos/generate.sh b/protos/generate.sh
index 2b01ade0345..93d14c09ac9 100755
--- a/protos/generate.sh
+++ b/protos/generate.sh
@@ -8,20 +8,30 @@ fi
 
 PROTO_PATH=$BENTOML_REPO/protos
 PY_OUT_PATH=$BENTOML_REPO/bentoml/yatai/proto
+# test YataiService Interceptor calls
+PROTO_TEST_PATH=$PROTO_PATH/tests
+PY_TEST_OUT_PATH=$BENTOML_REPO/tests/yatai/proto
 
 echo "Cleaning up existing proto generated py code.."
-rm -rf "$PY_OUT_PATH"
-mkdir -p "$PY_OUT_PATH"
-touch "$PY_OUT_PATH"/__init__.py
+rm -rf "$PY_OUT_PATH" "$PY_TEST_OUT_PATH"
+mkdir -p "$PY_OUT_PATH" "$PY_TEST_OUT_PATH"
+touch "$PY_OUT_PATH"/__init__.py "$PY_TEST_OUT_PATH"/__init__.py
 
 echo "Generate proto message code.."
-find "$PROTO_PATH"/ -name '*.proto' | while read -r protofile; do
+find "$PROTO_PATH"/ -name '*.proto' -not -path "${PROTO_TEST_PATH}/*" | while read -r protofile; do
 python -m grpc_tools.protoc \
   -I"$PROTO_PATH" \
   --python_out="$PY_OUT_PATH" \
   "$protofile"
 done
 
+echo "Generate grpc test service code.."
+python -m grpc_tools.protoc \
+  -I"$PROTO_TEST_PATH" \
+  --python_out="$PY_TEST_OUT_PATH" \
+  --grpc_python_out="$PY_TEST_OUT_PATH" \
+  "$PROTO_TEST_PATH"/mock_service.proto
+
 echo "Generate grpc service code.."
 python -m grpc_tools.protoc \
   -I"$PROTO_PATH" \
@@ -29,47 +39,61 @@ python -m grpc_tools.protoc \
   --grpc_python_out="$PY_OUT_PATH" \
   "$PROTO_PATH"/yatai_service.proto
 
+fix_grpc_service_code(){
+  PKGNAME=$1
+  OUT_DIR=$2
 
-echo "Fix imports in generated GRPC service code.."
-find "$PY_OUT_PATH"/ -name '*_pb2*.py' | while read -r pyfile; do
-sed -i'.old' 's/^import \([^ ]*\)_pb2 \(.*\)$/import bentoml.yatai.proto.\1_pb2 \2/' "$pyfile"
-sed -i'.old' 's/^from \([^ ]*\) import \([^ ]*\)_pb2\(.*\)$/from bentoml.yatai.proto.\1 import \2_pb2\3/' "$pyfile"
-# Fix google.protobuf package imports
-sed -i'.old' 's/^import bentoml.yatai.proto.google.\([^ ]*\)_pb2 as \([^ ]*\)$/import google.\1_pb2 as \2/' "$pyfile"
-sed -i'.old' 's/^from bentoml.yatai.proto.google.\([^ ]*\) import \([^ ]*\)_pb2 as \([^ ]*\)$/from google.\1 import \2_pb2 as \3/' "$pyfile"
-rm "$pyfile".old
-done
+  echo "Fix imports in generated GRPC service code.."
+  find "$OUT_DIR" -name '*_pb2*.py' | while read -r pyfile; do
+  sed -i'.old' "s/^import \([^ ]*\)_pb2 \(.*\)$/import $PKGNAME.yatai.proto.\1_pb2 \2/" "$pyfile"
+  sed -i'.old' "s/^from \([^ ]*\) import \([^ ]*\)_pb2\(.*\)$/from $PKGNAME.yatai.proto.\1 import \2_pb2\3/" "$pyfile"
+  # Fix google.protobuf package imports
+  sed -i'.old' "s/^import $PKGNAME.yatai.proto.google.\([^ ]*\)_pb2 as \([^ ]*\)$/import google.\1_pb2 as \2/" "$pyfile"
+  sed -i'.old' "s/^from $PKGNAME.yatai.proto.google.\([^ ]*\) import \([^ ]*\)_pb2 as \([^ ]*\)$/from google.\1 import \2_pb2 as \3/" "$pyfile"
+  rm "$pyfile".old
+  done
+}
 
 
-PKG_PATH=$PY_OUT_PATH
-PKGS=$(find "$PKG_PATH" -type d)
-for PKG in $PKGS; do
-    SUBPACKAGES=$(find "$PKG" -maxdepth 1 -type d | grep -E -v "${PKG}$" | sort)
-    MODULES=$(find "$PKG" -maxdepth 1 -iname "*.py" | grep -v "__init__.py" | sort)
-    MODULE_COUNT=$(echo "$MODULES" | wc -w)
-    PKG_INIT="${PKG}/__init__.py"
-    echo "Writing Python package exports"
-    echo "------------------------------"
-    echo "PKG: ${PKG} (${PKG_INIT})"
-    echo "SUBPACKAGES: ${SUBPACKAGES}"
-    echo "FOUND MODULES: $MODULE_COUNT"
-    # echo "MODULES: ${MODULES}"
-    echo ""
-    echo "__all__ = [" > "$PKG_INIT"
-    for MODULE in $MODULES; do
-        FILENAME=$(basename "$MODULE" .py)
-        echo "    \"${FILENAME}\"," >> "$PKG_INIT"
-    done
-    for SUBPKG in $SUBPACKAGES; do
-        SUBPKGNAME=$(basename "$SUBPKG")
-        echo "    \"${SUBPKGNAME}\"," >> "$PKG_INIT"
-    done
-    echo "]" >> "$PKG_INIT"
+fix_grpc_service_code "bentoml" "$PY_OUT_PATH"
+fix_grpc_service_code "tests" "$PY_TEST_OUT_PATH"
+
+
+edit_init() {
+  PKG_PATH=$1
+
+  PKGS=$(find "$PKG_PATH" -type d)
+  for PKG in $PKGS; do
+      SUBPACKAGES=$(find "$PKG" -maxdepth 1 -type d | grep -E -v "${PKG}$" | sort)
+      MODULES=$(find "$PKG" -maxdepth 1 -iname "*.py" | grep -v "__init__.py" | sort)
+      MODULE_COUNT=$(echo "$MODULES" | wc -w)
+      PKG_INIT="${PKG}/__init__.py"
+      echo "Writing Python package exports"
+      echo "------------------------------"
+      echo "PKG: ${PKG} (${PKG_INIT})"
+      echo "SUBPACKAGES: ${SUBPACKAGES}"
+      echo "FOUND MODULES: $MODULE_COUNT"
+      # echo "MODULES: ${MODULES}"
+      echo ""
+      echo "__all__ = [" > "$PKG_INIT"
+      for MODULE in $MODULES; do
+          FILENAME=$(basename "$MODULE" .py)
+          echo "    \"${FILENAME}\"," >> "$PKG_INIT"
+      done
+      for SUBPKG in $SUBPACKAGES; do
+          SUBPKGNAME=$(basename "$SUBPKG")
+          echo "    \"${SUBPKGNAME}\"," >> "$PKG_INIT"
+      done
+      echo "]" >> "$PKG_INIT"
   done
 
-echo "Done"
+  echo "Done"
+}
 
+edit_init "$PY_OUT_PATH"
+edit_init "$PY_TEST_OUT_PATH"
 
+# TODO: if run with generate-docker then we don't have to print this warning
 echo "Generate grpc code for javascript/typescript"
 echo "Please make sure protobufjs is installed on your system"
 echo "You can install with npm i -g protobufjs"
diff --git a/protos/tests/mock_service.proto b/protos/tests/mock_service.proto
new file mode 100644
index 00000000000..615a90bedea
--- /dev/null
+++ b/protos/tests/mock_service.proto
@@ -0,0 +1,18 @@
+syntax = "proto3";
+package bentoml;
+
+message MockRequest {
+  string input = 1;
+}
+
+message MockResponse {
+  string output = 1;
+}
+
+// A Simple MockService to test interceptors for each RPC calls
+service MockService {
+  rpc Execute (MockRequest) returns (MockResponse);
+  rpc ExecuteClientStream (stream MockRequest) returns (MockResponse);
+  rpc ExecuteServerStream (MockRequest) returns (stream MockResponse);
+  rpc ExecuteClientServerStream (stream MockRequest) returns (stream MockResponse);
+}
diff --git a/pylintrc b/pylintrc
index 3148e0cf614..f1c8fd9deb5 100644
--- a/pylintrc
+++ b/pylintrc
@@ -7,7 +7,7 @@ extension-pkg-whitelist=dependency_injector.containers,dependency_injector.provi
 
 # Add files or directories to the blacklist. They should be base names, not
 # paths.
-ignore=proto, _version.py
+ignore=tests/yatai/protos, proto, _version.py
 
 # Add files or directories matching the regex patterns to the blacklist. The
 # regex matches against base names, not paths.
diff --git a/pytest.ini b/pytest.ini
index 1ceab942957..ab669b4901e 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -1,2 +1,3 @@
 [pytest]
 addopts = -p no:warnings
+norecursedirs = tests/yatai/protos
diff --git a/setup.py b/setup.py
index 2adc86a2faa..6c5d0989726 100644
--- a/setup.py
+++ b/setup.py
@@ -71,6 +71,7 @@
     "pytest-cov>=2.7.1",
     "pytest>=5.4.0",
     "pytest-asyncio",
+    "parameterized",
     "scikit-learn",
 ]
 
diff --git a/tests/yatai/client/test_grpc_server_interceptor.py b/tests/yatai/client/test_grpc_server_interceptor.py
new file mode 100644
index 00000000000..e5d03d7fece
--- /dev/null
+++ b/tests/yatai/client/test_grpc_server_interceptor.py
@@ -0,0 +1,154 @@
+from unittest import TestCase
+from unittest.mock import Mock
+from parameterized import parameterized
+
+from bentoml.yatai.client.interceptor.prom_server_interceptor import (
+    PromServerInterceptor,
+    ServiceLatencyInterceptor,
+    _wrap_rpc_behaviour,
+)
+from bentoml.yatai.utils import parse_method_name, MethodName
+
+import requests
+
+from tests.yatai.grpc_testing_service import MockRequest, MockServerClient
+
+
+def test_method_name():
+    # Fields are correct and fully_qualified_service work.
+    mn = MethodName("foo.bar", "SearchService", "Search")
+    assert mn.package == "foo.bar"
+    assert mn.service == "SearchService"
+    assert mn.method == "Search"
+    assert mn.fully_qualified_service == "foo.bar.SearchService"
+
+
+def test_empty_package_method_name():
+    # fully_qualified_service works when there's no package
+    mn = MethodName("", "SearchService", "Search")
+    assert mn.fully_qualified_service == "SearchService"
+
+
+def test_parse_method_name():
+    mn, ok = parse_method_name("/foo.bar.SearchService/Search")
+    assert mn.package == "foo.bar"
+    assert mn.service == "SearchService"
+    assert mn.method == "Search"
+    assert ok
+
+
+def test_parse_empty_package():
+    # parse_method_name works with no package.
+    mn, _ = parse_method_name("/SearchService/Search")
+    assert mn.package == ""
+    assert mn.service == "SearchService"
+    assert mn.method == "Search"
+
+
+class TestPrometheusServerInterceptor(TestCase):
+    def setUp(self):
+        self.interceptor: PromServerInterceptor = PromServerInterceptor()
+
+    def test_handler_none(self):
+        mock_continuation = Mock(return_value=None)
+
+        ret = self.interceptor.intercept_service(
+            mock_continuation, Mock(return_value=None)
+        )
+
+        assert not ret
+
+    @parameterized.expand(
+        [(True, True, 1), (True, False, 1), (False, True, 1), (False, False, 2)],
+    )
+    def test_intercept_service_unary(
+        self, request_streaming, response_streaming, calls
+    ):
+        mock_handler_call_details = Mock(method="/grpc-service")
+        mock_handler = Mock(
+            request_streaming=request_streaming, response_streaming=response_streaming
+        )
+        mock_continuation = Mock(return_value=mock_handler)
+
+        self.interceptor.intercept_service(mock_continuation, mock_handler_call_details)
+        assert calls == mock_continuation.call_count
+
+    @parameterized.expand([("/grpc-service"), ("/grpc-service/grpc-method")])
+    def test_intercept_service(self, method):
+        mock_handler_call_details = Mock(method=method)
+        mock_handler = Mock(request_streaming=False, response_streaming=False)
+        mock_continuation = Mock(return_value=mock_handler)
+
+        self.interceptor.intercept_service(mock_continuation, mock_handler_call_details)
+        assert 2 == mock_continuation.call_count
+
+
+class TestServiceLatencyInterceptor(TestCase):
+    def setUp(self):
+        self.interceptor: ServiceLatencyInterceptor = ServiceLatencyInterceptor()
+
+    @parameterized.expand([("/grpc-latency"), ("/grpc-latency/grpc-method")])
+    def test_intercept_service(self, method):
+        mock_handler_call_details = Mock(method=method)
+        mock_handler = Mock(request_streaming=False, response_streaming=False)
+        mock_continuation = Mock(return_value=mock_handler)
+
+        self.interceptor.intercept_service(mock_continuation, mock_handler_call_details)
+        assert mock_continuation.call_count == 2
+
+
+class TestWrapRPCBehaviour(TestCase):
+    @parameterized.expand(
+        [
+            (True, True, "stream_stream"),
+            (True, False, "stream_unary"),
+            (False, True, "unary_stream"),
+            (False, False, "unary_unary"),
+        ]
+    )
+    def test_wrap_rpc_behaviour(self, request_streaming, response_streaming, behaviour):
+        mock_handler = Mock(
+            request_streaming=request_streaming, response_streaming=response_streaming
+        )
+        mock_fn = Mock()
+        res = _wrap_rpc_behaviour(mock_handler, mock_fn)
+        assert mock_fn.call_count == 1
+        assert getattr(res, behaviour) is not None
+
+    def test_wrap_rpc_behaviour_none(self):
+        assert not _wrap_rpc_behaviour(None, Mock())
+
+
+class TestMetrics(TestCase):
+    @parameterized.expand(
+        [
+            (
+                'grpc_server_started_total{grpc_method="Execute",grpc_service="bentoml.MockService",grpc_type="UNARY"}',  # noqa: E501
+                1.0,
+            ),
+            (
+                'grpc_server_started_total{grpc_method="Execute",grpc_service="bentoml.MockService",grpc_type="UNARY"}',  # noqa: E501
+                2.0,
+            ),
+            (
+                'grpc_server_started_total{grpc_method="Execute",grpc_service="bentoml.MockService",grpc_type="UNARY"}',  # noqa: E501
+                3.0,
+            ),
+            (
+                'grpc_server_handled_total{grpc_code="OK",grpc_method="Execute",grpc_service="bentoml.MockService",grpc_type="UNARY"}',  # noqa: E501
+                4.0,
+            ),
+        ]
+    )
+    def test_grpc_server_metrics(self, metric_name, value):
+        mock_server = MockServerClient(
+            special_cases="",
+            server_interceptors=[PromServerInterceptor(), ServiceLatencyInterceptor()],
+        )
+        with mock_server.mock_server_client() as client:
+            assert client.Execute(MockRequest(input="foo")).output == "foo"
+            r = requests.get(f"http://localhost:{mock_server.prom_port}")
+            assert (
+                f"{metric_name} {value}" in r.text
+            ), f"expected metrics {metric_name} {value}\
+                not found in server response:\n{r.text}"
diff --git a/tests/yatai/grpc_testing_service.py b/tests/yatai/grpc_testing_service.py
new file mode 100644
index 00000000000..c8f21c5e433
--- /dev/null
+++ b/tests/yatai/grpc_testing_service.py
@@ -0,0 +1,111 @@
+import contextlib
+from concurrent import futures
+from typing import Callable, Dict, Iterable, Optional, Union, List
+
+import grpc
+from prometheus_client import start_http_server
+
+from tests.yatai.proto.mock_service_pb2 import MockRequest, MockResponse
+from tests.yatai.proto.mock_service_pb2_grpc import MockServiceServicer
+from bentoml.utils import reserve_free_port
+
+SpecialCaseFunction = Callable[[str, grpc.ServicerContext], str]
+
+
+# this would also work for future interceptor if needed
+class MockService(MockServiceServicer):
+    """A gRPC service used for testing
+
+    Args:
+        special_cases: dict where keys are string, values are servicer_context
+        that take and return strings. This will allow testing exception"""
+
+    def __init__(self, special_cases: Union[str, Dict[str, SpecialCaseFunction]]):
+        self._special_cases = special_cases if isinstance(special_cases, dict) else ""
+
+    def Execute(
+        self, request: MockRequest, context: grpc.ServicerContext
+    ) -> MockResponse:
+        return MockResponse(output=self.__get_output(request, context))
+
+    def ExecuteClientStream(
+        self, request_iterator: Iterable[MockRequest], context: grpc.ServicerContext
+    ) -> MockResponse:
+        output = "".join(
+            self.__get_output(request, context) for request in request_iterator
+        )
+        return MockResponse(output=output)
+
+    def ExecuteServerStream(
+        self, request: MockRequest, context: grpc.ServicerContext
+    ) -> Iterable[MockResponse]:
+        for _o in self.__get_output(request, context):
+            yield MockResponse(output=_o)
+
+    def ExecuteClientServerStream(
+        self, request_iterator: Iterable[MockRequest], context: grpc.ServicerContext
+    ) -> Iterable[MockResponse]:
+        for request in request_iterator:
+            yield MockResponse(output=self.__get_output(request, context))
+
+    def __get_output(self, request: MockRequest, context: grpc.ServicerContext) -> str:
+        inp = request.input
+        output = inp
+
+        if isinstance(self._special_cases, dict):
+            if inp in self._special_cases:
+                output = self._special_cases[inp][inp, context]
+
+        return output
+
+
+class MockServerClient:
+    def __init__(
+        self,
+        special_cases: Union[str, Dict[str, SpecialCaseFunction]],
+        server_interceptors: Optional[List] = None,
+        prometheus_enabled: Optional[bool] = True,
+    ):
+        self.special_cases = special_cases
+        self.server_interceptors = server_interceptors
+        self.prometheus_enabled = prometheus_enabled
+        self.mock_server: grpc.Server
+        with reserve_free_port() as server_port:
+            self.server_port: int = server_port
+        with reserve_free_port() as prom_port:
+            self.prom_port: int = prom_port
+
+    @contextlib.contextmanager
+    def mock_server_client(self):
+        """A context manager returns a gRPC client connected with MockService"""
+        from tests.yatai.proto.mock_service_pb2_grpc import (
+            add_MockServiceServicer_to_server,
+            MockServiceStub,
+        )
+
+        if not self.server_interceptors:
+            interceptors = []
+        else:
+            interceptors = self.server_interceptors
+
+        self.mock_server = grpc.server(
+            futures.ThreadPoolExecutor(max_workers=1), interceptors=interceptors
+        )
+        mock_service = MockService(special_cases=self.special_cases)
+        add_MockServiceServicer_to_server(mock_service, self.mock_server)
+
+        # reserve a free port for windows, else we use unix domain socket
+
+        self.mock_server.add_insecure_port(f'[::]:{self.server_port}')
+        self.mock_server.start()
+
+        if self.prometheus_enabled:
+            start_http_server(self.prom_port)
+        channel = grpc.insecure_channel(f'localhost:{self.server_port}')
+
+        client_stub = MockServiceStub(channel)
+
+        try:
+            yield client_stub
+        finally:
+            self.mock_server.stop(None)
diff --git a/tests/yatai/proto/__init__.py b/tests/yatai/proto/__init__.py
new file mode 100644
index 00000000000..be8d77e8826
--- /dev/null
+++ b/tests/yatai/proto/__init__.py
@@ -0,0 +1,4 @@
+__all__ = [
+    "mock_service_pb2",
+    "mock_service_pb2_grpc",
+]
diff --git a/tests/yatai/proto/mock_service_pb2.py b/tests/yatai/proto/mock_service_pb2.py
new file mode 100644
index 00000000000..fd39f2d0435
--- /dev/null
+++ b/tests/yatai/proto/mock_service_pb2.py
@@ -0,0 +1,189 @@
+# -*- coding: utf-8 -*-
+# Generated by the protocol buffer compiler.  DO NOT EDIT!
+# source: mock_service.proto
+"""Generated protocol buffer code."""
+from google.protobuf import descriptor as _descriptor
+from google.protobuf import message as _message
+from google.protobuf import reflection as _reflection
+from google.protobuf import symbol_database as _symbol_database
+
+# @@protoc_insertion_point(imports)
+
+_sym_db = _symbol_database.Default()
+
+
+DESCRIPTOR = _descriptor.FileDescriptor(
+    name='mock_service.proto',
+    package='bentoml',
+    syntax='proto3',
+    serialized_options=None,
+    create_key=_descriptor._internal_create_key,
+    serialized_pb=b'\n\x12mock_service.proto\x12\x07\x62\x65ntoml\"\x1c\n\x0bMockRequest\x12\r\n\x05input\x18\x01 \x01(\t\"\x1e\n\x0cMockResponse\x12\x0e\n\x06output\x18\x01 \x01(\t2\x9f\x02\n\x0bMockService\x12\x36\n\x07\x45xecute\x12\x14.bentoml.MockRequest\x1a\x15.bentoml.MockResponse\x12\x44\n\x13\x45xecuteClientStream\x12\x14.bentoml.MockRequest\x1a\x15.bentoml.MockResponse(\x01\x12\x44\n\x13\x45xecuteServerStream\x12\x14.bentoml.MockRequest\x1a\x15.bentoml.MockResponse0\x01\x12L\n\x19\x45xecuteClientServerStream\x12\x14.bentoml.MockRequest\x1a\x15.bentoml.MockResponse(\x01\x30\x01\x62\x06proto3',
+)
+
+
+_MOCKREQUEST = _descriptor.Descriptor(
+    name='MockRequest',
+    full_name='bentoml.MockRequest',
+    filename=None,
+    file=DESCRIPTOR,
+    containing_type=None,
+    create_key=_descriptor._internal_create_key,
+    fields=[
+        _descriptor.FieldDescriptor(
+            name='input',
+            full_name='bentoml.MockRequest.input',
+            index=0,
+            number=1,
+            type=9,
+            cpp_type=9,
+            label=1,
+            has_default_value=False,
+            default_value=b"".decode('utf-8'),
+            message_type=None,
+            enum_type=None,
+            containing_type=None,
+            is_extension=False,
+            extension_scope=None,
+            serialized_options=None,
+            file=DESCRIPTOR,
+            create_key=_descriptor._internal_create_key,
+        ),
+    ],
+    extensions=[],
+    nested_types=[],
+    enum_types=[],
+    serialized_options=None,
+    is_extendable=False,
+    syntax='proto3',
+    extension_ranges=[],
+    oneofs=[],
+    serialized_start=31,
+    serialized_end=59,
+)
+
+
+_MOCKRESPONSE = _descriptor.Descriptor(
+    name='MockResponse',
+    full_name='bentoml.MockResponse',
+    filename=None,
+    file=DESCRIPTOR,
+    containing_type=None,
+    create_key=_descriptor._internal_create_key,
+    fields=[
+        _descriptor.FieldDescriptor(
+            name='output',
+            full_name='bentoml.MockResponse.output',
+            index=0,
+            number=1,
+            type=9,
+            cpp_type=9,
+            label=1,
+            has_default_value=False,
+            default_value=b"".decode('utf-8'),
+            message_type=None,
+            enum_type=None,
+            containing_type=None,
+            is_extension=False,
+            extension_scope=None,
+            serialized_options=None,
+            file=DESCRIPTOR,
+            create_key=_descriptor._internal_create_key,
+        ),
+    ],
+    extensions=[],
+    nested_types=[],
+    enum_types=[],
+    serialized_options=None,
+    is_extendable=False,
+    syntax='proto3',
+    extension_ranges=[],
+    oneofs=[],
+    serialized_start=61,
+    serialized_end=91,
+)
+
+DESCRIPTOR.message_types_by_name['MockRequest'] = _MOCKREQUEST
+DESCRIPTOR.message_types_by_name['MockResponse'] = _MOCKRESPONSE
+_sym_db.RegisterFileDescriptor(DESCRIPTOR)
+
+MockRequest = _reflection.GeneratedProtocolMessageType(
+    'MockRequest',
+    (_message.Message,),
+    {
+        'DESCRIPTOR': _MOCKREQUEST,
+        '__module__': 'mock_service_pb2'
+        # @@protoc_insertion_point(class_scope:bentoml.MockRequest)
+    },
+)
+_sym_db.RegisterMessage(MockRequest)
+
+MockResponse = _reflection.GeneratedProtocolMessageType(
+    'MockResponse',
+    (_message.Message,),
+    {
+        'DESCRIPTOR': _MOCKRESPONSE,
+        '__module__': 'mock_service_pb2'
+        # @@protoc_insertion_point(class_scope:bentoml.MockResponse)
+    },
+)
+_sym_db.RegisterMessage(MockResponse)
+
+
+_MOCKSERVICE = _descriptor.ServiceDescriptor(
+    name='MockService',
+    full_name='bentoml.MockService',
+    file=DESCRIPTOR,
+    index=0,
+    serialized_options=None,
+    create_key=_descriptor._internal_create_key,
+    serialized_start=94,
+    serialized_end=381,
+    methods=[
+        _descriptor.MethodDescriptor(
+            name='Execute',
+            full_name='bentoml.MockService.Execute',
+            index=0,
+            containing_service=None,
+            input_type=_MOCKREQUEST,
+            output_type=_MOCKRESPONSE,
+            serialized_options=None,
+            create_key=_descriptor._internal_create_key,
+        ),
+        _descriptor.MethodDescriptor(
+            name='ExecuteClientStream',
+            full_name='bentoml.MockService.ExecuteClientStream',
+            index=1,
+            containing_service=None,
+            input_type=_MOCKREQUEST,
+            output_type=_MOCKRESPONSE,
+            serialized_options=None,
+            create_key=_descriptor._internal_create_key,
+        ),
+        _descriptor.MethodDescriptor(
+            name='ExecuteServerStream',
+            full_name='bentoml.MockService.ExecuteServerStream',
+            index=2,
+            containing_service=None,
+            input_type=_MOCKREQUEST,
+            output_type=_MOCKRESPONSE,
+            serialized_options=None,
+            create_key=_descriptor._internal_create_key,
+        ),
+        _descriptor.MethodDescriptor(
+            name='ExecuteClientServerStream',
+            full_name='bentoml.MockService.ExecuteClientServerStream',
+            index=3,
+            containing_service=None,
+            input_type=_MOCKREQUEST,
+            output_type=_MOCKRESPONSE,
+            serialized_options=None,
+            create_key=_descriptor._internal_create_key,
+        ),
+    ],
+)
+_sym_db.RegisterServiceDescriptor(_MOCKSERVICE)
+
+DESCRIPTOR.services_by_name['MockService'] = _MOCKSERVICE
+
+# @@protoc_insertion_point(module_scope)
diff --git a/tests/yatai/proto/mock_service_pb2_grpc.py b/tests/yatai/proto/mock_service_pb2_grpc.py
new file mode 100644
index 00000000000..ea932dfe59f
--- /dev/null
+++ b/tests/yatai/proto/mock_service_pb2_grpc.py
@@ -0,0 +1,217 @@
+# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
+"""Client and server classes corresponding to protobuf-defined services."""
+import grpc
+
+import tests.yatai.proto.mock_service_pb2 as mock__service__pb2
+
+
+class MockServiceStub(object):
+    """A Simple MockService to test interceptors for each RPC calls
+    """
+
+    def __init__(self, channel):
+        """Constructor.
+
+        Args:
+            channel: A grpc.Channel.
+        """
+        self.Execute = channel.unary_unary(
+            '/bentoml.MockService/Execute',
+            request_serializer=mock__service__pb2.MockRequest.SerializeToString,
+            response_deserializer=mock__service__pb2.MockResponse.FromString,
+        )
+        self.ExecuteClientStream = channel.stream_unary(
+            '/bentoml.MockService/ExecuteClientStream',
+            request_serializer=mock__service__pb2.MockRequest.SerializeToString,
+            response_deserializer=mock__service__pb2.MockResponse.FromString,
+        )
+        self.ExecuteServerStream = channel.unary_stream(
+            '/bentoml.MockService/ExecuteServerStream',
+            request_serializer=mock__service__pb2.MockRequest.SerializeToString,
+            response_deserializer=mock__service__pb2.MockResponse.FromString,
+        )
+        self.ExecuteClientServerStream = channel.stream_stream(
+            '/bentoml.MockService/ExecuteClientServerStream',
+            request_serializer=mock__service__pb2.MockRequest.SerializeToString,
+            response_deserializer=mock__service__pb2.MockResponse.FromString,
+        )
+
+
+class MockServiceServicer(object):
+    """A Simple MockService to test interceptors for each RPC calls
+    """
+
+    def Execute(self, request, context):
+        """Missing associated documentation comment in .proto file."""
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+        context.set_details('Method not implemented!')
+        raise NotImplementedError('Method not implemented!')
+
+    def ExecuteClientStream(self, request_iterator, context):
+        """Missing associated documentation comment in .proto file."""
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+        context.set_details('Method not implemented!')
+        raise NotImplementedError('Method not implemented!')
+
+    def ExecuteServerStream(self, request, context):
+        """Missing associated documentation comment in .proto file."""
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+        context.set_details('Method not implemented!')
+        raise NotImplementedError('Method not implemented!')
+
+    def ExecuteClientServerStream(self, request_iterator, context):
+        """Missing associated documentation comment in .proto file."""
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)
+        context.set_details('Method not implemented!')
+        raise NotImplementedError('Method not implemented!')
+
+
+def add_MockServiceServicer_to_server(servicer, server):
+    rpc_method_handlers = {
+        'Execute': grpc.unary_unary_rpc_method_handler(
+            servicer.Execute,
+            request_deserializer=mock__service__pb2.MockRequest.FromString,
+            response_serializer=mock__service__pb2.MockResponse.SerializeToString,
+        ),
+        'ExecuteClientStream': grpc.stream_unary_rpc_method_handler(
+            servicer.ExecuteClientStream,
+            request_deserializer=mock__service__pb2.MockRequest.FromString,
+            response_serializer=mock__service__pb2.MockResponse.SerializeToString,
+        ),
+        'ExecuteServerStream': grpc.unary_stream_rpc_method_handler(
+            servicer.ExecuteServerStream,
+            request_deserializer=mock__service__pb2.MockRequest.FromString,
+            response_serializer=mock__service__pb2.MockResponse.SerializeToString,
+        ),
+        'ExecuteClientServerStream': grpc.stream_stream_rpc_method_handler(
+            servicer.ExecuteClientServerStream,
+            request_deserializer=mock__service__pb2.MockRequest.FromString,
+            response_serializer=mock__service__pb2.MockResponse.SerializeToString,
+        ),
+    }
+    generic_handler = grpc.method_handlers_generic_handler(
+        'bentoml.MockService', rpc_method_handlers
+    )
+    server.add_generic_rpc_handlers((generic_handler,))
+
+
+# This class is part of an EXPERIMENTAL API.
+class MockService(object):
+    """A Simple MockService to test interceptors for each RPC calls
+    """
+
+    @staticmethod
+    def Execute(
+        request,
+        target,
+        options=(),
+        channel_credentials=None,
+        call_credentials=None,
+        insecure=False,
+        compression=None,
+        wait_for_ready=None,
+        timeout=None,
+        metadata=None,
+    ):
+        return grpc.experimental.unary_unary(
+            request,
+            target,
+            '/bentoml.MockService/Execute',
+            mock__service__pb2.MockRequest.SerializeToString,
+            mock__service__pb2.MockResponse.FromString,
+            options,
+            channel_credentials,
+            insecure,
+            call_credentials,
+            compression,
+            wait_for_ready,
+            timeout,
+            metadata,
+        )
+
+    @staticmethod
+    def ExecuteClientStream(
+        request_iterator,
+        target,
+        options=(),
+        channel_credentials=None,
+        call_credentials=None,
+        insecure=False,
+        compression=None,
+        wait_for_ready=None,
+        timeout=None,
+        metadata=None,
+    ):
+        return grpc.experimental.stream_unary(
+            request_iterator,
+            target,
+            '/bentoml.MockService/ExecuteClientStream',
+            mock__service__pb2.MockRequest.SerializeToString,
+            mock__service__pb2.MockResponse.FromString,
+            options,
+            channel_credentials,
+            insecure,
+            call_credentials,
+            compression,
+            wait_for_ready,
+            timeout,
+            metadata,
+        )
+
+    @staticmethod
+    def ExecuteServerStream(
+        request,
+        target,
+        options=(),
+        channel_credentials=None,
+        call_credentials=None,
+        insecure=False,
+        compression=None,
+        wait_for_ready=None,
+        timeout=None,
+        metadata=None,
+    ):
+        return grpc.experimental.unary_stream(
+            request,
+            target,
+            '/bentoml.MockService/ExecuteServerStream',
+            mock__service__pb2.MockRequest.SerializeToString,
+            mock__service__pb2.MockResponse.FromString,
+            options,
+            channel_credentials,
+            insecure,
+            call_credentials,
+            compression,
+            wait_for_ready,
+            timeout,
+            metadata,
+        )
+
+    @staticmethod
+    def ExecuteClientServerStream(
+        request_iterator,
+        target,
+        options=(),
+        channel_credentials=None,
+        call_credentials=None,
+        insecure=False,
+        compression=None,
+        wait_for_ready=None,
+        timeout=None,
+        metadata=None,
+    ):
+        return grpc.experimental.stream_stream(
+            request_iterator,
+            target,
+            '/bentoml.MockService/ExecuteClientServerStream',
+            mock__service__pb2.MockRequest.SerializeToString,
+            mock__service__pb2.MockResponse.FromString,
+            options,
+            channel_credentials,
+            insecure,
+            call_credentials,
+            compression,
+            wait_for_ready,
+            timeout,
+            metadata,
+        )


Code review comments:
diff_hunk: @@ -143,6 +166,7 @@ def start_yatai_service_grpc_server(
         f'''* Web UI: {f"running on http://127.0.0.1:{ui_port}/{base_url}"
         if (with_ui and base_url!=".")
         else f"running on http://127.0.0.1:{ui_port}" if with_ui else "off"}\n'''
+        f'* Prometheus running on 127.0.0.1:{prometheus_port}\n'
path: bentoml/yatai/yatai_service.py
text: @user1:
@author why is this running on a separate port?  Is there a way to expose it on the web UI server? e.g. `http://127.0.0.1:{ui_port}/metrics`

diff_hunk: @@ -8,68 +8,92 @@ fi
 
 PROTO_PATH=$BENTOML_REPO/protos
 PY_OUT_PATH=$BENTOML_REPO/bentoml/yatai/proto
+# test YataiService Interceptor calls
+PROTO_TEST_PATH=$PROTO_PATH/tests
+PY_TEST_OUT_PATH=$BENTOML_REPO/tests/yatai/proto
 
 echo "Cleaning up existing proto generated py code.."
-rm -rf "$PY_OUT_PATH"
-mkdir -p "$PY_OUT_PATH"
-touch "$PY_OUT_PATH"/__init__.py
+rm -rf "$PY_OUT_PATH" "$PY_TEST_OUT_PATH"
+mkdir -p "$PY_OUT_PATH" "$PY_TEST_OUT_PATH"
+touch "$PY_OUT_PATH"/__init__.py "$PY_TEST_OUT_PATH"/__init__.py
 
 echo "Generate proto message code.."
-find "$PROTO_PATH"/ -name '*.proto' | while read -r protofile; do
+find "$PROTO_PATH"/ -name '*.proto' -not -path "${PROTO_TEST_PATH}/*" | while read -r protofile; do
 python -m grpc_tools.protoc \
   -I"$PROTO_PATH" \
   --python_out="$PY_OUT_PATH" \
   "$protofile"
 done
 
+echo "Generate grpc test service code.."
+python -m grpc_tools.protoc \
+  -I"$PROTO_TEST_PATH" \
+  --python_out="$PY_TEST_OUT_PATH" \
+  --grpc_python_out="$PY_TEST_OUT_PATH" \
+  "$PROTO_TEST_PATH"/mock_service.proto
+
 echo "Generate grpc service code.."
 python -m grpc_tools.protoc \
   -I"$PROTO_PATH" \
   --python_out="$PY_OUT_PATH" \
   --grpc_python_out="$PY_OUT_PATH" \
   "$PROTO_PATH"/yatai_service.proto
 
+fix_grpc_service_code(){
+  PKGNAME=$1
+  OUT_DIR=$2
 
-echo "Fix imports in generated GRPC service code.."
-find "$PY_OUT_PATH"/ -name '*_pb2*.py' | while read -r pyfile; do
-sed -i'.old' 's/^import \([^ ]*\)_pb2 \(.*\)$/import bentoml.yatai.proto.\1_pb2 \2/' "$pyfile"
-sed -i'.old' 's/^from \([^ ]*\) import \([^ ]*\)_pb2\(.*\)$/from bentoml.yatai.proto.\1 import \2_pb2\3/' "$pyfile"
-# Fix google.protobuf package imports
-sed -i'.old' 's/^import bentoml.yatai.proto.google.\([^ ]*\)_pb2 as \([^ ]*\)$/import google.\1_pb2 as \2/' "$pyfile"
-sed -i'.old' 's/^from bentoml.yatai.proto.google.\([^ ]*\) import \([^ ]*\)_pb2 as \([^ ]*\)$/from google.\1 import \2_pb2 as \3/' "$pyfile"
-rm "$pyfile".old
-done
+  echo "Fix imports in generated GRPC service code.."
+  find "$OUT_DIR" -name '*_pb2*.py' | while read -r pyfile; do
+  sed -i'.old' "s/^import \([^ ]*\)_pb2 \(.*\)$/import $PKGNAME.yatai.proto.\1_pb2 \2/" "$pyfile"
+  sed -i'.old' "s/^from \([^ ]*\) import \([^ ]*\)_pb2\(.*\)$/from $PKGNAME.yatai.proto.\1 import \2_pb2\3/" "$pyfile"
+  # Fix google.protobuf package imports
+  sed -i'.old' "s/^import $PKGNAME.yatai.proto.google.\([^ ]*\)_pb2 as \([^ ]*\)$/import google.\1_pb2 as \2/" "$pyfile"
+  sed -i'.old' "s/^from $PKGNAME.yatai.proto.google.\([^ ]*\) import \([^ ]*\)_pb2 as \([^ ]*\)$/from google.\1 import \2_pb2 as \3/" "$pyfile"
+  rm "$pyfile".old
+  done
+}
 
 
-PKG_PATH=$PY_OUT_PATH
-PKGS=$(find "$PKG_PATH" -type d)
-for PKG in $PKGS; do
-    SUBPACKAGES=$(find "$PKG" -maxdepth 1 -type d | grep -E -v "${PKG}$" | sort)
-    MODULES=$(find "$PKG" -maxdepth 1 -iname "*.py" | grep -v "__init__.py" | sort)
-    MODULE_COUNT=$(echo "$MODULES" | wc -w)
-    PKG_INIT="${PKG}/__init__.py"
-    echo "Writing Python package exports"
-    echo "------------------------------"
-    echo "PKG: ${PKG} (${PKG_INIT})"
-    echo "SUBPACKAGES: ${SUBPACKAGES}"
-    echo "FOUND MODULES: $MODULE_COUNT"
-    # echo "MODULES: ${MODULES}"
-    echo ""
-    echo "__all__ = [" > "$PKG_INIT"
-    for MODULE in $MODULES; do
-        FILENAME=$(basename "$MODULE" .py)
-        echo "    \"${FILENAME}\"," >> "$PKG_INIT"
-    done
-    for SUBPKG in $SUBPACKAGES; do
-        SUBPKGNAME=$(basename "$SUBPKG")
-        echo "    \"${SUBPKGNAME}\"," >> "$PKG_INIT"
-    done
-    echo "]" >> "$PKG_INIT"
+fix_grpc_service_code "bentoml" "$PY_OUT_PATH"
+fix_grpc_service_code "tests" "$PY_TEST_OUT_PATH"
+
+
+edit_init() {
+  PKG_PATH=$1
+
+  PKGS=$(find "$PKG_PATH" -type d)
+  for PKG in $PKGS; do
+      SUBPACKAGES=$(find "$PKG" -maxdepth 1 -type d | grep -E -v "${PKG}$" | sort)
+      MODULES=$(find "$PKG" -maxdepth 1 -iname "*.py" | grep -v "__init__.py" | sort)
+      MODULE_COUNT=$(echo "$MODULES" | wc -w)
+      PKG_INIT="${PKG}/__init__.py"
+      echo "Writing Python package exports"
+      echo "------------------------------"
+      echo "PKG: ${PKG} (${PKG_INIT})"
+      echo "SUBPACKAGES: ${SUBPACKAGES}"
+      echo "FOUND MODULES: $MODULE_COUNT"
+      # echo "MODULES: ${MODULES}"
+      echo ""
+      echo "__all__ = [" > "$PKG_INIT"
+      for MODULE in $MODULES; do
+          FILENAME=$(basename "$MODULE" .py)
+          echo "    \"${FILENAME}\"," >> "$PKG_INIT"
+      done
+      for SUBPKG in $SUBPACKAGES; do
+          SUBPKGNAME=$(basename "$SUBPKG")
+          echo "    \"${SUBPKGNAME}\"," >> "$PKG_INIT"
+      done
+      echo "]" >> "$PKG_INIT"
   done
 
-echo "Done"
+  echo "Done"
+}
 
+edit_init "$PY_OUT_PATH"
+edit_init "$PY_TEST_OUT_PATH"
 
+# TODO: if run with generate-docker then we don't have to print this warning
path: protos/generate.sh
text: @user1:
nit: remove todo, that's seems nice but should be fine

@author:
noted