Repository: deepset-ai/haystack
Pull request: feat: add health check endpoint to rest api
### Related Issues
- fixes  #3165

### Proposed Changes:
One FastAPI /health endpoint has been added. This endpoint's purpose is to allow faster diagnoses in the REST API environment and to use advanced monitoring tools to check the API status.

Users can set up alerts into monitoring tools to tell when the CPU, RAM, or GPU system usage is high or when the API uses too much of the available CPU and RAM.

### How did you test it?
`rest_api tests` have run, and a new test is included to check if the response is similar to the expected one.
https://github.com/danielbichuetti/haystack/actions/runs/2995987602

### Notes for the reviewer
Integration with advanced monitoring tools makes this endpoint a real plus for enterprise applications.

### Checklist
- [x] I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- [x] I have updated the related issue with new insights and changes
- [x] I added tests that demonstrate the correct behavior of the change
- [x] I've used the [conventional commit convention](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title
- [x] I documented my code
- [x] I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issue


Problem statement:
REST API Health endpoint
**Is your feature request related to a problem? Please describe.**
Today, when using the REST API deployed on a container or any other environment, there is no health endpoint for extensive health monitoring of the system.

Only /initialized provides some information regarding the system, but it misses some basic information about it.
There is also the endpoint that returns the haystack version.

**Describe the solution you'd like**
A health endpoint that can provide detailed information about the system. This helps with faster diagnostics and enables the use of advanced monitoring tools.

**Describe alternatives you've considered**
I would like to implement this. Some of the response fields I already have on a local branch are:

- Haystack version
- CPU usage over time
- Memory usage
- Disk usage
- GPU detailed utilization using the latest pynvml module
- System uptime

Output example:

```JSON
{"version":"1.8.0","cpu":{"avg_1m":"58.18%","avg_5m":"53.31%","avg_15m":"47.57%"},"memory":{"total":"15725.12MB","used":"11283.54MB","percent":"74.00%"},"gpu":[{"index":0,"usage":{"memory_total":"2048.00MB","memory_used":"906.19MB","memory_percent":"44.25%","kernel_usage":"14.00%"}}]}
```

Hints from issue discussion:
@masci I noticed you have been working on the REST API, what do you think about this endpoint?

@ZanSara You always have interesting points. Any opinions regarding it?

Commit message: docs: manual black run

Patch to review:
diff --git a/docs/_src/api/openapi/openapi-1.8.1rc0.json b/docs/_src/api/openapi/openapi-1.8.1rc0.json
index 4acf2a9c9d..ea79d2dd19 100644
--- a/docs/_src/api/openapi/openapi-1.8.1rc0.json
+++ b/docs/_src/api/openapi/openapi-1.8.1rc0.json
@@ -398,6 +398,28 @@
                     }
                 }
             }
+        },
+        "/health": {
+            "get": {
+                "tags": [
+                    "health"
+                ],
+                "summary": "Get Health Status",
+                "description": "This endpoint allows external systems to monitor the health of the Haystack REST API.",
+                "operationId": "get_health_status",
+                "responses": {
+                    "200": {
+                        "description": "Successful Response",
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "$ref": "#/components/schemas/HealthResponse"
+                                }
+                            }
+                        }
+                    }
+                }
+            }
         }
     },
     "components": {
@@ -511,6 +533,32 @@
                     }
                 }
             },
+            "CPUUsage": {
+                "title": "CPUUsage",
+                "required": [
+                    "avg_1m",
+                    "avg_5m",
+                    "avg_15m"
+                ],
+                "type": "object",
+                "properties": {
+                    "avg_1m": {
+                        "title": "Avg 1M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 1 minute"
+                    },
+                    "avg_5m": {
+                        "title": "Avg 5M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 5 minutes"
+                    },
+                    "avg_15m": {
+                        "title": "Avg 15M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 15 minutes"
+                    }
+                }
+            },
             "CreateLabelSerialized": {
                 "title": "CreateLabelSerialized",
                 "required": [
@@ -693,6 +741,62 @@
                 },
                 "additionalProperties": false
             },
+            "GPUInfo": {
+                "title": "GPUInfo",
+                "required": [
+                    "index",
+                    "usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "index": {
+                        "title": "Index",
+                        "type": "integer",
+                        "description": "GPU index"
+                    },
+                    "usage": {
+                        "title": "Usage",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/GPUUsage"
+                            }
+                        ],
+                        "description": "GPU usage details"
+                    }
+                }
+            },
+            "GPUUsage": {
+                "title": "GPUUsage",
+                "required": [
+                    "memory_total",
+                    "memory_used",
+                    "memory_percent",
+                    "kernel_usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "memory_total": {
+                        "title": "Memory Total",
+                        "type": "string",
+                        "description": "Total GPU memory in megabytes"
+                    },
+                    "memory_used": {
+                        "title": "Memory Used",
+                        "type": "string",
+                        "description": "Used GPU memory in megabytes"
+                    },
+                    "memory_percent": {
+                        "title": "Memory Percent",
+                        "type": "string",
+                        "description": "Percentage of used GPU memory"
+                    },
+                    "kernel_usage": {
+                        "title": "Kernel Usage",
+                        "type": "string",
+                        "description": "GPU kernel usage in percent"
+                    }
+                }
+            },
             "HTTPValidationError": {
                 "title": "HTTPValidationError",
                 "type": "object",
@@ -706,6 +810,60 @@
                     }
                 }
             },
+            "HaystackInfo": {
+                "title": "HaystackInfo",
+                "required": [
+                    "version",
+                    "cpu_usage",
+                    "memory_usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "version": {
+                        "title": "Version",
+                        "type": "string",
+                        "description": "Haystack version"
+                    },
+                    "cpu_usage": {
+                        "title": "Cpu Usage",
+                        "type": "string",
+                        "description": "REST API CPU usage details"
+                    },
+                    "memory_usage": {
+                        "title": "Memory Usage",
+                        "type": "string",
+                        "description": "REST API memory usage details"
+                    }
+                }
+            },
+            "HealthResponse": {
+                "title": "HealthResponse",
+                "required": [
+                    "haystack",
+                    "system"
+                ],
+                "type": "object",
+                "properties": {
+                    "haystack": {
+                        "title": "Haystack",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/HaystackInfo"
+                            }
+                        ],
+                        "description": "Haystack REST API health details"
+                    },
+                    "system": {
+                        "title": "System",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/SystemInfo"
+                            }
+                        ],
+                        "description": "System health details"
+                    }
+                }
+            },
             "Label": {
                 "title": "Label",
                 "required": [
@@ -774,6 +932,32 @@
                     }
                 }
             },
+            "MemoryUsage": {
+                "title": "MemoryUsage",
+                "required": [
+                    "total",
+                    "used",
+                    "percent"
+                ],
+                "type": "object",
+                "properties": {
+                    "total": {
+                        "title": "Total",
+                        "type": "string",
+                        "description": "Total memory in megabytes"
+                    },
+                    "used": {
+                        "title": "Used",
+                        "type": "string",
+                        "description": "Used memory in megabytes"
+                    },
+                    "percent": {
+                        "title": "Percent",
+                        "type": "string",
+                        "description": "Percentage of used memory"
+                    }
+                }
+            },
             "QueryRequest": {
                 "title": "QueryRequest",
                 "required": [
@@ -848,6 +1032,42 @@
                     }
                 }
             },
+            "SystemInfo": {
+                "title": "SystemInfo",
+                "required": [
+                    "cpu",
+                    "memory"
+                ],
+                "type": "object",
+                "properties": {
+                    "cpu": {
+                        "title": "Cpu",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/CPUUsage"
+                            }
+                        ],
+                        "description": "System CPU usage details"
+                    },
+                    "memory": {
+                        "title": "Memory",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/MemoryUsage"
+                            }
+                        ],
+                        "description": "System memory usage details"
+                    },
+                    "gpu_info": {
+                        "title": "Gpu Info",
+                        "type": "array",
+                        "items": {
+                            "$ref": "#/components/schemas/GPUInfo"
+                        },
+                        "description": "System GPU usage details"
+                    }
+                }
+            },
             "ValidationError": {
                 "title": "ValidationError",
                 "required": [
diff --git a/docs/_src/api/openapi/openapi.json b/docs/_src/api/openapi/openapi.json
index 4acf2a9c9d..ea79d2dd19 100644
--- a/docs/_src/api/openapi/openapi.json
+++ b/docs/_src/api/openapi/openapi.json
@@ -398,6 +398,28 @@
                     }
                 }
             }
+        },
+        "/health": {
+            "get": {
+                "tags": [
+                    "health"
+                ],
+                "summary": "Get Health Status",
+                "description": "This endpoint allows external systems to monitor the health of the Haystack REST API.",
+                "operationId": "get_health_status",
+                "responses": {
+                    "200": {
+                        "description": "Successful Response",
+                        "content": {
+                            "application/json": {
+                                "schema": {
+                                    "$ref": "#/components/schemas/HealthResponse"
+                                }
+                            }
+                        }
+                    }
+                }
+            }
         }
     },
     "components": {
@@ -511,6 +533,32 @@
                     }
                 }
             },
+            "CPUUsage": {
+                "title": "CPUUsage",
+                "required": [
+                    "avg_1m",
+                    "avg_5m",
+                    "avg_15m"
+                ],
+                "type": "object",
+                "properties": {
+                    "avg_1m": {
+                        "title": "Avg 1M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 1 minute"
+                    },
+                    "avg_5m": {
+                        "title": "Avg 5M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 5 minutes"
+                    },
+                    "avg_15m": {
+                        "title": "Avg 15M",
+                        "type": "string",
+                        "description": "Average CPU usage over the last 15 minutes"
+                    }
+                }
+            },
             "CreateLabelSerialized": {
                 "title": "CreateLabelSerialized",
                 "required": [
@@ -693,6 +741,62 @@
                 },
                 "additionalProperties": false
             },
+            "GPUInfo": {
+                "title": "GPUInfo",
+                "required": [
+                    "index",
+                    "usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "index": {
+                        "title": "Index",
+                        "type": "integer",
+                        "description": "GPU index"
+                    },
+                    "usage": {
+                        "title": "Usage",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/GPUUsage"
+                            }
+                        ],
+                        "description": "GPU usage details"
+                    }
+                }
+            },
+            "GPUUsage": {
+                "title": "GPUUsage",
+                "required": [
+                    "memory_total",
+                    "memory_used",
+                    "memory_percent",
+                    "kernel_usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "memory_total": {
+                        "title": "Memory Total",
+                        "type": "string",
+                        "description": "Total GPU memory in megabytes"
+                    },
+                    "memory_used": {
+                        "title": "Memory Used",
+                        "type": "string",
+                        "description": "Used GPU memory in megabytes"
+                    },
+                    "memory_percent": {
+                        "title": "Memory Percent",
+                        "type": "string",
+                        "description": "Percentage of used GPU memory"
+                    },
+                    "kernel_usage": {
+                        "title": "Kernel Usage",
+                        "type": "string",
+                        "description": "GPU kernel usage in percent"
+                    }
+                }
+            },
             "HTTPValidationError": {
                 "title": "HTTPValidationError",
                 "type": "object",
@@ -706,6 +810,60 @@
                     }
                 }
             },
+            "HaystackInfo": {
+                "title": "HaystackInfo",
+                "required": [
+                    "version",
+                    "cpu_usage",
+                    "memory_usage"
+                ],
+                "type": "object",
+                "properties": {
+                    "version": {
+                        "title": "Version",
+                        "type": "string",
+                        "description": "Haystack version"
+                    },
+                    "cpu_usage": {
+                        "title": "Cpu Usage",
+                        "type": "string",
+                        "description": "REST API CPU usage details"
+                    },
+                    "memory_usage": {
+                        "title": "Memory Usage",
+                        "type": "string",
+                        "description": "REST API memory usage details"
+                    }
+                }
+            },
+            "HealthResponse": {
+                "title": "HealthResponse",
+                "required": [
+                    "haystack",
+                    "system"
+                ],
+                "type": "object",
+                "properties": {
+                    "haystack": {
+                        "title": "Haystack",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/HaystackInfo"
+                            }
+                        ],
+                        "description": "Haystack REST API health details"
+                    },
+                    "system": {
+                        "title": "System",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/SystemInfo"
+                            }
+                        ],
+                        "description": "System health details"
+                    }
+                }
+            },
             "Label": {
                 "title": "Label",
                 "required": [
@@ -774,6 +932,32 @@
                     }
                 }
             },
+            "MemoryUsage": {
+                "title": "MemoryUsage",
+                "required": [
+                    "total",
+                    "used",
+                    "percent"
+                ],
+                "type": "object",
+                "properties": {
+                    "total": {
+                        "title": "Total",
+                        "type": "string",
+                        "description": "Total memory in megabytes"
+                    },
+                    "used": {
+                        "title": "Used",
+                        "type": "string",
+                        "description": "Used memory in megabytes"
+                    },
+                    "percent": {
+                        "title": "Percent",
+                        "type": "string",
+                        "description": "Percentage of used memory"
+                    }
+                }
+            },
             "QueryRequest": {
                 "title": "QueryRequest",
                 "required": [
@@ -848,6 +1032,42 @@
                     }
                 }
             },
+            "SystemInfo": {
+                "title": "SystemInfo",
+                "required": [
+                    "cpu",
+                    "memory"
+                ],
+                "type": "object",
+                "properties": {
+                    "cpu": {
+                        "title": "Cpu",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/CPUUsage"
+                            }
+                        ],
+                        "description": "System CPU usage details"
+                    },
+                    "memory": {
+                        "title": "Memory",
+                        "allOf": [
+                            {
+                                "$ref": "#/components/schemas/MemoryUsage"
+                            }
+                        ],
+                        "description": "System memory usage details"
+                    },
+                    "gpu_info": {
+                        "title": "Gpu Info",
+                        "type": "array",
+                        "items": {
+                            "$ref": "#/components/schemas/GPUInfo"
+                        },
+                        "description": "System GPU usage details"
+                    }
+                }
+            },
             "ValidationError": {
                 "title": "ValidationError",
                 "required": [
diff --git a/rest_api/pyproject.toml b/rest_api/pyproject.toml
index b4e9895e93..df7a38e88c 100644
--- a/rest_api/pyproject.toml
+++ b/rest_api/pyproject.toml
@@ -30,6 +30,8 @@ dependencies = [
     "uvicorn<1",
     "gunicorn<21",
     "python-multipart<1",  # optional FastAPI dependency for form data
+    "pynvml",
+    "psutil"
 ]
 dynamic = ["version"]
 
@@ -69,3 +71,7 @@ exclude_lines = [
   "if __name__ == .__main__.:",
   "if TYPE_CHECKING:",
 ]
+
+[tool.black]
+line-length = 120
+skip_magic_trailing_comma = true  # For compatibility with pydoc>=4.6, check if still needed.
diff --git a/rest_api/rest_api/controller/health.py b/rest_api/rest_api/controller/health.py
new file mode 100644
index 0000000000..fda2f8158a
--- /dev/null
+++ b/rest_api/rest_api/controller/health.py
@@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
+
+
+class GPUUsage(BaseModel):
+    memory_total: str = Field(..., description="Total GPU memory in megabytes")
+    memory_used: str = Field(..., description="Used GPU memory in megabytes")
+    memory_percent: str = Field(..., description="Percentage of used GPU memory")
+    kernel_usage: str = Field(..., description="GPU kernel usage in percent")
+
+
+class GPUInfo(BaseModel):
+    index: int = Field(..., description="GPU index")
+    usage: GPUUsage = Field(..., description="GPU usage details")
+
+
+class HaystackInfo(BaseModel):
+    version: str = Field(..., description="Haystack version")
+    cpu_usage: str = Field(..., description="REST API CPU usage details")
+    memory_usage: str = Field(..., description="REST API memory usage details")
+
+
+class SystemInfo(BaseModel):
+    cpu: CPUUsage = Field(..., description="System CPU usage details")
+    memory: MemoryUsage = Field(..., description="System memory usage details")
+    gpu_info: List[GPUInfo] = Field(default_factory=list, description="System GPU usage details")
+
+
+class HealthResponse(BaseModel):
+    haystack: HaystackInfo = Field(..., description="Haystack REST API health details")
+    system: SystemInfo = Field(..., description="System health details")
+
+
+@router.get("/health", response_model=HealthResponse, status_code=200)
+def get_health_status():
+    """
+    This endpoint allows external systems to monitor the health of the Haystack REST API.
+    """
+
+    cpu_1m, cpu_5m, cpu_15m = psutil.getloadavg()
+    cpu_count = float(os.cpu_count() or 0)
+    cpu_1m = (cpu_1m / cpu_count) * 100
+    cpu_5m = (cpu_5m / cpu_count) * 100
+    cpu_15m = (cpu_15m / cpu_count) * 100
+    cpu_current = CPUUsage(avg_1m=f"{cpu_1m:.2f}%", avg_5m=f"{cpu_5m:.2f}%", avg_15m=f"{cpu_15m:.2f}%")
+
+    memory_data = psutil.virtual_memory()
+    memory_total = memory_data.total / 1024 / 1024
+    memory_used = memory_data.used / 1024 / 1024
+    memory_percent = memory_data.percent
+    memory_usage = MemoryUsage(
+        total=f"{memory_total:.2f}MB", used=f"{memory_used:.2f}MB", percent=f"{memory_percent:.2f}%"
+    )
+
+    gpus: List[GPUInfo] = []
+
+    try:
+        pynvml.nvmlInit()
+        gpu_count = pynvml.nvmlDeviceGetCount()
+        gpu_mem_total = ""
+        gpu_mem_used = ""
+        gpu_mem_free = ""
+        for i in range(gpu_count):
+            handle = pynvml.nvmlDeviceGetHandleByIndex(i)
+            info = pynvml.nvmlDeviceGetMemoryInfo(handle)
+            gpu_mem_total = float(info.total) / 1024 / 1024
+            gpu_mem_used = float(info.used) / 1024 / 1024
+            gpu_info = GPUInfo(
+                index=i,
+                usage=GPUUsage(
+                    memory_total=f"{gpu_mem_total:.2f}MB",
+                    memory_used=f"{gpu_mem_used:.2f}MB",
+                    memory_percent=f"{(gpu_mem_used/gpu_mem_total)*100:.2f}%",
+                    kernel_usage=f"{pynvml.nvmlDeviceGetUtilizationRates(handle).gpu:.2f}%",
+                ),
+            )
+            gpus.append(gpu_info)
+    except pynvml.NVMLError:
+        logger.warning("No NVIDIA GPU found.")
+
+    p_cpu_usage = 0
+    p_memory_usage = 0
+    p = psutil.Process()
+    with p.oneshot():
+        p_cpu_usage = p.cpu_percent() / cpu_count
+        p_memory_usage = p.memory_percent()
+
+    return HealthResponse(
+        haystack=HaystackInfo(
+            version=haystack.__version__, cpu_usage=f"{p_cpu_usage:.2f}%", memory_usage=f"{p_memory_usage:.2f}%"
+        ),
+        system=SystemInfo(cpu=cpu_current, memory=memory_usage, gpu_info=gpus),
+    )
diff --git a/rest_api/rest_api/utils.py b/rest_api/rest_api/utils.py
index cc496ca290..174913b5f4 100644
--- a/rest_api/rest_api/utils.py
+++ b/rest_api/rest_api/utils.py
@@ -25,13 +25,14 @@ def get_app() -> FastAPI:
     app = FastAPI(title="Haystack REST API", debug=True, version=haystack_version, root_path=ROOT_PATH)
 
     # Creates the router for the API calls
-    from rest_api.controller import file_upload, search, feedback, document
+    from rest_api.controller import file_upload, search, feedback, document, health
 
     router = APIRouter()
     router.include_router(search.router, tags=["search"])
     router.include_router(feedback.router, tags=["feedback"])
     router.include_router(file_upload.router, tags=["file-upload"])
     router.include_router(document.router, tags=["document"])
+    router.include_router(health.router, tags=["health"])
 
     # This middleware enables allow all cross-domain requests to the API from a browser. For production
     # deployments, it could be made more restrictive.
diff --git a/rest_api/test/test_rest_api.py b/rest_api/test/test_rest_api.py
index f358e41d99..07c6bd58cf 100644
--- a/rest_api/test/test_rest_api.py
+++ b/rest_api/test/test_rest_api.py
@@ -11,6 +11,7 @@
 import pytest
 from fastapi.testclient import TestClient
 from haystack import Document, Answer
+import haystack
 from haystack.nodes import BaseReader, BaseRetriever
 from haystack.document_stores import BaseDocumentStore
 from haystack.schema import Label
@@ -499,3 +500,9 @@ def test_get_feedback_malformed_query(client, feedback):
     feedback["unexpected_field"] = "misplaced-value"
     response = client.post(url="/feedback", json=feedback)
     assert response.status_code == 422
+
+
+def test_get_health_check(client):
+    response = client.get(url="/health")
+    assert response.status_code == 200
+    assert response.json()["haystack"]["version"] == haystack.__version__


Code review comments:
diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
path: rest_api/rest_api/controller/health.py
text: @user1:
On a second thought, I think using `str` here simply to be able to add the `%` sign makes it unnecessary hard to hook this endpoint to a monitoring tool. I understand this health endpoint is more for human consumption than for machines, but sticking to fractions of 1 instead of using formatted percentages has the nice side effect of making it easier to dump the data in some nice plot if there's the need.

What do you think? Are you ok changing these (and most of the other fields below) back to `float`?

@author:
Float is easier for machine readability. The consumer won't need to parse it. 

And the original thoughts weren't really for human. You are right. Let's change it.

diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
+
+
+class GPUUsage(BaseModel):
+    memory_total: str = Field(..., description="Total GPU memory in megabytes")
+    memory_used: str = Field(..., description="Used GPU memory in megabytes")
+    memory_percent: str = Field(..., description="Percentage of used GPU memory")
+    kernel_usage: str = Field(..., description="GPU kernel usage in percent")
+
+
+class GPUInfo(BaseModel):
+    index: int = Field(..., description="GPU index")
+    usage: GPUUsage = Field(..., description="GPU usage details")
+
+
+class HaystackInfo(BaseModel):
+    version: str = Field(..., description="Haystack version")
+    cpu_usage: str = Field(..., description="REST API CPU usage details")
+    memory_usage: str = Field(..., description="REST API memory usage details")
+
+
+class SystemInfo(BaseModel):
+    cpu: CPUUsage = Field(..., description="System CPU usage details")
+    memory: MemoryUsage = Field(..., description="System memory usage details")
+    gpu_info: List[GPUInfo] = Field(default_factory=list, description="System GPU usage details")
+
+
+class HealthResponse(BaseModel):
+    haystack: HaystackInfo = Field(..., description="Haystack REST API health details")
+    system: SystemInfo = Field(..., description="System health details")
+
+
+@router.get("/health", response_model=HealthResponse, status_code=200)
+def get_health_status():
+    """
+    This endpoint allows external systems to monitor the health of the Haystack REST API.
+    """
+
+    cpu_1m, cpu_5m, cpu_15m = psutil.getloadavg()
+    cpu_count = float(os.cpu_count() or 0)
+    cpu_1m = (cpu_1m / cpu_count) * 100
+    cpu_5m = (cpu_5m / cpu_count) * 100
+    cpu_15m = (cpu_15m / cpu_count) * 100
+    cpu_current = CPUUsage(avg_1m=f"{cpu_1m:.2f}%", avg_5m=f"{cpu_5m:.2f}%", avg_15m=f"{cpu_15m:.2f}%")
+
+    memory_data = psutil.virtual_memory()
+    memory_total = memory_data.total / 1024 / 1024
+    memory_used = memory_data.used / 1024 / 1024
+    memory_percent = memory_data.percent
+    memory_usage = MemoryUsage(
+        total=f"{memory_total:.2f}MB", used=f"{memory_used:.2f}MB", percent=f"{memory_percent:.2f}%"
+    )
+
+    gpus: List[GPUInfo] = []
+
+    try:
+        pynvml.nvmlInit()
+        gpu_count = pynvml.nvmlDeviceGetCount()
+        gpu_mem_total = ""
+        gpu_mem_used = ""
+        gpu_mem_free = ""
+        for i in range(gpu_count):
+            handle = pynvml.nvmlDeviceGetHandleByIndex(i)
+            info = pynvml.nvmlDeviceGetMemoryInfo(handle)
+            gpu_mem_total = float(info.total) / 1024 / 1024
+            gpu_mem_used = float(info.used) / 1024 / 1024
+            gpu_info = GPUInfo(
+                index=i,
+                usage=GPUUsage(
+                    memory_total=f"{gpu_mem_total:.2f}MB",
+                    memory_used=f"{gpu_mem_used:.2f}MB",
+                    memory_percent=f"{(gpu_mem_used/gpu_mem_total)*100:.2f}%",
+                    kernel_usage=f"{pynvml.nvmlDeviceGetUtilizationRates(handle).gpu:.2f}%",
+                ),
+            )
+            gpus.append(gpu_info)
+    except pynvml.NVMLError:
+        logger.warning("No NVIDIA GPU found.")
+
+    p_cpu_usage = 0
+    p_memory_usage = 0
+    p = psutil.Process()
+    with p.oneshot():
+        p_cpu_usage = p.cpu_percent() / cpu_count
+        p_memory_usage = p.memory_percent()
path: rest_api/rest_api/controller/health.py
text: @user1:
I think we can get the detailed CPU and MEM stats, like you have gathered for the system, for the Haystack process as well. Or there is some technical issue with that?

@author:
In the process itself, we don't have total memory, as it is only a system parameter. And the CPU average usage isn't directly available. The application would have to do snapshots and save them in memory. Do you want me to implement this?


@author:
Some system-wide data has been removed as per suggestions.

diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
path: rest_api/rest_api/controller/health.py
text: @user1:
Referring to the comment above: here you're specifying the unit in the description, so it should also be ok to simply return the float.

diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
+
+
+class GPUUsage(BaseModel):
+    memory_total: str = Field(..., description="Total GPU memory in megabytes")
+    memory_used: str = Field(..., description="Used GPU memory in megabytes")
+    memory_percent: str = Field(..., description="Percentage of used GPU memory")
+    kernel_usage: str = Field(..., description="GPU kernel usage in percent")
+
+
+class GPUInfo(BaseModel):
+    index: int = Field(..., description="GPU index")
+    usage: GPUUsage = Field(..., description="GPU usage details")
+
+
+class HaystackInfo(BaseModel):
+    version: str = Field(..., description="Haystack version")
+    cpu_usage: str = Field(..., description="REST API CPU usage details")
+    memory_usage: str = Field(..., description="REST API memory usage details")
+
+
+class SystemInfo(BaseModel):
+    cpu: CPUUsage = Field(..., description="System CPU usage details")
+    memory: MemoryUsage = Field(..., description="System memory usage details")
+    gpu_info: List[GPUInfo] = Field(default_factory=list, description="System GPU usage details")
+
+
+class HealthResponse(BaseModel):
+    haystack: HaystackInfo = Field(..., description="Haystack REST API health details")
+    system: SystemInfo = Field(..., description="System health details")
+
+
+@router.get("/health", response_model=HealthResponse, status_code=200)
+def get_health_status():
+    """
+    This endpoint allows external systems to monitor the health of the Haystack REST API.
+    """
+
+    cpu_1m, cpu_5m, cpu_15m = psutil.getloadavg()
+    cpu_count = float(os.cpu_count() or 0)
+    cpu_1m = (cpu_1m / cpu_count) * 100
+    cpu_5m = (cpu_5m / cpu_count) * 100
+    cpu_15m = (cpu_15m / cpu_count) * 100
+    cpu_current = CPUUsage(avg_1m=f"{cpu_1m:.2f}%", avg_5m=f"{cpu_5m:.2f}%", avg_15m=f"{cpu_15m:.2f}%")
+
+    memory_data = psutil.virtual_memory()
+    memory_total = memory_data.total / 1024 / 1024
+    memory_used = memory_data.used / 1024 / 1024
+    memory_percent = memory_data.percent
+    memory_usage = MemoryUsage(
+        total=f"{memory_total:.2f}MB", used=f"{memory_used:.2f}MB", percent=f"{memory_percent:.2f}%"
+    )
+
+    gpus: List[GPUInfo] = []
+
+    try:
+        pynvml.nvmlInit()
+        gpu_count = pynvml.nvmlDeviceGetCount()
+        gpu_mem_total = ""
+        gpu_mem_used = ""
+        gpu_mem_free = ""
path: rest_api/rest_api/controller/health.py
text: @user1:
I think you don't need these three

```suggestion
```

@author:
There were remnants of other tests. Sorry for not removing it 😢

diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
+
+
+class GPUUsage(BaseModel):
+    memory_total: str = Field(..., description="Total GPU memory in megabytes")
+    memory_used: str = Field(..., description="Used GPU memory in megabytes")
+    memory_percent: str = Field(..., description="Percentage of used GPU memory")
+    kernel_usage: str = Field(..., description="GPU kernel usage in percent")
+
+
+class GPUInfo(BaseModel):
+    index: int = Field(..., description="GPU index")
+    usage: GPUUsage = Field(..., description="GPU usage details")
+
+
+class HaystackInfo(BaseModel):
+    version: str = Field(..., description="Haystack version")
+    cpu_usage: str = Field(..., description="REST API CPU usage details")
+    memory_usage: str = Field(..., description="REST API memory usage details")
+
+
+class SystemInfo(BaseModel):
+    cpu: CPUUsage = Field(..., description="System CPU usage details")
+    memory: MemoryUsage = Field(..., description="System memory usage details")
+    gpu_info: List[GPUInfo] = Field(default_factory=list, description="System GPU usage details")
path: rest_api/rest_api/controller/health.py
text: @user2:
This might be a bit out of scope for the Haystack REST API... I'll ask @user1 about his opinion. If there was a way to get the usage of GPU by the Haystack process (and add it to the `HaystackInfo` object) that would be super-cool tho :slightly_smiling_face: 

@author:
There is a XmlDeviceQuery method on pynvml. It would help with this.

@author:
Do you think that maybe only the available memory? Usually RAM and GPU available are some concerns when doing NLP.

@author:
I implemented the Haystack-specific GPU usage and removed some of the system-wide details. They are restricted now.

diff_hunk: @@ -0,0 +1,126 @@
+from typing import List
+
+import logging
+
+import os
+import pynvml
+import psutil
+
+from pydantic import BaseModel, Field
+
+from fastapi import FastAPI, APIRouter
+
+import haystack
+
+from rest_api.utils import get_app
+from rest_api.config import LOG_LEVEL
+
+logging.getLogger("haystack").setLevel(LOG_LEVEL)
+logger = logging.getLogger("haystack")
+
+
+router = APIRouter()
+app: FastAPI = get_app()
+
+
+class CPUUsage(BaseModel):
+    avg_1m: str = Field(..., description="Average CPU usage over the last 1 minute")
+    avg_5m: str = Field(..., description="Average CPU usage over the last 5 minutes")
+    avg_15m: str = Field(..., description="Average CPU usage over the last 15 minutes")
+
+
+class MemoryUsage(BaseModel):
+    total: str = Field(..., description="Total memory in megabytes")
+    used: str = Field(..., description="Used memory in megabytes")
+    percent: str = Field(..., description="Percentage of used memory")
+
+
+class GPUUsage(BaseModel):
+    memory_total: str = Field(..., description="Total GPU memory in megabytes")
+    memory_used: str = Field(..., description="Used GPU memory in megabytes")
+    memory_percent: str = Field(..., description="Percentage of used GPU memory")
+    kernel_usage: str = Field(..., description="GPU kernel usage in percent")
+
+
+class GPUInfo(BaseModel):
+    index: int = Field(..., description="GPU index")
+    usage: GPUUsage = Field(..., description="GPU usage details")
+
+
+class HaystackInfo(BaseModel):
+    version: str = Field(..., description="Haystack version")
+    cpu_usage: str = Field(..., description="REST API CPU usage details")
+    memory_usage: str = Field(..., description="REST API memory usage details")
+
+
+class SystemInfo(BaseModel):
+    cpu: CPUUsage = Field(..., description="System CPU usage details")
+    memory: MemoryUsage = Field(..., description="System memory usage details")
+    gpu_info: List[GPUInfo] = Field(default_factory=list, description="System GPU usage details")
+
+
+class HealthResponse(BaseModel):
+    haystack: HaystackInfo = Field(..., description="Haystack REST API health details")
+    system: SystemInfo = Field(..., description="System health details")
+
+
+@router.get("/health", response_model=HealthResponse, status_code=200)
+def get_health_status():
+    """
+    This endpoint allows external systems to monitor the health of the Haystack REST API.
+    """
+
+    cpu_1m, cpu_5m, cpu_15m = psutil.getloadavg()
+    cpu_count = float(os.cpu_count() or 0)
path: rest_api/rest_api/controller/health.py
text: @user1:
if `cpu_count()` fails and `cpu_count == 0`, the math below won't work. Let's check for `cpu_count == 0` and manage the resulting situation where there's no CPU info. 

Additionally, I don't think you need to specify `float()`. Division of a float by an int returns a float result.

@author:
I think I have changed this here and set 1 because of the division.