Package: kserve
KServe Python SDKPython SDK for KServe Server and Client.InstallationKServe Python SDK can be installed bypiporpoetry.pip installpipinstallkserveTo install Kserve with storage supportpipinstallkserve[storage]PoetryInstall viaPoetry.makedev_installTo install Kserve with storage supportpoetryinstall-Estorageorpoetryinstall--extras"storage"KServe Python ServerKServe's python server libraries implement a standardized library that is extended by model serving frameworks such as Scikit Learn, XGBoost and PyTorch. It encapsulates data plane API definitions and storage retrieval for models.It provides many functionalities, including among others:Registering a model and starting the serverPrediction HandlerPre/Post Processing HandlerLiveness HandlerReadiness HandlersIt supports the following storage providers:Google Cloud Storage with a prefix: "gs://"By default, it usesGOOGLE_APPLICATION_CREDENTIALSenvironment variable for user authentication.IfGOOGLE_APPLICATION_CREDENTIALSis not provided, anonymous client will be used to download the artifacts.S3 Compatible Object Storage with a prefix "s3://"By default, it usesS3_ENDPOINT,AWS_ACCESS_KEY_ID, andAWS_SECRET_ACCESS_KEYenvironment variables for user authentication.Azure Blob Storage with the format: "https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}"By default, it uses anonymous client to download the artifacts.For e.g.https://kfserving.blob.core.windows.net/triton/simple_string/Local filesystem either without any prefix or with a prefix "file://". For example:Absolute path:/absolute/pathorfile:///absolute/pathRelative path:relative/pathorfile://relative/pathFor local filesystem, we recommended to use relative path without any prefix.Persistent Volume Claim (PVC) with the format "pvc://{$pvcname}/[path]".Thepvcnameis the name of the PVC that contains the model.The[path]is the relative path to the model on the PVC.For e.g.pvc://mypvcname/model/path/on/pvcGeneric URI, over eitherHTTP, prefixed withhttp://orHTTPS, prefixed withhttps://. For example:https://<some_url>.com/model.joblibhttp://<some_url>.com/model.joblibMetricsFor latency metrics, send a request to/metrics. Prometheus latency histograms are emitted for each of the steps (pre/postprocessing, explain, predict).
Additionally, the latencies of each step are logged per request.Metric NameDescriptionTyperequest_preprocess_secondspre-processing request latencyHistogramrequest_explain_secondsexplain request latencyHistogramrequest_predict_secondsprediction request latencyHistogramrequest_postprocess_secondspre-processing request latencyHistogramKServe ClientGetting StartedKServe's python client interacts with KServe control plane APIs for executing operations on a remote KServe cluster, such as creating, patching and deleting of a InferenceService instance. See theSample for Python SDK Clientto get started.Documentation for Client APIPlease reviewKServe Client APIdocs.Documentation For ModelsKnativeAddressableKnativeConditionKnativeURLKnativeVolatileTimeNetUrlUserinfoV1alpha1InferenceGraphV1alpha1InferenceGraphListV1alpha1InferenceGraphSpecV1alpha1InferenceGraphStatusV1alpha1InferenceRouterV1alpha1InferenceStepV1alpha1InferenceTargetV1beta1AlibiExplainerSpecV1beta1BatcherV1beta1ComponentExtensionSpecV1beta1ComponentStatusSpecV1beta1CustomExplainerV1beta1CustomPredictorV1beta1CustomTransformerV1beta1ExplainerConfigV1beta1ExplainerSpecV1beta1ExplainersConfigV1beta1InferenceServiceV1beta1InferenceServiceListV1beta1InferenceServiceSpecV1beta1InferenceServiceStatusV1beta1InferenceServicesConfigV1beta1IngressConfigV1beta1LoggerSpecV1beta1ModelSpecV1beta1ONNXRuntimeSpecV1beta1PodSpecV1beta1PredictorConfigV1beta1PredictorExtensionSpecV1beta1PredictorSpecV1beta1PredictorsConfigV1beta1SKLearnSpecV1beta1TFServingSpecV1beta1TorchServeSpecV1beta1TrainedModelV1beta1TrainedModelListV1beta1TrainedModelSpecV1beta1TrainedModelStatusV1beta1TransformerConfigV1beta1TransformerSpecV1beta1TransformersConfigV1beta1TritonSpecV1beta1XGBoostSpec