Package: mlservicewrapper-core
InstallingInstall using pippip install mlservicewrapper-coreInstall directly fromthe latest GitHub Release.Install from sourcegit clone https://github.com/ml-service-wrapper/ml-service-wrapper-core.git
cd ml-service-wrapper
pip install .Implementing an ML serviceWrite a class that matches the interface defined byService:importmlservicewrapperimportmlservicewrapper.core.servicesclassSampleService(mlservicewrapper.core.services.Service):asyncdefload(self,ctx:mlservicewrapper.core.contexts.ServiceContext):passasyncdefprocess(self,ctx:mlservicewrapper.core.contexts.ProcessContext):passdefdispose(self):passThe three functions describe the three phases of service lifetime:loadis called when the service is first initializing, and should load models into memory and do necessary pre-work. It makes sense to parse and storeServiceContextparameters in this function, as they won't be accessible later.processwill be called many times for eachload, and is where data should actually be handled. SeeProcessContextfor how to process data.disposemay or may not be called,and is optional, but should be used for cleanup tasks during service shutdown, e.g. disposal of handles or temporary files.ContextsServiceContextAServiceContextobject will be passed to theloadfunction when the service is first initialized. It exposes a single function:get_parameter_value(name: str, required: bool = True, default: str = None)Used to get a parameter from the environment. These parameters may be sourced from:A configuration file (using theparametersproperty)Environment variablesOther, environment-specific key-value storesNote that all parameter values are either typestrorNone. It is the implementation's responsibility to parse string input and handle missing values, potentially with use of thedefaultparameter. Numbers will not be parsed.Service parameters are considered required unlessrequiredis specified asFalse,ora non-Nonevalue is passed as adefault.ProcessContextAProcessContextobject is passed to theprocessfunction, and exposes key details about a particular execution. It has more functions than aServiceContext:get_input_dataframe(name: str, required: bool = True)Returns a PandasDataFrameobject containing the named input dataset.Note that an optional parameterrequiredmay be set toFalsein rare cases when an input dataset is optional.set_output_dataframe(self, name: str, df: pd.DataFrame)Set the named output dataset using an existing PandasDataFrameget_parameter_value(name: str, required: bool = True, default: str = None) -> strReturns execution-specific parameters,not includingthose defined in theServiceContext. To use service-level parameters, store them on the service instance.Process parameters are considered required unlessrequiredis specified asFalse,ora non-Nonevalue is passed as adefault.Heads up:most implementations will not use execution parameters. Consider usingServiceContextparameters instead. It's also advisable to provide sensible default values, either in-code or throughServiceContextparameters.Depending on the deployment environment, input and output datasets may be sourced from:Local CSV files,SQL tables or queries,JSON documents or request bodies, orOther sources...ValidationValidating input and raising appropriate errors helps callers understand usage of the ML service. Some built-in errors may have special behaviors in supporting environments. Use the one that most specifically describes the problem.As best practice, work to validate input datasets and parameters as early as possible. For example, test that all required categorical fields are presentbeforeperforming work to preprocess text ones.ParametersMissingParameterError(name: str, message: str = None)Used internally when a parameter is requested via theget_parameter_valuefunction, but cannot be found on the request. Similarly to theMissingDatasetError, logic is likely best left to therequiredparameter on that function.BadParameterError(name: str, message: str = None)Raise for all other parameter validation errors, e.g. when a string is not parsable.DatasetsMissingDatasetFieldError(dataset_name: str, field_name: str, message: str = None)Used when a required field is missing from an input dataset. For example:DatasetFieldError(dataset_name: str, field_name: str, message: str = None)Used when a dataset fieldispresent, but is otherwise invalid. Use is implementation-specific, but could describe an unparsable number field, a duplicate value in an expected-unique field, or other like input inconsistencies.MissingDatasetError(dataset_name: str, message: str = None)Thrown internally when a call toget_input_dataframeis made when no dataset exists by the name. It is unlikely implementations will benefit from calling this error directly, and should defer to theProcessContextusing therequiredparameter onget_input_dataframe.BadDatasetError(dataset_name: str, message: str = None)Base class for other errors, callable when a dataset does not match the agreed-upon contract.Configuration filesEach service is accompanied by a simple JSON configuration file, that tells the wrapper some basic details about the service.modulePathThe path,relative to the configuration file,where theServiceinheritor is defined.classNameThe name of theServiceinheritor itself. Note that this class must be initializable with no parameters; those should be saved for theloadfunction.serviceInstanceNameIn cases when you choose to instantiate theServiceyourself, the name of the instance exposed by the module.Only used ifclassNameis omitted.parametersAn optional dictionary of configuration-specific key-value pairs, which should be passed via theServiceContextparameters. This is useful when multiple configuration files can be used for the sameService.metaApplication-level metadata, not passed to theService. Useful for managing configurations internally.Note thatmodulePathis required, along witheitherclassNameorserviceInstanceName.Naming ConventionsBy nature of its configurability, external type-checking is not possible.Validationcan help to identify errors, but only at runtime. As consequence, a consistent naming schema is important to reducing unnecessary cycles.Follow simple rules when naming parameters and datasets:Use Pascal Case (e.g.MaxValue)Use only letters and numbersBe concise, but descriptiveBe consistent with casingDebugging a serviceExamples below use the sample service, and consequently require cloning of this repository to run.Write your own debug scriptSee./sample/1_simple/src/debug.pyfor an example.Test end-to-end using a configuration fileCall the debug module directly. The provided configuration file is parsed, just like it would be in a production environment.python-mmlservicewrapper.core.debug\--config"./sample/1_simple/src/config.json"\--input-dir"./sample/1_simple/data/input"\--load-paramsModBy=3ParameterRequired?DescriptionExample--configYesPath to service configuration file--config "./sample/1_simple/src/config.json"--input-dirNoPath to input directory. Input datasets will be looked up within the directory.--input-dir "./sample/1_simple/data/input"--input-pathsNoPath mapping for input datasets. Multiple values can be comma-delimited, and each value takes the form of<dataset name>=<path>. Explicitly-pathed datasets will override any files from--input-dir, if both are provided.--input-paths Data=./sample/1_simple/data/input/Data.csv--output-dirNoPath to output directory. Output datasets will be looked up within the directory.--output-dir "./sample/1_simple/data/output"--output-pathsNoPath mapping for output datasets. Multiple values can be comma-delimited, and each value takes the form of<dataset name>=<path>. Explicitly-pathed datasets will override any files from--output-dir, if both are provided.--output-paths Data=./sample/1_simple/data/output/Data.csv--load-paramsNoValues forServiceContextparameters. Multiple values can be comma-delimited, and each value takes the form of<key>=<value>--load-params ModBy=3--run-paramsNoValues forProcessContextparameters. Multiple values can be comma-delimited, and each value takes the form of<key>=<value>--run-params RuntimeOption=OptionValue--split-dataset-for-perfNoRuns performance analysis against the named dataset. This involves splitting the dataset into one-row chunks and callingprocessagainst each row independently, timing each execution, and reporting statistics. This is meant to help simulate a runtime environment where only one row is received at a time (e.g. for HTTP clients that don't use batching.)--split-dataset-for-perf Data--assess-accuracyNoCalculates classification accuracy for the named input and output fields. This is useful when testing on a pre-labeled data file that contains truth labels which are ignored by theprocesscall itself, for example a file used for training. Accuracy is calculated as the percentage of time the value in the input dataset field is exactly equal to that in the output dataset, meaning formatting (including case and whitespace) must be taken into consideration.--assess-accuracy InputData.ActualLabel=OutputData.PredictedLabel--profile-processing-to-fileNoRuns acProfile profileragainst the serviceprocesscall, and outputs the results to the specified file path. These results can then be read withpstats.Stats. This can be particularly useful when paired with--split-dataset-for-perf, to highlight bottlenecks that occur across many executions.--profile-processing-to-file "./profile"Host and DeployUse a host to run the service in production. A host will use its own implementation for contexts, which could pull input datasets from unexpected locations.At present, there is only one host natively supported:ml-service-wrapper-host-http, which wraps the ML service as an HTTP endpoint, exposing it with JSON endpoints.