Package: label-studio-ml
What is the Label Studio ML backend?The Label Studio ML backend is an SDK that lets you wrap your machine learning code and turn it into a web server.
You can then connect that server to a Label Studio instance to perform 2 tasks:Dynamically pre-annotate data based on model inference resultsRetrain or fine-tune a model based on recently annotated dataIf you just need to load static pre-annotated data into Label Studio, running an ML backend might be overkill for you. Instead, you canimport preannotated data.How it worksGet your model codeWrap it with the Label Studio SDKCreate a running server scriptLaunch the scriptConnect Label Studio to ML backend on the UIQuickstartFollow this example tutorial to run an ML backend with a simple text classifier:Clone the repogitclonehttps://github.com/heartexlabs/label-studio-ml-backendSetup environmentIt is highly recommended to usevenv,virtualenvorcondapython environments. You can use the same environment as Label Studio does.Read moreabout creating virtual environments viavenv.cdlabel-studio-ml-backend# Install label-studio-ml and its dependenciespipinstall-U-e.# Install example dependenciespipinstall-rlabel_studio_ml/examples/requirements.txtInitialize an ML backend based on an example script:label-studio-mlinitmy_ml_backend--scriptlabel_studio_ml/examples/simple_text_classifier/simple_text_classifier.pyThis ML backend is an example provided by Label Studio. Seehow to create your own ML backend.Start ML backend serverlabel-studio-mlstartmy_ml_backendStart Label Studio and connect it to the running ML backend on the project settings page.Create your own ML backendFollow this tutorial to wrap existing machine learning model code with the Label Studio ML SDK to use it as an ML backend with Label Studio.Before you start, determine the following:The expected inputs and outputs for your model. In other words, the type of labeling that your model supports in Label Studio, which informs theLabel Studio labeling config. For example, text classification labels of "Dog", "Cat", or "Opossum" could be possible inputs and outputs.Theprediction formatreturned by your ML backend server.This example tutorial outlines how to wrap a simple text classifier based on thescikit-learnframework with the Label Studio ML SDK.Start by creating a class declaration. You can create a Label Studio-compatible ML backend server in one command by inheriting it fromLabelStudioMLBase.fromlabel_studio_ml.modelimportLabelStudioMLBaseclassMyModel(LabelStudioMLBase):Then, define loaders & initializers in the__init__method.def__init__(self,**kwargs):# don't forget to initialize base class...super(MyModel,self).__init__(**kwargs)self.model=self.load_my_model()There are special variables provided by the inherited class:self.parsed_label_configis a Python dict that provides a Label Studio project config structure. Seeref for details. Use might want to use this to align your model input/output with Label Studio labeling configuration;self.label_configis a raw labeling config string;self.train_outputis a Python dict with the results of the previous model training runs (the output of thefit()method described bellow) Use this if you want to load the model for the next updates for active learning and model fine-tuning.After you define the loaders, you can define two methods for your model: an inference call and a training call.Inference callUse an inference call to get pre-annotations from your model on-the-fly. You must update the existing predict method in the example ML backend scripts to make them work for your specific use case. Write your own code to override thepredict(tasks, **kwargs)method, which takesJSON-formatted Label Studio tasksand returns predictions in theformat accepted by Label Studio.Exampledefpredict(self,tasks,**kwargs):predictions=[]# Get annotation tag first, and extract from_name/to_name keys from the labeling config to make predictionsfrom_name,schema=list(self.parsed_label_config.items())[0]to_name=schema['to_name'][0]fortaskintasks:# for each task, return classification results in the form of "choices" pre-annotationspredictions.append({'result':[{'from_name':from_name,'to_name':to_name,'type':'choices','value':{'choices':['My Label']}}],# optionally you can include prediction scores that you can use to sort the tasks and do active learning'score':0.987})returnpredictionsTraining callUse the training call to update your model with new annotations. You don't need to use this call in your code, for example if you just want to pre-annotate tasks without retraining the model. If you do want to retrain the model based on annotations from Label Studio, use this method.Write your own code to override thefit(annotations, **kwargs)method, which takesJSON-formatted Label Studio annotationsand returns an arbitrary dict where some information about the created model can be stored.Exampledeffit(self,completions,workdir=None,**kwargs):# ... do some heavy computations, get your model and store checkpoints and resourcesreturn{'checkpoints':'my/model/checkpoints'}# <-- you can retrieve this dict as self.train_output in the subsequent callsAfter you wrap your model code with the class, define the loaders, and define the methods, you're ready to run your model as an ML backend with Label Studio.For other examples of ML backends, refer to theexamples in this repository. These examples aren't production-ready, but can help you set up your own code as a Label Studio ML backend.Deploy your ML backend to GCPBefore you start:InstallgcloudInit billing for account if it's notactivatedInit gcloud, type the following commands and login in browser:gcloudauthloginActivate your Cloud Build APIFind your GCP project ID(Optional) Add GCP_REGION with your default region to your ENV variablesTo start deployment:Create your own ML backendStart deployment to GCP:label-studio-mldeploygcp{ml-backend-local-dir}\--from={model-python-script}\--gcp-project-id{gcp-project-id}\--label-studio-host{https://app.heartex.com}\--label-studio-api-key{YOUR-LABEL-STUDIO-API-KEY}After label studio deploys the model - you will get model endpoint in console.