Package: model-inference-fastapi
model-inference-fastapiRunning inference on ML model over http using fastapiInstallationpipinstallmodel-inference-fastapiUsageCreate a model service filemodel_service.pyimplementing theModelServiceBaseinterface like so.fromtypingimportDict,List,Optionalfrommodel_inference_fastapi.model_service_baseimportModelServiceBaseclassModelService(ModelServiceBase):__model=None@staticmethoddefvalidate(data:Optional[List[Dict]]=None,image_files:Optional[List[bytes]]=None)->Optional[str]:return@staticmethoddefpredict(data:Optional[List[Dict]]=None,image_files:Optional[List[bytes]]=None)->List[Dict]:return[{"status":"Ready to Go!!!"}]For instance if you are trying to deploy an image classification model (
e.g.ResNet50) usingtorch. You will need
the followingmodel_service.pyexample in pytorch.importjsonfromtypingimportDict,List,Optionalimporttorchfromtorchvision.modelsimportresnet50fromtorchvision.transformsimportResize,Compose,CenterCrop,ToTensor,Normalizefrommodel_inference_fastapi.model_service_baseimportModelServiceBasefrommodel_inference_fastapi.utilsimportconvert_file_to_pil_image# derived from https://keras.io/api/applications/classModelService(ModelServiceBase):__model=resnet50(weights="IMAGENET1K_V2")__transform=Compose([Resize(size=256),CenterCrop(size=224),ToTensor(),Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])])withopen("imagenet1000_clsidx_to_labels.txt")asf:__class_names=eval(f.read())@staticmethoddefvalidate(*,data:Optional[List[Dict]]=None,image_files:Optional[List[bytes]]=None):iflen(image_files)!=1:return"There must be at least one image file in image_files"@staticmethoddefpredict(*,data:Optional[List[Dict]]=None,image_files:Optional[List[bytes]]=None)->List[Dict]:test_image_tensor=ModelService.__transform(convert_file_to_pil_image(image_files[0]))test_image_tensor=test_image_tensor.view(1,3,224,224)withtorch.no_grad():ModelService.__model.eval()# Model outputs log probabilitiesout=ModelService.__model(test_image_tensor)ps=torch.exp(out)top_k,top_class=ps.topk(1,dim=1)print("Output class :  ",ModelService.__class_names[top_class.cpu().numpy()[0][0]])return[{"class":ModelService.__class_names[top_class.cpu().numpy()[0][0]]}]Run Serveruvicornmodel_inference_fastapi.main:appVisit the API testing page powered by FastAPI herehttp://127.0.0.1:8000/docs/.Navigate to the/predict/AKAPrediction RoutedropdownClick theTry it out buttonFor theimage_filesarrayclick Add string item buttonChoose a file to uploadClickexecuteScroll down to see the server responseTestingmodel_service.pyHere is a sample test file for image classificationmodel_service.py# replace this import with import for you own `model_service.py`frommodel_inference_fastapi.model_serviceimportModelServicedeftest_predict():withopen('/path/to/image.jpg','rb')asfile:prediction=ModelService.predict(image_files=[file.read()])assertprediction[0]["class"]=="dog"Using the docker imageSee example dir for more info.create aDockerfilelike soFROMrobotstech/model-inference-fastapiEXPOSE8000WORKDIR/appCOPYrequirements.txt./requirements.txtRUNpipinstall--upgradepipRUNpipinstall-rrequirements.txt# move model_service.pyCOPY./model_service.py./model_inference_fastapi/# move data or other resources to root dir of model-inferenceCOPY./imagenet1000_clsidx_to_labels.txt.The key info here is to move themodel_service.pyanddata/resourceto the right directory