Package: pytorch-deploy
torch-deployInstallationTo install:pip install pytorch-deployYou also have to installtorchandtorchvision. You can do sohere.UsageDeploying a pretrained ResNet-18:importtorchimporttorchvision.modelsasmodelsfromtorch_deployimportdeployresnet18=models.resnet18(pretrained=True)resnet18.eval()deploy(resnet18,pre=torch.tensor)The default host and port is 0.0.0.0:8000.EndpointsYou can access the docs for the endpoints at "host:port/docs" after runningpython server.py./predictRequest body: application/jsonResponse body: application/jsonHere's an example of how to use to use the /predict endpoint.importrequestsfromPILimportImageimportnumpyasnpfromtorchvisionimporttransformsim=Image.open('palm.jpg')resize=transforms.Resize(224)to_tensor=transforms.ToTensor()normalize=transforms.Normalize(mean=[0.485,0.456,0.406],std=[0.229,0.224,0.225])tensor=normalize(to_tensor(resize(im))).unsqueeze(0)body={"inputs":tensor.tolist()}r=requests.post("http://127.0.0.1:8000/predict",json=body)response=r.json()output=np.array(response["output"])Notethat you need to send the model input in the request JSON body under the field "inputs".
If you want to send a tensor or a numpy array in the request, you need to turn it into a list first.The output of the model will be in the response JSON body under the "output" field.Sample response format:response={"output":(yournumpyarrayasalisthere)}/predict_imageRequest body: multipart/form-dataResponse body: application/jsonHere's an example of how to use to use the /predict_image endpoint.importrequestsimportnumpyasnpfilename="../palm.jpg"files={'file':open(filename,"rb")}r=requests.post("http://127.0.0.1:8000/predict_image",files=files)response=r.json()output=np.array(response["output"])print(np.argmax(output))The file is uploaded with the content type "multipart/form-data". This requires minimal work on the client side and is compatible with standard file upload requests.Documentationtorch_deploy.deploy(model:nn.Module,pre:Union[List[Callable],Callable]=None,post:Union[List[Callable],Callable]=None,host:str="0.0.0.0",port:int=8000,ssl_keyfile:str=None,ssl_certfile:str=None,ssl_ca_certs:str=None,logdir:str="./deploy_logs/",inference_fn:str=None)Easily converts a pytorch model to API for production usage.model: A PyTorch model which subclasses nn.Module and is callable. Model used for the API.pre: A function or list of functions to be applied to the input.post: Function or list of functions applied to model output before being sent as a response.host: The address for serving the model.port: The port for serving the model.ssl_keyfile,ssl_certfile,ssl_ca_certs: SSL configurations that are passed to uvicornlogfile: Filename to create a file that stores date, ip address, and size of input for each access of the API. IfNone, no file will be created.inference_fn: Name of the method of the model that should be called for the inputs. IfNone, the model itself will be called (Ifmodelis ann.Modulethen it's equivalent to callingmodel.forward(inputs)).ExamplesThere are some sample code in the examples/ directory.Currently In ProgressStill working on an OAuth2 login system that requires correct user credentials to use torch-deploy.Dependenciestorch, torchvision, fastapi, uvicorn, requests, numpy, pydantic