Package: nanowire-service-py
nanowire-service-pyWrapper for interacting with Nanowire platformUsageInstall the library viapip install nanowire-service-py, or by adding it to requirements file and runningpip install -r requirements.txtThis library is designed for tight integration with Nanowire platform (created by Spotlight Data).The library does not have a hardcode requirement for a specific web server, so a another framework like django or flask could be utilised, however, I'd recommend usingfastapidue to it's simplicity and speedEnvironmentThe following environment variables need to be supplied:classEnvironment(BaseModel):# Dapr spectDAPR_HTTP_PORT:intDAPR_APP_ID:strPUB_SUB:str# Where /pending requests get madeSCHEDULER_PUB_SUB:str# Dapr related properties# Whether we should wait for DAPR server to be active before loadingNO_WAIT:bool=False# Whether the service should publish to schduler# This shouldn't be done if we have an "executor" workerNO_PUBLISH:bool=FalseLOG_LEVEL:Union[str,int]="DEBUG"# Postgres connection detailsPOSTGRES_URL:strPOSTGRES_SCHEMA:str# Utilised for healthchecks and identifying the podSERVICE_ID:str=str(uuid.uuid4())This will be verified on service startup.EntrypointThe primary code logic should be placed in a sub-class ofBaseHandler. User is expected to implementvalidate_argsas well ashandle_bodymethods:importosfromdotenvimportload_dotenvfromfastapiimportFastAPI,ResponsefrompydanticimportBaseModel,validatorfromtypingimportAny,List,Optionalimportpandasaspdfromnanowire_service_pyimportBaseHandler,create,TaskBodyfromtoolboximportClusterToolload_dotenv()allowed_methods=["HDBSCAN","DBSCAN"]# pydantic used to verify function bodyclassArguments(BaseModel):contentUrl:strtextCol:strindexCol:strclusterSize:float=0.2nLabels:int=10method:str="DBSCAN"customStops:Optional[List[str]]=[]maxVocab:int=5000memSave:bool=FalsewithAnomalous:bool=False@validator('method')defmethod_check(cls,method):ifmethodnotinallowed_methods:raiseValueError("Method has to be one of:{}, received:{}".format(",".join(allowed_methods),method))returnmethod# Our custom handlerclassMyHandler(BaseHandler):def__init__(self,*args):super().__init__(*args)self.cluster_tool=ClusterTool(self.logger)defvalidate_args(self,args:Any,task_id:str)->Arguments:returnArguments(**args)defhandle_body(self,args:Arguments,meta:Any,task_id:str):df=pd.read_csv(args.contentUrl,dtype='unicode')ifargs.textColnotindf.columns:raiseRuntimeError("Could not find text column '{}' in CSV".format(args.textCol),{"origin":"CSV"})ifargs.indexColnotindf.columns:raiseRuntimeError("Could not find index column '{}' in CSV".format(args.indexCol),{"origin":"CSV"})result=self.cluster_tool.main(df,args)return(result,meta)# Always handled by the library, pass environment directlyexecutor=create(os.environ,MyHandler)app=FastAPI()# Let's DAPR know which topics should be subscribed to@app.get("/dapr/subscribe")defsubscribe():returnexecutor.subscriptions# Primary endpoint, where request will be delivered to# TaskBody type here verifies the post body@app.post("/subscription")defsubscription(body:TaskBody,response:Response):status=executor.handle_request(body.data.id)response.status_code=status# Return empty body so dapr doesn't freak outreturn{}# Start heartbeat thread, which will periodically send updates to databaseexecutor.heartbeat()Assuming the filename ismain.pythe server can then be started viauvicorn main:appHandling failureThe primary validation happens withinvalidate_argsfunction bypydanticmodels. This is where anything related to input should be checked.If at any point you want the current task to fail, raiseRuntimeErrororException. This will indicate the library, that we should fail and not retry again. For example:CSV missing columns or having incorrect text formatNot enough data passedAnything else that raises for a retryable error, should be raised viaRetryError.VersioningVersioning is based onsemver, however, it primarily applies to thecreatefunction exposed by the package.
If you're using any of the internal system parts, make sure to validate before updating the version.ContributingReadCONTRIBUTING.md🛡 LicenseThis project is licensed under the terms of theMITlicense. SeeLICENSEfor more details.CreditsThis project was generated withpython-package-template.