Package: delphai-backend-utils
Delphai backend common facilities [Pakage]DevelopmentGithub action deploys to pypi when a new tag is created.
Also make sure the version insetup.pyis equal to the version on GithubNOTEPYPI wont allow version overriding, Means make sure the pakage works well before changing the version insetup.pyInstallationInclude the reference to this library into yourPipfile[packages]
delphai-backend-utils = "*"The following packages will be installed automatically, and you do not need to include them intoPipfile:omegaconf = "*"memoization = "*"azure-identity = "==1.4.0b3"azure-keyvault = "*"python-dotenv = "*"kubernetes = "*"coloredlogs = "*"If you are going to use the moduledb_access, please also includepymongointoPipfile:[packages]
delphai-backend-utils = "*"
pymongo = "*"If you are going to use the modulegrpc_calls, please also include gRPC dependencies intoPipfile:[packages]
delphai-backend-utils = "*"
grpcio-reflection = "*"For full functionality:[packages]
delphai-backend-utils = "*"
pymongo = "*"
grpcio = "*"
grpcio-tools = "*"
grpcio-reflection = "*"
tornado = "*"To update this library in your development environment:pipenvupdateModuleconfigLoads and provides configuration data fromconfig/*.ymlfiles, including resolved values from environment variables stored into.envfile, and Kubernetes and Azure secrets.Configuration files used to compile an actual working configuration:config/default.yml- basic default configurationconfig/development.yml- redefines default configuration in a development environmentconfig/review.yml- redefines default configuration in the review environmentconfig/staging.yml- redefines default configuration in the staging environmentconfig/production.yml- redefines default configuration in the production environment.env- used only in development environmentImporting:fromdelphai_backend_utils.configimportget_configUsage example 1- retrieving a single value:address=get_config('server.host_and_port')If value is not redefined in a environment-specific config file, it will be extracted from default.yml:server:
  host_and_port: 0.0.0.0:8080Usage example 2- retrieving and using a whole chapter:server_config=get_config('server')address=server_config['host_and_port']# Be careful, a KeyError might happenUsage example 3- retrieving the whole configuraion as an OmegaConf object:config=get_config(None)# Notice "None"db_name=config.db.name# "MissingMandatoryValue: Missing mandatory value: db.connection_string" can happen, and it might be a desirable behaviorModuledb_configEstablishing connections to MongoDB.Importing:fromdelphai_backend_utils.db_configimportget_own_db_connectionPleasedo not forgetto includepymongointo yourPipfile(see above in the "Installation" section).Usage example- printing out a "companies" collection size:db_conn=get_own_db_connection()print(db_conn.companies.estimated_document_count())There is one more functionchunksin this module which is hugely useful when we need to implement chunked reads and updates.ModuleloggingConfigurable logging. In addition to standard logging operations implementserror_with_tracebackfunction that logs a last occured exception.Importing:fromdelphai_backend_utilsimportloggingUsage example- printing out a "companies" collection size:fromdelphai_backend_utilsimportloggingdeffail_here():return1/0defcalculate_result():try:returnfail_here()exceptException:logging.error_with_traceback()returnNoneModulegrpc_callsThe natural way to call other microservices. Functions:call_grpc - calls other grpc serviceasync_call_grpc - awaitable version of call_grpc. Use if caller is "async def".get_description - to get other grpc's service descriptionasync_get_description - awaitable version of get_description. Use if caller is "async def".Importing:fromdelphai_backend_utilsimportgrpc_callsPleasedo not forgetto include gRPC dependencies into yourPipfile(see above in the "Installatio" section).Usage example- printing out a "companies" collection size:suggestions=grpc_calls.call_grpc('delphai.typeahead.Typeahead.get_suggestions',{'search_query':'siemens'})foridx,suggestioninenumerate(suggestions['matches']):print(f'{idx+1}.{suggestion["value"]}{suggestion.get("url")}')ModuleusersThe natural way to call other microservices. Functions:call_grpc - calls other grpc serviceasync_call_grpc - awaitable version of call_grpc. Use if caller is "async def".get_description - to get other grpc's service descriptionasync_get_description - awaitable version of get_description. Use if caller is "async def".Importing:fromdelphai_backend_utils.userimportget_userUsage example:try:user=get_user(context)# context is a gRPC context that usually contains user description in its metadatauser_id_str=user.get('https://delphai.com/mongo_user_id')exceptException:raiseException('Failed get user info')ifnotuser_id_str:raiseException('User profile is not properly configured')ModuleformattingCommon forms and formatting standards. Functions:clean_url - clean an urlImporting:fromdelphai_backend_utils.formattingimportclean_urlUsage example:url=clean_url(url,keep_www=True)Moduleown_gatewayImplements an individual HTTP1–>gRPC (HTTP2) gateway for gRPC microservices.Usage example(servefunction inserver.pymodule):defserve():server=grpc.server(futures.ThreadPoolExecutor(max_workers=10))service_pb2_grpc.add_GreetingServicer_to_server(Greeting(),server)# the reflection service will be aware of "Greeter" and "ServerReflection" services.service_names=(service_pb2.DESCRIPTOR.services_by_name['Greeting'].full_name,reflection.SERVICE_NAME,)reflection.enable_server_reflection(service_names,server)# Start own gateway if configured                                  # <<<<run_own_gateway=get_config('server.run_own_gateway')orFalse# <<<<ifrun_own_gateway:# <<<<fromdelphai_backend_utils.own_gatewayimportrun_own_gateway# <<<<run_own_gateway(service_pb2.DESCRIPTOR,server)# <<<<address=get_config('server.host_and_port')or'0.0.0.0:8080'server.add_insecure_port(address)server.start()logging.info(f'Started server{address}')try:server.wait_for_termination()exceptKeyboardInterrupt:logging.error('Interrupted')Lines to add marked with "# <<<<".Additionally:Includetornadointo yourPipfile(see above in the "Installation" section).Turn on this feature in your configuration file (default.ymlor some other configuration file):server:# host_and_port: 0.0.0.0:8080  # "0.0.0.0:8080" is a default value. Uncomment and change if you need another.run_own_gateway:true# Set to true if you need an individual gateway to be runned together with gRPC handler.# gateway_host_and_port: 0.0.0.0:7070  # "0.0.0.0:7070" is a default value.Moduleapi_callsA wrapper for calling APIs. Contains only a single function for Azure Machine Learning Endpoints right now (call_azure_endpoint).Importing:fromdelphai_backend_utils.api_callsimportcall_azure_endpointUsage example- calling word embedding model to retrieve similar keywords:api_url='http://51.145.149.205:80/api/v1/service/similar-keywords/score'api_key=''# has to be setkeywords_resp=call_azure_endpoint(api_url,api_key,'urban mobility')ifkeywords_resp['success']:print(', '.join([keywordforkeyword,_inkeywords_resp['content']]))