Package: serviceprovider
Python RangerBefore you start, you might wanna checkRangerfor more details. You'll need it
to follow some jargon being used in this readme.There are 2 sections in here.Ranger Service ProviderRanger DaemonRanger Service ProviderA service provider in Ranger is something can be used to broadcast that a service is available at some host:port, where
clients can connect and request services (make http calls). This broadcast is essentially done using zookeeper. The
following python class helps you do the same for any python based service/tool. It follows the same models as present in
the main ranger java library.Similar details can be found atPyPiInstallationpython3.9-mpipinstallserviceproviderUsagefromserviceprovider.ranger_modelsimport*fromserviceproviderimportRangerServiceProvider,HealthCheck# Create the ranger service providerranger=RangerServiceProvider(cluster_details=ClusterDetails(zk_string='localhost:2181',update_interval=1),service_details=ServiceDetails(host='localhost',port=12211,environment='stage',namespace='myorg',service_name='python-test'),health_check=HealthCheck(url='localhost:12211/health',scheme=UrlScheme.GET))## Start the updates in background (this will update zookeeper at regular intervals)ranger.start()## You may also start the updates and block your current thread (until we hit an interrupt)ranger.start(block=True)## When you wish to stop updatesranger.stop()DetailsThe above sample shows how to set up a background thread, that does the job of publishing regular updates to zk. You can
optionally provide a healthcheck url, which will receive a ping at regular intervals. A HEALTHY broadcast will only be
done if the ping check was successful. You can check HealthCheck to customize the URL to your needs.Ranger Daemon setupThis section deals with using the code as a simple light daemon that can run alongside your software (but outside it) to
provide regular service discovery updates to zookeeper. As usual, checkRangerfor more details.IntentIdeally, you would directly use the standard Ranger java client to deeply integrate the service's health updates with
ranger.In scenarios where you can't do the above, you can rely on this daemon. Say you need discovery updates to be published
for a service written in a langauge other than java, or you are unable to add the ranger dependency directly, in your
java application.The intent of this daemon is to run along-side your software and publish updates, as long as your software is up and
healthy. Currently, support has been added for a dockerized setup, as well as an import based custom setup. Currently,
Support has been provided for the following:Simple usageImport based usageDocker multi-container setup1. Simple usageIf you just wish to invoke the script directly, clone the project and follow along the helper. You command will look
like sopython3.9serviceprovider/ranger_daemon.py-zklocalhost:2181-smyapp-hostlocalhost-p12211-norg-estage-hcu'http://localhost:12211/healthcheck?pretty=true'2. Import Based UsageYou can also choose to run the daemon from within another python file, by forwarding the command line arguments. Install
the package first, as shown belowpython3.9-mpipinstallserviceproviderimportsysfromserviceprovider.ranger_daemonimportranger_daemon_triggerranger_daemon_trigger(sys.argv[1:])3. Docker BasedImagine a scenario where you already have a docker application, but you want to run this daemon alongside the container,
to make the existing container discoverable, without having to code up an integration with ranger. The following is a
solution to this problem. You can use docker compose to run your service and this daemon as a multi container docker
application.After this, your existing container should be ready for service discovery.Docker containers are available on
theDockerHub.The following docker command can be used to start the daemon, using environment variables. The table below explains the
various environment variables required to run the scriptEnv VariableDescriptionHOSTHostnamePORTPortRANGER_ZKZookeeper connection stringSERVICE_NAMEName of serviceENVEnvironment (stage/prod)NAMESPACENamespace in zookeeperHEALTH_CHECK[optional] GET healthcheck URL to be used for pingsREGION[optional] Region valueTAGS[optional] Comma separated tagsdockerrun--rm-d-eRANGER_ZK=<zookeeper_info>-eSERVICE_NAME=<name_of_service>-eHOST=<host_of_machine>-ePORT=<port>-eENV=<environment>-eNAMESPACE=<namespace>-eHEALTH_CHECK=<health_check_url>--namepython-ranger-daemontusharknaik/python-ranger-daemon:1.7Here is an example for running it on a Mac machine, assuming your zookeeper is already running onlocalhost:2181(
notice the network being set tohostand zookeeper being sent ashost.docker.internalfor connecting to localhost
from within docker)dockerrun--rm-d--networkhost-eRANGER_ZK=host.docker.internal:2181-eSERVICE_NAME=python-test-eHOST=localhost-ePORT=12211-eENV=stage-eNAMESPACE=myorg-eHEALTH_CHECK="localhost:12211/health"--namepython-ranger-daemontusharknaik/python-ranger-daemon:1.7Under the hoodThe daemon/thread will write data to zookeeper in the following format (datamodel from ranger):{"host":"localhost","port":12211,"nodeData":{"environment":"stage","tags":["identity","auth"],"region":"IN-nm"},"healthcheckStatus":"healthy","lastUpdatedTimeStamp":1639044989841}Updates will be published in the path: /$NAMESPACE/$SERVICE_NAME at a periodic intervals of --interval (default: 1
second)The following will be taken care of:Infinite retry and connection reattempts in case of zk connection issuesProper cleanup of zk connections to get rid of ephemeral nodesProper loggingDoes continuous health check pings on a particular health check url if required [optional]