Package: kube-kombu
Kube KombuThis project wraps thekombuconsumer of python for the use with writing consumer for RabbitMQ custom pubsub.
Since Kombu Consumer doesn't come with liveness check this package provides liveness check on the top
of kombu consumers. This package exposes a TCP port which can be added tokubernetes livenessprobe.Logic for liveness probe is as follows:-Open an asnycio server TCP port in the same process as Consumer.On Message Received of TCP checks for liveness of the threads and connection of the Kombu Consumers.If the rabbit consumers are found to be inactive this closes the TCP port.Once the port is closed liveness checks will fail the next time leading to restart of podSetup and Running Kombu consumersInstallation Steps:Install python 3.9 or greater on your system usingpyenvActivate your project's virtual environment for installing this library$source<virtualenv-path>/bin/activateInstall consumer library by running$pipinstallkube_kombuImplementation StepsIf you are using django you'll need to setup the django project before running thestart_consumer.Example :-importdjangofromdjango.confimportsettingsimportosos.environ.setdefault("DJANGO_SETTINGS_MODULE","project.settings")django.setup()There are three variables that can be defined in your django settings file or in environment variables or as constants in your project:RABBITMQ: This can be a dictionary containing rabbitmq related variables
####Example:RABBITMQ={"URL":"<RABBIT_URL>","EXCHANGE":"<RABBIT_EXCHANGE>","EXCHANGE_TYPE":"<RABBIT_EXCHANGE_TYPE>","ROUTING_KEY":"<RABBIT_ROUTING_KEY>","QUEUE":"<RABBIT_QUEUE>"}Once you have defined the rabbit config you need to define consumer_config :-fromkube_kombu.consumer_configimportConsumerConfigconsumer_config=ConsumerConfig("URL","EXCHANGE","EXCHANGE_TYPE","ROUTING_KEY","QUEUE",)AbstractConsumerAdapterdefines the abstract methodcallbackwhich you need to extend your class with and implement your own adapter on what you want to do on receiving the message.Sample Adapter Can be written as :-importjsonimportloggingfromkube_kombu.abstract_consumer_adapterimportAbstractConsumerAdapterLOGGER=logging.getLogger(__name__)classSampleConsumerAdapter(AbstractConsumerAdapter):@classmethoddefhandle_event1(cls,data):pass@classmethoddefhandle_event2(cls,data):passdefcallback(self,data,message):try:ifisinstance(data,str):data=json.loads(data)ifdata["event"]=="event1":SampleConsumerAdapter.handle_event1(data["payload"])elifdata["event"]=="event2":SampleConsumerAdapter.handle_event2(data["payload"])exceptExceptionase:LOGGER.exception(f"Exception in callback, Error:{e}")message.ack()DONOT FORGET TO ACK THE MESSAGE at the end of callbackOnce you have implemented the Adapters and config of your own you will now need to start the consumer which can be done as follows:-fromkube_kombu.start_consumerimportstart_consumerstart_consumer(consumer_config,SampleConsumerAdapter,host,port)At the end if you want to run the Kombu Consumers on Pod you can implement the__main__as follows:-if__name__=="__main__":importargparseparser=argparse.ArgumentParser(description='Setup Host and Port for Kube Liveness check')parser.add_argument('--port',type=int,metavar='path',default=8988,help='Post to start TCP healthCheck server on. Default is 8988')parser.add_argument('--host',metavar='path',default="0.0.0.0",help='IP host to start health check server on. Default is 0.0.0.0')args=parser.parse_args()main(args.host,args.port)This will help you to pass the post and host from dockerRUNcommand instead.HEALTHCHECK_HOST: Host on which the consumer thread will open a port fot liveness check. Keep it0.0.0.0for use with Kubernetes liveness check.HEALTHCHECK_PORT: Port which will be opened by consumer thread for liveness check. Use the same port as you would use withEXPOSEcommand in dockerExampleSample scripts are written in thesampledirectory for defining and initializing the consumercheck_liveness_probe.pyfile is for testing the socket connection locallyYou should now be able to use tcp liveness probe in kubernetes for liveness check of the pod.