Package: django-jaiminho
jaiminhoA broker agnostic implementation of the outbox and other message resilience patterns for Django apps.Getting StartedTo use jaiminho with your project, you just need to do 6 steps:1 - Install itpython-mpipinstalljaiminho2 - Add jaiminho to the INSTALLED_APPSINSTALLED_APPS=[..."jaiminho"]3 - Run migrationspythonmanage.pymigrate4 - Configure jaiminho options in Django settings.py:JAIMINHO_CONFIG={"PERSIST_ALL_EVENTS":False,"DELETE_AFTER_SEND":True,"DEFAULT_ENCODER":DjangoJSONEncoder,"PUBLISH_STRATEGY":"publish-on-commit",}5 - Decorate your functions with @save_to_outboxfromjaiminho.sendimportsave_to_outbox@save_to_outboxdefany_external_call(**kwargs):# do somethingreturn6 - Run the relay events commandpython manage.py events_relay --run-in-loop --loop-interval 1If you don't use--run-in-loopoption, the relay command will run only 1 time. This is useful in case you want to configure it as a cronjob.DetailsJaiminho@save_to_outboxdecorator willinterceptdecorated function andpersistit in adatabase tablein the sametransactionthat is active in the decorated function context. The event relaycommand, is aseparated processthat fetches the rows from this table and execute the functions. When an outage happens, the event relay command willkeep retrying until it succeeds. This way,eventual consistency is ensuredby design.Configuration optionsPUBLISH_STRATEGY- Strategy used to publish events (publish-on-commit, keep-order)PERSIST_ALL_EVENTS- Saves all events and not only the ones that fail, default isFalse. Only applicable for{ "PUBLISH_STRATEGY": "publish-on-commit" }since all events needs to be stored on keep-order strategy.DELETE_AFTER_SEND- Delete the event from the outbox table immediately, after a successful sendDEFAULT_ENCODER- Default Encoder for the payload (overwritable in the function call)StrategiesKeep OrderThis strategy is similar to transactional outboxdescribed by Chris Richardson. The decorated function intercepts the function call and saves it on the local DB to be executed later. A separate command relayer will keep polling local DB and executing those functions in the same order it was stored.
Be carefully with this approach,if any execution fails, the relayer will get stuckas it would not be possible to guarantee delivery order.Publish on commitThis strategy will always execute the decorated function after current transaction commit. With this approach, we don't depend on a relayer (separate process / cronjob) to execute the decorated function and deliver the message. Failed items will only be retried
through relayer. Although this solution has a better performance as only failed items is delivered by the relay command,we cannot guarantee delivery order.Relay CommandWe already provide a command to relay items from DB,EventRelayCommand. The way you should configure depends on the strategy you choose.
For example, onPublish on Commit Strategyyou can configure a cronjob to run every a couple of minutes since only failed items are published by the command relay. If you are usingKeep Order Strategy, you should run relay command in loop mode as all items will be published by the command, e.gcall_command(events_relay.Command(), run_in_loop=True, loop_interval=0.1).How to clean older eventsYou can use Jaiminho'sEventCleanerCommandin order to do that. It will query for all events that were sent before a given time interval (e.g. last 7 days) and will delete them from the outbox table.The default time interval is7 days. You can use theTIME_TO_DELETEsetting to change it. It should be added toJAIMINHO_CONFIGand must be a validtimedelta.Relay per stream and Overwrite publish strategyDifferent streams can have different requirements. You can save separate events per streams by using the@save_to_outbox_streamdecorator:@save_to_outbox_stream("my-stream")defany_external_call(payload,**kwargs):# do somethingpassyou can also overwrite publish strategy configure on settings:@save_to_outbox_stream("my-stream",PublishStrategyType.KEEP_ORDER)defany_external_call(payload,**kwargs):# do somethingpassAnd then, run relay command with stream filter optionpythonmanage.pyrelay_eventTrue0.1my-streamIn the example above,Trueis the option for run_in_loop;0.1for loop_interval; andmy_streamis the name of the stream.SignalsJaiminho triggers the following Django signals:SignalDescriptionevent_publishedTriggered when an event is sent successfullyevent_failed_to_publishTriggered when an event is not sent, being added to the Outbox table queueHow to collect metrics from Jaiminho?You could use the Django signals triggered by Jaiminho to collect metrics.
Consider the following code as example:fromdjango.dispatchimportreceiver@receiver(event_published)defon_event_sent(sender,event_payload,**kwargs):metrics.count(f"event_sent_successfully{event_payload.get('type')}")@receiver(event_failed_to_publish)defon_event_send_error(sender,event_payload,**kwargs):metrics.count(f"event_failed{event_payload.get('type')}")DevelopmentCreate a virtualenvvirtualenvvenv
pipinstall-rrequirements-dev.txt
tox-epy39CollaborationIf you want to improve or suggest improvements, check ourCONTRIBUTING.mdfile.LicenseThis project is licensed under MIT License.SecurityIf you have any security concern or report feel free to reach out tosecurity@loadsmart.com;