Package: django-camel-spitter
django-camel-spitterHi. I am a very rude camel 🐫 and I like to spit logs 💦 directly into your database 🗄️.IntroductionProject django-camel-spitter adds a new handler to standard django logging system.Purpose of this handler is to store logs straight to the database, primary with simplicity and opportunity to easy extend this solution.Installation# pippipinstalldjango-camel-spitter# pipenvpipenvinstalldjango-camel-spitter# poetrypoetryadddjango-camel-spitterSetup1. Addingcamel_spittertosettings.INSTALLED_APPS:INSTALLED_APPS=('django.contrib.auth','django.contrib.contenttypes','django.contrib.sessions','django.contrib.messages','camel_spitter')2. Adding additionalloggingdb connection tosettings.DATABASES:This additional connection is needed for handling DB transaction atomicity.
Exception without own connection cannot create DB log, when rollback was made.
Transactions in django are default handled on default DB connection.
So rollback will stop default connection execution, but logging connection not.DATABASES={'default':{'ENGINE':'django.db.backends.postgresql','HOST':os.getenv('DATABASE_HOST'),'PORT':os.getenv('DATABASE_PORT',5432),'NAME':os.getenv('DATABASE_NAME'),'USER':os.getenv('DATABASE_USER'),'PASSWORD':os.getenv('DATABASE_PASSWORD',None)},'logging':{'ENGINE':'django.db.backends.postgresql','HOST':os.getenv('DATABASE_HOST'),'PORT':os.getenv('DATABASE_PORT',5432),'NAME':os.getenv('DATABASE_NAME'),'USER':os.getenv('DATABASE_USER'),'PASSWORD':os.getenv('DATABASE_PASSWORD',None)}}3. Addingmodel,filterandhandlertosettings.LOGGING:LOGGING={'version':1,'disable_existing_loggers':True,'filters':{'db_filter':{'()':'camel_spitter.db_filter.DBFilter',},},'handlers':{'db':{'level':'INFO','class':'camel_spitter.db_handler.DBHandler','model':'tests.models.BasicLogEntry',# path to your custom model'filters':['db_filter']}},'loggers':{'logger':{'handlers':['db'],'level':'INFO'}}}4. Creating a log model:Only importance is inheritance fromcamel_spitter.models.BaseLogModel.fromcamel_spitter.modelsimportBaseLogModelclassBasicLogEntry(BaseLogModel):classMeta:app_label='tests'db_table='log_entries'default_permissions=()Example1. Quick useIf you did all setup steps, you are ready to log to the database.importloggingfromapp.modelsimportBasicLogEntrylogging.getLogger('logger').error('Foo Bar Error')logged_information=BasicLogEntry.objects.get(message='Foo Bar Error')# logged_information = {BasicLogEntry}BasicLogEntry object (1)2. Example of extended modelIf you like to log some additional data, for example: [request.body, user_name], you need to first add these fields to model:fromcamel_spitter.modelsimportBaseLogModelfromdjango.dbimportmodelsclassExtendedLogEntry(BaseLogModel):classMeta:app_label='tests'db_table='extended_log_entries'default_permissions=()request_body=models.JSONField(null=True)user_name=models.CharField(max_length=100,null=True)As a second, you need to add these additional data to logging:importlogginglogging.getLogger('logger').error('Foo Bar Error',extra={'request_body':json.loads(request.body),'user_name':'Foo Bar'})Important notesTesting with pytest and file DBWhen tests are made with pytest library and file DB like SQLite,
tests will make two separate database files from specified connections even though path and engine are the same.
So retrieving log is needed to be executed with using likeBasicLogEntry.objects.using('logging').get(message='Foo Bar').
To avoid this issue you can specify mirror database in test settings.'default':{'ENGINE':'django.db.backends.sqlite3','NAME':DB_PATH,},'logging':{'ENGINE':'django.db.backends.sqlite3','NAME':DB_PATH,'TEST':{'MIRROR':'default',}},Made with ❤ byAdam Žúrek&BACKBONE s.r.o.0.1.0 : 2020-06-27Initial release0.2.0 : 2020-06-29Changed python version requirement from "^0.8" to "^0.6"0.3.0 : 2020-08-04DB logging is successful even if it is wrapped in a transaction, which made rollback.0.3.1 : 2020-11-02Changed django version requirement from "==3.0.*" to ">=2.0"0.3.2 : 2021-05-10🐪 Added id AutoField to BaseLogModel (According to Django 3.2+ recommendations)