Package: django-celery-errorlog
django-celery-errorlogReuseable app for django to collect the unexpted exception and generate
comprehansive report just like what you get in debug mode and store in
database from celery taskIntroductionThis is a extension ofdjango-errorlogto
bring support of celery task with some other features. I love Celery,
but sometimes when there are bugs hiding in the celery tasks code, and
the code has already running as deamon in the background of the
production server, you will find it’s hard to track and debug the error.
More importantly, when a unhandled exception happened, that’s also means
the task failed, and the arguments send to this task will be losed. To
solve this problem, this app wrap you task function inside a database
transaction, when unhandled exception been catched by the wrapper, it
will do this following stuff:rollback this transactionget the exception and traceback to generate a HTML error report as
same as the Django buildin 500 page whenDEBUG=True, which
contains the stack trace also the varibles in each stack. That make
it much easier to debug.record the arguments of this taskget the error categorizedSo, after error happened, you can check theCeleryError.unfixed_errors, then fix the code, restart the worker,
then runerror.fix()to send this task back to the queue again to
make sure all task will be run.If you doerror.fix()before you fixed code in place, it’s doesn’t
matter, because even the old error has been mark as fixed, since the
same error will be raised again, so you will get new error with same
parameters.Change Logs2016-12-04: 0.1.0 Initial submit. Split the code from the online
project. Write the documents, and add the tests.Installpipinstalldjango-celery-errorlogThen modify the settingsFollow the instruction ofdjango-errorlogto
set it up properlyset up djcelery properlyAdddjcelery-errorlogintoINSTALLED_APPSaftererrorlogThen dopython manage.py migrateto setup the database table.UsageIn tasks.py, you have to change the import ofshared_taskandperiodic_taskfromcelerytodjcelery_errorlogto make
errorlog work, here is the example# -*- coding: utf-8 -*-fromdjcelery_errorlogimportshared_task@shared_task(name="tests")deftests(**kwargs):raiseValueError(kwargs)That’s it, now when there is uncatched exception been throwed,
CeleryError will record the invoke parameter and stack trace.buildin shell command(same as django-errorlog)>>>fromdjcelery_errorlog.modelsimportCeleryError>>>CeleryError.unfixed_errors{0:<CeleryError:1-test1-ValueError:A>,1:<CeleryError:4-test2-ValueError:B>}>>>error=CeleryError.unfixed_errors[1]>>>error1:<CeleryError:4-test1-ValueError:B>>>># in this repr, the first number is the index to make it easy to select;>>># the second number 4 is the the count of the same error happened;>>># test1 is the name of the task;>>># ValueError is the exception type;>>># B is the args in the exception.>>>error.vcs_rev# the git/hg version of error, for hg, it's the incremental number that is orderable"1">>>error.ignore()# this command ignore the whole 4 error logsFix the errorAfter you fixed the code base on the error been tracked, you can doerror.fix([queue="queue_name"])to get the task with same parameter
run again. Remember, when you run fix on one error, all same errors will
be send to the celery worker and marked as fixed.