Package: django-consistency-model
Django Consistency ModelDCM is a set of tools that helps you to keep your data in your Django Models consistent.MotivationYou have a lot of legacy and inconsistent data in your project and you need to clean it outYou want to monitor the broken dataYou are looking for a very simple solution.Quick StartInstall the package:pipinstalldjango-consistency-modelAdd new app intoINSTALLED_APPS:INSTALLED_APPS=(# ..."consistency_model",)Add your first validator using decorator consistency_validator:fromdecimalimportDecimalfromdjango.dbimportmodelsfromconsistency_modelimportconsistency_validatorclassOrder(models.Model):total=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)refund=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)revenue=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)@consistency_validatordefvalidate_revenue(self):assertself.revenue==self.total-self.refund,"revenue = total - refund"Run command to check validators:./manage.pyconsistency_model_checkWhat if I need to check more than one condition in one validatorThe first thing you may think of is using more than one validator, and it is common to have more than one validator (for example, one validator per field).Sometimes, you want to check more than one aspect in one validator or have a complex calculation you don't want to do for every validator.For those cases, you may want to use functionconsistency_error. It shows the system an error without raising an exception, so one validator can generate more than one error.fromdecimalimportDecimalfromdjango.dbimportmodelsfromconsistency_modelimportconsistency_validator,consistency_errorclassOrder(models.Model):total=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)refund=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)revenue=models.DecimalField(default=Decimal("0.00"),decimal_places=2,max_digits=10)@consistency_validatordefvalidate_total(self):assertself.total>=0,"can't be negative"@consistency_validatordefvalidate_revenue(self):ifself.revenue<0:consistency_error("can't be negative","negative")ifself.revenue!=self.total-self.refund:consistency_error("revenue = total - refund","formula")As you can see, one validator (validate_revenue) checks two factors of the field revenue.The functionconsistency_errorhas two arguments - message and name(optional). The name is a unique value for the validator and will be used in monitoring.I don't want to check all of the data, but only one model instead.When you add a new validator, you don't want to check all the data. You want to test only one validator instead.Argument--filtercan help you with that./manage.pyconsistency_model_check--filterstoreapp.Order.validate_revenueCheck only one model./manage.pyconsistency_model_check--filterstoreapp.OrderCheck the model but excluding one validator. Argument--excludeexcludes validator from validation circle../manage.pyconsistency_model_check--filterstoreapp.Order--excludestoreapp.Order.validate_revenueCheck only one object. Using--objectyou can check a specific object in db../manage.pyconsistency_model_check--objectstoreapp.Order.56You can combine--objectwith--filterand--excludeas well.I want to monitor my DB on consistency constantly.The idea of consistency monitoring is very simple. You add the commandconsistency_model_monitoringto your cron. The command checks DB and saves all of the errors inConsistencyFail. Nothing is too complicated.As the result, you can see all of the inconsistency errors in admin panel. Or you can connectpre_savesignal toconsistency_model.ConsistencyFailand send an email notification in case of any new inconsistency.Monitoring configuration.A typical situation is when you don't want to monitor all the data but only recently added/updated data. By default, the system checks only 10k recent IDs, but you have a lot of flexibility to change that with functionregister_consistency.Let's take a look of how one can be used.For modelOrderyou want to check only 10 last ids.fromconsistency_modelimportregister_consistencyregister_consistency(Order,limit=10)register_consistencycan be used as class decoratorfromconsistency_modelimportregister_consistency@register_consistency(limit=10)classOrder(models.Model):# ...you can order not by id, butmodified_onfieldfromconsistency_modelimportregister_consistencyregister_consistency(Order,order_by='modified_on')you can use a consistency checker class to overwrite the whole query for consistency checkfromdjango.dbimportmodelsfromconsistency_modelimportregister_consistency,ConsistencyCheckerclassOrder(models.Model):is_legacy=models.BooleanField(dafult=False)# ...classOrderConsistencyChecker(ConsistencyChecker):limit=None# I don't want to have any limitationorder_by='modified_on'defget_queryset(self):returnself.cls.objects.filter(is_legacy=False)register_consistency(Order,OrderConsistencyChecker)Again, it is possible to be used as class decorator for any  on both classes.For Model:fromdjango.dbimportmodelsfromconsistency_modelimportregister_consistency,ConsistencyCheckerclassOrderConsistencyChecker(ConsistencyChecker):# ...@register_consistency(OrderConsistencyChecker)classOrder(models.Model):is_legacy=models.BooleanField(dafult=False)# ...For Checker:fromdjango.dbimportmodelsfromconsistency_modelimportregister_consistency,ConsistencyCheckerclassOrder(models.Model):is_legacy=models.BooleanField(dafult=False)# ...@register_consistency(Order)classOrderConsistencyChecker(ConsistencyChecker):# ...SettingsCONSISTENCY_DEFAULT_MONITORING_LIMIT(default:10_000) - default limit rows per modelCONSISTENCY_DEFAULT_ORDER_BY(default:"-id") - defaul model ordering for monitoringCONSISTENCY_DEFAULT_CHECKER(default:"consistency_model.tools.ConsistencyChecker") - default class for consistency monitoringIf you havepidpackage installed, one will be used for monitoring command to prevent running multiple monitpring process. The following settings will be used for monitoringCONSISTENCY_PID_MONITORING_FILENAME(default:"consistency_monitoring")CONSISTENCY_PID_MONITORING_FOLDER(default:None) - folder the pid file is stored.tempfile.gettempdir()is using if it isNoneContributingWe’re looking to grow the project and get more contributors especially to support more languages/versions. We’d also like to get the .pre-commit-hooks.yaml files added to popular linters without maintaining forks / mirrors.Feel free to submit bug reports, pull requests, and feature requests.Tools:toxpre-commitblack