Package: pydata-checks
Data ChecksCreate, schedule, and deploy data quality checks.OverviewExisiting data observability solutions are painfully static.data_checksprovides a dynamic data observability framework that allows you to reuse existing Python code and/or write new Python code to define data quality checks that can then be easily scheduled and monitored. Inspired by Python'sunittest, data_checks allows you to write data quality checks as easily and seamlessly as you would write unittests on your code.Quickstart1) InstallationInstall the latest version of data_checks using pip:pipinstallpydata-checks2) Start a Data Check projectInitialize a new data_checks project by using theinitcommand from your project directory (/Users/USERNAME/Desktop/PROJECT_NAME):python-mdata_checks.initThis will start a series of prompts that will guide you through the process of initializing a new data_checks project. For example:$python-mdata_checks.init
Entertherelativefilepathofthedirectorywheresuiteswillbestored:my_first_data_checks_project/suites
Directory'/Users/USERNAME/Desktop/PROJECT_NAME/my_first_data_checks_project/suites'doesnotexist.
Wouldyouliketocreateit?[y/n]:y
Entertherelativefilepathofthedirectorywherecheckswillbestored:my_first_data_checks_project/checks
Directory'/Users/USERNAME/Desktop/PROJECT_NAME/my_first_data_checks_project/checks'doesnotexist.
Wouldyouliketocreateit?[y/n]:y
EnterthedefaultCRONschedule:*****
EnterthedatabaseURL:database_url
EnterthealertingendpointURL:
check_settings.pygenerated.
my_first_data_check.pygenerated.This will create a new directory with the following structure:PROJECT_NAME
├── my_first_data_checks_project
│   ├── __init__.py
│   ├── checks
│   │   ├── __init__.py
│   │   └── my_first_data_check.py
│   ├── suites
│   │   ├── __init__.py
├── check_settings.py3) Set theCHECK_SETTINGS_MODULEto point to thecheck_settings.pyfileexportCHECK_SETTINGS_MODULE=check_settings4) Run the autogenerated data checkpython-mdata_checks.do.run_checkMyFirstDataCheckOutput:[1/1checks]MyFirstDataCheck[1/2Rules]rule_my_first_failed_rule
Thisrulefailed
DataCheckException(severity=1.0,exception=Thisrulefailed,metadata={'rule':'rule_my_first_failed_rule','params':{'args':(),'kwargs':{}}})[2/2Rules]rule_my_first_successful_rulerule_my_first_successful_ruletook0.0seconds5) Modify the autogenerated data checkOpen up themy_first_data_checks_project/checks.my_first_data_check.pyfile and customize the data check to your liking. For instance, you can modify therule_my_first_failed_ruleto always pass by removing the exception:fromdata_checks.classes.data_checkimportDataCheckclassMyFirstDataCheck(DataCheck):...defrule_my_first_failed_rule(self):# This rule will now succeedassertTrue,"This rule now succeeds"...Rerun the data check:python-mdata_checks.do.run_checkMyFirstDataCheckOutput:[1/1checks]MyFirstDataCheck[1/2Rules]rule_my_first_successful_rulerule_my_first_successful_ruletook9.5367431640625e-07seconds[2/2Rules]rule_my_first_failed_rulerule_my_first_failed_ruletook9.5367431640625e-07seconds:tada: Congrats! :tada: You've created and executed your first data check! See thedocumentationfor more information on how writing more advanced checks, suites, and other features like scheduling and alerting.