Package: pandas-validity
pandas-validityWhat is it?pandas-validityis a Python library for the validation of pandas DataFrames. It provides aDataFrameValidatorclass that serves as a context manager. Within this context, you can perform multiple validations and checks. Any encountered errors are collected and raised at the end of the process. TheDataFrameValidatorraises aValidationErrorsGroupexception to summarize the errors.InstallationYou can easily install the latest released version using binary installers from thePython Package Index (PyPI):pipinstallpandas-validityDevelopment InstallationPrerequisites:poetryfor environment managementThe source code is currently hosted on GitHub atohmycoffe/pandas-validity. To get the development version:gitclonegit@github.com:ohmycoffe/pandas-validity.gitTo install the project and development dependencies:makeinstallTo run tests:maketestTo view all possible commands, use:makehelpUsageimportpandasaspdimportdatetimefrompandas_validityimportDataFrameValidator# Create a sample DataFramedf=pd.DataFrame({"A":[1,2,3],"B":["a",None,"c"],"C":[2.3,4.5,9.2],"D":[datetime.datetime(2023,1,1,1),datetime.datetime(2023,1,1,2),datetime.datetime(2023,1,1,3),],})# Define your expectations and data type mappingsexpected_columns=['A','B','C','E']data_types_mapping={"A":'float',"D":'datetime'}# Use DataFrameValidator for validationwithDataFrameValidator(df)asvalidator:validator.is_empty()validator.has_required_columns(expected_columns)validator.has_no_redundant_columns(expected_columns)validator.has_valid_data_types(data_types_mapping)validator.has_no_missing_data()Output:Erroroccurred:(<class'pandas_validity.exceptions.ValidationError'>)Thedataframehasmissingcolumns:['E']Erroroccurred:(<class'pandas_validity.exceptions.ValidationError'>)Thedataframehasredundantcolumns:['D']Erroroccurred:(<class'pandas_validity.exceptions.ValidationError'>)Column'A'hasaninvaliddatatype:'int64'Erroroccurred:(<class'pandas_validity.exceptions.ValidationError'>)Found1missingvalue:[{'index':1,'column':'B','value':None}]+ExceptionGroupTraceback(mostrecentcalllast):
...|pandas_validity.exceptions.ValidationErrorsGroup:Validationerrorsfound:4.(4sub-exceptions)+-+----------------1----------------|pandas_validity.exceptions.ValidationError:Thedataframehasmissingcolumns:['E']+----------------2----------------|pandas_validity.exceptions.ValidationError:Thedataframehasredundantcolumns:['D']+----------------3----------------|pandas_validity.exceptions.ValidationError:Column'A'hasaninvaliddatatype:'int64'+----------------4----------------|pandas_validity.exceptions.ValidationError:Found1missingvalue:[{'index':1,'column':'B','value':None}]+------------------------------------The library supports the following data types for validation:predefined:"str","int","float","datetime","bool"or anyCallablethat accepts a datatype/dtypeobject and returns a boolean value to indicate the validation status - example:pd.api.types.is_string_dtypeDevelopmentPrerequisites:poetryfor environment managementThe source code is currently hosted on GitHub at:https://github.com/ohmycoffe/pandas-validitygitclonegit@github.com:ohmycoffe/pandas-validity.gitTo install the project and development dependencies:makeinstallTo run tests:maketestTo view all possible commands, use:makeLicenseThis project is licensed under the terms of theMITlicense.