Package: opulent-pandas
Opulent-PandasOpulent-Pandas is a schema validation packages aimed specifically at validating the schema of pandas dataframes.
It takes heavy inspiration fromvoluptuous, and tries to stay as close as possible to the API defined in this package. Opulent-Pandas
is different from voluptuous in that it heavily relies onPandasto perform the validation. This makes Opulent-Pandas considerably faster
than voluptuous on larger datasets. It does, however, mean that the input format is also a Pandas DataFrame, rather than a dict (as is the case for voluptuous)
A performance comparison of voluptuous and Opulent-Pandas will be added to this readme soon!ExampleDefining a schema in Opulent-Pandas is very similar to how you would in voluptuous. To make the similarities and differences clear, let's walk through the same example as is done in the voluptuous readme.Twitter'suser search APIaccepts
query URLs like:$ curl 'https://api.twitter.com/1.1/users/search.json?q=python&per_page=20&page=1'To validate this we might use a schema like:>>>fromopulent_pandasimportSchema,TypeValidator,Required>>>schema=Schema({...Required('q'):[TypeValidator(str)],...Required('per_page'):[TypeValidator(int)],...Required('page'):[TypeValidator(int)],...})Comparing with voluptuous, you'll notice that the validators per field are always specified as a list. Other than that,
it's very similar to how you would define the schema with voluptuousIf we look at the more complex schema, as defined in the readme of voluptuous, we see very similar schemas:>>>fromopulent_pandas.validatorimportRequired,RangeValidator,TypeValidator,ValueLengthValidator>>>schema=Schema({...Required('q'):[TypeValidator(str),ValueLengthValidator(min_length=1)],...Required('per_page'):[TypeValidator(int),RangeValidator(min=1,max=20)],...Required('page'):[TypeValidator(int),RangeValidator(min=0)],...})One difference between Opulent-Pandas and voluptuous is that Opulent-Pandas has avalidatefunction that can be used
to validate a given data structure rather tha voluptuous' approach of passing the data directly to your schema as a parameter.If you pass data in that does not satisfy the requirements specified in your Opulent-Pandas schema, you'll get a corresponding error message. Walking
through the examples provided in the voluptuous readme:There are 3 required fields:
TODO: this example should also tell you which columns are missing. Seems to be a bug.>>>fromopulent_pandasimportMissingColumnError>>>try:...schema.validate({})...raiseAssertionError('MissingColumnError not raised')...exceptMissingColumnErrorase:...exc=e>>>str(exc)=="Columns missing"Trueqmust be a string:>>>fromopulent_pandasimportInvalidTypeError>>>try:...schema.validate(pd.DataFrame({'q':[123],'per_page':[10],'page':[1]})...raiseAssertionError('InvalidTypeError not raised')...exceptInvalidTypeErrorase:...exc=e>>>str(exc)=="Invalid data type found for column: q. Required: <class 'str'>"True...and must be at least one character in length:>>>fromopulent_pandasimportValueLengthError>>>try:...schema.validate(pd.DataFrame({'q':[''],'per_page':5,'page':12}))...raiseAssertionError('ValueLengthError not raised')...exceptValueLengthErrorase:...exc=e>>>str(exc)=="Value found with length smaller than enforced minimum length for column: q. Minimum Length: 1"True"per_page" is a positive integer no greater than 20:>>>fromopulent_pandasimportRangeError>>>try:...schema.validate(pd.DataFrame({'q':['#topic'],'per_page':[900],'page':[12]}))...raiseAssertionError('RangeError not raised')...exceptRangeErrorase:...exc=e>>>str(exc)=="Value found larger than enforced maximum for column: per_page. Required maximum: 20"True>>>try:...schema.validate(pd.DataFrame({'q':['#topic'],'per_page':[-10],'page':[12]}))...raiseAssertionError('RangeError not raised')...exceptRangeErrorase:...exc=e>>>str(exc)=="Value found larger than enforced minimum for column: per_page. Required minimum: 1"True"page" is an integer >= 0:>>>try:...schema.validate(pd.DataFrame({'q':['#topic'],'per_page':['one']})...raiseAssertionError('InvalidTypeError not raised')...exceptInvalidTypeErrorase:...exc=e>>>str(exc)=="Invalid data type found for column: page. Required type: <class 'int'>"True