Package: tableschema
tableschema-pyA Python implementation of theTable Schemastandard.[Important Notice]We have releasedFrictionless Framework. This framework provides improvedtableschemafunctionality extended to be a complete data solution. The change in not breaking for the existing software so no actions are required. Please read theMigration Guidefromtableschemato Frictionless Framework.we continue to bug-fixtableschema@1.xin thisrepositoryas well as it's available onPyPias it was beforeplease note thatfrictionless@3.xversion's API, we're working on at the moment, is not stablewe will releasefrictionless@4.xby the end of 2020 to be the first SemVer/stable versionFeaturesTableto work with data tables described by Table SchemaSchemarepresenting Table SchemaFieldrepresenting Table Schema fieldvalidateto validate Table Schemainferto infer Table Schema from databuilt-in command-line interface to validate and infer schemasstorage/plugins system to connect tables to different storage backends like SQL DatabaseContentsGetting StartedInstallationDocumentationIntroductionWorking with TableWorking with SchemaWorking with FieldAPI ReferencecliTableSchemaFieldStoragevalidateinferFailedCastDataPackageExceptionTableSchemaExceptionLoadErrorValidationErrorCastErrorIntegrityErrorUniqueKeyErrorRelationErrorUnresolvedFKErrorStorageErrorExperimentalContributingChangelogGetting StartedInstallationThe package uses semantic versioning. It means that major versions  could include breaking changes. It's highly recommended to specifytableschemaversion range in yoursetup/requirementsfile e.g.tableschema>=1.0,<2.0.$pipinstalltableschemaDocumentationIntroductionLet's start with a simple example:fromtableschemaimportTable# Create tabletable=Table('path.csv',schema='schema.json')# Print schema descriptorprint(table.schema.descriptor)# Print cast rows in a dict formforkeyed_rowintable.iter(keyed=True):print(keyed_row)Working with TableA table is a core concept in a tabular data world. It represents data with metadata (Table Schema). Let's see how we can use it in practice.Consider we have some local csv file. It could be inline data or from a remote link - all supported by theTableclass (except local files for in-brower usage of course). But say it'sdata.csvfor now:city,location
london,"51.50,-0.11"
paris,"48.85,2.30"
rome,N/ALet's create and read a table instance. We use the staticTable.loadmethod and thetable.readmethod with thekeyedoption to get an array of keyed rows:table=Table('data.csv')table.headers# ['city', 'location']table.read(keyed=True)# [#   {city: 'london', location: '51.50,-0.11'},#   {city: 'paris', location: '48.85,2.30'},#   {city: 'rome', location: 'N/A'},# ]As we can see, our locations are just strings. But they should be geopoints. Also, Rome's location is not available, but it's just a stringN/Ainstead ofNone. First we have to infer Table Schema:table.infer()table.schema.descriptor# { fields:#   [ { name: 'city', type: 'string', format: 'default' },#     { name: 'location', type: 'geopoint', format: 'default' } ],#  missingValues: [ '' ] }table.read(keyed=True)# Fails with a data validation errorLet's fix the "not available" location. There is amissingValuesproperty in Table Schema specification. As a first try we setmissingValuestoN/Aintable.schema.descriptor. The schema descriptor can be changed in-place, but all changes should also be committed usingtable.schema.commit():table.schema.descriptor['missingValues']='N/A'table.schema.commit()table.schema.valid# falsetable.schema.errors# [<ValidationError: "'N/A' is not of type 'array'">]As a good citizens we've decided to check our schema descriptor's validity. And it's not valid! We should use an array for themissingValuesproperty. Also, don't forget to include "empty string" as a valid missing value:table.schema.descriptor['missingValues']=['','N/A']table.schema.commit()table.schema.valid# trueAll good. It looks like we're ready to read our data again:table.read(keyed=True)# [#   {city: 'london', location: [51.50,-0.11]},#   {city: 'paris', location: [48.85,2.30]},#   {city: 'rome', location: null},# ]Now we see that:locations are arrays with numeric latitude and longitudeRome's location is a native PythonNoneAnd because there are no errors after reading, we can be sure that our data is valid against our schema. Let's save it:table.schema.save('schema.json')table.save('data.csv')Ourdata.csvlooks the same because it has been stringified back tocsvformat. But now we haveschema.json:{"fields":[{"name":"city","type":"string","format":"default"},{"name":"location","type":"geopoint","format":"default"}],"missingValues":["","N/A"]}If we decide to improve it even more we could update the schema file and then open it again. But now providing a schema path:table=Table('data.csv',schema='schema.json')# Continue the workAs already mentioned a given schema can be used tovalidatedata (see theSchemasection for schema specification details). In default mode invalid data rows immediately trigger anexceptionin thetable.iter()/table.write()methods.Suppose this schema-invalid local fileinvalid_data.csv:key,value
zero,0
one,not_an_integer
two,2We're going to validate the data against the following schema:table=Table('invalid_data.csv',schema={'fields':[{'name':'key'},{'name':'value','type':'integer'}]})Iterating over the data triggers an exception due to the failed cast of'not_an_integer'toint:forrowintable.iter():print(row)# Traceback (most recent call last):# ...# tableschema.exceptions.CastError: There are 1 cast errors (see exception.errors) for row "3"Hint: The row number count starts with 1 and also includes header lines.(Note: You can optionally switch offiter()/read()value casting using the cast parameter, see reference below.)By providing a custom exception handler (a callable) to those methods you can treat occurring exceptions at your own discretion, i.e. to "fail late" and e.g. gather a validation report on the whole data:errors=[]defexc_handler(exc,row_number=None,row_data=None,error_data=None):errors.append((exc,row_number,row_data,error_data))forrowintable.iter(exc_handler=exc_handler):print(row)# ['zero', 0]# ['one', FailedCast('not_an_integer')]# ['two', 2]print(errors)# [(CastError('There are 1 cast errors (see exception.errors) for row "3"',),#   3,#   OrderedDict([('key', 'one'), ('value', 'not_an_integer')]),#   OrderedDict([('value', 'not_an_integer')]))]Note thatData rows are yielded even though the data is schema-invalid; this is due to our custom expression handler choosing not to raise exceptions (but rather collect them in the errors list).Data field values that can't get casted properly (ifiter()/read()cast parameter is set to True, which is the default) are wrapped into aFailedCast"value holder". This allows for distinguishing uncasted values from successfully casted values on the data consumer side.FailedCastinstances can only get yielded when custom exception handling is in place.The custom exception handler callable must support a function signature as specified in theiter()/read()sections of theTableclass API reference.Working with SchemaA model of a schema with helpful methods for working with the schema and supported data. Schema instances can be initialized with a schema source as a url to a JSON file or a JSON object. The schema is initially validated (seevalidatebelow). By default validation errors will be stored inschema.errorsbut in a strict mode it will be instantly raised.Let's create a blank schema. It's not valid becausedescriptor.fieldsproperty is required by theTable Schemaspecification:schema=Schema()schema.valid# falseschema.errors# [<ValidationError: "'fields' is a required property">]To avoid creating a schema descriptor by hand we will use aschema.infermethod to infer the descriptor from given data:schema.infer([['id','age','name'],['1','39','Paul'],['2','23','Jimmy'],['3','36','Jane'],['4','28','Judy'],])schema.valid# trueschema.descriptor#{ fields:#   [ { name: 'id', type: 'integer', format: 'default' },#     { name: 'age', type: 'integer', format: 'default' },#     { name: 'name', type: 'string', format: 'default' } ],#  missingValues: [ '' ] }Now we have an inferred schema and it's valid. We can cast data rows against our schema. We provide a string input which will be cast correspondingly:schema.cast_row(['5','66','Sam'])# [ 5, 66, 'Sam' ]But if we try provide some missing value to theagefield, the cast will fail because the only valid "missing" value is an empty string. Let's update our schema:schema.cast_row(['6','N/A','Walt'])# Cast errorschema.descriptor['missingValues']=['','N/A']schema.commit()schema.cast_row(['6','N/A','Walt'])# [ 6, None, 'Walt' ]We can save the schema to a local file, and resume work on it at any time by loading it from that file:schema.save('schema.json')schema=Schema('schema.json')Working with FieldfromtableschemaimportField# Init fieldfield=Field({'name':'name','type':'number'})# Cast a valuefield.cast_value('12345')# -> 12345Data values can be cast to native Python objects with a Field instance. Type instances can be initialized withfield descriptors. This allows formats and constraints to be defined.Casting a value will check the value is of the expected type, is in the correct format, and complies with any constraints imposed by a schema. E.g. a date value (in ISO 8601 format) can be cast with a DateType instance. Values that can't be cast will raise anInvalidCastErrorexception.Casting a value that doesn't meet the constraints will raise aConstraintErrorexception.API Referenceclicli()Command-line interfaceUsage: tableschema [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  infer     Infer a schema from data.
  info      Return info on this version of Table Schema
  validate  Validate that a supposed schema is in fact a Table Schema.TableTable(self,source,schema=None,strict=False,post_cast=[],storage=None,**options)Table representationArgumentssource (str/list[]): data source one of:local file (path)remote file (url)array of arrays representing the rowsschema (any): data schema in all forms supported bySchemaclassstrict (bool): strictness option to pass toSchemaconstructorpost_cast (function[]): list of post cast processorsstorage (None): storage name likesqlorbigqueryoptions (dict):tabulatoror storage's optionsRaisesTableSchemaException: raises on any errortable.hashTable's SHA256 hash if it's available.If it's already read using e.g.table.read, otherwise returnsNone.
In the middle of an iteration it returns hash of already read contentsReturnsstr/None: SHA256 hashtable.headersTable's headers is availableReturnsstr[]: headerstable.schemaReturns schema class instance if availableReturnsSchema: schematable.sizeTable's size in BYTES if it's availableIf it's already read using e.g.table.read, otherwise returnsNone.
In the middle of an iteration it returns size of already read contentsReturnsint/None: size in BYTEStable.itertable.iter(keyed=False,extended=False,cast=True,integrity=False,relations=False,foreign_keys_values=False,exc_handler=None)Iterates through the table data and emits rows cast based on table schema.Argumentskeyed (bool):
    yield keyed rows in a form of `{header1: value1, header2: value2}`
    (default is false; the form of rows is `[value1, value2]`)

extended (bool):
    yield extended rows in a for of `[rowNumber, [header1, header2], [value1, value2]]`
    (default is false; the form of rows is `[value1, value2]`)

cast (bool):
    disable data casting if false
    (default is true)

integrity (dict):
    dictionary in a form of `{'size': <bytes>, 'hash': '<sha256>'}`
    to check integrity of the table when it's read completely.
    Both keys are optional.

relations (dict):
    dictionary of foreign key references in a form
    of `{resource1: [{field1: value1, field2: value2}, ...], ...}`.
    If provided, foreign key fields will checked and resolved
    to one of their references (/!\ one-to-many fk are not completely resolved).

foreign_keys_values (dict):
    three-level dictionary of foreign key references optimized
    to speed up validation process in a form of
    `{resource1: {(fk_field1, fk_field2): {(value1, value2): {one_keyedrow}, ... }}}`.
    If not provided but relations is true, it will be created
    before the validation process by *index_foreign_keys_values* method

exc_handler (func):
    optional custom exception handler callable.
    Can be used to defer raising errors (i.e. "fail late"), e.g.
    for data validation purposes. Must support the signature belowCustom exception handlerdefexc_handler(exc,row_number=None,row_data=None,error_data=None):'''Custom exception handler (example)# Arguments:exc(Exception):Deferred exception instancerow_number(int):Data row number that triggers exception excrow_data(OrderedDict):Invalid data row source dataerror_data(OrderedDict):Data row source data field subset responsible for the error, ifapplicable (e.g. invalid primary or foreign key fields). May beidentical to row_data.'''# ...RaisesTableSchemaException: base class of any errorCastError: data cast errorIntegrityError: integrity checking errorUniqueKeyError: unique key constraint violationUnresolvedFKError: unresolved foreign key reference errorReturnsIterator[list]: yields rowstable.readtable.read(keyed=False,extended=False,cast=True,limit=None,integrity=False,relations=False,foreign_keys_values=False,exc_handler=None)Read the whole table and return as array of rowsIt has the same API astable.iterexcept forArgumentslimit (int): limit count of rows to read and returnReturnslist[]: returns rowstable.infertable.infer(limit=100,confidence=0.75,missing_values=[''],guesser_cls=None,resolver_cls=None)Infer a schema for the table.It will infer and set Table Schema totable.schemabased on table data.Argumentslimit (int): limit rows sample sizeconfidence (float): how many casting errors are allowed (as a ratio, between 0 and 1)missing_values (str[]): list of missing values (by default[''])guesser_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]resolver_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]Returnsdict: Table Schema descriptortable.savetable.save(target,storage=None,**options)Save data source to file locally in CSV format with,(comma) delimiterTo save schema usetable.schema.save()Argumentstarget (str): saving target (e.g. file path)storage (None/str): storage name likesqlorbigqueryoptions (dict):tabulatoror storage optionsRaisesTableSchemaException: raises an error if there is saving problemReturnsTrue/Storage: returns true or storage instancetable.index_foreign_keys_valuestable.index_foreign_keys_values(relations)Creates a three-level dictionary of foreign key referencesWe create them optimized to speed up validation process in a form of{resource1: {(fk_field1, fk_field2): {(value1, value2): {one_keyedrow}, ... }}}.For each foreign key of the schema it will iterate through the correspondingrelations['resource']to create an index (i.e. a dict) of existing values
for the foreign fields and store on keyed row for each value combination.The optimization relies on the indexation of possible values for one foreign key
in a hashmap to later speed up resolution.This method is public to allow creating the index once to apply it
on multiple tables charing the same schema
(typicallygrouped resources in datapackage)Notesthe second key of the output is a tuple of the foreign fields,
a proxy identifier of the foreign keythe same relation resource can be indexed multiple times
as a schema can contain more than one Foreign Keys
pointing to the same resourceArgumentsrelations (dict):
dict of foreign key references in a form of{resource1: [{field1: value1, field2: value2}, ...], ...}.
It must contain all resources pointed in the foreign keys schema definition.Returnsdict:
returns a three-level dictionary of foreign key references
optimized to speed up validation process in a form of{resource1: {(fk_field1, fk_field2): {(value1, value2): {one_keyedrow}, ... }}})SchemaSchema(self,descriptor={},strict=False)Schema representationArgumentsdescriptor (str/dict): schema descriptor one of:
- local path
- remote url
- dictionarystrict (bool): flag to specify validation behaviour:
- if false, errors will not be raised but instead collected inschema.errors- if true, validation errors are raised immediatelyRaisesTableSchemaException: raise any error that occurs during the processschema.descriptorSchema's descriptorReturnsdict: descriptorschema.errorsValidation errorsAlways empty in strict mode.ReturnsException[]: validation errorsschema.field_namesSchema's field namesReturnsstr[]: an array of field namesschema.fieldsSchema's fieldsReturnsField[]: an array of field instancesschema.foreign_keysSchema's foreign keysReturnsdict[]: foreign keysschema.headersSchema's field namesReturnsstr[]: an array of field namesschema.missing_valuesSchema's missing valuesReturnsstr[]: missing valuesschema.primary_keySchema's primary keysReturnsstr[]: primary keysschema.validValidation statusAlways true in strict mode.Returnsbool: validation statusschema.get_fieldschema.get_field(name)Get schema's field by name.Usetable.update_fieldif you want to modify the field descriptorArgumentsname (str): schema field nameReturnsField/None:Fieldinstance orNoneif not foundschema.get_fieldschema.get_field(name)Get schema's field by name.Usetable.update_fieldif you want to modify the field descriptorArgumentsname (str): schema field nameReturnsField/None:Fieldinstance orNoneif not foundschema.add_fieldschema.add_field(descriptor)Add new field to schema.The schema descriptor will be validated with newly added field descriptor.Argumentsdescriptor (dict): field descriptorRaisesTableSchemaException: raises any error that occurs during the processReturnsField/None: addedFieldinstance orNoneif not addedschema.update_fieldschema.update_field(name,update)Update existing descriptor field by nameArgumentsname (str): schema field nameupdate (dict): update to apply to field's descriptorReturnsbool: true on success and false if no field is found to be modifiedschema.remove_fieldschema.remove_field(name)Remove field resource by name.The schema descriptor will be validated after field descriptor removal.Argumentsname (str): schema field nameRaisesTableSchemaException: raises any error that occurs during the processReturnsField/None: removedFieldinstances orNoneif not foundschema.cast_rowschema.cast_row(row,fail_fast=False,row_number=None,exc_handler=None)Cast row based on field types and formats.Argumentsrow (any[]: data row as an array of valuesReturnsany[]: returns cast data rowschema.inferschema.infer(rows,headers=1,confidence=0.75,guesser_cls=None,resolver_cls=None)Infer and setschema.descriptorbased on data sample.Argumentsrows (list[]): array of arrays representing rows.headers (int/str[]): data sample headers (one of):
- row number containing headers (rowsshould contain headers rows)
- array of headers (rowsshould NOT contain headers rows)confidence (float): how many casting errors are allowed (as a ratio, between 0 and 1)guesser_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]resolver_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]Returnsdict: Table Schema descriptorschema.commitschema.commit(strict=None)Update schema instance if there are in-place changes in the descriptor.ExamplefromtableschemaimportSchemadescriptor={'fields':[{'name':'my_field','title':'My Field','type':'string'}]}schema=Schema(descriptor)print(schema.get_field('my_field').descriptor['type'])# string# Update descriptor by field positionschema.descriptor['fields'][0]['type']='number'# Update descriptor by field nameschema.update_field('my_field',{'title':'My Pretty Field'})# True# Change are not committedprint(schema.get_field('my_field').descriptor['type'])# stringprint(schema.get_field('my_field').descriptor['title'])# My Field# Commit changeschema.commit()print(schema.get_field('my_field').descriptor['type'])# numberprint(schema.get_field('my_field').descriptor['title'])# My Pretty FieldArgumentsstrict (bool): alterstrictmode for further workRaisesTableSchemaException: raises any error that occurs during the processReturnsbool: true on success and false if not modifiedschema.saveschema.save(target,ensure_ascii=True)Save schema descriptor to target destination.Argumentstarget (str): path where to save a descriptorRaisesTableSchemaException: raises any error that occurs during the processReturnsbool: true on successFieldField(self,descriptor,missing_values=[''],schema=None)Field representaionArgumentsdescriptor (dict): schema field descriptormissingValues (str[]): an array with string representing missing valuesRaisesTableSchemaException: raises any error that occurs during the processfield.constraintsField constraintsReturnsdict: dict of field constraintsfield.descriptorFields's descriptorReturnsdict: descriptorfield.formatField formatReturnsstr: field formatfield.missing_valuesField's missing valuesReturnsstr[]: missing valuesfield.nameField nameReturnsstr: field namefield.requiredWhether field is requiredReturnsbool: true if requiredfield.schemaReturns a schema instance if the field belongs to some schemaReturnsSchema: field's schemafield.typeField typeReturnsstr: field typefield.cast_valuefield.cast_value(value,constraints=True)Cast given value according to the field type and format.Argumentsvalue (any): value to cast against fieldconstraints (boll/str[]): gets constraints configuration
- it could be set to true to disable constraint checks
- it could be an Array of constraints to check e.g. ['minimum', 'maximum']RaisesTableSchemaException: raises any error that occurs during the processReturnsany: returns cast valuefield.test_valuefield.test_value(value,constraints=True)Test whether value is compliant to the field.Argumentsvalue (any): value to cast against fieldconstraints (bool/str[]): constraints configurationReturnsbool: returns if value is compliant to the fieldStorageStorage(self,**options)Storage factory/interfaceFor usersUseStorage.connectto instantiate a storageFor instantiation of concrete storage instances,tableschema.Storageprovides a unified factory methodconnect(which uses the plugin system under the hood):# pip install tableschema_sqlfromtableschemaimportStoragestorage=Storage.connect('sql',**options)storage.create('bucket',descriptor)storage.write('bucket',rows)storage.read('bucket')For integratorsThe library includes interface declaration to implement tabularStorage.
This interface allow to use different data storage systems like SQL
withtableschema.Tableclass (load/save) as well as on the data package level:An implementor must followtableschema.Storageinterface
to write his own storage backend. Concrete storage backends
could include additional functionality specific to conrete storage system.
Seepluginsbelow to know how to integrate custom storage plugin into your workflow.storage.bucketsReturn list of storage bucket names.Abucketis a special term which has almost the same meaning astable.
You should considerbucketas atablestored in thestorage.Raisesexceptions.StorageError: raises on any errorReturnsstr[]: return list of bucket namesstorage.connectstorage.connect(name,**options)Create tabularstoragebased on storage name.This method is statis:Storage.connect()Argumentsname (str): storage name likesqloptions (dict): concrete storage optionsRaisesStorageError: raises on any errorReturnsStorage: returnsStorageinstancestorage.createstorage.create(bucket,descriptor,force=False)Create one/multiple buckets.Argumentsbucket (str/list): bucket name or list of bucket namesdescriptor (dict/dict[]): schema descriptor or list of descriptorsforce (bool): whether to delete and re-create already existing bucketsRaisesexceptions.StorageError: raises on any errorstorage.deletestorage.delete(bucket=None,ignore=False)Delete one/multiple/all buckets.Argumentsbucket (str/list/None): bucket name or list of bucket names to delete.
IfNone, all buckets will be deleteddescriptor (dict/dict[]): schema descriptor or list of descriptorsignore (bool): don't raise an error on non-existent bucket deletionRaisesexceptions.StorageError: raises on any errorstorage.describestorage.describe(bucket,descriptor=None)Get/set bucket's Table Schema descriptorArgumentsbucket (str): bucket namedescriptor (dict/None): schema descriptor to setRaisesexceptions.StorageError: raises on any errorReturnsdict: returns Table Schema descriptorstorage.iterstorage.iter(bucket)Return an iterator of typed values based on the schema of this bucket.Argumentsbucket (str): bucket nameRaisesexceptions.StorageError: raises on any errorReturnslist[]: yields data rowsstorage.readstorage.read(bucket)Read typed values based on the schema of this bucket.Argumentsbucket (str): bucket nameRaisesexceptions.StorageError: raises on any errorReturnslist[]: returns data rowsstorage.writestorage.write(bucket,rows)This method writes data rows intostorage.It should store values of unsupported types as strings internally (like csv does).Argumentsbucket (str): bucket namerows (list[]): data rows to writeRaisesexceptions.StorageError: raises on any errorvalidatevalidate(descriptor)Validate descriptorArgumentsdict: descriptorRaisesValidationError: on validation errorsReturnsbool: Trueinferinfer(source,headers=1,limit=100,confidence=0.75,missing_values=[''],guesser_cls=None,resolver_cls=None,**options)Infer source schema.Argumentssource (any): source as path, url or inline dataheaders (int/str[]): headers rows number or headers listconfidence (float): how many casting errors are allowed (as a ratio, between 0 and 1)missing_values (str[]): list of missing values (by default[''])guesser_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]resolver_cls (class): you can implement inferring strategies by
providing type-guessing and type-resolving classes [experimental]RaisesTableSchemaException: raises any error that occurs during the processReturnsdict: returns schema descriptorFailedCastFailedCast(self,value)Wrap an original data field value that failed to be properly casted.FailedCast allows for further processing/yielding values but still be able
to distinguish uncasted values on the consuming side.Delegates attribute access and the basic rich comparison methods to the
underlying object. Supports default user-defined classes hashability i.e.
is hashable based on object identity (not based on the wrapped value).Argumentsvalue (any): valueDataPackageExceptionDataPackageException(self,message,errors=[])Base class for all DataPackage/TableSchema exceptions.If there are multiple errors, they can be read from the exception object:try:# lib actionexceptDataPackageExceptionasexception:ifexception.multiple:forerrorinexception.errors:# handle errordatapackageexception.errorsList of nested errorsReturnsDataPackageException[]: list of nested errorsdatapackageexception.multipleWhether it's a nested exceptionReturnsbool: whether it's a nested exceptionTableSchemaExceptionTableSchemaException(self,message,errors=[])Base class for all TableSchema exceptions.LoadErrorLoadError(self,message,errors=[])All loading errors.ValidationErrorValidationError(self,message,errors=[])All validation errors.CastErrorCastError(self,message,errors=[])All value cast errors.IntegrityErrorIntegrityError(self,message,errors=[])All integrity errors.UniqueKeyErrorUniqueKeyError(self,message,errors=[])Unique key constraint violation (CastError subclass)RelationErrorRelationError(self,message,errors=[])All relations errors.UnresolvedFKErrorUnresolvedFKError(self,message,errors=[])Unresolved foreign key reference error (RelationError subclass).StorageErrorStorageError(self,message,errors=[])All storage errors.ExperimentalThis API is experimental and can be changed/removed in the futureThere is an experimental environment variableTABLESCHEMA_PRESERVE_MISSING_VALUESwhich, if it is set, affects how data casting works.By default, missing values are resolved toNonevalues. When this flag is set, missing values are passed through as it is. For example:missing_values.pyfromtableschemaimportFieldfield=Field({'type':'number'},missing_values=['-'])print(field.cast_value('3'))print(field.cast_value('-'))Running this script in different modes:$pythonmissing_values.py3None
$TABLESCHEMA_PRESERVE_MISSING_VALUES=1pythonmissing_values.py3-The flags affects all the library's APIs and software built on top oftableschema. For example, Data Package Pipelines:$TABLESCHEMA_PRESERVE_MISSING_VALUES=1dpprun./my_pipelineContributingThe project follows theOpen Knowledge International coding standards.Recommended way to get started is to create and activate a project virtual environment.
To install package and development dependencies into active environment:$makeinstallTo run tests with linting and coverage:$maketestChangelogHere described only breaking and the most important changes. The full changelog and documentation for all released versions can be found in the nicely formattedcommit history.v1.20Added --json flag to the CLI (#287)v1.19Deduplicate field names if guessing in inferv1.18Publishfield.ERROR/cast_function/check_functionsv1.17Addedschema.missing_valuesandfield.missing_valuesv1.16Fixed the way we parsegeopoint:as a string it can be in 3 forms ("default", "array", "object") BUTas native object it can only be a list/tuplev1.15Added an experimentalTABLESCHEMA_PRESERVE_MISSING_VALUESenvironment variable flagv1.14Allow providing custom guesser and resolver totable.inferandinferv1.13Addedmissing_valuesargument to theinferfunction (#269)v1.12Support optional custom exception handling for table.iter/read (#259)v1.11Addedpreserve_missing_valuesparameter tofield.cast_valuev1.10Added an ability to check table's integrity while readingv1.9Implemented thetable.sizeandtable.hashpropertiesv1.8Addedtable.index_foreign_keys_valuesand improved foreign key checks performancev1.7Addedfield.schemapropertyv1.6Instrictmode raise an exception if there are problems in field constructionv1.5Allow providing custom guesser and resolver to schema inferv1.4Addedschema.update_fieldmethodv1.3Support datetime with no time for date castingv1.2Support floats like 1.0 for integer castingv1.1Added theconfidenceparameter toinferv1.0The library has been rebased on the Frictionless Data specs v1 -https://frictionlessdata.io/specs/table-schema/