Package: django-data-schema
Django Data SchemaDjango data schema is a lightweight Django app for defining the schema for a model, dictionary, or list.
By describing a schema on a piece of data, this allows other applications to easily reference
fields of models or fields in dictionaries (or their related json fields).Django data schema also takes care of all conversions under the hood, such as parsing datetime strings, converting strings to numeric values, using default values when values don't exist, and so on.InstallationModel OverviewExamplesInstallationpipinstalldjango-data-schemaModel OverviewDjango data schema defines three models for building schemas on data. These models areDataSchema,FieldSchema, andFieldOptional.TheDataSchemamodel provides amodel_content_typefield that points to a DjangoContentTypemodel.
This field represents which object this schema is modeling. If the field is None, it is assumed that
this schema models an object such as a dictionary or list.After the enclosingDataSchemahas been defined, variousFieldSchemamodels can reference the main
data schema.FieldSchemamodels provide the following attributes:field_key: The name of the field. Used to identify a field in a dictionary or model.field_position: The position of the field. Used to identify a field in a list.uniqueness_order: The order of this field in the uniqueness constraint of the schema. Defaults to None.field_type: The type of field. More on the field types below.field_format: An optional formatting string for the field. Used differently depending on the field type and documented more below.default_value: If the field returns None, this default value will be returned instead.AFieldSchemaobject must specify its data type. While data of a given type can be stored in different formats,
django-data-schema normalizes the data when accessing it throughget_value, described below. The available
types are listed in theFieldSchemaTypeclass. These types are listed here, with the type they normalize to:FieldSchemaType.DATE: A pythondateobject from thedatetimemodule. Currently returned as adatetimeobject.FieldSchemaType.DATETIME: A pythondatetimeobject from thedatetimemodule.FieldSchemaType.INT: A pythonint.FieldSchemaType.FLOAT: A pythonfloat.FieldSchemaType.STRING: A pythonstr.FieldSchemaType.BOOLEAN: A pythonbool.These fields provide the necessary conversion mechanisms when accessing data viaFieldSchema.get_value. Differences in how theget_valuefunction operates are detailed below.Using get_value on DATE or DATETIME fieldsTheget_valuefunction has the following behavior on DATE and DATETIME fields:If called on a Pythonintorfloatvalue, the numeric value will be passed to thedatetime.utcfromtimestampfunction.If called on astringorunicodevalue, the string will be stripped of all trailing and leading whitespace. If the string is empty, the default value (or None) will be used. If the string is not empty, it will be passed to dateutil'sparsefunction. If thefield_formatfield is specified on theFieldSchemaobject, it will be passed to thestrptimefunction instead.If called on an aware datetime object (or a string with a timezone), it will be converted to naive UTC time.If called on None, the default value (or None) is returned.Using get_value on INT or FLOAT fieldsTheget_valuefunction has the following behavior on INT and FLOAT fields:If called on astringorunicodevalue, the string will be stripped of all non-numeric numbers except for periods. If the string is blank, the default value (or None) will be returned. If not, the string will then be passed toint()orfloat().If called on anintorfloat, the value will be passed to theint()orfloat()function.No other values can be converted. Thefield_formatparameter is ignored.If called on None, the default value (or None) is returned.Using get_value on a STRING fieldTheget_valuefunction has the following behavior on a STRING field:If called on astringorunicodevalue, the string will be stripped of all trailing and leading whitespace. If afield_formatis specified, the string is then be matched to the regex. If it passes, the string is returned. If not, None is returned and the default value is used (or None).All other types are passed to thestr()function.If called on None, the default value (or None) is returned.Using get_value on a BOOLEAN fieldTheget_valuefunction has the following behavior on a BOOLEAN field:Bool data types will return True or FalseTruthy looking string values return True ('t', 'T', 'true', 'True', 'TRUE', 1, '1')Falsy looking string values return False ('f', 'F', 'false', 'False', 'FALSE', 0, '0')If called on None, the default value (or None) is returned.ExamplesA data schema can be created like the following:fromdata_schemaimportDataSchema,FieldSchema,FieldSchemaTypeuser_login_schema=DataSchema.objects.create()user_id_field=FieldSchema.objects.create(data_schema=user_login_schema,field_key='user_id',uniqueness_order=1,field_type=FieldSchemaType.STRING)login_time_field=FieldSchema.objects.create(data_schema=user_login_schema,field_key='login_time',field_type=FieldSchemaType.DATETIME)The above example represents the schema of a user login. In this schema, the user id field provides the uniqueness
constraint of the data. The uniqueness constraint can then easily be accessed by simply doing the following.unique_fields=user_login_schema.get_unique_fields()The above function returns the unique fields in the order in which they were specified, allowing the user to
generate a unique ID for the data.To obtain values of data using the schema, one can use theget_valuefunction as follows:data={'user_id':'my_user_id','login_time':1396396800,}printlogin_time_field.get_value(data)2014-04-0200:00:00Note that theget_valuefunction looks at the type of data object and uses the proper access method. If the
data object is adict, it accesses it usingdata[field_key]. If it is an object, it accesses it withgetattr(data, field_key). An array is accessed asdata[field_position].Here's another example of parsing datetime objects in an array with a format string.string_time_field_schema=FieldSchema.objects.create(data_schema=data_schema,field_key='time',field_position=1,field_type=FieldSchemaType.DATETIME,field_format='%Y-%m-%d%H:%M:%S')printstring_time_field_schema.get_value(['value','2013-04-12 12:12:12'])2013-04-1212:12:12Note that if you are parsing numerical fields, Django data schema will strip out any non-numerical values, allowing the user to get values of currency-based numbers and other formats.revenue_field_schema=FieldSchema.objects.create(data_schema=data_schema,field_key='revenue',field_type=FieldSchemaType.FLOAT)printrevenue_field_schema.get_value({'revenue':'$15,000,456.23'})15000456.23Note thatFieldSchemaobjects have an analogousset_valuemethod for setting the value of a field.
Theset_valuemethod does not do any data conversions, so when calling this method, be sure to use a value
that is in the correct format.