Package: schema-enforcer
Schema EnforcerSchema Enforcer provides a framework for testing structured data against schema definitions usingJSONSchema.Getting StartedInstallSchema Enforcer is a python library which is available on PyPi. It requires a python version of 3.8 or greater. Once a supported version of python is installed on your machine, pip can be used to install the tool by using the commandpython -m pip install schema-enforcer.python -m pip install schema-enforcerOverviewSchema Enforcer requires that two different elements be defined by the user:Schema Definition Files: These are files which define the schema to which a given set of data should adhere.Structured Data Files: These are files which contain data that should adhere to the schema defined in one (or multiple) of the schema definition files.Note: Data which needs to be validated against a schema definition can come in the form of Structured Data Files or Ansible host vars. Ansible is not installed by default when schema-enforcer is installed. In order to use Ansible features, ansible must already be available or must be declared as an optional dependency when schema-enforcer upon installation. In the interest of brevity and simplicity, this README.md contains discussion only of Structured Data Files -- for more information on how to useschema-enforcerwith ansible host vars, seethe ansible_command READMEWhenschema-enforcerruns, it assumes directory hierarchy which should be in place from the folder in which the tool is run.schema-enforcerwill search forschema definition filesnested inside of./schema/schemas/which end in.yml,.yaml, or.json.schema-enforcerwill do a recursive search forstructured data filesstarting in the current working diretory (./). It does this by searching all directories (including the current working directory) for files ending in.yml,.yaml, or.json. Theschemafolder and it's subdirectories are excluded from this search by default.bash$ cd examples/example1
bash$ tree
.
├── chi-beijing-rt1
│   ├── dns.yml
│   └── syslog.yml
├── eng-london-rt1
│   ├── dns.yml
│   └── ntp.yml
└── schema
    └── schemas
        ├── dns.yml
        ├── ntp.yml
        └── syslog.yml

4 directories, 7 filesIn the above example,chi-beijing-rt1is a directory with structured data files containing some configuration for a router namedchi-beijing-rt1. There are two structured data files inside of this folder,dns.ymlandsyslog.yml. Similarly, theeng-london-rt1directory contains definition files for a router namedeng-london-rt1--dns.ymlandntp.yml.The filechi-beijing-rt1/dns.ymldefines the DNS serverschi-beijing.rt1should use. The data in this file includes a simple hash-type data structure with a key ofdns_serversand a value of an array. Each element in this array is a hash-type object with a key ofaddressand a value which is the string of an IP address.bash$ cat chi-beijing-rt1/dns.yml# jsonschema:schemas/dns_servers---dns_servers:-address:"10.1.1.1"-address:"10.2.2.2"Note: The line# jsonschema: schemas/dns_serverstellsschema-enforcerthe ID of the schema which the structured data defined in the file should be validated against. The schema ID is defined by the$idtop level key in a schema definition. More information on how the structured data is mapped to a schema ID to which it should adhere can be found in themapping_schemas READMEThe fileschema/schemas/dns.ymlis a schema definition file. It contains a schema definition for ntp servers written in JSONSchema. The data inchi-beijing-rt1/dns.ymlandeng-london-rt1/dns.ymlshould adhere to the schema defined in this schema definition file.bash$ cat schema/schemas/dns.yml---$schema:"http://json-schema.org/draft-07/schema#"$id:"schemas/dns_servers"description:"DNSServerConfigurationschema."type:"object"properties:dns_servers:type:"array"items:type:"object"properties:name:type:"string"address:type:"string"format:"ipv4"vrf:type:"string"required:-"address"uniqueItems:truerequired:-"dns_servers"Note: The cat of the schema definition file may be a little scary if you haven't seen JSONSchema before. Don't worry too much if it is difficult to parse right now. The important thing to note is that this file contains a schema definition to which the structured data in the fileschi-beijing-rt1/dns.ymlandeng-london-rt1/dns.ymlshould adhere.Basic usageOnce schema-enforcer has been installed, theschema-enforcer validatecommand can be used run schema validations of YAML/JSON instance files against the defined schema.bash$schema-enforcer--help
Usage:schema-enforcer[OPTIONS]COMMAND[ARGS]...

Options:--helpShowthismessageandexit.

Commands:ansibleValidatethehostvarforallhostswithinanAnsible...schemaManageyourschemasvalidateValidatesinstancefilesagainstdefinedschemaTo run the schema validations, the commandschema-enforcer validatecan be run.bash$schema-enforcervalidate
schema-enforcervalidate
ALLSCHEMAVALIDATIONCHECKSPASSEDTo acquire more context regarding what files specifically passed schema validation, the--show-passflag can be passed in.bash$schema-enforcervalidate--show-pass
PASS[FILE]./eng-london-rt1/ntp.yml
PASS[FILE]./eng-london-rt1/dns.yml
PASS[FILE]./chi-beijing-rt1/syslog.yml
PASS[FILE]./chi-beijing-rt1/dns.yml
ALLSCHEMAVALIDATIONCHECKSPASSEDIf we modify one of the addresses in thechi-beijing-rt1/dns.ymlfile so that it's value is the booleantrueinstead of an IP address string, then run theschema-enforcertool, the validation will fail with an error message.bash$ cat chi-beijing-rt1/dns.yml# jsonschema:schemas/dns_servers---dns_servers:-address:true-address:"10.2.2.2"bash$test-schemavalidate
FAIL|[ERROR]Trueisnotoftype'string'[FILE]./chi-beijing-rt1/dns.yml[PROPERTY]dns_servers:0:address
bash$echo$?1When a structured data file fails schema validation,schema-enforcerexits with a code of 1.Configuration SettingsSchema enforcer will work with default settings, however, apyproject.tomlfile can be placed at the root of the path in whichschema-enforceris run in order to override default settings or declare configuration for more advanced features. Inside of thispyproject.tomlfile,tool.schema_enforcersections can be used to declare settings for schema enforcer. Take for example thepyproject.tomlfile in example 2.bash$cdexamples/example2&&tree-L2.
├──README.md
├──hostvars
│├──chi-beijing-rt1
│├──eng-london-rt1
│└──ger-berlin-rt1
├──invalid
├──pyproject.toml
└──schema├──definitions└──schemas8directories,2filesIn this toml file, a schema mapping is declared which tells schema enforcer which structured data files should be checked by which schema IDs.bash$catpyproject.toml[tool.schema_enforcer.schema_mapping]# Map structured data filename to schema IDs'dns_v1.yml'=['schemas/dns_servers']'dns_v2.yml'=['schemas/dns_servers_v2']'syslog.yml'=['schemas/syslog_servers']More information on available configuration settings can be found in theconfiguration READMESupported FormatsBy default, schema enforcer installs the jsonschemaformat_nongplextra (in version <1.2.0) orformat-nongpl(in versions >=1.2.0). This extra allows the use of formats that can be used in schema definitions (e.g. ipv4, hostname...etc). Theformat_nongplorformat-nongplextra only installs transitive dependencies that are not licensed under GPL. Theiriandiri-referenceformats are defined by therfc3987transitive dependency which is licensed under GPL. As such,iriandiri-referenceformats arenotsupported byformat-nongpl/format_nongpl. If you have a need to useiriand/oriri-referenceformats, you can do so by running the following pip command (or it's poetry equivalent):pip install 'jsonschema[rfc3987]'See the "Validating Formats" section in thejsonschema documentationfor more information.Where To Go NextDetailed documentation can be found in the README.md files inside of thedocs/directory."Introducing Schema Enforcer" blog postUsing a pyproject.toml file for configurationMapping Structured Data Files to Schema FilesTheansiblecommandThevalidatecommandTheschemacommandImplementing custom validators