Package: hepdata-validator
JSON schema and validation code (in Python 3) for HEPData submissionsDocumentation:https://hepdata-validator.readthedocs.ioInstallationIf you can, installLibYAML(a C library for parsing and emitting YAML) on your machine.
This will allow for the use ofCSafeLoader(instead of PythonSafeLoader) for faster loading of YAML files.
Not a big deal for small files, but performs markedly better on larger documents.Install fromPyPIusingpip:$pipinstall--userhepdata-validator$hepdata-validate--helpIf you would like to use LibYAML, you may need an additional step if running on an M1 Mac, to ensure pyyaml is built
with the LibYAML bindings. Run the following after installing LibYAML via Homebrew:$LDFLAGS="-L$(brew--prefix)/lib"CFLAGS="-I$(brew--prefix)/include"pipinstall--global-option="--with-libyaml"--forcepyyamlDevelopersDevelopers should install from GitHub in avirtual environment:$gitclonehttps://github.com/HEPData/hepdata-validator$cdhepdata-validator$python3.9-mvenvvenv$sourcevenv/bin/activate(venv)$pipinstall--upgradepip(venv)$pipinstall--upgrade-e".[all]"Tests should be run both with and without LibYAML, as error messages from the different YAML parsers vary:(venv)$USE_LIBYAML=Truepytesttestsuite(venv)$USE_LIBYAML=FalsepytesttestsuiteUsageThehepdata-validatorpackage allows you to validate (via the command line or Python):A full directory of submission and data filesAn archive file (.zip, .tar, .tar.gz, .tgz) containing all of the files (full details)Asingle .yaml or .yaml.gz file(butnotsubmission.yamlor a YAML data file)Asubmission.yamlfile or individual YAML data file (via Python only, not via the command line)The same package is used for validating uploads made tohepdata.net, therefore
first validating offline can be more efficient in checking your submission is valid before uploading.Command lineInstalling thehepdata-validatorpackage adds the commandhepdata-validateto your path, which allows you to validate aHEPData submissionoffline.ExamplesTo validate a submission comprising of multiple files in the current directory:$hepdata-validateTo validate a submission comprising of multiple files in another directory:$hepdata-validate-d../TestHEPSubmissionTo validate an archive file (.zip, .tar, .tar.gz, .tgz) in the current directory:$hepdata-validate-aTestHEPSubmission.zipTo validate a single YAML file in the current directory:$hepdata-validate-fsingle_yaml_file.yamlUsage options$hepdata-validate--helpUsage:hepdata-validate[OPTIONS]Offlinevalidationofsubmission.yamlandYAMLdatafiles.Cancheckeitheradirectory,anarchivefile,orthesingleYAMLfileformat.Options:-d,--directoryTEXTDirectorytocheck(defaultstocurrentworkingdirectory)-f,--fileTEXTSingle.yamlor.yaml.gzfile(butnotsubmission.yamloraYAMLdatafile)tocheck-seehttps://hepdata-submission.readthedocs.io/en/latest/single_yaml.html.(Overridesdirectory)-a,--archiveTEXTArchivefile(.zip,.tar,.tar.gz,.tgz)tocheck.(Overridesdirectoryandfile)--helpShowthismessageandexit.PythonValidating a full submissionTo validate a full submission, instantiate aFullSubmissionValidatorobject:fromhepdata_validator.full_submission_validatorimportFullSubmissionValidator,SchemaTypefull_submission_validator=FullSubmissionValidator()# validate a directoryis_dir_valid=full_submission_validator.validate(directory='TestHEPSubmission')# or uncomment to validate an archive file# is_archive_valid = full_submission_validator.validate(archive='TestHEPSubmission.zip')# or uncomment to validate a single file# is_file_valid = full_submission_validator.validate(file='single_yaml_file.yaml')# if there are any error messages, they are retrievable through this callfull_submission_validator.get_messages()# the error messages can be printed for each filefull_submission_validator.print_errors('submission.yaml')# the list of valid files can be retrieved via the valid_files property, which is a# dict mapping SchemaType (e.g. SUBMISSION, DATA, SINGLE_YAML, REMOTE) to lists of# valid filesfull_submission_validator.valid_files[SchemaType.SUBMISSION]full_submission_validator.valid_files[SchemaType.DATA]# full_submission_validator.valid_files[SchemaType.SINGLE_YAML]# if a remote schema is used, valid_files is a list of tuples (schema, file)# full_submission_validator.valid_files[SchemaType.REMOTE]# the list of valid files can be printedfull_submission_validator.print_valid_files()Validating individual filesTo validate submission files, instantiate aSubmissionFileValidatorobject:fromhepdata_validator.submission_file_validatorimportSubmissionFileValidatorsubmission_file_validator=SubmissionFileValidator()submission_file_path='submission.yaml'# the validate method takes a string representing the file pathis_valid_submission_file=submission_file_validator.validate(file_path=submission_file_path)# if there are any error messages, they are retrievable through this callsubmission_file_validator.get_messages()# the error messages can be printedsubmission_file_validator.print_errors(submission_file_path)To validate data files, instantiate aDataFileValidatorobject:fromhepdata_validator.data_file_validatorimportDataFileValidatordata_file_validator=DataFileValidator()# the validate method takes a string representing the file pathdata_file_validator.validate(file_path='data.yaml')# if there are any error messages, they are retrievable through this calldata_file_validator.get_messages()# the error messages can be printeddata_file_validator.print_errors('data.yaml')Optionally, if you have already loaded the YAML object, then you can pass it through
as adataobject. You must also pass through thefile_pathsince this is used as a key
for the error message lookup map.fromhepdata_validator.data_file_validatorimportDataFileValidatorimportyamlfile_contents=yaml.safe_load(open('data.yaml','r'))data_file_validator=DataFileValidator()data_file_validator.validate(file_path='data.yaml',data=file_contents)data_file_validator.get_messages('data.yaml')data_file_validator.print_errors('data.yaml')For the analogous case of theSubmissionFileValidator:fromhepdata_validator.submission_file_validatorimportSubmissionFileValidatorimportyamlsubmission_file_path='submission.yaml'# convert a generator returned by yaml.safe_load_all into a listdocs=list(yaml.safe_load_all(open(submission_file_path,'r')))submission_file_validator=SubmissionFileValidator()is_valid_submission_file=submission_file_validator.validate(file_path=submission_file_path,data=docs)submission_file_validator.print_errors(submission_file_path)Schema VersionsWhen consideringnative HEPData JSON schemas, there are multipleversions.
In most cases you should use thelatestversion (the default). If you need to use a different version,
you can pass a keyword argumentschema_versionwhen initialising the validator:submission_file_validator=SubmissionFileValidator(schema_version='0.1.0')data_file_validator=DataFileValidator(schema_version='0.1.0')Remote SchemasWhen usingremotely defined schemas, versions depend on the organization providing those schemas,
and it is their responsibility to offer a way of keeping track of different schema versions.TheJsonSchemaResolverobject resolves$refin the JSON schema. TheHTTPSchemaDownloaderobject retrieves
schemas from a remote location, and optionally saves them in the local file system, following the structure:schemas_remote/<org>/<project>/<version>/<schema_name>. An example may be:fromhepdata_validator.data_file_validatorimportDataFileValidatordata_validator=DataFileValidator()# Split remote schema path and schema nameschema_path='https://scikit-hep.org/pyhf/schemas/1.0.0/'schema_name='workspace.json'# Create JsonSchemaResolver object to resolve $ref in JSON schemafromhepdata_validator.schema_resolverimportJsonSchemaResolverpyhf_resolver=JsonSchemaResolver(schema_path)# Create HTTPSchemaDownloader object to validate against remote schemafromhepdata_validator.schema_downloaderimportHTTPSchemaDownloaderpyhf_downloader=HTTPSchemaDownloader(pyhf_resolver,schema_path)# Retrieve and save the remote schema in the local pathpyhf_type=pyhf_downloader.get_schema_type(schema_name)pyhf_spec=pyhf_downloader.get_schema_spec(schema_name)pyhf_downloader.save_locally(schema_name,pyhf_spec)# Load the custom schema as a custom typeimportospyhf_path=os.path.join(pyhf_downloader.schemas_path,schema_name)data_validator.load_custom_schema(pyhf_type,pyhf_path)# Validate a specific schema instancedata_validator.validate(file_path='pyhf_workspace.json',file_type=pyhf_type)The native HEPData JSON schema are provided as part of thehepdata-validatorpackage and it is not necessary to
download them. However, in principle, for testing purposes, note that the same mechanism above could be used with:schema_path='https://hepdata.net/submission/schemas/1.1.1/'schema_name='data_schema.json'and passing a HEPData YAML data file as thefile_pathargument of thevalidatemethod.