fireworks-schema 1.4.2
JSON Schema for FireWorks
License: BSD-3-Clause
Requires Python: >=3.6
Keywords: workflow system, json schema, fireworks

# JSON Schema for FireWorks
This package provides a [JSON schema](https://json-schema.org/) for
the [FireWorks](https://github.com/materialsproject/fireworks) package.

## Why should I use JSON schema?

The input for FireWorks is often provided in JSON or YAML formats and generated by
third-party software that is unaware of the valid data types in FireWorks. Latent
mismatches of data types may produce run-time errors, such as missing keywords
or wrong data types, that are more difficult to handle than a validation of the
initial input.

The *fireworks_schema* package provides a formal human- and machine-readable description of
the data types used in classes in FireWorks. Additionally, a function is provided
that checks the validity of JSON and YAML inputs immediately before deserialization.


## Installing *fireworks_schema*

The recommended way to install this package into your virtual environment is
using ``pip`` (in any folder):

```
python -m pip install fireworks-schema
```

Alternatively you can download a release from the GitHub
[repository](https://github.com/ikondov/fireworks_schema), unpack the archive,
change into the top-level folder containing ``setup.cfg`` and run:

```
python -m pip install .[test]
```

After the installation you can run the tests (from the top-level folder):

```
cd fireworks_schema/tests && pytest
```


## Using *fireworks_schema* to validate input for FireWorks

There are two ways to perform JSON schema validation:

* Call the schema validator explicitly
* Activate automatic schema validation


### Call the schema validator explicitly

This is the case when you use Python but read JSON/YAML serialized objects
provided externally. In the following example, a serialized workflow object is
loaded from a YAML file and validated against the Workflow schema:

```python
  import yaml
  import fireworks_schema
  from fireworks import Workflow

  with open('empty_fws.yaml', 'rt') as yf:
      dct = yaml.safe_load(yf)
  fireworks_schema.validate(dct, 'Workflow')
  wf = Workflow.from_dict(dct)
```

### Activate automatic schema validation

To activate automatic schema validation you must specify:

```yaml
  JSON_SCHEMA_VALIDATE: true
```

in your FWConfig file. For more details about managing your FWConfig file see the
[FW Config tutorial](https://materialsproject.github.io/fireworks/config_tutorial.html).

The default value of ``JSON_SCHEMA_VALIDATE`` is ``false``.

If automatic validation is turned on, i.e. ``JSON_SCHEMA_VALIDATE`` is ``true``,
then validation is performed only for the classes specified in the list
``JSON_SCHEMA_VALIDATE_LIST``, whenever an object of these
classes is loaded from file or from string.
There is no default for ``JSON_SCHEMA_VALIDATE_LIST``
and therefore you must set ``JSON_SCHEMA_VALIDATE_LIST`` in your FWConfig file.
For example, to turn on automatic validation for serialized ``Firework`` and
``Workflow`` objects these two lines must be added to the FWConfig file:

```yaml
  JSON_SCHEMA_VALIDATE: true
  JSON_SCHEMA_VALIDATE_LIST: [Firework, Workflow]
```

FireWorks performs automatic validation only when `from_file()` or `from_format()` methods
of the listed classes are called, i.e. only on reading from files and strings. This implies
that the utility function `load_object_from_file` also performs validation. Otherwise FireWorks
will not perform validation in scenarios such as dealing with documents from/to a database
or from/to a URI. To validate after having read a document from a database / URI and before
deserializing one should use `fw_schema_deserialize` to decorate the `from_dict()` method.
To validate after serialization and before writing to a database / URI one should decorate
the `to_dict()` method with `fw_schema_serialize`, e.g.:

```python
from fireworks_schema import fw_schema_deserialize, fw_schema_serialize

class MyClass(FWSerializable):
    ...

    @classmethod
    @fw_schema_deserialize
    def from_dict(cls, dct):
        ...
        return cls(...)

    @fw_schema_serialize
    def to_dict(self):
        dct = ...
        return dct
```

Note that the decorators honor the `JSON_SCHEMA_VALIDATE` setting, which is `False`
(default) or `True`, but not the `JSON_SCHEMA_VALIDATE_LIST`.


### Exception handling

When a custom schema is not valid then `SchemaError` is raised. When an instance
does not validate against a (valid) schema then `ValidationError` is raised. The
local schema and instance are displayed after the error description. Nevertheless,
the schema and the instance with which the validator has been called are not
displayed. These are known if the schema validator is called explicitly, i.e. by
calling `fireworks_schema.validate(instance, schema_name)`. But when automatic
validation is activated (either by the `JSON_SCHEMA_VALIDATE_LIST` or the decorator
approaches outlined above) then these are hard to detect. For this purpose, the
decorator accepts an optional `debug` parameter (`False` by default). By setting it
to `True` the schema name and the instance are concatenated to the original message
of the raised `ValidationError`.

Example:

```python
class MyClass(FWSerializable):
    ...

    @classmethod
    @fw_schema_deserialize(debug=True)
    def from_dict(cls, dct):
        ...
```


### Register external schemas

Schemas for all relevant classes in FireWorks are provided with this package and
they work "out of the box". It is possible to use custom schemas for further classes,
that are sub-classes of `fireworks.utilities.fw_serializers.FWSerializable`, provided
by other packages but not FireWorks. For this, every schema has to be registered
like this:

```python
  import fireworks_schema
  fireworks_schema.register_schema('/absolute/path/to/the/schema.json')
```

After that, the schema can be used for both explicit and automatic validation as
described in the previous sections.

Currently, these restrictions/conventions apply:
* The filename, e.g. `schema.json` as in the example, must be different from
the file names of already registered schemas. This restriction comes about because
the `$id` schema property is currently not used.
* With `jsonschema < 4.18.0` all referenced schemas must be installed in the
same directory. This implies that the default schemas provided with `fireworks_schema`
cannot be used from custom schemas if `jsonschema < 4.18.0`.
* The schema filename is lower case of the schema name which in turn is the class name.
* Only JSON schema [draft-07](https://json-schema.org/draft-07) must be used.

Here an example for a custom schema with filename `fwintegerarray.json`:

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$ref": "#/FWIntegerArray",
  "FWIntegerArray": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "_fw_name": {
        "const": "{{FWIntegerArray}}"
      },
      "data": {
        "type": "array",
        "items": {
          "type": "integer"
        }
      }
    },
    "required": [
      "_fw_name"
    ]
  }
}
```