Repository: Materials-Consortia/optimade-python-tools

Issue:
Chemical formulae are not properly validated on model creation
Currently, any rubbish can be placed in the chemical formula field when creating a model as we didn't write any validators (!) for them. This is not a massive deal as its an easy thing for implementations to get right, but e.g. `chemical_formula_anonymous` has strict rules on order that we should enforce.

Discussion:
ml-evs: Have just discovered that all of our test data has the wrong element ordering for `chemical_formula_reduced`! Will fix this evening.
codecov[bot]: # [Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=h1) Report
> Merging [#547](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=desc) into [master](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/commit/004686c705bf76d0dd228ba84b9a91a7d62d53a7?el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `97.67%`.

[![Impacted file tree graph](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/graphs/tree.svg?width=650&height=150&src=pr&token=UJAtmqkZZO)](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #547      +/-   ##
==========================================
+ Coverage   91.47%   91.53%   +0.05%     
==========================================
  Files          62       62              
  Lines        3110     3153      +43     
==========================================
+ Hits         2845     2886      +41     
- Misses        265      267       +2     
```

| Flag | Coverage Δ | |
|---|---|---|
| #project | `91.53% <97.67%> (+0.05%)` | :arrow_up: |
| #validator | `63.68% <74.41%> (+0.14%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [optimade/models/structures.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3N0cnVjdHVyZXMucHk=) | `94.90% <96.96%> (-0.18%)` | :arrow_down: |
| [optimade/models/utils.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3V0aWxzLnB5) | `91.56% <100.00%> (+1.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=footer). Last update [004686c...235271d](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

ml-evs: I think that's caught them all now...
ml-evs: Penny for anyone's thoughts? @CasperWA @shyamd 
ml-evs: From a resolved suggestion above:

> Since fields are not cross-validated, I've added an extra test case with both C and H in them as "deformities". Do you have any opinions on my second point above? Should we add pairwise validators for all correlated fields?


ml-evs: I think that's dealt with everything, except my question above (which is for another PR).

Are we happy to add the regexp to our OpenAPI specification for now, but potentially demote it down to a validator property pending the meeting next week?
CasperWA: Also, the changes in this PR introduces a lot of `ValueError` raises that are not tested. Do you think it's reasonable to add tests for these or simply leave it be? In an ideal world it would be nice to know that the raises are valid exits for the validators and it's not a code block that will never actually be reached.
ml-evs: > Also, the changes in this PR introduces a lot of `ValueError` raises that are not tested. Do you think it's reasonable to add tests for these or simply leave it be? In an ideal world it would be nice to know that the raises are valid exits for the validators and it's not a code block that will never actually be reached.

`ValueError`s get turned into the pydantic `ValidationError`s that we are testing for with all our entry tests right? At the very least we have a match on the message that it returns an error corresponding to the field we think is incorrect. Everything I've added here should be tested in the new tests that use the `good_structure` fixture, plus a lot of things we weren't testing directly before
CasperWA: > > Also, the changes in this PR introduces a lot of `ValueError` raises that are not tested. Do you think it's reasonable to add tests for these or simply leave it be? In an ideal world it would be nice to know that the raises are valid exits for the validators and it's not a code block that will never actually be reached.
> 
> `ValueError`s get turned into the pydantic `ValidationError`s that we are testing for with all our entry tests right? At the very least we have a match on the message that it returns an error corresponding to the field we think is incorrect. Everything I've added here should be tested in the new tests that use the `good_structure` fixture, plus a lot of things we weren't testing directly before

Right. It just seems 6 new lines are missed for the `Structure` model validators, possibly not being tested? See [here](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3N0cnVjdHVyZXMucHk=).
ml-evs: > Right. It just seems 6 new lines are missed for the `Structure` model validators, possibly not being tested? See [here](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3N0cnVjdHVyZXMucHk=).

Ah, those are the checks that aren't triggered as they are already caught by the overall field regexp. How much do we want to placate the coverage gods?
CasperWA: > > Right. It just seems 6 new lines are missed for the `Structure` model validators, possibly not being tested? See [here](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3N0cnVjdHVyZXMucHk=).
> 
> Ah, those are the checks that aren't triggered as they are already caught by the overall field regexp. How much do we want to placate the coverage gods?

So what you're saying is that they are now never reached?... :)
ml-evs: > > > Right. It just seems 6 new lines are missed for the `Structure` model validators, possibly not being tested? See [here](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/547/diff?src=pr&el=tree#diff-b3B0aW1hZGUvbW9kZWxzL3N0cnVjdHVyZXMucHk=).
> > 
> > 
> > Ah, those are the checks that aren't triggered as they are already caught by the overall field regexp. How much do we want to placate the coverage gods?
> 
> So what you're saying is that they are now never reached?... :)

Well, the checks themselves are, if they raise an error then we can diagnose any future problems that we have introduced into the regexp. Equally, if we decide tomorrow that we shouldn't include the regexp in the schema, then we'll have to rely on these checks again (or at least stop fastapi from putting the regexp in the schema itself)
CasperWA: > > So what you're saying is that they are now never reached?... :)
> 
> Well, the checks themselves are, if they raise an error then we can diagnose any future problems that we have introduced into the regexp. Equally, if we decide tomorrow that we shouldn't include the regexp in the schema, then we'll have to rely on these checks again (or at least stop fastapi from putting the regexp in the schema itself)

Personally, I'd prefer the cleaner option; to cut away the fat/non-used code.
This will diminish confusion and if we wish to revert, we can always do this from the git history.
ml-evs: > > > So what you're saying is that they are now never reached?... :)
> > 
> > 
> > Well, the checks themselves are, if they raise an error then we can diagnose any future problems that we have introduced into the regexp. Equally, if we decide tomorrow that we shouldn't include the regexp in the schema, then we'll have to rely on these checks again (or at least stop fastapi from putting the regexp in the schema itself)
> 
> Personally, I'd prefer the cleaner option; to cut away the fat/non-used code.
> This will diminish confusion and if we wish to revert, we can always do this from the git history.

Understood; let's park this until the meeting tomorrow, in case they don't like the regexp.

- If we can adopt the regexp in the spec, then the extra checks can be removed. 
- Alternatively, w could run the specific field validators with `pre=True` or `always=True` so they are triggered even when the regexp doesn't match, to provide a more informative error
- 
ml-evs: In the OPTIMADE meeting today it was agreed that putting the chemical formula regexp into the schema is good, so we can proceed here and remove the extra validators that never fail. The discussion also included the possibility of expanding the regexp to also check for element symbols.

Patch:
diff --git a/openapi/openapi.json b/openapi/openapi.json
index ad33b329..98f07741 100644
--- a/openapi/openapi.json
+++ b/openapi/openapi.json
@@ -3050,16 +3050,19 @@
           },
           "chemical_formula_reduced": {
             "title": "Chemical Formula Reduced",
+            "pattern": "^([A-Z][a-z]?\\d*)*$",
             "type": "string",
             "description": "The reduced chemical formula for a structure as a string with element symbols and integer chemical proportion numbers.\nThe proportion number MUST be omitted if it is 1.\n\n- **Type**: string\n\n- **Requirements/Conventions**:\n    - **Support**: SHOULD be supported by all implementations, i.e., SHOULD NOT be `null`.\n    - **Query**: MUST be a queryable property.\n      However, support for filters using partial string matching with this property is OPTIONAL (i.e., BEGINS WITH, ENDS WITH, and CONTAINS).\n      Intricate queries on formula components are instead suggested to be formulated using set-type filter operators on the multi valued `elements` and `elements_ratios` properties.\n    - Element names MUST have proper capitalization (e.g., `\"Si\"`, not `\"SI\"` for \"silicon\").\n    - Elements MUST be placed in alphabetical order, followed by their integer chemical proportion number.\n    - For structures with no partial occupation, the chemical proportion numbers are the smallest integers for which the chemical proportion is exactly correct.\n    - For structures with partial occupation, the chemical proportion numbers are integers that within reasonable approximation indicate the correct chemical proportions. The precise details of how to perform the rounding is chosen by the API implementation.\n    - No spaces or separators are allowed.\n\n- **Examples**:\n    - `\"H2NaO\"`\n    - `\"ClNa\"`\n    - `\"CCaO3\"`\n\n- **Query examples**:\n    - A filter that matches an exactly given formula is `chemical_formula_reduced=\"H2NaO\"`."
           },
           "chemical_formula_hill": {
             "title": "Chemical Formula Hill",
+            "pattern": "^([A-Z][a-z]?\\d*)*$",
             "type": "string",
             "description": "The chemical formula for a structure in [Hill form](https://dx.doi.org/10.1021/ja02046a005) with element symbols followed by integer chemical proportion numbers. The proportion number MUST be omitted if it is 1.\n\n- **Type**: string\n\n- **Requirements/Conventions**:\n    - **Support**: OPTIONAL support in implementations, i.e., MAY be `null`.\n    - **Query**: Support for queries on this property is OPTIONAL.\n      If supported, only a subset of the filter features MAY be supported.\n    - The overall scale factor of the chemical proportions is chosen such that the resulting values are integers that indicate the most chemically relevant unit of which the system is composed.\n      For example, if the structure is a repeating unit cell with four hydrogens and four oxygens that represents two hydroperoxide molecules, `chemical_formula_hill` is `\"H2O2\"` (i.e., not `\"HO\"`, nor `\"H4O4\"`).\n    - If the chemical insight needed to ascribe a Hill formula to the system is not present, the property MUST be handled as unset.\n    - Element names MUST have proper capitalization (e.g., `\"Si\"`, not `\"SI\"` for \"silicon\").\n    - Elements MUST be placed in [Hill order](https://dx.doi.org/10.1021/ja02046a005), followed by their integer chemical proportion number.\n      Hill order means: if carbon is present, it is placed first, and if also present, hydrogen is placed second.\n      After that, all other elements are ordered alphabetically.\n      If carbon is not present, all elements are ordered alphabetically.\n    - If the system has sites with partial occupation and the total occupations of each element do not all sum up to integers, then the Hill formula SHOULD be handled as unset.\n    - No spaces or separators are allowed.\n\n- **Examples**:\n    - `\"H2O2\"`\n\n- **Query examples**:\n    - A filter that matches an exactly given formula is `chemical_formula_hill=\"H2O2\"`."
           },
           "chemical_formula_anonymous": {
             "title": "Chemical Formula Anonymous",
+            "pattern": "^([A-Z][a-z]?\\d*)*$",
             "type": "string",
             "description": "The anonymous formula is the `chemical_formula_reduced`, but where the elements are instead first ordered by their chemical proportion number, and then, in order left to right, replaced by anonymous symbols A, B, C, ..., Z, Aa, Ba, ..., Za, Ab, Bb, ... and so on.\n\n- **Type**: string\n\n- **Requirements/Conventions**:\n    - **Support**: SHOULD be supported by all implementations, i.e., SHOULD NOT be `null`.\n    - **Query**: MUST be a queryable property.\n      However, support for filters using partial string matching with this property is OPTIONAL (i.e., BEGINS WITH, ENDS WITH, and CONTAINS).\n\n- **Examples**:\n    - `\"A2B\"`\n    - `\"A42B42C16D12E10F9G5\"`\n\n- **Querying**:\n    - A filter that matches an exactly given formula is `chemical_formula_anonymous=\"A2B\"`."
           },
diff --git a/optimade/models/structures.py b/optimade/models/structures.py
index bb5483f7..4203824a 100644
--- a/optimade/models/structures.py
+++ b/optimade/models/structures.py
@@ -1,4 +1,5 @@
 # pylint: disable=no-self-argument,line-too-long,no-name-in-module
+import re
 from enum import IntEnum, Enum
 from sys import float_info
 from typing import List, Optional, Union
@@ -12,6 +13,8 @@ from optimade.models.utils import (
     OptimadeField,
     StrictField,
     SupportLevel,
+    ANONYMOUS_ELEMENTS,
+    CHEMICAL_FORMULA_REGEXP,
 )
 
 EXTENDED_CHEMICAL_SYMBOLS = CHEMICAL_SYMBOLS + EXTRA_SYMBOLS
@@ -366,6 +369,7 @@ The proportion number MUST be omitted if it is 1.
     - A filter that matches an exactly given formula is `chemical_formula_reduced="H2NaO"`.""",
         support=SupportLevel.SHOULD,
         queryable=SupportLevel.MUST,
+        regex=CHEMICAL_FORMULA_REGEXP,
     )
 
     chemical_formula_hill: Optional[str] = OptimadeField(
@@ -396,6 +400,7 @@ The proportion number MUST be omitted if it is 1.
     - A filter that matches an exactly given formula is `chemical_formula_hill="H2O2"`.""",
         support=SupportLevel.OPTIONAL,
         queryable=SupportLevel.OPTIONAL,
+        regex=CHEMICAL_FORMULA_REGEXP,
     )
 
     chemical_formula_anonymous: str = OptimadeField(
@@ -417,6 +422,7 @@ The proportion number MUST be omitted if it is 1.
     - A filter that matches an exactly given formula is `chemical_formula_anonymous="A2B"`.""",
         support=SupportLevel.SHOULD,
         queryable=SupportLevel.MUST,
+        regex=CHEMICAL_FORMULA_REGEXP,
     )
 
     dimension_types: conlist(Periodicity, min_items=3, max_items=3) = OptimadeField(
@@ -758,6 +764,62 @@ The properties of the species are found in the property `species`.
         queryable=SupportLevel.MUST,
     )
 
+    @validator("chemical_formula_reduced", "chemical_formula_hill")
+    def check_ordered_formula(cls, v, field):
+        if v is None:
+            return v
+
+        numbers = re.findall(r"(\d+\.?)+", v)
+        elements = re.findall(r"[A-Z][a-z]?", v)
+        expected_elements = sorted(elements)
+
+        if field.name == "chemical_formula_hill":
+            for elem in ("C", "H"):
+                if elem in expected_elements:
+                    expected_elements.pop(expected_elements.index(elem))
+                    expected_elements.insert(0, elem)
+
+        if any(elem not in CHEMICAL_SYMBOLS for elem in elements):
+            raise ValueError(
+                f"Cannot use unknown chemical symbols {[elem for elem in elements if elem not in CHEMICAL_SYMBOLS]} in {field.name!r}"
+            )
+
+        if "1" in numbers:
+            raise ValueError(
+                "Must omit proportion '1' from formula {v} in {field.name!r}"
+            )
+        if expected_elements != elements:
+            order = "Hill" if field == "chemical_formula_hill" else "alphabetical"
+            raise ValueError(
+                f"Elements in {field.name!r} must appear in {order} order: {expected_elements} not {elements}."
+            )
+
+        return v
+
+    @validator("chemical_formula_anonymous")
+    def check_anonymous_formula(cls, v):
+        elements = tuple(re.findall(r"[A-Z][a-z]*", v))
+        numbers = [int(n.strip()) for n in re.split(r"[A-Z][a-z]*", v) if n.strip()]
+
+        if any(n == 1 for n in numbers):
+            raise ValueError(
+                "'chemical_formula_anonymous' {v} must omit proportion '1'"
+            )
+
+        expected_labels = ANONYMOUS_ELEMENTS[: len(elements)]
+        expected_numbers = sorted(numbers, reverse=True)
+
+        if expected_numbers != numbers:
+            raise ValueError(
+                f"'chemical_formula_anonymous' {v} has wrong order: elements with highest proportion should appear first: {numbers} vs expected {expected_numbers}"
+            )
+        if elements != expected_labels:
+            raise ValueError(
+                f"'chemical_formula_anonymous' {v} has wrong labels: {elements} vs expected {expected_labels}."
+            )
+
+        return v
+
     @validator("elements", each_item=True)
     def element_must_be_chemical_symbol(cls, v):
         if v not in CHEMICAL_SYMBOLS:
@@ -778,12 +840,6 @@ The properties of the species are found in the property `species`.
             )
         return v
 
-    @validator("chemical_formula_reduced", "chemical_formula_hill")
-    def no_spaces_in_reduced(cls, v):
-        if v and " " in v:
-            raise ValueError(f"Spaces are not allowed, you passed: {v}")
-        return v
-
     @validator("nperiodic_dimensions")
     def check_periodic_dimensions(cls, v, values):
         if values.get("dimension_types", []) and v is None:
diff --git a/optimade/models/utils.py b/optimade/models/utils.py
index 5f02f3c1..f3765e41 100644
--- a/optimade/models/utils.py
+++ b/optimade/models/utils.py
@@ -1,6 +1,7 @@
 import inspect
 import warnings
 import re
+import itertools
 from enum import Enum
 from typing import Optional
 
@@ -182,6 +183,22 @@ class SemanticVersion(str):
         return f"{self.major}.{self.minor}.{self.patch}"
 
 
+def anonymous_element_generator():
+    """ Generator that yields the next symbol in the A, B, Aa, ... Az naming scheme. """
+    from string import ascii_lowercase
+
+    for size in itertools.count(1):
+        for s in itertools.product(ascii_lowercase, repeat=size):
+            s = list(s)
+            s[0] = s[0].upper()
+            yield "".join(s)
+
+
+ANONYMOUS_ELEMENTS = tuple(itertools.islice(anonymous_element_generator(), 150))
+""" Returns the first 150 values of the anonymous element generator. """
+
+CHEMICAL_FORMULA_REGEXP = r"^([A-Z][a-z]?\d*)*$"
+
 EXTRA_SYMBOLS = ["X", "vacancy"]
 
 CHEMICAL_SYMBOLS = [
diff --git a/setup.py b/setup.py
index 9db85cfd..a222f209 100644
--- a/setup.py
+++ b/setup.py
@@ -89,7 +89,7 @@ setup(
     install_requires=[
         "lark-parser~=0.10.1",
         "fastapi~=0.61.1",
-        "pydantic~=1.6",
+        "pydantic~=1.6.1",
         "email_validator~=1.1",
         "requests~=2.24",
         'typing-extensions~=3.7;python_version<"3.8"',


Test patch:
diff --git a/optimade/server/data/test_structures.json b/optimade/server/data/test_structures.json
index 65bf537a..6bd7d291 100644
--- a/optimade/server/data/test_structures.json
+++ b/optimade/server/data/test_structures.json
@@ -64,11 +64,14 @@
     "structure_features": [],
     "task_id": "mpf_1",
     "relationships": {
-        "references": {
-            "data": [
-                {"type": "references", "id": "dijkstra1968"}
-            ]
-        }
+      "references": {
+        "data": [
+          {
+            "type": "references",
+            "id": "dijkstra1968"
+          }
+        ]
+      }
     }
   },
   {
@@ -176,11 +179,14 @@
     "structure_features": [],
     "task_id": "mpf_2",
     "relationships": {
-        "references": {
-            "data": [
-                {"type": "references", "id": "dijkstra1968"}
-            ]
-        }
+      "references": {
+        "data": [
+          {
+            "type": "references",
+            "id": "dijkstra1968"
+          }
+        ]
+      }
     }
   },
   {
@@ -288,11 +294,14 @@
     "structure_features": [],
     "task_id": "mpf_3",
     "relationships": {
-        "references": {
-            "data": [
-                {"type": "references", "id": "maddox1988"}
-            ]
-        }
+      "references": {
+        "data": [
+          {
+            "type": "references",
+            "id": "maddox1988"
+          }
+        ]
+      }
     }
   },
   {
@@ -556,7 +565,7 @@
     ],
     "nelements": 4,
     "nsites": 5,
-    "pretty_formula": "AcCuO2F",
+    "pretty_formula": "AcCuFO2",
     "species": [
       {
         "chemical_symbols": [
@@ -798,7 +807,7 @@
     ],
     "nelements": 4,
     "nsites": 16,
-    "pretty_formula": "Ag10Te4BrCl",
+    "pretty_formula": "Ag10BrClTe4",
     "species": [
       {
         "chemical_symbols": [
@@ -960,7 +969,7 @@
     ],
     "nelements": 6,
     "nsites": 10,
-    "pretty_formula": "Ag2CSNClO4",
+    "pretty_formula": "Ag2CClNO4S",
     "species": [
       {
         "chemical_symbols": [
@@ -1202,7 +1211,7 @@
     ],
     "nelements": 5,
     "nsites": 24,
-    "pretty_formula": "Ag2H12C6N3Cl",
+    "pretty_formula": "Ag2C6ClH12N3",
     "species": [
       {
         "chemical_symbols": [
@@ -1454,7 +1463,7 @@
     ],
     "nelements": 5,
     "nsites": 25,
-    "pretty_formula": "Ag2H2C2N6O13",
+    "pretty_formula": "Ag2C2H2N6O13",
     "species": [
       {
         "chemical_symbols": [
@@ -1701,7 +1710,7 @@
     ],
     "nelements": 7,
     "nsites": 23,
-    "pretty_formula": "Ag2H8C2S2N5ClO3",
+    "pretty_formula": "Ag2C2ClH8N5O3S2",
     "species": [
       {
         "chemical_symbols": [
@@ -1937,7 +1946,7 @@
     ],
     "nelements": 6,
     "nsites": 18,
-    "pretty_formula": "Ag4Hg5S5IBrCl2",
+    "pretty_formula": "Ag4BrCl2Hg5IS5",
     "species": [
       {
         "chemical_symbols": [
@@ -2443,7 +2452,7 @@
     ],
     "nelements": 8,
     "nsites": 74,
-    "pretty_formula": "AgB10P2H40C15NCl2O3",
+    "pretty_formula": "AgB10C15Cl2H40NO3P2",
     "species": [
       {
         "chemical_symbols": [
@@ -2796,7 +2805,7 @@
     ],
     "nelements": 7,
     "nsites": 29,
-    "pretty_formula": "AgH14C3S3N6ClO",
+    "pretty_formula": "AgC3ClH14N6OS3",
     "species": [
       {
         "chemical_symbols": [
@@ -3172,7 +3181,7 @@
     ],
     "nelements": 8,
     "nsites": 44,
-    "pretty_formula": "Ba2NaCe2Ti2FeSi8HO27",
+    "pretty_formula": "Ba2Ce2FeHNaO27Si8Ti2",
     "species": [
       {
         "chemical_symbols": [
@@ -3574,7 +3583,7 @@
     ],
     "nelements": 9,
     "nsites": 44,
-    "pretty_formula": "Ba2NaTi2MnRe2Si8HO26F",
+    "pretty_formula": "Ba2FHMnNaO26Re2Si8Ti2",
     "species": [
       {
         "chemical_symbols": [
@@ -3708,9 +3717,12 @@
     "task_id": "mpf_3819",
     "relationships": {
       "references": {
-          "data": [
-              {"type": "references", "id": "dummy/2019"}
-          ]
+        "data": [
+          {
+            "type": "references",
+            "id": "dummy/2019"
+          }
+        ]
       }
     }
   }
diff --git a/tests/adapters/structures/raw_test_structures.json b/tests/adapters/structures/raw_test_structures.json
index 082d55e0..be13ed70 100644
--- a/tests/adapters/structures/raw_test_structures.json
+++ b/tests/adapters/structures/raw_test_structures.json
@@ -506,7 +506,7 @@
         0.4
       ],
       "chemical_formula_descriptive": "AcCuO2F",
-      "chemical_formula_reduced": "AcCuO2F",
+      "chemical_formula_reduced": "AcCuFO2",
       "chemical_formula_anonymous": "A2BCD",
       "dimension_types": [
         1,
@@ -691,7 +691,7 @@
         0.25
       ],
       "chemical_formula_descriptive": "Ag10Te4BrCl",
-      "chemical_formula_reduced": "Ag10Te4BrCl",
+      "chemical_formula_reduced": "Ag10BrClTe4",
       "chemical_formula_anonymous": "A10B4CD",
       "dimension_types": [
         1,
@@ -882,7 +882,7 @@
         0.1
       ],
       "chemical_formula_descriptive": "Ag2CSNClO4",
-      "chemical_formula_reduced": "Ag2CSNClO4",
+      "chemical_formula_reduced": "Ag2CClNO4S",
       "chemical_formula_anonymous": "A4B2CDEF",
       "dimension_types": [
         1,
@@ -1053,7 +1053,7 @@
         0.125
       ],
       "chemical_formula_descriptive": "Ag2H12C6N3Cl",
-      "chemical_formula_reduced": "Ag2H12C6N3Cl",
+      "chemical_formula_reduced": "Ag2C6ClH12N3",
       "chemical_formula_anonymous": "A12B6C3D2E",
       "dimension_types": [
         1,
@@ -1299,7 +1299,7 @@
         0.52
       ],
       "chemical_formula_descriptive": "Ag2H2C2N6O13",
-      "chemical_formula_reduced": "Ag2H2C2N6O13",
+      "chemical_formula_reduced": "Ag2C2H2N6O13",
       "chemical_formula_anonymous": "A13B6C2D2E2",
       "dimension_types": [
         1,
@@ -1555,7 +1555,7 @@
         0.08695652173913043
       ],
       "chemical_formula_descriptive": "Ag2H8C2S2N5ClO3",
-      "chemical_formula_reduced": "Ag2H8C2S2N5ClO3",
+      "chemical_formula_reduced": "Ag2C2ClH8N5O3S2",
       "chemical_formula_anonymous": "A8B5C3D2E2F2G",
       "dimension_types": [
         1,
@@ -1815,7 +1815,7 @@
         0.2777777777777778
       ],
       "chemical_formula_descriptive": "Ag4Hg5S5IBrCl2",
-      "chemical_formula_reduced": "Ag4Hg5S5IBrCl2",
+      "chemical_formula_reduced": "Ag4BrCl2Hg5IS5",
       "chemical_formula_anonymous": "A5B5C4D2EF",
       "dimension_types": [
         1,
@@ -2040,7 +2040,7 @@
         0.02702702702702703
       ],
       "chemical_formula_descriptive": "AgB10P2H40C15NCl2O3",
-      "chemical_formula_reduced": "AgB10P2H40C15NCl2O3",
+      "chemical_formula_reduced": "AgB10C15Cl2H40NO3P2",
       "chemical_formula_anonymous": "A40B15C10D3E2F2GH",
       "dimension_types": [
         1,
@@ -2617,7 +2617,7 @@
         0.10344827586206896
       ],
       "chemical_formula_descriptive": "AgH14C3S3N6ClO",
-      "chemical_formula_reduced": "AgH14C3S3N6ClO",
+      "chemical_formula_reduced": "AgC3ClH14N6OS3",
       "chemical_formula_anonymous": "A14B6C3D3EFG",
       "dimension_types": [
         1,
@@ -2917,7 +2917,7 @@
         0.045454545454545456
       ],
       "chemical_formula_descriptive": "Ba2NaCe2Ti2FeSi8HO27",
-      "chemical_formula_reduced": "Ba2NaCe2Ti2FeSi8HO27",
+      "chemical_formula_reduced": "Ba2Ce2FeHNaO27Si8Ti2",
       "chemical_formula_anonymous": "A27B8C2D2E2FGH",
       "dimension_types": [
         1,
@@ -3318,7 +3318,7 @@
         0.045454545454545456
       ],
       "chemical_formula_descriptive": "Ba2NaTi2MnRe2Si8HO26F",
-      "chemical_formula_reduced": "Ba2NaTi2MnRe2Si8HO26F",
+      "chemical_formula_reduced": "Ba2FHMnNaO26Re2Si8Ti2",
       "chemical_formula_anonymous": "A26B8C2D2E2FGHI",
       "dimension_types": [
         1,
diff --git a/tests/models/conftest.py b/tests/models/conftest.py
index cb964f4b..ff0daebe 100644
--- a/tests/models/conftest.py
+++ b/tests/models/conftest.py
@@ -63,3 +63,49 @@ def starting_links() -> dict:
         "aggregate": "test",
         "no_aggregate_reason": "This is a test database",
     }
+
+
+@pytest.fixture(scope="function")
+def good_structure() -> dict:
+    """Returns a 'good' structure that does not need to be mapped, which can be
+    deformed to test different validators.
+
+    """
+    import datetime
+
+    return {
+        "id": "db/1234567",
+        "type": "structures",
+        "attributes": {
+            "last_modified": datetime.datetime.now(),
+            "elements": ["Ge", "Si"],
+            "nsites": 3,
+            "nelements": 2,
+            "elements_ratios": [0.5, 0.5],
+            "chemical_formula_reduced": "GeSi",
+            "chemical_formula_hill": "GeSi",
+            "chemical_formula_descriptive": "GeSi",
+            "chemical_formula_anonymous": "AB",
+            "dimension_types": [1, 1, 1],
+            "nperiodic_dimensions": 3,
+            "lattice_vectors": [[4.0, 0.0, 0.0], [0.0, 4.0, 0.0], [0.0, 1.0, 4.0]],
+            "cartesian_site_positions": [[0, 0, 0], [0, 0, 0], [0, 0, 0]],
+            "species": [
+                {"name": "Si", "chemical_symbols": ["Si"], "concentration": [1.0]},
+                {"name": "Ge", "chemical_symbols": ["Ge"], "concentration": [1.0]},
+                {
+                    "name": "vac",
+                    "chemical_symbols": ["vacancy"],
+                    "concentration": [1.0],
+                },
+            ],
+            "species_at_sites": ["Si", "Ge", "vac"],
+            "assemblies": [
+                {
+                    "sites_in_groups": [[0], [1], [2]],
+                    "group_probabilities": [0.3, 0.5, 0.2],
+                }
+            ],
+            "structure_features": ["assemblies"],
+        },
+    }
diff --git a/tests/models/test_data/test_bad_structures.json b/tests/models/test_data/test_bad_structures.json
index 12697659..100feb90 100644
--- a/tests/models/test_data/test_bad_structures.json
+++ b/tests/models/test_data/test_bad_structures.json
@@ -772,7 +772,7 @@
     ],
     "nelements": 4,
     "nsites": 16,
-    "pretty_formula": "Ag10Te4BrCl",
+    "pretty_formula": "Ag10BrClTe4",
     "species": [
       {
         "chemical_symbols": [
@@ -934,7 +934,7 @@
     ],
     "nelements": 6,
     "nsites": 10,
-    "pretty_formula": "Ag2CSNClO4",
+    "pretty_formula": "Ag2CClNO4S",
     "species": [
       {
         "chemical_symbols": [
@@ -1176,7 +1176,7 @@
     ],
     "nelements": 5,
     "nsites": 24,
-    "pretty_formula": "Ag2H12C6N3Cl",
+    "pretty_formula": "Ag2C6ClH12N3",
     "species": [
       {
         "chemical_symbols": [
@@ -1428,7 +1428,7 @@
     ],
     "nelements": 5,
     "nsites": 25,
-    "pretty_formula": "Ag2H2C2N6O13",
+    "pretty_formula": "Ag2C2H2N6O13",
     "species": [
       {
         "chemical_symbols": [
diff --git a/tests/models/test_structures.py b/tests/models/test_structures.py
index 31ad9d2f..24d63877 100644
--- a/tests/models/test_structures.py
+++ b/tests/models/test_structures.py
@@ -39,3 +39,44 @@ def test_bad_structures(bad_structures, mapper):
         print(f"Trying structure number {index} from 'test_bad_structures.json'")
         with pytest.raises(ValidationError):
             StructureResource(**mapper(MAPPER).map_back(structure))
+
+
+deformities = (
+    None,
+    {"chemical_formula_anonymous": "AaBC"},
+    {"chemical_formula_anonymous": "BC"},
+    {"chemical_formula_anonymous": "A1B1"},
+    {"chemical_formula_anonymous": "BC1"},
+    {"chemical_formula_anonymous": "A9C"},
+    {"chemical_formula_anonymous": "A9.2B"},
+    {"chemical_formula_anonymous": "A2B90"},
+    {"chemical_formula_anonymous": "A2B10"},
+    {"chemical_formula_hill": "SiGe"},
+    {"chemical_formula_hill": "GeHSi"},
+    {"chemical_formula_hill": "CGeHSi"},
+    {"chemical_formula_reduced": "Ge1.0Si1.0"},
+    {"chemical_formula_reduced": "GeSi2.0"},
+    {"chemical_formula_reduced": "GeSi.2"},
+    {"chemical_formula_reduced": "Ge1Si"},
+    {"chemical_formula_reduced": "GeSi1"},
+    {"chemical_formula_reduced": "SiGe2"},
+    {"chemical_formula_reduced": "FaKeElEmENtS"},
+    {"chemical_formula_reduced": "abcd"},
+    {"chemical_formula_reduced": "a2BeH"},
+    {"chemical_formula_reduced": "............"},
+    {"chemical_formula_reduced": "Ag6 Cl2"},
+)
+
+
+@pytest.mark.parametrize("deformity", deformities)
+def test_structure_deformities(good_structure, deformity):
+    """Make specific checks upon performing single invalidating deformations
+    of the data of a good structure.
+
+    """
+    if deformity is None:
+        StructureResource(**good_structure)
+    else:
+        good_structure["attributes"].update(deformity)
+        with pytest.raises(ValidationError, match=fr".*{list(deformity.keys())[0]}.*"):
+            StructureResource(**good_structure)
diff --git a/tests/models/test_utils.py b/tests/models/test_utils.py
index 279d4d68..61d10d47 100644
--- a/tests/models/test_utils.py
+++ b/tests/models/test_utils.py
@@ -1,5 +1,5 @@
 import pytest
-from pydantic import BaseModel
+from pydantic import BaseModel, ValidationError, Field
 from optimade.models.utils import OptimadeField, StrictField, SupportLevel
 from typing import List, Callable
 
@@ -74,3 +74,40 @@ def test_compatible_strict_optimade_field():
     strict_schema = CorrectModelWithStrictField.schema()
     strict_schema["title"] = optimade_schema["title"]
     assert strict_schema == optimade_schema
+
+
+def test_formula_regexp():
+    """This test checks some simple chemical formulae with the
+    `CHEMICAL_FORMULA_REGEXP`.
+
+    """
+    import re
+    from optimade.models.utils import CHEMICAL_FORMULA_REGEXP
+
+    class DummyModel(BaseModel):
+        formula: str = Field(regex=CHEMICAL_FORMULA_REGEXP)
+
+    good_formulae = (
+        "AgCl",
+        "H5F",
+        "LiP5",
+        "Jn7Qb4",  # Regexp does not care about the actual existence of elements
+        "A5B213CeD3E65F12G",
+        "A1Be2",
+    )
+
+    bad_formulae = (
+        "Ag...Cl",
+        "123123",
+        "Ag   Cl",
+        "abcd",
+        "6F7G",
+    )
+
+    for formula in good_formulae:
+        assert re.match(CHEMICAL_FORMULA_REGEXP, formula)
+        assert DummyModel(formula=formula)
+
+    for formula in bad_formulae:
+        with pytest.raises(ValidationError):
+            assert DummyModel(formula=formula)
diff --git a/tests/server/query_params/test_filter.py b/tests/server/query_params/test_filter.py
index 213ca68b..2633d162 100644
--- a/tests/server/query_params/test_filter.py
+++ b/tests/server/query_params/test_filter.py
@@ -208,13 +208,13 @@ def test_string_contains(check_response):
 
 
 def test_string_start(check_response):
-    request = '/structures?filter=chemical_formula_descriptive STARTS WITH "Ag2CSNCl"'
+    request = '/structures?filter=chemical_formula_descriptive STARTS WITH "Ag2CClN"'
     expected_ids = ["mpf_259"]
     check_response(request, expected_ids)
 
 
 def test_string_end(check_response):
-    request = '/structures?filter=chemical_formula_descriptive ENDS WITH "NClO4"'
+    request = '/structures?filter=chemical_formula_descriptive ENDS WITH "ClNO4S"'
     expected_ids = ["mpf_259"]
     check_response(request, expected_ids)
 
@@ -237,7 +237,7 @@ def test_awkward_not_queries(check_response, client):
     """
     expected_ids = ["mpf_3819"]
     request = (
-        '/structures?filter=chemical_formula_descriptive="Ba2NaTi2MnRe2Si8HO26F" AND '
+        '/structures?filter=chemical_formula_descriptive="Ba2FHMnNaO26Re2Si8Ti2" AND '
         'chemical_formula_anonymous = "A26B8C2D2E2FGHI" '
     )
     check_response(request, expected_ids)
@@ -260,7 +260,7 @@ def test_awkward_not_queries(check_response, client):
     request = (
         "/structures?filter="
         "NOT ( "
-        'chemical_formula_descriptive = "Ba2NaTi2MnRe2Si8HO26F" AND '
+        'chemical_formula_descriptive = "Ba2FHMnNaO26Re2Si8Ti2" AND '
         'chemical_formula_anonymous = "A26B8C2D2E2FGHI" OR '
         'chemical_formula_anonymous = "A2BC" AND '
         'NOT chemical_formula_descriptive = "Ac2AgPb" '