Repository: Materials-Consortia/optimade-python-tools
Pull request #287 (2020-06-02 16:48:25)

Create template filtertransformer BaseTransformer
Here's a first pass at a `BaseTransformer`. I think we need to discuss what to do with the Django, JSON and TransformerSkeleton and elasticsearch transformers, plus all the grammar variants... I haven't really gotten my head around how lark interacts with these magic-word methods, and how we currently have different method names between al the transformers.

For now, I've made most of the `BaseTransformer` methods abstract, apart from those that do very simple things (i.e. take in a value and return value).

- [x] Add deprecation warning for `Lark2Django`/`DjangoTransformer`, which is working from 0.9.7 grammar and will be tricky to adapt
- [x] Remove `JSONTransformer` and `DebugTransformer` (and updated docs)
- [x] Created abstract `BaseTransformer` and pulled mongo functionality out
- [x] Made file for `v1.0.0` grammar that links to `v0.10.1`

codecov[bot]: # [Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=h1) Report
> Merging [#287](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=desc) into [master](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/commit/d29f7a1c1e89d69aaeffee87e2e46093814c4d3c&el=desc) will **increase** coverage by `0.14%`.
> The diff coverage is `98.66%`.

[![Impacted file tree graph](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/graphs/tree.svg?width=650&height=150&src=pr&token=UJAtmqkZZO)](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #287      +/-   ##
==========================================
+ Coverage   90.05%   90.20%   +0.14%     
==========================================
  Files          54       56       +2     
  Lines        2273     2307      +34     
==========================================
+ Hits         2047     2081      +34     
  Misses        226      226              
```

| Flag | Coverage Δ | |
|---|---|---|
| #unittests | `90.20% <98.66%> (+0.14%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [optimade/filtertransformers/base\_transformer.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvZmlsdGVydHJhbnNmb3JtZXJzL2Jhc2VfdHJhbnNmb3JtZXIucHk=) | `98.18% <98.18%> (ø)` | |
| [optimade/filtertransformers/\_\_init\_\_.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvZmlsdGVydHJhbnNmb3JtZXJzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | |
| [optimade/filtertransformers/django.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvZmlsdGVydHJhbnNmb3JtZXJzL2RqYW5nby5weQ==) | `91.37% <100.00%> (ø)` | |
| [optimade/filtertransformers/elasticsearch.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvZmlsdGVydHJhbnNmb3JtZXJzL2VsYXN0aWNzZWFyY2gucHk=) | `87.89% <100.00%> (+0.15%)` | :arrow_up: |
| [optimade/filtertransformers/mongo.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvZmlsdGVydHJhbnNmb3JtZXJzL21vbmdvLnB5) | `97.12% <100.00%> (+0.09%)` | :arrow_up: |
| [optimade/server/mappers/entries.py](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287/diff?src=pr&el=tree#diff-b3B0aW1hZGUvc2VydmVyL21hcHBlcnMvZW50cmllcy5weQ==) | `98.24% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=footer). Last update [d29f7a1...59d693e](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/pull/287?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

fekad: The `TransformerSkeleton` can be removed for sure. That was just a template for the "developers" and used for debugging.

Patch:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ad42c154..d65a1f3b 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -4,22 +4,22 @@ The [Materials Consortia](https://github.com/Materials-Consortia) is very open t
 
 This may be anything from simple feedback and raising [new issues](https://github.com/Materials-Consortia/optimade-python-tools/issues/new) to creating [new PRs](https://github.com/Materials-Consortia/optimade-python-tools/compare).
 
-We have below recommendations for setting up an environment in which one may develop the package further.
+Recommendations for setting up a development environment can be found in the [Installation instructions](https://www.optimade.org/optimade-python-tools/install/#full-development-installation).
 
 ## Getting Started with Filter Parsing and Transforming
 
 Example use:
 
 ```python
-from optimade.filterparser import Parser
+from optimade.filterparser import LarkParser
 
-p = Parser(version=(0,9,7))
+p = LarkParser(version=(1, 0, 0))
 tree = p.parse("nelements<3")
 print(tree)
 ```
 
 ```shell
-Tree(start, [Tree(expression, [Tree(term, [Tree(atom, [Tree(comparison, [Token(VALUE, 'nelements'), Token(OPERATOR, '<'), Token(VALUE, '3')])])])])])
+Tree('filter', [Tree('expression', [Tree('expression_clause', [Tree('expression_phrase', [Tree('comparison', [Tree('property_first_comparison', [Tree('property', [Token('IDENTIFIER', 'nelements')]), Tree('value_op_rhs', [Token('OPERATOR', '<'), Tree('value', [Tree('number', [Token('SIGNED_INT', '3')])])])])])])])])])
 ```
 
 ```python
@@ -27,14 +27,17 @@ print(tree.pretty())
 ```
 
 ```shell
-start
+filter
   expression
-    term
-      atom
+    expression_clause
+      expression_phrase
         comparison
-          nelements
-          <
-          3
+          property_first_comparison
+            property	nelements
+            value_op_rhs
+              <
+              value
+                number	3
 ```
 
 ```python
@@ -43,36 +46,31 @@ print(tree.pretty())
 ```
 
 ```shell
-start
+filter
   expression
-    term
-      term
-        atom
-          comparison
-            _mp_bandgap
-            >
-            5.0
-      AND
-      atom
+    expression_clause
+      expression_phrase
         comparison
-          _cod_molecular_weight
-          <
-          350
-```
-
-```python
-# Assumes graphviz installed on system (e.g. `conda install -c anaconda graphviz`) and `pip install pydot`
-from lark.tree import pydot__tree_to_png
-
-pydot__tree_to_png(tree, "exampletree.png")
+          property_first_comparison
+            property	_mp_bandgap
+            value_op_rhs
+              >
+              value
+                number	5.0
+      expression_phrase
+        comparison
+          property_first_comparison
+            property	_cod_molecular_weight
+            value_op_rhs
+              <
+              value
+                number	350
 ```
 
-![example tree](images/exampletree.png)
-
 ### Flow for Parsing User-Supplied Filter and Converting to Backend Query
 
-`optimade.filterparser.Parser` will take user input to generate a `lark.Tree` and feed that to a `lark.Transformer`.
-E.g., `optimade.filtertransformers.mongo.MongoTransformer` will turn the tree into something useful for your MondoDB backend:
+`optimade.filterparser.LarkParser` will take user input to generate a `lark.Tree` and feed that to a `lark.Transformer`.
+E.g., `optimade.filtertransformers.mongo.MongoTransformer` will turn the tree into something useful for your MongoDB backend:
 
 ```python
 # Example: Converting to MongoDB Query Syntax
@@ -85,55 +83,23 @@ query = transformer.transform(tree)
 print(query)
 ```
 
-```python
-{'$and': [{'_mp_bandgap': {'$gt': 5.0}}, {'_cod_molecular_weight': {'$lt': 350.0}}]}
+```json
+{
+    "$and": [
+        {"_mp_bandgap": {"$gt": 5.0}},
+        {"_cod_molecular_weight": {"$lt": 350.0}}
+    ]
+}
 ```
 
-There is also a [basic JSON transformer][optimade.filtertransformers.json] you can use as a simple example for developing your own transformer.
-You can also use the JSON output it produces as an easy-to-parse input for a "transformer" in your programming language of choice.
-
-```python
-class JSONTransformer(Transformer):
-    def __init__(self, compact=False):
-        self.compact = compact
-        super().__init__()
-
-    def __default__(self, data, children):
-        items = []
-        for c in children:
-            if isinstance(c, Token):
-                token_repr = {
-                    "@module": "lark.lexer",
-                    "@class": "Token",
-                    "type_": c.type,
-                    "value": c.value,
-                }
-                if self.compact:
-                    del token_repr["@module"]
-                    del token_repr["@class"]
-                items.append(token_repr)
-            elif isinstance(c, dict):
-                items.append(c)
-            else:
-                raise ValueError(f"Unknown type {type(c)} for tree child {c}")
-        tree_repr = {
-            "@module": "lark",
-            "@class": "Tree",
-            "data": data,
-            "children": items,
-        }
-        if self.compact:
-            del tree_repr["@module"]
-            del tree_repr["@class"]
-        return tree_repr
-```
 
 ### Developing New Filter Transformers
 
-If you would like to add a new transformer, please add:
+If you would like to add a new transformer, please raise an issue to signal your intent (in case someone else is already working on this).
+Adding a transformer requires the following:
 
-1. A module (.py file) in the `optimade/filtertransformers` folder.
-2. Any additional Python requirements must be optional and provided as a separate "`extra_requires`" entry in `setup.py`.
+1. A new submodule (`.py` file) in the `optimade/filtertransformers` folder containing an implementation of the transformer object, preferably one that extends `optimade.filtertransformers.base_transformer.BaseTransformer`.
+2. Any additional Python requirements must be optional and provided as a separate "`extra_requires`" entry in `setup.py` and in the `requirements.txt` file.
 3. Tests in `optimade/filtertransformers/tests` that are skipped if the required packages fail to import.
 
 For examples, please check out existing filter transformers.
diff --git a/docs/api_reference/filtertransformers/debug.md b/docs/api_reference/filtertransformers/debug.md
deleted file mode 100644
index 99ff427d..00000000
--- a/docs/api_reference/filtertransformers/debug.md
+++ /dev/null
@@ -1,3 +0,0 @@
-# debug
-
-::: optimade.filtertransformers.debug
diff --git a/docs/api_reference/filtertransformers/json.md b/docs/api_reference/filtertransformers/json.md
deleted file mode 100644
index 41370588..00000000
--- a/docs/api_reference/filtertransformers/json.md
+++ /dev/null
@@ -1,3 +0,0 @@
-# json
-
-::: optimade.filtertransformers.json
diff --git a/optimade/filtertransformers/__init__.py b/optimade/filtertransformers/__init__.py
index e69de29b..6526f52d 100644
--- a/optimade/filtertransformers/__init__.py
+++ b/optimade/filtertransformers/__init__.py
@@ -0,0 +1,5 @@
+""" This module implements filter transformer classes for different backends. These
+classes typically parse the filter with Lark and produce an appropriate query for the
+given backend.
+
+"""
diff --git a/optimade/filtertransformers/base_transformer.py b/optimade/filtertransformers/base_transformer.py
new file mode 100644
index 00000000..7f1120ae
--- /dev/null
+++ b/optimade/filtertransformers/base_transformer.py
@@ -0,0 +1,179 @@
+import abc
+from lark import Transformer, v_args
+from typing import Dict
+from optimade.server.mappers import BaseResourceMapper
+
+__all__ = ("BaseTransformer",)
+
+
+class BaseTransformer(abc.ABC, Transformer):
+    """Generic filter transformer that handles various
+    parts of the grammar in a backend non-specific way.
+
+    """
+
+    # map from standard comparison operators to the backend-specific version
+    operator_map: Dict[str, str] = {
+        "<": None,
+        "<=": None,
+        ">": None,
+        ">=": None,
+        "!=": None,
+        "=": None,
+    }
+
+    # map from back-end specific operators to their inverse
+    # e.g. {"$lt": "$gt"} for MongoDB.
+    reversed_operator_map: Dict[str, str] = {}
+
+    def __init__(self, mapper: BaseResourceMapper = None):
+        """Initialise the transformer object, optionally loading in a
+        resource mapper for use when post-processing.
+
+        """
+        self.mapper = mapper
+
+    def postprocess(self, query):
+        """Post-process the query according to the rules defined for
+        the backend.
+
+        """
+        return query
+
+    def transform(self, tree):
+        """ Transform the query using the Lark transformer then post-process. """
+        return self.postprocess(super().transform(tree))
+
+    def __default__(self, data, children, meta):
+        raise NotImplementedError(
+            f"Calling __default__, i.e., unknown grammar concept. data: {data}, children: {children}, meta: {meta}"
+        )
+
+    def filter(self, arg):
+        """ filter: expression* """
+        return arg[0] if arg else None
+
+    @v_args(inline=True)
+    def constant(self, value):
+        """ constant: string | number """
+        # Note: Return as is.
+        return value
+
+    @v_args(inline=True)
+    def value(self, value):
+        """ value: string | number | property """
+        # Note: Return as is.
+        return value
+
+    @v_args(inline=True)
+    def non_string_value(self, value):
+        """ non_string_value: number | property """
+        # Note: Return as is.
+        return value
+
+    @v_args(inline=True)
+    def not_implemented_string(self, value):
+        """not_implemented_string: value
+
+        Raises:
+            NotImplementedError: For further information, see Materials-Consortia/OPTIMADE issue 157:
+                https://github.com/Materials-Consortia/OPTIMADE/issues/157
+
+        """
+        raise NotImplementedError("Comparing strings is not yet implemented.")
+
+    def property(self, arg):
+        """ property: IDENTIFIER ( "." IDENTIFIER )* """
+        return ".".join(arg)
+
+    @v_args(inline=True)
+    def string(self, string):
+        """ string: ESCAPED_STRING """
+        return string.strip('"')
+
+    @v_args(inline=True)
+    def signed_int(self, number):
+        """ signed_int : SIGNED_INT """
+        return int(number)
+
+    @v_args(inline=True)
+    def number(self, number):
+        """ number: SIGNED_INT | SIGNED_FLOAT """
+        if number.type == "SIGNED_INT":
+            type_ = int
+        elif number.type == "SIGNED_FLOAT":
+            type_ = float
+        return type_(number)
+
+    @v_args(inline=True)
+    def comparison(self, value):
+        """ comparison: constant_first_comparison | property_first_comparison """
+        # Note: Return as is.
+        return value
+
+    @abc.abstractmethod
+    def value_list(self, arg):
+        """ value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )* """
+
+    @abc.abstractmethod
+    def value_zip(self, arg):
+        """ value_zip: [ OPERATOR ] value ":" [ OPERATOR ] value (":" [ OPERATOR ] value)* """
+
+    @abc.abstractmethod
+    def value_zip_list(self, arg):
+        """ value_zip_list: value_zip ( "," value_zip )* """
+
+    @abc.abstractmethod
+    def expression(self, arg):
+        """ expression: expression_clause ( OR expression_clause ) """
+
+    @abc.abstractmethod
+    def expression_clause(self, arg):
+        """ expression_clause: expression_phrase ( AND expression_phrase )* """
+
+    @abc.abstractmethod
+    def expression_phrase(self, arg):
+        """ expression_phrase: [ NOT ] ( comparison | "(" expression ")" ) """
+
+    @abc.abstractmethod
+    def property_first_comparison(self, arg):
+        """property_first_comparison: property ( value_op_rhs | known_op_rhs | fuzzy_string_op_rhs | set_op_rhs |
+        set_zip_op_rhs | length_op_rhs )
+
+        """
+
+    @abc.abstractmethod
+    def constant_first_comparison(self, arg):
+        """ constant_first_comparison: constant OPERATOR ( non_string_value | not_implemented_string ) """
+
+    @v_args(inline=True)
+    @abc.abstractmethod
+    def value_op_rhs(self, operator, value):
+        """ value_op_rhs: OPERATOR value """
+
+    @abc.abstractmethod
+    def known_op_rhs(self, arg):
+        """ known_op_rhs: IS ( KNOWN | UNKNOWN ) """
+
+    @abc.abstractmethod
+    def fuzzy_string_op_rhs(self, arg):
+        """ fuzzy_string_op_rhs: CONTAINS value | STARTS [ WITH ] value | ENDS [ WITH ] value """
+
+    @abc.abstractmethod
+    def set_op_rhs(self, arg):
+        """ set_op_rhs: HAS ( [ OPERATOR ] value | ALL value_list | ANY value_list | ONLY value_list ) """
+
+    @abc.abstractmethod
+    def length_op_rhs(self, arg):
+        """ length_op_rhs: LENGTH [ OPERATOR ] value """
+
+    @abc.abstractmethod
+    def set_zip_op_rhs(self, arg):
+        """set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list |
+        ANY value_zip_list )
+
+        """
+
+    @abc.abstractmethod
+    def property_zip_addon(self, arg):
+        """ property_zip_addon: ":" property (":" property)* """
diff --git a/optimade/filtertransformers/debug.py b/optimade/filtertransformers/debug.py
deleted file mode 100644
index b67a35aa..00000000
--- a/optimade/filtertransformers/debug.py
+++ /dev/null
@@ -1,183 +0,0 @@
-from lark import Transformer
-
-
-class DebugTransformer(Transformer):  # pragma: no cover
-    def __init__(self):
-        super().__init__()
-
-    def __default__(self, data, children, meta):
-        print("Node: ", data, children)
-        return data
-
-
-class TransformerSkeleton(Transformer):  # pragma: no cover
-    """Prints out all the nodes and its arguments during the walk-through of the tree."""
-
-    def __init__(self):
-        super().__init__()
-
-    def filter(self, arg):
-        # filter: expression*
-        print("Node: ", "filter", arg)
-        return "filter"
-
-    def constant(self, arg):
-        # constant: string | number
-        print("Node: ", "constant", arg)
-        return "constant"
-
-    def value(self, arg):
-        # value: string | number | property
-        print("Node: ", "value", arg)
-        return "value"
-
-    def value_list(self, arg):
-        # value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )*
-        print("Node: ", "value_list", arg)
-        return "value_list"
-
-    def value_zip(self, arg):
-        # value_zip: [ OPERATOR ] value ":" [ OPERATOR ] value (":" [ OPERATOR ] value)*
-        print("Node: ", "value_zip", arg)
-        return "value_zip"
-
-    def value_zip_list(self, arg):
-        # value_zip_list: value_zip ( "," value_zip )*
-        print("Node: ", "value_zip_list", arg)
-        return "value_zip_list"
-
-    def expression(self, arg):
-        # expression: expression_clause [ OR expression ]
-        print("Node: ", "expression", arg)
-        return "expression"
-
-    def expression_clause(self, arg):
-        # expression_clause: expression_phrase [ AND expression_clause ]
-        print("Node: ", "expression_clause", arg)
-        return "expression_clause"
-
-    def expression_phrase(self, arg):
-        # expression_phrase: [ NOT ] ( comparison | predicate_comparison | "(" expression ")" )
-        print("Node: ", "expression_phrase", arg)
-        return "expression_phrase"
-
-    def comparison(self, arg):
-        # comparison: constant_first_comparison | property_first_comparison
-        # Note: Do nothing!
-        print("Node: ", "comparison", arg)
-        return "comparison"
-
-    def property_first_comparison(self, arg):
-        # property_first_comparison: property ( value_op_rhs | known_op_rhs | fuzzy_string_op_rhs | set_op_rhs |
-        # set_zip_op_rhs )
-        print("Node: ", "property_first_comparison", arg)
-        return "property_first_comparison"
-
-    def constant_first_comparison(self, arg):
-        # constant_first_comparison: constant value_op_rhs
-        print("Node: ", "constant_first_comparison", arg)
-        return "constant_first_comparison"
-
-    def predicate_comparison(self, arg):
-        # predicate_comparison: length_comparison
-        print("Node: ", "predicate_comparison", arg)
-        return "predicate_comparison"
-
-    def value_op_rhs(self, arg):
-        # value_op_rhs: OPERATOR value
-        print("Node: ", "value_op_rhs", arg)
-        return "value_op_rhs"
-
-    def known_op_rhs(self, arg):
-        # known_op_rhs: IS ( KNOWN | UNKNOWN )
-        print("Node: ", "known_op_rhs", arg)
-        return "known_op_rhs"
-
-    def fuzzy_string_op_rhs(self, arg):
-        # fuzzy_string_op_rhs: CONTAINS string | STARTS [ WITH ] string | ENDS [ WITH ] string
-        print("Node: ", "fuzzy_string_op_rhs", arg)
-        return "fuzzy_string_op_rhs"
-
-    def set_op_rhs(self, arg):
-        # set_op_rhs: HAS ( [ OPERATOR ] value | ALL value_list | ANY value_list | ONLY value_list )
-        print("Node: ", "set_op_rhs", arg)
-        return "set_op_rhs"
-
-    def set_zip_op_rhs(self, arg):
-        # set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list |
-        # ANY value_zip_list )
-        print("Node: ", "set_zip_op_rhs", arg)
-        return "set_zip_op_rhs"
-
-    def length_comparison(self, arg):
-        # length_comparison: LENGTH property OPERATOR value
-        print("Node: ", "length_comparison", arg)
-        return "length_comparison"
-
-    def property_zip_addon(self, arg):
-        # property_zip_addon: ":" property (":" property)*
-        print("Node: ", "property_zip_addon", arg)
-        return "property_zip_addon"
-
-    def property(self, arg):
-        # property: IDENTIFIER ( "." IDENTIFIER )*
-        print("Node: ", "property", arg)
-        return "property"
-
-    def string(self, arg):
-        # string: ESCAPED_STRING
-        print("Node: ", "string", arg)
-        return "string"
-
-    def number(self, arg):
-        # number: SIGNED_INT | SIGNED_FLOAT
-        print("Node: ", "number", arg)
-        return "number"
-
-    def __default__(self, data, children, meta):
-        print("Node: ", data, children)
-        return data
-
-
-if __name__ == "__main__":  # pragma: no cover
-    from optimade.filterparser import LarkParser
-
-    p = LarkParser(version=(0, 10, 0))
-    # t = DebugTransformer()
-    t = TransformerSkeleton()
-
-    # f = 'a.a = "text" OR a<a AND NOT b>=8'
-
-    # single list
-
-    f = "list HAS < 3"
-
-    f = "list HAS < 3, > 4"  # -> error
-    f = "list HAS ALL < 3, > 4"
-
-    # multiple lists
-
-    f = "list1:list2 HAS < 3 : > 4"
-    f = "list1:list2 HAS ALL < 3 : > 4"
-
-    f = "list1:list2 HAS < 3 : > 4, < 2 : > 5"  # -> error
-    f = "list1:list2 HAS ALL < 3 : > 4, < 2 : > 5"
-    f = "list1:list2 HAS ALL < 3, < 2 : > 4, > 5"  # -> error
-
-    # f = 'list1:list2 HAS < 3, > 4'  # -> error
-    # f = 'list1:list2 HAS ALL < 3, > 4'  # -> error
-
-    f = 'elements:elements_ratios HAS ALL "Al":>0.3333, "Al":<0.3334'
-    f = 'elements:elements_ratios HAS ALL "Al":>0.3333 AND elements_ratio<0.3334'
-    f = 'elements:elements_ratios HAS ALL "Al" : >0.3333, <0.3334'  # -> error
-
-    f = "list1:list2 HAS ALL < 3 : > 4, < 2 : > 5 : > 4, < 2 : > 5"  # valid but wrong
-    f = "ghf.flk<gh"  # valid but wrong
-
-    # f = ''
-
-    tree = p.parse(f)
-    print(tree)
-    print(tree.pretty())
-
-    t.transform(tree)
diff --git a/optimade/filtertransformers/django.py b/optimade/filtertransformers/django.py
index 3a1daf73..2e101c3a 100644
--- a/optimade/filtertransformers/django.py
+++ b/optimade/filtertransformers/django.py
@@ -1,3 +1,11 @@
+import warnings
+
+warnings.warn(
+    "Django functionality is deprecated and will be removed in later versions (unless support is requested).",
+    DeprecationWarning,
+    stacklevel=2,
+)
+
 import operator
 from optimade.filterparser import LarkParser
 from lark import Tree
@@ -5,6 +13,9 @@ from lark.lexer import Token
 from django.db.models import Q
 
 
+__all__ = ("DjangoTransformer",)
+
+
 class DjangoQueryError(Exception):
     pass
 
@@ -19,7 +30,16 @@ django_db_keys = {
 }
 
 
-class Lark2Django:
+class DjangoTransformer:
+    """Filter transformer for implementations using Django.
+
+    !!! warning "Warning"
+        This transformer is deprecated as it only supports
+        the 0.9.7 grammar version, and works different to other
+        filter transformers in this package.
+
+    """
+
     def __init__(self):
         self.opers = {
             "=": self.eq,
diff --git a/optimade/filtertransformers/elasticsearch.py b/optimade/filtertransformers/elasticsearch.py
index f788b5ba..6dd900ef 100644
--- a/optimade/filtertransformers/elasticsearch.py
+++ b/optimade/filtertransformers/elasticsearch.py
@@ -4,6 +4,11 @@ import lark
 from elasticsearch_dsl import Q, Text, Keyword, Integer, Field
 from optimade.models import CHEMICAL_SYMBOLS, ATOMIC_NUMBERS
 
+__all__ = (
+    "ElasticTransformer",
+    "Transformer",
+)
+
 
 _cmp_operators = {">": "gt", ">=": "gte", "<": "lt", "<=": "lte"}
 _rev_cmp_operators = {">": "<", ">=": "<=", "<": ">", "<=": "=>"}
@@ -65,8 +70,8 @@ class Quantity:
         return self.name
 
 
-class Transformer(lark.Transformer):
-    """Transformer that transforms ``v0.10.0`` grammer parse trees into queries.
+class ElasticTransformer(lark.Transformer):
+    """Transformer that transforms ``v0.10.0`` grammar parse trees into queries.
 
     Uses elasticsearch_dsl and will produce a `Q` instance.
     """
@@ -286,3 +291,7 @@ class Transformer(lark.Transformer):
 
     def string_literal(self, args):
         return args[0].strip('"')
+
+
+# added for potential backward compatibility
+Transformer = ElasticTransformer
diff --git a/optimade/filtertransformers/json.py b/optimade/filtertransformers/json.py
deleted file mode 100644
index 5427be2d..00000000
--- a/optimade/filtertransformers/json.py
+++ /dev/null
@@ -1,37 +0,0 @@
-from lark import Transformer
-from lark.lexer import Token
-
-
-class JSONTransformer(Transformer):  # pragma: no cover
-    def __init__(self, compact=False):
-        self.compact = compact
-        super().__init__()
-
-    def __default__(self, data, children, meta):
-        items = []
-        for c in children:
-            if isinstance(c, Token):
-                token_repr = {
-                    "@module": "lark.lexer",
-                    "@class": "Token",
-                    "type_": c.type,
-                    "value": c.value,
-                }
-                if self.compact:
-                    del token_repr["@module"]
-                    del token_repr["@class"]
-                items.append(token_repr)
-            elif isinstance(c, dict):
-                items.append(c)
-            else:
-                raise ValueError(f"Unknown type {type(c)} for tree child {c}")
-        tree_repr = {
-            "@module": "lark",
-            "@class": "Tree",
-            "data": data,
-            "children": items,
-        }
-        if self.compact:
-            del tree_repr["@module"]
-            del tree_repr["@class"]
-        return tree_repr
diff --git a/optimade/filtertransformers/mongo.py b/optimade/filtertransformers/mongo.py
index 3500f575..9b09844a 100755
--- a/optimade/filtertransformers/mongo.py
+++ b/optimade/filtertransformers/mongo.py
@@ -1,11 +1,18 @@
 import copy
-from lark import Transformer, v_args, Token
-from optimade.server.mappers import BaseResourceMapper
+from lark import v_args, Token
+from optimade.filtertransformers.base_transformer import BaseTransformer
 from optimade.server.exceptions import BadRequest
 
+__all__ = ("MongoTransformer",)
 
-class MongoTransformer(Transformer):
-    """Support for grammar v0.10.1"""
+
+class MongoTransformer(BaseTransformer):
+    """Transformer for MongoDB backend. Parses lark tree into
+    a dictionary to be passed to pymongo/mongomock. Uses
+    post-processing functions to handle aliasing and some
+    specific edge-cases for MongoDB.
+
+    """
 
     operator_map = {
         "<": "$lt",
@@ -24,14 +31,6 @@ class MongoTransformer(Transformer):
         "$eq": "$eq",
     }
 
-    def __init__(self, mapper: BaseResourceMapper = None):
-        """Initialise the object, optionally loading in a
-        resource mapper for use when post-processing.
-
-        """
-        self.mapper = mapper
-        super().__init__()
-
     def postprocess(self, query):
         """ Used to post-process the final parsed query. """
         if self.mapper:
@@ -46,41 +45,6 @@ class MongoTransformer(Transformer):
 
         return query
 
-    def transform(self, tree):
-        return self.postprocess(super().transform(tree))
-
-    def filter(self, arg):
-        # filter: expression*
-        return arg[0] if arg else None
-
-    @v_args(inline=True)
-    def constant(self, value):
-        # constant: string | number
-        # Note: Do nothing!
-        return value
-
-    @v_args(inline=True)
-    def value(self, value):
-        # value: string | number | property
-        # Note: Do nothing!
-        return value
-
-    @v_args(inline=True)
-    def non_string_value(self, value):
-        """ non_string_value: number | property """
-        # Note: Do nothing!
-        return value
-
-    @v_args(inline=True)
-    def not_implemented_string(self, value):
-        """not_implemented_string: value
-
-        Raise NotImplementedError.
-        For further information, see Materials-Consortia/OPTIMADE issue 157:
-        https://github.com/Materials-Consortia/OPTIMADE/issues/157
-        """
-        raise NotImplementedError("Comparing strings is not yet implemented.")
-
     def value_list(self, arg):
         # value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )*
         # NOTE: no support for optional OPERATOR, yet, so this takes the
@@ -115,12 +79,6 @@ class MongoTransformer(Transformer):
         # expression_phrase: [ NOT ] ( comparison | "(" expression ")" )
         return self._recursive_expression_phrase(arg)
 
-    @v_args(inline=True)
-    def comparison(self, value):
-        # comparison: constant_first_comparison | property_first_comparison
-        # Note: Do nothing!
-        return value
-
     def property_first_comparison(self, arg):
         # property_first_comparison: property ( value_op_rhs | known_op_rhs | fuzzy_string_op_rhs | set_op_rhs |
         # set_zip_op_rhs | length_op_rhs )
@@ -204,34 +162,6 @@ class MongoTransformer(Transformer):
         # property_zip_addon: ":" property (":" property)*
         raise NotImplementedError
 
-    def property(self, arg):
-        # property: IDENTIFIER ( "." IDENTIFIER )*
-        return ".".join(arg)
-
-    @v_args(inline=True)
-    def string(self, string):
-        # string: ESCAPED_STRING
-        return string.strip('"')
-
-    @v_args(inline=True)
-    def signed_int(self, number):
-        # signed_int : SIGNED_INT
-        return int(number)
-
-    @v_args(inline=True)
-    def number(self, number):
-        # number: SIGNED_INT | SIGNED_FLOAT
-        if number.type == "SIGNED_INT":
-            type_ = int
-        elif number.type == "SIGNED_FLOAT":
-            type_ = float
-        return type_(number)
-
-    def __default__(self, data, children, meta):
-        raise NotImplementedError(
-            f"Calling __default__, i.e., unknown grammar concept. data: {data}, children: {children}, meta: {meta}"
-        )
-
     def _recursive_expression_phrase(self, arg):
         """Helper function for parsing `expression_phrase`. Recursively sorts out
         the correct precedence for `$and`, `$or` and `$nor`.
diff --git a/optimade/grammar/v0.10.1.lark b/optimade/grammar/v0.10.1.lark
deleted file mode 100644
index 573b1d6a..00000000
--- a/optimade/grammar/v0.10.1.lark
+++ /dev/null
@@ -1,111 +0,0 @@
-// optimade v0.10.1 grammar spec in lark grammar format
-
-?start: filter
-filter: expression*
-
-// Values
-constant: string | number
-// Note: support for property in value is OPTIONAL
-value: string | number | property
-
-// Note: not_implemented_string is only here to help Transformers
-non_string_value: number | property
-not_implemented_string: string
-
-// Note:  support for OPERATOR in value_list is OPTIONAL
-value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )*
-// Note: support for OPERATOR in value_zip is OPTIONAL
-value_zip: [ OPERATOR ] value ":" [ OPERATOR ] value (":" [ OPERATOR ] value)*
-value_zip_list: value_zip ( "," value_zip )*
-
-// Expressions
-expression: expression_clause ( _OR expression_clause )*
-expression_clause: expression_phrase ( _AND expression_phrase )*
-expression_phrase: [ NOT ] ( comparison | "(" expression ")" )
-// Note: support for constant_first_comparison is OPTIONAL
-comparison: constant_first_comparison | property_first_comparison
-
-// Note: support for set_zip_op_rhs in comparison is OPTIONAL
-property_first_comparison: property ( value_op_rhs
-                                    | known_op_rhs
-                                    | fuzzy_string_op_rhs
-                                    | set_op_rhs
-                                    | set_zip_op_rhs
-                                    | length_op_rhs )
-
-constant_first_comparison: constant OPERATOR ( non_string_value | not_implemented_string )
-
-value_op_rhs: OPERATOR value
-known_op_rhs: IS ( KNOWN | UNKNOWN )
-fuzzy_string_op_rhs: CONTAINS value
-                   | STARTS [ WITH ] value
-                   | ENDS [ WITH ] value
-// Note: support for ONLY in set_op_rhs is OPTIONAL
-// Note: support for [ OPERATOR ] in set_op_rhs is OPTIONAL
-// set_op_rhs: HAS [ ALL | ANY | ONLY] value_list
-set_op_rhs: HAS ( [ OPERATOR ] value
-                 | ALL value_list
-                 | ANY value_list
-                 | ONLY value_list )
-
-// Note: support for [ OPERATOR ] is OPTIONAL
-length_op_rhs: LENGTH [ OPERATOR ] signed_int
-
-set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list | ANY value_zip_list )
-property_zip_addon: ":" property (":" property)*
-
-// Property syntax
-property: IDENTIFIER ( "." IDENTIFIER )*
-
-// String syntax
-string: ESCAPED_STRING
-
-// Number token syntax
-number: SIGNED_INT | SIGNED_FLOAT
-
-// Custom signed int
-signed_int: SIGNED_INT
-
-// Tokens
-
-// Boolean relations
-_AND:     "AND"
-_OR:      "OR"
-NOT:      "NOT"
-
-IS:       "IS"
-KNOWN:    "KNOWN"
-UNKNOWN:  "UNKNOWN"
-
-CONTAINS: "CONTAINS"
-STARTS:   "STARTS"
-ENDS:     "ENDS"
-WITH:     "WITH"
-
-LENGTH:   "LENGTH"
-HAS:      "HAS"
-ALL:      "ALL"
-ONLY:     "ONLY"
-ANY:      "ANY"
-
-// Comparison OPERATORs
-OPERATOR: ( "<" ["="] | ">" ["="] | ["!"] "=" )
-
-IDENTIFIER: ( "_" | LCASE_LETTER ) ( "_" | LCASE_LETTER | DIGIT )*
-LCASE_LETTER: "a".."z"
-DIGIT: "0".."9"
-
-// Strings
-
-_STRING_INNER: /(.|[\t\f\r\n])*?/
-_STRING_ESC_INNER: _STRING_INNER /(?<!\\)(\\\\)*?/
-
-ESCAPED_STRING : "\"" _STRING_ESC_INNER "\""
-
-
-%import common.SIGNED_INT
-%import common.SIGNED_FLOAT
-
-// White-space
-%import common.WS
-%ignore WS
diff --git a/optimade/grammar/v0.10.1.lark b/optimade/grammar/v0.10.1.lark
new file mode 120000
index 00000000..c5582225
--- /dev/null
+++ b/optimade/grammar/v0.10.1.lark
@@ -0,0 +1,1 @@
+v1.0.0.lark
\ No newline at end of file
diff --git a/optimade/grammar/v1.0.0.lark b/optimade/grammar/v1.0.0.lark
new file mode 100644
index 00000000..974225a6
--- /dev/null
+++ b/optimade/grammar/v1.0.0.lark
@@ -0,0 +1,111 @@
+// optimade v1.0.0 (also valid for v0.10.1) grammar spec in lark grammar format
+
+?start: filter
+filter: expression*
+
+// Values
+constant: string | number
+// Note: support for property in value is OPTIONAL
+value: string | number | property
+
+// Note: not_implemented_string is only here to help Transformers
+non_string_value: number | property
+not_implemented_string: string
+
+// Note:  support for OPERATOR in value_list is OPTIONAL
+value_list: [ OPERATOR ] value ( "," [ OPERATOR ] value )*
+// Note: support for OPERATOR in value_zip is OPTIONAL
+value_zip: [ OPERATOR ] value ":" [ OPERATOR ] value (":" [ OPERATOR ] value)*
+value_zip_list: value_zip ( "," value_zip )*
+
+// Expressions
+expression: expression_clause ( _OR expression_clause )*
+expression_clause: expression_phrase ( _AND expression_phrase )*
+expression_phrase: [ NOT ] ( comparison | "(" expression ")" )
+// Note: support for constant_first_comparison is OPTIONAL
+comparison: constant_first_comparison | property_first_comparison
+
+// Note: support for set_zip_op_rhs in comparison is OPTIONAL
+property_first_comparison: property ( value_op_rhs
+                                    | known_op_rhs
+                                    | fuzzy_string_op_rhs
+                                    | set_op_rhs
+                                    | set_zip_op_rhs
+                                    | length_op_rhs )
+
+constant_first_comparison: constant OPERATOR ( non_string_value | not_implemented_string )
+
+value_op_rhs: OPERATOR value
+known_op_rhs: IS ( KNOWN | UNKNOWN )
+fuzzy_string_op_rhs: CONTAINS value
+                   | STARTS [ WITH ] value
+                   | ENDS [ WITH ] value
+// Note: support for ONLY in set_op_rhs is OPTIONAL
+// Note: support for [ OPERATOR ] in set_op_rhs is OPTIONAL
+// set_op_rhs: HAS [ ALL | ANY | ONLY] value_list
+set_op_rhs: HAS ( [ OPERATOR ] value
+                 | ALL value_list
+                 | ANY value_list
+                 | ONLY value_list )
+
+// Note: support for [ OPERATOR ] is OPTIONAL
+length_op_rhs: LENGTH [ OPERATOR ] signed_int
+
+set_zip_op_rhs: property_zip_addon HAS ( value_zip | ONLY value_zip_list | ALL value_zip_list | ANY value_zip_list )
+property_zip_addon: ":" property (":" property)*
+
+// Property syntax
+property: IDENTIFIER ( "." IDENTIFIER )*
+
+// String syntax
+string: ESCAPED_STRING
+
+// Number token syntax
+number: SIGNED_INT | SIGNED_FLOAT
+
+// Custom signed int
+signed_int: SIGNED_INT
+
+// Tokens
+
+// Boolean relations
+_AND:     "AND"
+_OR:      "OR"
+NOT:      "NOT"
+
+IS:       "IS"
+KNOWN:    "KNOWN"
+UNKNOWN:  "UNKNOWN"
+
+CONTAINS: "CONTAINS"
+STARTS:   "STARTS"
+ENDS:     "ENDS"
+WITH:     "WITH"
+
+LENGTH:   "LENGTH"
+HAS:      "HAS"
+ALL:      "ALL"
+ONLY:     "ONLY"
+ANY:      "ANY"
+
+// Comparison OPERATORs
+OPERATOR: ( "<" ["="] | ">" ["="] | ["!"] "=" )
+
+IDENTIFIER: ( "_" | LCASE_LETTER ) ( "_" | LCASE_LETTER | DIGIT )*
+LCASE_LETTER: "a".."z"
+DIGIT: "0".."9"
+
+// Strings
+
+_STRING_INNER: /(.|[\t\f\r\n])*?/
+_STRING_ESC_INNER: _STRING_INNER /(?<!\\)(\\\\)*?/
+
+ESCAPED_STRING : "\"" _STRING_ESC_INNER "\""
+
+
+%import common.SIGNED_INT
+%import common.SIGNED_FLOAT
+
+// White-space
+%import common.WS
+%ignore WS
diff --git a/optimade/server/entry_collections/mongo.py b/optimade/server/entry_collections/mongo.py
index ea9705be..2e68a41d 100644
--- a/optimade/server/entry_collections/mongo.py
+++ b/optimade/server/entry_collections/mongo.py
@@ -65,9 +65,7 @@ class MongoCollection(EntryCollection):
             MongoTransformer(mapper=resource_mapper),
         )
 
-        self.parser = LarkParser(
-            version=(0, 10, 1), variant="default"
-        )  # The MongoTransformer only supports v0.10.1 as the latest grammar
+        self.parser = LarkParser(version=(1, 0, 0), variant="default")
 
         # check aliases do not clash with mongo operators
         self._check_aliases(self.resource_mapper.all_aliases())
diff --git a/optimade/validator/__init__.py b/optimade/validator/__init__.py
index a6df19a3..c14ae58d 100644
--- a/optimade/validator/__init__.py
+++ b/optimade/validator/__init__.py
@@ -5,7 +5,7 @@ from .validator import ImplementationValidator
 __all__ = ["ImplementationValidator", "validate"]
 
 
-def validate():
+def validate():  # pragma: no cover
     import argparse
     import sys
     import os


Test patch:
diff --git a/tests/filterparser/test_filterparser.py b/tests/filterparser/test_filterparser.py
index f4ef123b..d4a40374 100644
--- a/tests/filterparser/test_filterparser.py
+++ b/tests/filterparser/test_filterparser.py
@@ -40,8 +40,8 @@ class TestParserV0_9_5:
         assert repr(self.parser) is not None
 
 
-class TestParserV0_10_1:
-    version = (0, 10, 1)
+class TestParserV1_0_0:
+    version = (1, 0, 0)
     variant = "default"
 
     @pytest.fixture(autouse=True)
diff --git a/tests/filtertransformers/test_django.py b/tests/filtertransformers/test_django.py
index 55e90b92..8a969b38 100644
--- a/tests/filtertransformers/test_django.py
+++ b/tests/filtertransformers/test_django.py
@@ -16,12 +16,12 @@ test_data = [
 ]
 
 
-class TestLark2Django:
+class TestDjangoTransformer:
     @pytest.fixture(autouse=True)
     def set_up_class(self):
-        from optimade.filtertransformers.django import Lark2Django
+        from optimade.filtertransformers.django import DjangoTransformer
 
-        self.Transformer = Lark2Django()
+        self.Transformer = DjangoTransformer()
 
     def test_query_conversion(self):
         for raw_q, dj_q in test_data:
diff --git a/tests/filtertransformers/test_mongo.py b/tests/filtertransformers/test_mongo.py
index c9462baf..6bcc9892 100644
--- a/tests/filtertransformers/test_mongo.py
+++ b/tests/filtertransformers/test_mongo.py
@@ -7,7 +7,7 @@ from optimade.server.exceptions import BadRequest
 
 
 class TestMongoTransformer:
-    version = (0, 10, 1)
+    version = (1, 0, 0)
     variant = "default"
 
     @pytest.fixture(autouse=True)