Repository: dbt-checkpoint/dbt-checkpoint (language: python)

Issue:
Add integration tests
**Describe the feature you'd like**
I would like to add integration tests to test `pre-commit-dbt` if the new version of `dbt` is released.



Discussion / hints:

Fix patch:
diff --git a/.pre-commit-hooks.yaml b/.pre-commit-hooks.yaml
index 5452e353..8763b765 100644
--- a/.pre-commit-hooks.yaml
+++ b/.pre-commit-hooks.yaml
@@ -4,6 +4,12 @@
     entry: check-column-desc-are-same
     language: python
     files: '.*\.(yml|yaml)$'
+-   id: check-column-name-contract
+    name: Check column name contract
+    description: Check column name abides to contract.
+    entry: check-column-name-contract
+    language: python
+    types: [sql]
 -   id: check-macro-has-description
     name: Check the macro has description
     description: Ensures that the macro has description in properties file.
diff --git a/HOOKS.md b/HOOKS.md
index f5aa1922..dbed1126 100644
--- a/HOOKS.md
+++ b/HOOKS.md
@@ -4,6 +4,7 @@
 
 **Model checks:**
  * [`check-column-desc-are-same`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-column-desc-are-same): Check column descriptions are the same.
+ * [`check-column-name-contract`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-column-name-contract): Check column name abides to contract.
  * [`check-model-columns-have-desc`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-columns-have-desc): Check the model columns have description.
  * [`check-model-has-all-columns`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-has-all-columns): Check the model has all columns in the properties file.
  * [`check-model-has-description`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-has-description): Check the model has description.
@@ -97,6 +98,48 @@ E.g. in two of your models, you have `customer_id` with the description `This is
 - If any column in the found model has different descriptions than others, the hook fails.
 - The description must be in either the yml file **or** the manifest.
 
+-----
+### `check-column-name-contract`
+
+Check that column name abides to a contract, as described in [this blog post](https://emilyriederer.netlify.app/post/column-name-contracts/) by Emily Riederer. A contract consists on a regex pattern and a data type.
+
+#### Arguments
+
+`--pattern`: Regex pattern to match column names.
+`--dtype`: Data type.
+
+#### Example
+```
+repos:
+- repo: https://github.com/offbi/pre-commit-dbt
+ rev: v1.0.0
+ hooks:
+ - id: check-column-name-contract
+   args: [--patter, "(is|has|do)_.*", --dtype, boolean]
+```
+
+#### When to use it
+
+You want to make sure your columns follow a contract, e.g. all your boolean columns start with the prefixes `is_`, `has_` or `do_`.
+
+#### Requirements
+
+| Model exists in `manifest.json` <sup id="a1">[1](#f1)</sup> | Model exists in `catalog.json` <sup id="a2">[2](#f2)</sup> |
+| :----: | :----------: |
+| :x: Not needed | :white_check_mark: Yes |
+
+<sup id="f1">1</sup> It means that you need to run `dbt run`, `dbt compile` before run this hook.<br/>
+<sup id="f2">2</sup> It means that you need to run `dbt docs generate` before run this hook.
+
+#### How it works
+
+- Hook takes all changed `SQL` files.
+- The model name is obtained from the `SQL` file name.
+- The catalog is scanned for a model.
+- If any column in the found model matches the regex pattern and it's data type does not match the contract's data type, the hook fails.
+- If any column in the found model matches the contract's data type and does not match the regex pattern, the hook fails.
+
+
 -----
 ### `check-model-columns-have-desc`
 
diff --git a/README.md b/README.md
index cb41f3e8..eead1fa7 100644
--- a/README.md
+++ b/README.md
@@ -33,6 +33,7 @@ If this is the case, `pre-commit-dbt` is here to help you!
 
 **Model checks:**
  * [`check-column-desc-are-same`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-column-desc-are-same): Check column descriptions are the same.
+ * [`check-column-name-contract`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-column-name-contract): Check column name abides to contract.
  * [`check-model-columns-have-desc`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-columns-have-desc): Check the model columns have description.
  * [`check-model-has-all-columns`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-has-all-columns): Check the model has all columns in the properties file.
  * [`check-model-has-description`](https://github.com/offbi/pre-commit-dbt/blob/main/HOOKS.md#check-model-has-description): Check the model has description.
diff --git a/pre_commit_dbt/check_column_name_contract.py b/pre_commit_dbt/check_column_name_contract.py
new file mode 100644
index 00000000..c4518463
--- /dev/null
+++ b/pre_commit_dbt/check_column_name_contract.py
@@ -0,0 +1,84 @@
+import argparse
+import re
+from typing import Any
+from typing import Dict
+from typing import Optional
+from typing import Sequence
+
+from pre_commit_dbt.utils import add_catalog_args
+from pre_commit_dbt.utils import add_filenames_args
+from pre_commit_dbt.utils import get_filenames
+from pre_commit_dbt.utils import get_json
+from pre_commit_dbt.utils import get_models
+from pre_commit_dbt.utils import JsonOpenError
+
+
+def check_column_name_contract(
+    paths: Sequence[str], pattern: str, dtype: str, catalog: Dict[str, Any]
+) -> int:
+    status_code = 0
+    sqls = get_filenames(paths, [".sql"])
+    filenames = set(sqls.keys())
+    models = get_models(catalog, filenames)
+
+    for model in models:
+        for col in model.node.get("columns", []).values():
+            col_name = col.get("name")
+            col_type = col.get("type")
+
+            # Check all files of type dtype follow naming pattern
+            if dtype == col_type:
+                if re.match(pattern, col_name) is None:
+                    status_code = 1
+                    print(
+                        f"{col_name}: column is of type {dtype} and "
+                        f"does not match regex pattern {pattern}."
+                    )
+
+            # Check all files with naming pattern are of type dtype
+            elif re.match(pattern, col_name):
+                status_code = 1
+                print(
+                    f"{col_name}: column name matches regex pattern {pattern} "
+                    f"and is of type {col_type} instead of {dtype}."
+                )
+
+    return status_code
+
+
+def main(argv: Optional[Sequence[str]] = None) -> int:
+    parser = argparse.ArgumentParser()
+    add_filenames_args(parser)
+    add_catalog_args(parser)
+
+    parser.add_argument(
+        "--pattern",
+        type=str,
+        required=True,
+        help="Regex pattern to match column names.",
+    )
+    parser.add_argument(
+        "--dtype",
+        type=str,
+        required=True,
+        help="Expected data type for the matching columns.",
+    )
+
+    args = parser.parse_args(argv)
+
+    try:
+        catalog = get_json(args.catalog)
+    except JsonOpenError as e:
+        print(f"Unable to load catalog file ({e})")
+        return 1
+
+    return check_column_name_contract(
+        paths=args.filenames,
+        pattern=args.pattern,
+        dtype=args.dtype,
+        catalog=catalog,
+    )
+
+
+if __name__ == "__main__":
+    exit(main())
diff --git a/setup.cfg b/setup.cfg
index 118c1137..fe57b6d7 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -27,6 +27,7 @@ python_requires = >=3.7.1
 [options.entry_points]
 console_scripts =
     check-column-desc-are-same = pre_commit_dbt.check_column_desc_are_same:main
+    check-column-name-contract = pre_commit_dbt.check_column_name_contract:main
     check-macro-has-description = pre_commit_dbt.check_macro_has_description:main
     check-model-columns-have-desc = pre_commit_dbt.check_model_columns_have_desc:main
     check-model-has-all-columns = pre_commit_dbt.check_model_has_all_columns:main


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 6e72fdda..19e2bdff 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -205,6 +205,34 @@
             },
         },
         "model.test.only_model_cols": {"metadata": {}, "columns": {}},
+        "model.test.with_boolean_column_with_prefix": {
+            "metadata": {},
+            "columns": {
+                "is_boolean": {"type": "boolean", "name": "is_boolean"},
+                "COL2": {"type": "TEXT", "name": "COL2"},
+            },
+        },
+        "model.test.with_boolean_column_without_prefix": {
+            "metadata": {},
+            "columns": {
+                "COL1": {"type": "boolean", "name": "COL1"},
+                "COL2": {"type": "TEXT", "name": "COL2"},
+            },
+        },
+        "model.test.without_boolean_column_with_prefix": {
+            "metadata": {},
+            "columns": {
+                "is_not_boolean": {"type": "TEXT", "name": "is_not_boolean"},
+                "COL2": {"type": "TEXT", "name": "COL2"},
+            },
+        },
+        "model.test.without_boolean_column_without_prefix": {
+            "metadata": {},
+            "columns": {
+                "COL1": {"type": "TEXT", "name": "COL1"},
+                "COL2": {"type": "TEXT", "name": "COL2"},
+            },
+        },
     },
     "sources": {
         "source.test.ff.with_catalog_columns": {},
diff --git a/tests/unit/test_check_column_name_contract.py b/tests/unit/test_check_column_name_contract.py
new file mode 100644
index 00000000..ba5799c5
--- /dev/null
+++ b/tests/unit/test_check_column_name_contract.py
@@ -0,0 +1,27 @@
+import pytest
+
+from pre_commit_dbt.check_column_name_contract import main
+
+
+# Input args, valid manifest, expected return value
+TESTS = (
+    (["aa/bb/with_boolean_column_with_prefix.sql"], "is_.*", "boolean", True, 0),
+    (["aa/bb/with_boolean_column_with_prefix.sql"], "is_.*", "boolean", False, 1),
+    (["aa/bb/with_boolean_column_without_prefix.sql"], "is_.*", "boolean", True, 1),
+    (["aa/bb/without_boolean_column_with_prefix.sql"], "is_.*", "boolean", True, 1),
+    (["aa/bb/without_boolean_column_without_prefix.sql"], "is_.*", "boolean", True, 0),
+)
+
+
+@pytest.mark.parametrize(
+    ("input_args", "pattern", "dtype", "valid_catalog", "expected_status_code"), TESTS
+)
+def test_check_column_name_contract(
+    input_args, pattern, dtype, valid_catalog, expected_status_code, catalog_path_str
+):
+    if valid_catalog:
+        input_args.extend(["--catalog", catalog_path_str])
+    input_args.extend(["--pattern", pattern])
+    input_args.extend(["--dtype", dtype])
+    status_code = main(input_args)
+    assert status_code == expected_status_code