Repository: datacontract/datacontract-cli
Pull request #767: Adding mermaid format for export and adding mermaid figure in html export
Author: login: mathispernias
Created: 2025-05-20T18:12:09Z | Merged: 2025-05-22T18:27:48Z

- [x] Tests pass
- [x] ruff format
- [x] README.md updated (if relevant)
- [x] CHANGELOG.md entry added



Diff:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f98bbf003..8e4039c43 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## Unreleased
+### Added
+- `datacontract export --format mermaid` Export to [Mermaid](https://mermaid-js.github.io/mermaid/#/) 
+
+### Changed
+- Adding the mermaid figure to the html export 
+- ODCS export: Export physical type if the physical type is configured in config object
 
 ### Changed
 =======
@@ -23,7 +29,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [0.10.26] - 2025-05-16
 
 ### Changed
-
 - Databricks: Add support for Variant type (#758)
 - `datacontract export --format odcs`: Export physical type if the physical type is configured in
   config object (#757)
diff --git a/README.md b/README.md
index 370b1a768..f3766c6e5 100644
--- a/README.md
+++ b/README.md
@@ -828,10 +828,10 @@ models:
 │ *  --format          [jsonschema|pydantic-model|sodacl|db  The export format. [default: None]    │
 │                      t|dbt-sources|dbt-staging-sql|odcs|r  [required]                            │
 │                      df|avro|protobuf|great-expectations|                                        │
-│                      terraform|avro-idl|sql|sql-query|htm                                        │
-│                      l|go|bigquery|dbml|spark|sqlalchemy|                                        │
-│                      data-caterer|dcs|markdown|iceberg|cu                                        │
-│                      stom]                                                                       │
+│                      terraform|avro-idl|sql|sql-query|mer                                        │
+│                      maid|html|go|bigquery|dbml|spark|sql                                        │
+│                      alchemy|data-caterer|dcs|markdown|ic                                        │
+│                      eberg|custom]                                                               │
 │    --output          PATH                                  Specify the file path where the       │
 │                                                            exported data will be saved. If no    │
 │                                                            path is provided, the output will be  │
diff --git a/datacontract/catalog/catalog.py b/datacontract/catalog/catalog.py
index 03284defb..50a92aecc 100644
--- a/datacontract/catalog/catalog.py
+++ b/datacontract/catalog/catalog.py
@@ -6,7 +6,7 @@
 from jinja2 import Environment, PackageLoader, select_autoescape
 
 from datacontract.data_contract import DataContract
-from datacontract.export.html_export import get_version
+from datacontract.export.html_exporter import get_version
 from datacontract.model.data_contract_specification import DataContractSpecification
 
 
diff --git a/datacontract/export/exporter.py b/datacontract/export/exporter.py
index f0b9d2f3e..2c3864b38 100644
--- a/datacontract/export/exporter.py
+++ b/datacontract/export/exporter.py
@@ -33,6 +33,7 @@ class ExportFormat(str, Enum):
     avro_idl = "avro-idl"
     sql = "sql"
     sql_query = "sql-query"
+    mermaid = "mermaid"
     html = "html"
     go = "go"
     bigquery = "bigquery"
diff --git a/datacontract/export/exporter_factory.py b/datacontract/export/exporter_factory.py
index 4804f7993..c9d8419ce 100644
--- a/datacontract/export/exporter_factory.py
+++ b/datacontract/export/exporter_factory.py
@@ -89,6 +89,12 @@ def load_module_class(module_path, class_name):
     class_name="DbtExporter",
 )
 
+exporter_factory.register_lazy_exporter(
+    name=ExportFormat.mermaid,
+    module_path="datacontract.export.mermaid_exporter",
+    class_name="MermaidExporter",
+)
+
 exporter_factory.register_lazy_exporter(
     name=ExportFormat.dbt_sources,
     module_path="datacontract.export.dbt_converter",
@@ -127,7 +133,7 @@ def load_module_class(module_path, class_name):
 
 exporter_factory.register_lazy_exporter(
     name=ExportFormat.html,
-    module_path="datacontract.export.html_export",
+    module_path="datacontract.export.html_exporter",
     class_name="HtmlExporter",
 )
 
diff --git a/datacontract/export/html_export.py b/datacontract/export/html_exporter.py
similarity index 93%
rename from datacontract/export/html_export.py
rename to datacontract/export/html_exporter.py
index bc1b1c101..77d66f9d1 100644
--- a/datacontract/export/html_export.py
+++ b/datacontract/export/html_exporter.py
@@ -17,6 +17,8 @@ def export(self, data_contract, model, server, sql_server_type, export_args) ->
 
 
 def to_html(data_contract_spec: DataContractSpecification) -> str:
+    from datacontract.export.mermaid_exporter import to_mermaid
+
     # Load templates from templates folder
     package_loader = PackageLoader("datacontract", "templates")
     env = Environment(
@@ -54,6 +56,9 @@ def to_html(data_contract_spec: DataContractSpecification) -> str:
     formatted_date = now.strftime("%d %b %Y %H:%M:%S UTC")
     datacontract_cli_version = get_version()
 
+    # Get the mermaid diagram
+    mermaid_diagram = to_mermaid(data_contract_spec)
+
     # Render the template with necessary data
     html_string = template.render(
         datacontract=data_contract_spec,
@@ -62,6 +67,7 @@ def to_html(data_contract_spec: DataContractSpecification) -> str:
         datacontract_yaml=datacontract_yaml,
         formatted_date=formatted_date,
         datacontract_cli_version=datacontract_cli_version,
+        mermaid_diagram=mermaid_diagram,
     )
 
     return html_string
diff --git a/datacontract/export/mermaid_exporter.py b/datacontract/export/mermaid_exporter.py
new file mode 100644
index 000000000..ea0b0b215
--- /dev/null
+++ b/datacontract/export/mermaid_exporter.py
@@ -0,0 +1,32 @@
+from datacontract.export.exporter import Exporter
+from datacontract.model.data_contract_specification import DataContractSpecification
+
+
+class MermaidExporter(Exporter):
+    def export(self, data_contract, model, server, sql_server_type, export_args) -> dict:
+        return to_mermaid(data_contract)
+
+
+def to_mermaid(data_contract_spec: DataContractSpecification) -> str | None:
+    mmd_entity = "erDiagram\n\t"
+    mmd_references = []
+    try:
+        for model_name, model in data_contract_spec.models.items():
+            entity_block = ""
+            for field_name, field in model.fields.items():
+                entity_block += f"\t{field_name.replace('#', 'Nb').replace(' ', '_').replace('/', 'by')}{'🔑' if field.primaryKey or (field.unique and field.required) else ''}{'⌘' if field.references else ''} {field.type}\n"
+                if field.references:
+                    mmd_references.append(
+                        f'"📑{field.references.split(".")[0] if "." in field.references else ""}"'
+                        + "}o--{ ||"
+                        + f'"📑{model_name}"'
+                    )
+            mmd_entity += f'\t"**{model_name}**"' + "{\n" + entity_block + "}\n"
+
+        if mmd_entity == "":
+            return None
+        else:
+            return f"{mmd_entity}\n"
+    except Exception as e:
+        print(f"error : {e}")
+        return None
diff --git a/datacontract/export/sql_type_converter.py b/datacontract/export/sql_type_converter.py
index a89948a3e..75168e5c3 100644
--- a/datacontract/export/sql_type_converter.py
+++ b/datacontract/export/sql_type_converter.py
@@ -159,7 +159,11 @@ def convert_to_dataframe(field: Field) -> None | str:
 # https://docs.databricks.com/en/sql/language-manual/sql-ref-datatypes.html
 def convert_to_databricks(field: Field) -> None | str:
     type = field.type
-    if field.config and "databricksType" in field.config and type.lower() not in ["array", "object", "record", "struct"]:
+    if (
+        field.config
+        and "databricksType" in field.config
+        and type.lower() not in ["array", "object", "record", "struct"]
+    ):
         return field.config["databricksType"]
     if type is None:
         return None
diff --git a/datacontract/templates/datacontract.html b/datacontract/templates/datacontract.html
index 31aa60a28..41c393f9f 100644
--- a/datacontract/templates/datacontract.html
+++ b/datacontract/templates/datacontract.html
@@ -5,6 +5,8 @@
   <meta charset="UTF-8">
   <meta name="viewport" content="width=device-width, initial-scale=1.0">
   {# <script src="https://cdn.tailwindcss.com"></script> #}
+  <script src="https://unpkg.com/@panzoom/panzoom@4.6.0/dist/panzoom.min.js"></script>
+
   <style>
     {{ style | safe }}
   </style>
@@ -29,7 +31,6 @@
       </div>
     </div>
   </nav>
-
   <main class="pb-7">
 
     <div class="pt-5 mx-auto max-w-7xl sm:px-6 lg:px-8">
@@ -77,7 +78,6 @@ <h2 class="text-2xl font-bold leading-7 text-gray-900 sm:truncate sm:text-3xl sm
             {{ render_partial('partials/datacontract_information.html', datacontract = datacontract) }}
           </section>
 
-
           {% if datacontract.servers %}
           <section id="servers">
             <div class="px-4 sm:px-0">
@@ -103,6 +103,52 @@ <h1 class="text-base font-semibold leading-6 text-gray-900" id="servers">Servers
           </section>
           {% endif %}
 
+          <section id="diagram" class="mt-6">
+            <div class="px-4 sm:px-0">
+              <h1 class="text-base font-semibold leading-6 text-gray-900">Entity Relationship
+                Diagram</h1>
+              <p class="text-sm text-gray-500">Visual representation of data model relationships</p>
+            </div>
+            <div class="mt-3">
+              <div class="overflow-hidden bg-white shadow-sm ring-1 ring-gray-900/5 sm:rounded-lg">
+                <div class="diagram-container p-4 w-full" id="diagram-container">
+                  <pre class="mermaid">
+                    {{ mermaid_diagram }}
+                  </pre>
+                </div>
+              </div>
+            </div>
+            <script type="module">
+              import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.esm.min.mjs';
+
+              mermaid.initialize({
+                startOnLoad: false,
+                theme: 'neutral'
+              });
+              await mermaid.run({
+                querySelector: '.mermaid',
+                postRenderCallback: (id) => {
+                  const container = document.getElementById("diagram-container");
+                  const svgElement = container.querySelector("svg");
+
+                  if (svgElement) {
+                    // Initialize Panzoom
+                    const panzoomInstance = Panzoom(svgElement, {
+                      maxScale: 5,
+                      minScale: 0.5,
+                      step: 0.1,
+                    });
+
+                    // Mouse wheel zoom
+                    container.addEventListener("wheel", (event) => {
+                      event.preventDefault();
+                      panzoomInstance.zoomWithWheel(event);
+                    });
+                  }
+                }
+              });
+            </script>
+          </section>
 
           <section id="models">
             <div class="flex justify-between">
diff --git a/tests/test_export_mermaid.py b/tests/test_export_mermaid.py
new file mode 100644
index 000000000..9b1e31bd4
--- /dev/null
+++ b/tests/test_export_mermaid.py
@@ -0,0 +1,56 @@
+import os
+from pathlib import Path
+
+from typer.testing import CliRunner
+
+from datacontract.cli import app
+
+
+def test_cli():
+    runner = CliRunner()
+    result = runner.invoke(app, ["export", "./fixtures/export/datacontract.yaml", "--format", "mermaid"])
+    assert result.exit_code == 0
+
+
+def test_cli_with_output(tmp_path: Path):
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            "./fixtures/export/datacontract.yaml",
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+    assert os.path.exists(tmp_path / "datacontract.mermaid")
+
+
+def test_mermaid_structure(tmp_path: Path):
+    datacontract_file = "fixtures/export/datacontract.yaml"
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            datacontract_file,
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+
+    with open(tmp_path / "datacontract.mermaid") as file:
+        content = file.read()
+
+    # Check structure
+    assert "erDiagram" in content
+    assert "orders" in content
+    assert "order_id" in content
+    assert "order_total" in content
+    assert "order_status" in content


Reviews:
user: login: simonharrer
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T07:08:00Z
user: login: simonharrer
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T07:09:03Z
user: login: mathispernias
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T13:32:02Z
user: login: mathispernias
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T13:54:45Z
user: login: jochenchrist
body: Thanks for your contribution!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T18:24:14Z

Review comments:
diff_hunk: @@ -54,6 +56,9 @@ def to_html(data_contract_spec: DataContractSpecification) -> str:
     formatted_date = now.strftime("%d %b %Y %H:%M:%S UTC")
     datacontract_cli_version = get_version()
 
+    # Obtenir le diagramme Mermaid
path: datacontract/export/html_export.py
user: login: simonharrer
body: comments in english, please. :-)
created_at: 2025-05-21T07:08:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,77 @@
+import os
+from pathlib import Path
+
+from typer.testing import CliRunner
+
+from datacontract.cli import app
+
+
+def test_cli():
+    runner = CliRunner()
+    result = runner.invoke(app, ["export", "./fixtures/export/datacontract.yaml", "--format", "mermaid"])
+    assert result.exit_code == 0
+
+
+def test_cli_with_output(tmp_path: Path):
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            "./fixtures/export/datacontract.yaml",
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+    assert os.path.exists(tmp_path / "datacontract.mermaid")
+
+
+def test_mermaid_content(tmp_path: Path):
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            "./fixtures/export/datacontract.yaml",
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+
+    # Read the generated file
+    with open(tmp_path / "datacontract.mermaid", "r") as f:
+        content = f.read()
+
+    # Basic content validation
+    assert "erDiagram" in content
+
+
+def test_mermaid_expected(tmp_path: Path):
+    expected = (
+        """erDiagram\n\t\t"ðŸ“‘orders"{\n\torder_idðŸ”‘ varchar\n\torder_total bigint\n\torder_status text\n}\n\n"""
path: tests/test_export_mermaid.py
user: login: simonharrer
body: I stumbled upon ðŸ. What do they mean? Do we need them?
created_at: 2025-05-21T07:09:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,77 @@
+import os
+from pathlib import Path
+
+from typer.testing import CliRunner
+
+from datacontract.cli import app
+
+
+def test_cli():
+    runner = CliRunner()
+    result = runner.invoke(app, ["export", "./fixtures/export/datacontract.yaml", "--format", "mermaid"])
+    assert result.exit_code == 0
+
+
+def test_cli_with_output(tmp_path: Path):
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            "./fixtures/export/datacontract.yaml",
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+    assert os.path.exists(tmp_path / "datacontract.mermaid")
+
+
+def test_mermaid_content(tmp_path: Path):
+    runner = CliRunner()
+    result = runner.invoke(
+        app,
+        [
+            "export",
+            "./fixtures/export/datacontract.yaml",
+            "--format",
+            "mermaid",
+            "--output",
+            tmp_path / "datacontract.mermaid",
+        ],
+    )
+    assert result.exit_code == 0
+
+    # Read the generated file
+    with open(tmp_path / "datacontract.mermaid", "r") as f:
+        content = f.read()
+
+    # Basic content validation
+    assert "erDiagram" in content
+
+
+def test_mermaid_expected(tmp_path: Path):
+    expected = (
+        """erDiagram\n\t\t"ðŸ“‘orders"{\n\torder_idðŸ”‘ varchar\n\torder_total bigint\n\torder_status text\n}\n\n"""
path: tests/test_export_mermaid.py
user: login: mathispernias
body: The characters like ðŸ"' and âŒ˜ "' are supposed to be emojis that serve as visual markers in the ER diagrams. I removed this test and added just a check of structural elements rather than exact string matching
created_at: 2025-05-21T13:32:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -54,6 +56,9 @@ def to_html(data_contract_spec: DataContractSpecification) -> str:
     formatted_date = now.strftime("%d %b %Y %H:%M:%S UTC")
     datacontract_cli_version = get_version()
 
+    # Obtenir le diagramme Mermaid
path: datacontract/export/html_export.py
user: login: mathispernias
body: i changed it !
created_at: 2025-05-21T13:54:45Z
author_association: CONTRIBUTOR

PR comments:
user: login: jochenchrist
created_at: 2025-05-20T19:26:47Z
body: Not sure if I missed something, but this does not look final yet:
<img width="1512" alt="image" src="https://github.com/user-attachments/assets/17c38322-d377-4be1-a58e-f4a39e6671f5" />

author_association: CONTRIBUTOR
user: login: mathispernias
created_at: 2025-05-20T20:59:12Z
body: ![Capture d’écran 2025-05-20 165421](https://github.com/user-attachments/assets/976e9c4c-b7c3-45c3-9ddc-de8398d21f73)
This the version I had I pushed the wrong one my bad. 
author_association: CONTRIBUTOR