Repository: joshuadavidthomas/django-bird (python)

Issue:
Create `discover_components` management command
The `discover_components` method of the component registry is awfully heavy, requiring scanning and parsing all of a project's templates to discover all the uses of a component. I do this so we can know which assets to serve for a given template across extends/includes. I think being able to front load this and not have it happen at app ready time would be a big win. 

Perhaps the management command could be run before `collectstatic` and generate a manifest that could be collected next to the component templates. The asset templatetags could then read the manifest file instead of the registry. 

Pull request description:
Add asset manifest system for better performance
closes #207 
## Summary
- Adds asset manifest system to optimize template-component relationship tracking and asset loading
- Implements production-only optimization with automatic fallback in development mode
- Adds management command for pre-computing component usage in templates

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1c012ec..4defd51 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,6 +22,10 @@ and this project attempts to adhere to [Semantic Versioning](https://semver.org/
 
 - Added `get_component_names_used_in_template` method to ComponentRegistry to access component names used in a template
 - Added `get_component_assets` utility function to staticfiles.py to get assets for a component with optional filtering
+- Added asset manifest system for optimized startup and rendering performance
+- Added `manifest.py` module for generating, loading, and saving asset manifests
+- Added management command `generate_asset_manifest` for pre-computing template-component relationships
+- Added automatic fallback to component scanning in development mode (when DEBUG=True)
 
 ## [0.16.2]
 
diff --git a/docs/assets.md b/docs/assets.md
index 8914818..4b9ee6a 100644
--- a/docs/assets.md
+++ b/docs/assets.md
@@ -133,3 +133,31 @@ python manage.py collectstatic
 ```
 
 This will collect all component assets into your static files directory, allowing you to serve them via your web server, [WhiteNoise](https://whitenoise.readthedocs.io), or a CDN.
+
+## Asset Manifest
+
+For production deployments, django-bird provides a management command to generate an asset manifest:
+
+```bash
+python manage.py generate_asset_manifest
+```
+
+This command creates a manifest file at `STATIC_ROOT/django_bird/manifest.json` that maps templates to their used components. In production mode, this manifest is used to load assets without scanning templates at runtime.
+
+### Integration with collectstatic
+
+For optimal deployment, follow this sequence:
+
+1. Run `python manage.py collectstatic` first to collect all component assets
+2. Then run `python manage.py generate_asset_manifest` to create the manifest file in the collected static files
+
+This ensures that:
+- All component assets are properly collected by the Django staticfiles system
+- The manifest is generated with up-to-date component information
+- The manifest file is placed in the correct location within your static files directory
+
+For automated deployments, you can combine these commands:
+
+```bash
+python manage.py collectstatic --noinput && python manage.py generate_asset_manifest
+```
diff --git a/src/django_bird/management/__init__.py b/src/django_bird/management/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/django_bird/management/commands/__init__.py b/src/django_bird/management/commands/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/django_bird/management/commands/generate_asset_manifest.py b/src/django_bird/management/commands/generate_asset_manifest.py
new file mode 100644
index 0000000..520be69
--- /dev/null
+++ b/src/django_bird/management/commands/generate_asset_manifest.py
@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+from argparse import ArgumentParser
+from typing import Any
+from typing import final
+
+from django.core.management.base import BaseCommand
+
+from django_bird._typing import override
+from django_bird.manifest import default_manifest_path
+from django_bird.manifest import generate_asset_manifest
+from django_bird.manifest import save_asset_manifest
+
+
+@final
+class Command(BaseCommand):
+    help: str = (
+        "Generates a manifest of component usage in templates for loading assets"
+    )
+
+    @override
+    def add_arguments(self, parser: ArgumentParser) -> None:
+        parser.add_argument(
+            "--output",
+            type=str,
+            default=None,
+            help="Path where the manifest file should be saved. Defaults to STATIC_ROOT/django_bird/manifest.json",
+        )
+
+    @override
+    def handle(self, *args: Any, **options: Any) -> None:
+        manifest_data = generate_asset_manifest()
+        output_path = options["output"] or default_manifest_path()
+        save_asset_manifest(manifest_data, output_path)
+        self.stdout.write(
+            self.style.SUCCESS(
+                f"Asset manifest generated successfully at {output_path}"
+            )
+        )
diff --git a/src/django_bird/manifest.py b/src/django_bird/manifest.py
new file mode 100644
index 0000000..f246e98
--- /dev/null
+++ b/src/django_bird/manifest.py
@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+import json
+import logging
+from pathlib import Path
+
+from django.conf import settings
+
+from django_bird.templates import gather_bird_tag_template_usage
+
+logger = logging.getLogger(__name__)
+
+_manifest_cache = None
+
+
+def load_asset_manifest() -> dict[str, list[str]] | None:
+    """Load asset manifest from the default location.
+
+    Returns a simple dict mapping template paths to lists of component names.
+    If the manifest cannot be loaded, returns None and falls back to runtime scanning.
+
+    Returns:
+        dict[str, list[str]] | None: Manifest data or None if not found or invalid
+    """
+    global _manifest_cache
+
+    if _manifest_cache is not None:
+        return _manifest_cache
+
+    if hasattr(settings, "STATIC_ROOT") and settings.STATIC_ROOT:
+        manifest_path = default_manifest_path()
+        if manifest_path.exists():
+            try:
+                with open(manifest_path) as f:
+                    manifest_data = json.load(f)
+                    _manifest_cache = manifest_data
+                    return manifest_data
+            except json.JSONDecodeError:
+                logger.warning(
+                    f"Asset manifest at {manifest_path} contains invalid JSON. Falling back to registry."
+                )
+                return None
+            except (OSError, PermissionError) as e:
+                logger.warning(
+                    f"Error reading asset manifest at {manifest_path}: {str(e)}. Falling back to registry."
+                )
+                return None
+
+    # No manifest found, will fall back to registry
+    return None
+
+
+def generate_asset_manifest() -> dict[str, list[str]]:
+    """Generate a manifest by scanning templates for component usage.
+
+    Returns:
+        dict[str, list[str]]: A dictionary mapping template paths to lists of component names.
+    """
+    template_component_map: dict[str, set[str]] = {}
+    for template_path, component_names in gather_bird_tag_template_usage():
+        # Convert Path objects to strings for JSON
+        template_component_map[str(template_path)] = component_names
+
+    manifest: dict[str, list[str]] = {
+        template: sorted(list(components))
+        for template, components in template_component_map.items()
+    }
+
+    return manifest
+
+
+def save_asset_manifest(manifest_data: dict[str, list[str]], path: Path | str) -> None:
+    """Save asset manifest to a file.
+
+    Args:
+        manifest_data: The manifest data to save
+        path: Path where to save the manifest
+    """
+    path_obj = Path(path)
+    path_obj.parent.mkdir(parents=True, exist_ok=True)
+
+    with open(path_obj, "w") as f:
+        json.dump(manifest_data, f, indent=2)
+
+
+def default_manifest_path() -> Path:
+    """Get the default manifest path.
+
+    Returns:
+        Path: The default path for the asset manifest file
+    """
+    if hasattr(settings, "STATIC_ROOT") and settings.STATIC_ROOT:
+        return Path(settings.STATIC_ROOT) / "django_bird" / "manifest.json"
+    else:
+        # Fallback for when STATIC_ROOT is not set
+        return Path("django_bird-asset-manifest.json")
diff --git a/src/django_bird/templatetags/tags/asset.py b/src/django_bird/templatetags/tags/asset.py
index 02fb1c3..1307511 100644
--- a/src/django_bird/templatetags/tags/asset.py
+++ b/src/django_bird/templatetags/tags/asset.py
@@ -5,11 +5,13 @@ from enum import Enum
 from typing import final
 
 from django import template
+from django.conf import settings
 from django.template.base import Parser
 from django.template.base import Token
 from django.template.context import Context
 
 from django_bird._typing import override
+from django_bird.manifest import load_asset_manifest
 
 
 class AssetTag(Enum):
@@ -35,18 +37,40 @@ class AssetNode(template.Node):
     @override
     def render(self, context: Context) -> str:
         from django_bird.components import components
+        from django_bird.staticfiles import Asset
+        from django_bird.staticfiles import get_component_assets
 
         template = getattr(context, "template", None)
         if not template:
             return ""
-        used_components = components.get_component_usage(template.origin.name)
-        assets = set(
-            asset
-            for component in used_components
-            for asset in component.assets
-            if asset.type.tag == self.asset_tag
-        )
+
+        template_path = template.origin.name
+
+        used_components = []
+
+        # Only use manifest in production mode
+        if not settings.DEBUG:
+            manifest = load_asset_manifest()
+            if manifest and template_path in manifest:
+                # Use manifest for component names in production
+                component_names = manifest[template_path]
+                used_components = [
+                    components.get_component(name) for name in component_names
+                ]
+
+        # If we're in development or there was no manifest data, use registry
+        if not used_components:
+            used_components = list(components.get_component_usage(template_path))
+
+        assets: set[Asset] = set()
+        for component in used_components:
+            component_assets = get_component_assets(component)
+            assets.update(
+                asset for asset in component_assets if asset.type.tag == self.asset_tag
+            )
+
         if not assets:
             return ""
+
         rendered = [asset.render() for asset in sorted(assets, key=lambda a: a.path)]
         return "\n".join(rendered)


Test patch:
diff --git a/tests/templatetags/test_asset.py b/tests/templatetags/test_asset.py
index 09e2cf9..09ecf1a 100644
--- a/tests/templatetags/test_asset.py
+++ b/tests/templatetags/test_asset.py
@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+import shutil
+
 import pytest
 from django.template.base import Parser
 from django.template.base import Token
@@ -7,7 +9,10 @@ from django.template.base import TokenType
 from django.template.context import Context
 from django.template.exceptions import TemplateSyntaxError
 from django.template.loader import get_template
+from django.test import override_settings
 
+from django_bird.manifest import generate_asset_manifest
+from django_bird.manifest import save_asset_manifest
 from django_bird.staticfiles import CSS
 from django_bird.staticfiles import JS
 from django_bird.templatetags.tags.asset import AssetNode
@@ -380,3 +385,173 @@ class TestNode:
         node = AssetNode(CSS)
         context = Context({})
         assert node.render(context) == ""
+
+
+class TestManifest:
+    @pytest.fixture
+    def static_root(self, tmp_path):
+        static_dir = tmp_path / "static"
+        static_dir.mkdir()
+
+        with override_settings(STATIC_ROOT=str(static_dir)):
+            yield static_dir
+
+        shutil.rmtree(static_dir)
+
+    def test_asset_tag_with_manifest_in_production(
+        self, create_template, templates_dir, static_root, registry
+    ):
+        button = TestComponent(
+            name="button", content="<button>{{ slot }}</button>"
+        ).create(templates_dir)
+
+        button_css = TestAsset(
+            component=button,
+            content=".button { color: blue; }",
+            asset_type=CSS,
+        ).create()
+
+        template_path = templates_dir / "manifest_test.html"
+        template_path.write_text("""
+        <html>
+        <head>
+            <title>Test</title>
+            {% bird:css %}
+        </head>
+        <body>
+            {% bird button %}Test Button{% endbird %}
+        </body>
+        </html>
+        """)
+
+        registry.discover_components()
+
+        manifest_data = generate_asset_manifest()
+
+        manifest_path = static_root / "django_bird"
+        manifest_path.mkdir(parents=True)
+        save_asset_manifest(manifest_data, manifest_path / "manifest.json")
+
+        with override_settings(DEBUG=False):
+            template = create_template(template_path)
+            rendered = template.render({})
+
+            assert (
+                f'<link rel="stylesheet" href="/static/bird/{button_css.file.name}">'
+                in rendered
+            )
+
+    def test_asset_tag_in_debug_mode(
+        self, create_template, templates_dir, static_root, registry
+    ):
+        button = TestComponent(
+            name="button", content="<button>{{ slot }}</button>"
+        ).create(templates_dir)
+
+        button_css = TestAsset(
+            component=button,
+            content=".button { color: blue; }",
+            asset_type=CSS,
+        ).create()
+
+        template_path = templates_dir / "debug_test.html"
+        template_path.write_text("""
+        <html>
+        <head>
+            <title>Test</title>
+            {% bird:css %}
+        </head>
+        <body>
+            {% bird button %}Test Button{% endbird %}
+        </body>
+        </html>
+        """)
+
+        registry.discover_components()
+
+        manifest_data = {"not/a/real/template.html": ["not-button"]}
+
+        manifest_path = static_root / "django_bird"
+        manifest_path.mkdir(parents=True)
+        save_asset_manifest(manifest_data, manifest_path / "manifest.json")
+
+        with override_settings(DEBUG=True):
+            template = create_template(template_path)
+            rendered = template.render({})
+
+            assert (
+                f'<link rel="stylesheet" href="/static/bird/{button_css.file.name}">'
+                in rendered
+            )
+
+    def test_asset_tag_fallback_when_template_not_in_manifest(
+        self, create_template, templates_dir, static_root, registry
+    ):
+        button = TestComponent(
+            name="button", content="<button>{{ slot }}</button>"
+        ).create(templates_dir)
+
+        button_css = TestAsset(
+            component=button,
+            content=".button { color: blue; }",
+            asset_type=CSS,
+        ).create()
+
+        template_path = templates_dir / "not_in_manifest.html"
+        template_path.write_text("""
+        <html>
+        <head>
+            <title>Test</title>
+            {% bird:css %}
+        </head>
+        <body>
+            {% bird button %}Test Button{% endbird %}
+        </body>
+        </html>
+        """)
+
+        # Create a different template to put in the manifest
+        other_path = templates_dir / "other_template.html"
+        other_path.write_text("""
+        <html><body>Other template</body></html>
+        """)
+
+        registry.discover_components()
+
+        manifest_data = {str(other_path): ["button"]}
+
+        manifest_path = static_root / "django_bird"
+        manifest_path.mkdir(parents=True)
+        save_asset_manifest(manifest_data, manifest_path / "manifest.json")
+
+        with override_settings(DEBUG=False):
+            template = create_template(template_path)
+            rendered = template.render({})
+
+            assert (
+                f'<link rel="stylesheet" href="/static/bird/{button_css.file.name}">'
+                in rendered
+            )
+
+    def test_asset_tag_renders_nothing_when_no_component_found(
+        self, create_template, templates_dir
+    ):
+        template_path = templates_dir / "no_components.html"
+        template_path.write_text("""
+        <html>
+        <head>
+            <title>Test</title>
+            {% bird:css %}
+        </head>
+        <body>
+            No bird components here
+            {% bird:js %}
+        </body>
+        </html>
+        """)
+
+        template = create_template(template_path)
+        rendered = template.render({})
+
+        assert '<link rel="stylesheet"' not in rendered
+        assert "<script src=" not in rendered
diff --git a/tests/test_manifest.py b/tests/test_manifest.py
new file mode 100644
index 0000000..22064e4
--- /dev/null
+++ b/tests/test_manifest.py
@@ -0,0 +1,305 @@
+from __future__ import annotations
+
+import json
+import shutil
+from io import StringIO
+from pathlib import Path
+
+import pytest
+from django.core.management import call_command
+from django.test import override_settings
+
+from django_bird.manifest import default_manifest_path
+from django_bird.manifest import generate_asset_manifest
+from django_bird.manifest import load_asset_manifest
+from django_bird.manifest import save_asset_manifest
+from tests.utils import TestComponent
+
+
+@pytest.fixture(autouse=True)
+def reset_manifest_cache():
+    # Access and reset the private module-level cache variable
+    import django_bird.manifest
+
+    django_bird.manifest._manifest_cache = None
+    yield
+    django_bird.manifest._manifest_cache = None
+
+
+@pytest.fixture
+def static_root(tmp_path):
+    static_dir = tmp_path / "static"
+    static_dir.mkdir()
+
+    with override_settings(STATIC_ROOT=str(static_dir)):
+        yield static_dir
+
+    shutil.rmtree(static_dir)
+
+
+def test_load_asset_manifest_nonexistent():
+    with override_settings(STATIC_ROOT="/nonexistent/path/"):
+        loaded_manifest = load_asset_manifest()
+        assert loaded_manifest is None
+
+
+def test_load_asset_manifest_invalid_json(static_root):
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "manifest.json"
+    manifest_path.write_text("{ this is not valid JSON }")
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest is None
+
+
+def test_load_asset_manifest_permission_error(static_root, monkeypatch):
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "manifest.json"
+    manifest_path.write_text("{}")
+
+    # Mock open to raise a permission error
+    def mock_open_with_error(*args, **kwargs):
+        if str(manifest_path) in str(args[0]):
+            raise PermissionError("Permission denied")
+        return open(*args, **kwargs)
+
+    monkeypatch.setattr("builtins.open", mock_open_with_error)
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest is None
+
+
+def test_load_asset_manifest_os_error(static_root, monkeypatch):
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "asset-manifest.json"
+    manifest_path.write_text("{}")
+
+    # Mock open to raise an OS error
+    def mock_open_with_error(*args, **kwargs):
+        if str(manifest_path) in str(args[0]):
+            raise OSError("I/O error")
+        return open(*args, **kwargs)
+
+    monkeypatch.setattr("builtins.open", mock_open_with_error)
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest is None
+
+
+def test_load_asset_manifest_from_static_root(static_root):
+    test_manifest_data = {
+        "/path/to/template1.html": ["button", "card"],
+        "/path/to/template2.html": ["accordion", "tab"],
+    }
+
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "manifest.json"
+    with open(manifest_path, "w") as f:
+        json.dump(test_manifest_data, f)
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest == test_manifest_data
+
+
+def test_load_asset_manifest_static_root_invalid_json(static_root):
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "manifest.json"
+    manifest_path.write_text("{ invalid json here }")
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest is None
+
+
+def test_load_asset_manifest_static_root_permission_error(static_root, monkeypatch):
+    django_bird_dir = static_root / "django_bird"
+    django_bird_dir.mkdir()
+
+    manifest_path = django_bird_dir / "manifest.json"
+    manifest_path.write_text("{}")
+
+    # Mock open to raise permission error for this specific path
+    def mock_open_with_error(*args, **kwargs):
+        if str(manifest_path) in str(args[0]):
+            raise PermissionError("Permission denied")
+        return open(*args, **kwargs)
+
+    monkeypatch.setattr("builtins.open", mock_open_with_error)
+
+    loaded_manifest = load_asset_manifest()
+    assert loaded_manifest is None
+
+
+def test_generate_asset_manifest(templates_dir, registry):
+    template1 = templates_dir / "test_manifest1.html"
+    template1.write_text("""
+    <html>
+    <head>
+        <title>Test</title>
+    </head>
+    <body>
+        {% bird button %}Button{% endbird %}
+        {% bird card %}Card{% endbird %}
+    </body>
+    </html>
+    """)
+
+    template2 = templates_dir / "test_manifest2.html"
+    template2.write_text("""
+    <html>
+    <head>
+        <title>Test</title>
+    </head>
+    <body>
+        {% bird accordion %}Accordion{% endbird %}
+        {% bird tab %}Tab{% endbird %}
+    </body>
+    </html>
+    """)
+
+    TestComponent(name="button", content="<button>{{ slot }}</button>").create(
+        templates_dir
+    )
+    TestComponent(name="card", content="<div class='card'>{{ slot }}</div>").create(
+        templates_dir
+    )
+    TestComponent(
+        name="accordion", content="<div class='accordion'>{{ slot }}</div>"
+    ).create(templates_dir)
+    TestComponent(name="tab", content="<div class='tab'>{{ slot }}</div>").create(
+        templates_dir
+    )
+
+    registry.discover_components()
+
+    manifest = generate_asset_manifest()
+
+    all_keys = list(manifest.keys())
+    template1_key = [k for k in all_keys if str(template1) in k][0]
+    template2_key = [k for k in all_keys if str(template2) in k][0]
+
+    assert sorted(manifest[template1_key]) == sorted(["button", "card"])
+    assert sorted(manifest[template2_key]) == sorted(["accordion", "tab"])
+
+
+def test_save_and_load_asset_manifest(tmp_path):
+    test_manifest_data = {
+        "/path/to/template1.html": ["button", "card"],
+        "/path/to/template2.html": ["accordion", "tab"],
+    }
+
+    output_path = tmp_path / "test-manifest.json"
+
+    save_asset_manifest(test_manifest_data, output_path)
+
+    assert output_path.exists()
+
+    with open(output_path) as f:
+        loaded_data = json.load(f)
+
+    assert loaded_data == test_manifest_data
+
+
+def test_default_manifest_path():
+    with override_settings(STATIC_ROOT="/path/to/static"):
+        path = default_manifest_path()
+        assert path == Path("/path/to/static/django_bird/manifest.json")
+
+    with override_settings(STATIC_ROOT=None):
+        path = default_manifest_path()
+        assert path == Path("django_bird-asset-manifest.json")
+
+
+class TestManagementCommand:
+    """Tests for the generate_asset_manifest management command."""
+
+    def test_generate_asset_manifest_command_default(
+        self, static_root, templates_dir, registry
+    ):
+        TestComponent(name="test_cmd", content="<div>{{ slot }}</div>").create(
+            templates_dir
+        )
+
+        template_path = templates_dir / "manifest_cmd_test.html"
+        template_path.write_text("""
+        <html>
+        <body>
+            {% bird test_cmd %}Test Command{% endbird %}
+        </body>
+        </html>
+        """)
+
+        registry.discover_components()
+
+        stdout = StringIO()
+
+        call_command("generate_asset_manifest", stdout=stdout)
+
+        output = stdout.getvalue()
+
+        assert "Asset manifest generated successfully" in output
+
+        manifest_path = static_root / "django_bird" / "manifest.json"
+        assert manifest_path.exists()
+
+        with open(manifest_path) as f:
+            manifest_data = json.load(f)
+
+        template_keys = [
+            k for k in manifest_data.keys() if "manifest_cmd_test.html" in k
+        ]
+        assert len(template_keys) == 1
+        assert "test_cmd" in manifest_data[template_keys[0]]
+
+    def test_generate_asset_manifest_command_with_options(
+        self, tmp_path, templates_dir, registry
+    ):
+        TestComponent(name="test_cmd2", content="<div>{{ slot }}</div>").create(
+            templates_dir
+        )
+
+        template_path = templates_dir / "manifest_cmd_options.html"
+        template_path.write_text("""
+        <html>
+        <body>
+            {% bird test_cmd2 %}Test Command 2{% endbird %}
+        </body>
+        </html>
+        """)
+
+        registry.discover_components()
+
+        custom_output = tmp_path / "custom" / "manifest.json"
+
+        stdout = StringIO()
+
+        call_command(
+            "generate_asset_manifest",
+            output=str(custom_output),
+            stdout=stdout,
+        )
+
+        output = stdout.getvalue()
+
+        assert f"Asset manifest generated successfully at {custom_output}" in output
+        assert custom_output.exists()
+
+        with open(custom_output) as f:
+            content = f.read()
+            manifest_data = json.loads(content)
+
+        template_keys = [
+            k for k in manifest_data.keys() if "manifest_cmd_options.html" in k
+        ]
+
+        assert len(template_keys) == 1
+        assert "test_cmd2" in manifest_data[template_keys[0]]