Repository: Chainlit/chainlit
Issue category: Performance Issue

Issue:
Prevent attempting to setup data layer on every call toget_data_layer() when no data layer's defined.
**Is your feature request related to a problem? Please describe.**

Currently, the `get_data_layer()` function attempts to set up the data layer on every call, even when no data layer is defined. This can lead to unnecessary checks and potential performance issues if the setup logic is executed repeatedly.

**Describe the solution you'd like**

Modify the `get_data_layer()` function to return early if the `_data_layer` is already set. This prevents re-initialization by checking if `_data_layer` is `None` before attempting to set it up. The current implementation already includes this logic:

```python
def get_data_layer():
    global _data_layer

    if not _data_layer:
        if api_key := os.environ.get("LITERAL_API_KEY"):
            from .literalai import LiteralDataLayer

            # support legacy LITERAL_SERVER variable as fallback
            server = os.environ.get("LITERAL_API_URL") or os.environ.get(
                "LITERAL_SERVER"
            )
            _data_layer = LiteralDataLayer(api_key=api_key, server=server)

    return _data_layer
```

**Describe alternatives you've considered**

We should discriminate between there intentionally not being a data layer setup and there being a default data layer (e.g. LiteralAI). Perhaps we could add a 'null' data layer or something similar?

An alternative could be to refactor the data layer setup logic into a separate initialization function that is explicitly called once during application startup, rather than being embedded within `get_data_layer()`.

**Additional context**

The current implementation already prevents re-initialization by checking if `_data_layer` is `None` before setting it up. If `_data_layer` is already initialized, it simply returns the existing instance <sup>[[1]](https://github.com/Chainlit/chainlit/blob/main/backend/chainlit/data/__init__.py)</sup>. Just when no data layer is used, it will run this logic every time `get_data_layer()` is called.


Hints:
@dosu Would you help me fill out the full issue report based on the title and the contents of `backend/data/__init__.py`?
<!-- Answer -->
To fill out the full issue report based on the title and the contents of `backend/data/__init__.py`, you can use the following template:

---

**Is your feature request related to a problem? Please describe.**

Currently, the `get_data_layer()` function attempts to set up the data layer on every call, even when no data layer is defined. This can lead to unnecessary checks and potential performance issues if the setup logic is executed repeatedly.

**Describe the solution you'd like**

Modify the `get_data_layer()` function to return early if the `_data_layer` is already set. This prevents re-initialization by checking if `_data_layer` is `None` before attempting to set it up. The current implementation already includes this logic:

```python
def get_data_layer():
    global _data_layer

    if not _data_layer:
        if api_key := os.environ.get("LITERAL_API_KEY"):
            from .literalai import LiteralDataLayer

            # support legacy LITERAL_SERVER variable as fallback
            server = os.environ.get("LITERAL_API_URL") or os.environ.get(
                "LITERAL_SERVER"
            )
            _data_layer = LiteralDataLayer(api_key=api_key, server=server)

    return _data_layer
```

**Describe alternatives you've considered**

An alternative could be to refactor the data layer setup logic into a separate initialization function that is explicitly called once during application startup, rather than being embedded within `get_data_layer()`.

**Additional context**

The current implementation already prevents re-initialization by checking if `_data_layer` is `None` before setting it up. If `_data_layer` is already initialized, it simply returns the existing instance <sup>[[1]](https://github.com/Chainlit/chainlit/blob/main/backend/chainlit/data/__init__.py)</sup>.

--- 

This template should help in clearly communicating the issue and the proposed solution.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Patch:
diff --git a/backend/chainlit/data/__init__.py b/backend/chainlit/data/__init__.py
index dfc359c24b..513bc5975d 100644
--- a/backend/chainlit/data/__init__.py
+++ b/backend/chainlit/data/__init__.py
@@ -7,26 +7,38 @@
 )
 
 _data_layer: Optional[BaseDataLayer] = None
+_data_layer_initialized = False
 
 
 def get_data_layer():
-    global _data_layer
-    print("Getting data layer", _data_layer)
-
-    if not _data_layer:
-        from chainlit.config import config
-
-        if config.code.data_layer:
-            # When @data_layer is configured, call it to get data layer.
-            _data_layer = config.code.data_layer()
-        elif api_key := os.environ.get("LITERAL_API_KEY"):
-            # When LITERAL_API_KEY is defined, use LiteralAI data layer
-            from .literalai import LiteralDataLayer
-
-            # support legacy LITERAL_SERVER variable as fallback
-            server = os.environ.get("LITERAL_API_URL") or os.environ.get(
-                "LITERAL_SERVER"
+    global _data_layer, _data_layer_initialized
+
+    if not _data_layer_initialized:
+        if _data_layer:
+            # Data layer manually set, warn user that this is deprecated.
+            import warnings
+
+            warnings.warn(
+                "Setting data layer manually is deprecated. Use @data_layer instead.",
+                DeprecationWarning,
             )
-            _data_layer = LiteralDataLayer(api_key=api_key, server=server)
+
+        else:
+            from chainlit.config import config
+
+            if config.code.data_layer:
+                # When @data_layer is configured, call it to get data layer.
+                _data_layer = config.code.data_layer()
+            elif api_key := os.environ.get("LITERAL_API_KEY"):
+                # When LITERAL_API_KEY is defined, use LiteralAI data layer
+                from .literalai import LiteralDataLayer
+
+                # support legacy LITERAL_SERVER variable as fallback
+                server = os.environ.get("LITERAL_API_URL") or os.environ.get(
+                    "LITERAL_SERVER"
+                )
+                _data_layer = LiteralDataLayer(api_key=api_key, server=server)
+
+        _data_layer_initialized = True
 
     return _data_layer


Test patch: