Repository: comet-ml/opik
Pull request: Updated behavior with Opik tracking is disabled
## Details

Improvements to the `OPIK_TRACK_DISABLE` behavior:

1. No longer display the message about configuring the SDK
2. Fix the LangChain integration to not track traces and spans

Partially resolves #1169

Problem statement:
[FR]: Some form of transparent delegation or null-object pattern for cases of shared code bases with users gradually onboarding opik
### Proposal summary

My team is still in the process of onboarding use of opik, which means some people will have running instances of opik whereas others won't.

I would like opik to not be obstruction to those who have not yet onboarded without having to add conditional logic across my code.

For example:
Instead of having to write:

```
FOO_PROMPT: str = "My prompt template..."
if has_opik_running():
  prompt = opik.Prompt(name="foo", prompt=FOO_PROMPT)
else:
  prompt = FOO_PROMPT
```

I'd want to just have `prompt = opik.Prompt(name="foo", prompt=FOO_PROMPT)` return `FOO_PROMPT`.

### Motivation

I am trying to solve the problem of gradually onboarding team members who work in a distributed way, some of which are currently experimenting with locally running instances of opik.

Here is another example of how I am handling this problem:

```
# the default logging handler will be our custom stdout one, but if we have a local opik deploy, we will use it
tracer = LLMLoggingHandler()
OPIK_BASE_URL = os.getenv("OPIK_BASE_URL")
if OPIK_BASE_URL:
    OPIK_LOCAL = "localhost" in OPIK_BASE_URL.lower()
    opik.configure(use_local=OPIK_LOCAL, force=True)
    username = subprocess.check_output("whoami", shell=True).decode("utf-8").strip()
    OPIK_PROJECT_NAME = f"{username}-concierge"

    tracer = OpikTracer(project_name=OPIK_PROJECT_NAME)

chat_model = ChatBedrockConverse(
    client=get_bedrock_runtime(),
    model=self.recommendation_model_id,
    temperature=0,
    max_tokens=4096,
    additional_model_request_fields={
        "top_k": 1,
    },
    # top_p=0.001,
    callbacks=[tracer],
)

...

# update tracing tags to group conversations
try:
    traces = tracer.created_traces()

    for trace in traces:
        trace.update(metadata={"session_id": chat_session_id})
except: ...
    # fail gracefully for any other "tracer" (e.g. the LLMLoggingHandler)
```

Hints from issue discussion:
@donatoaz Good suggestion, let me take a look later today
Hi @donatoaz,

I've taken a closer look and indeed I think we can improve the user experience when a user doesn't have Opik configured / installed. We currently have the `OPIK_TRACK_DISABLE` environment variable that disables the logging of data to Opik but it still displays a message recommending a user configures the SDK.

On the other hand, I think updating the `Prompt` method will be confusing as it's main purpose is to maintain a sync between the prompts stored in code and the BE. Silently not syncing it with the backend will lead to confusing behavior where data might not be synced even though a user expects it to
I did notice the LangChain integration doesn't respect the `OPIK_TRACK_DISABLE` parameter, looking into it now

Commit message: Updated behavior with Opik tracking is disabled

Patch to review:
diff --git a/sdks/python/src/opik/api_objects/opik_client.py b/sdks/python/src/opik/api_objects/opik_client.py
index 9c8016897f..5de7ff8267 100644
--- a/sdks/python/src/opik/api_objects/opik_client.py
+++ b/sdks/python/src/opik/api_objects/opik_client.py
@@ -68,6 +68,7 @@ def __init__(
             api_key=api_key,
         )
         config.check_for_misconfiguration(config_)
+        self._config = config_
 
         self._workspace: str = config_.workspace
         self._project_name: str = config_.project_name
diff --git a/sdks/python/src/opik/config.py b/sdks/python/src/opik/config.py
index 881733f40c..cc9fe68922 100644
--- a/sdks/python/src/opik/config.py
+++ b/sdks/python/src/opik/config.py
@@ -261,8 +261,13 @@ def check_for_misconfiguration(config: OpikConfig) -> None:
     )  # does not detect all OSS installations
     workspace_is_default = config.workspace == OPIK_WORKSPACE_DEFAULT_NAME
     api_key_configured = config.api_key is not None
+    tracking_disabled = config.track_disable
 
-    if cloud_installation and (not api_key_configured or workspace_is_default):
+    if (
+        cloud_installation
+        and (not api_key_configured or workspace_is_default)
+        and not tracking_disabled
+    ):
         print()
         LOGGER.error(
             "========================\n"
@@ -273,7 +278,7 @@ def check_for_misconfiguration(config: OpikConfig) -> None:
         )
         return
 
-    if localhost_installation and not workspace_is_default:
+    if localhost_installation and not workspace_is_default and not tracking_disabled:
         print()
         LOGGER.error(
             "========================\n"
diff --git a/sdks/python/src/opik/integrations/langchain/opik_tracer.py b/sdks/python/src/opik/integrations/langchain/opik_tracer.py
index c5b12f81d6..7314067768 100644
--- a/sdks/python/src/opik/integrations/langchain/opik_tracer.py
+++ b/sdks/python/src/opik/integrations/langchain/opik_tracer.py
@@ -286,38 +286,72 @@ def created_traces(self) -> List[trace.Trace]:
         """
         return self._created_traces
 
+    def _skip_tracking(self) -> bool:
+        config = self._opik_client._config
+        if config.track_disable:
+            return True
+
+        return False
+
     def _on_llm_start(self, run: "Run") -> None:
         """Process the LLM Run upon start."""
+        if self._skip_tracking():
+            return
+
         self._process_start_span(run)
 
     def _on_llm_end(self, run: "Run") -> None:
         """Process the LLM Run."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span(run)
 
     def _on_llm_error(self, run: "Run") -> None:
         """Process the LLM Run upon error."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span_with_error(run)
 
     def _on_chain_start(self, run: "Run") -> None:
         """Process the Chain Run upon start."""
+        if self._skip_tracking():
+            return
+
         self._process_start_span(run)
 
     def _on_chain_end(self, run: "Run") -> None:
         """Process the Chain Run."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span(run)
 
     def _on_chain_error(self, run: "Run") -> None:
         """Process the Chain Run upon error."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span_with_error(run)
 
     def _on_tool_start(self, run: "Run") -> None:
         """Process the Tool Run upon start."""
+        if self._skip_tracking():
+            return
+
         self._process_start_span(run)
 
     def _on_tool_end(self, run: "Run") -> None:
         """Process the Tool Run."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span(run)
 
     def _on_tool_error(self, run: "Run") -> None:
         """Process the Tool Run upon error."""
+        if self._skip_tracking():
+            return
+
         self._process_end_span_with_error(run)
diff --git a/sdks/python/tests/library_integration/langchain/test_langchain.py b/sdks/python/tests/library_integration/langchain/test_langchain.py
index 4dd34bde68..c9afe1e54a 100644
--- a/sdks/python/tests/library_integration/langchain/test_langchain.py
+++ b/sdks/python/tests/library_integration/langchain/test_langchain.py
@@ -4,6 +4,7 @@
 from langchain.llms import fake
 from langchain.prompts import PromptTemplate
 
+import os
 import opik
 from opik import context_storage
 from opik.api_objects import opik_client, span, trace
@@ -750,3 +751,27 @@ def f():
     assert len(fake_backend.span_trees) == 1
     assert len(callback.created_traces()) == 0
     assert_equal(EXPECTED_SPANS_TREE, fake_backend.span_trees[0])
+
+
+def test_langchain_callback__disabled_tracking(fake_backend):
+    os.environ["OPIK_TRACK_DISABLE"] = "true"
+
+    llm = fake.FakeListLLM(
+        responses=["I'm sorry, I don't think I'm talented enough to write a synopsis"]
+    )
+
+    template = "Given the title of play, write a synopsys for that. Title: {title}."
+
+    prompt_template = PromptTemplate(input_variables=["title"], template=template)
+
+    synopsis_chain = prompt_template | llm
+    test_prompts = {"title": "Documentary about Bigfoot in Paris"}
+
+    callback = OpikTracer()
+    synopsis_chain.invoke(input=test_prompts, config={"callbacks": [callback]})
+
+    callback.flush()
+    os.environ["OPIK_TRACK_DISABLE"] = "false"
+
+    assert len(fake_backend.trace_trees) == 0
+    assert len(callback.created_traces()) == 0


Code review comments:
diff_hunk: @@ -286,38 +286,72 @@ def created_traces(self) -> List[trace.Trace]:
         """
         return self._created_traces
 
+    def _skip_tracking(self) -> bool:
+        config = self._opik_client._config
path: sdks/python/src/opik/integrations/langchain/opik_tracer.py
text: @user1:
Don't access private attributes.
If we want to be able to get the configuration used during the `Opik` object instantiation - we can make a public property for that.

It's probably better to make it read-only then, so that `self._opik_client.config.SOME_CONFIG_NAME=XXX` update operation would not be possible (safety reasons).

diff_hunk: @@ -750,3 +751,27 @@ def f():
     assert len(fake_backend.span_trees) == 1
     assert len(callback.created_traces()) == 0
     assert_equal(EXPECTED_SPANS_TREE, fake_backend.span_trees[0])
+
+
+def test_langchain_callback__disabled_tracking(fake_backend):
+    os.environ["OPIK_TRACK_DISABLE"] = "true"
path: sdks/python/tests/library_integration/langchain/test_langchain.py
text: @user1:
You should use `testlib.patch_environ` context manager for that to make sure the environment will return back to its initial state after the test has finished (no matter how).

Otherwise, if this test fails in the middle, it may affect the environment and can potentially harm other tests (especially with THAT environment variable :).