Repository: vibrantlabsai/ragas (python)

Issue:
Please Help me ! When using the LLM evaluation, there is a bug that prevents the use of the LLM client.

[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
After I upgraded ragas to version V0.4.3, when I followed the instructions to use the latest lllm_factory to build the evaluation model client, I encountered a call error and didn't know what to do.
I hope that using our locally deployed model can also support the evaluation process. However, when I used the OpenAI client and Langchain's langchain_openai, both failed and reported the same issue.

Ragas version: 0.4.3
Python version: 3.12.5

**Code to Reproduce**
The code is as follows：
```
from ragas import experiment
from ragas.metrics.collections import Faithfulness, ContextRecall
from pydantic import BaseModel
from ragas.llms import llm_factory
from openai import AsyncOpenAI
import asyncio
from langchain_openai import ChatOpenAI
# Define experiment result structure
class ExperimentResult(BaseModel):
    faithfulness: float
    answer_relevancy: float

client = AsyncOpenAI(
    api_key="sk-",
    base_url="xxxxxxx"
)
llm = llm_factory(model="deepseek-r1:70b", provider="deepseek",client=client)
# Create experiment function
@experiment(ExperimentResult)
async def run_evaluation(row):
    faithfulness = Faithfulness(llm=llm)
    answer_relevancy = ContextRecall(llm=llm)
    print(row)
    faith_result = await faithfulness.ascore(
        user_input=row.user_input,
        response=row.response,
        retrieved_contexts=row.retrieved_contexts
    )

    relevancy_result = await answer_relevancy.ascore(
        user_input=row.user_input,
        retrieved_contexts=row.retrieved_contexts,
        reference=row.reference
    )

    print(faith_result,relevancy_result)
    return ExperimentResult(
        faithfulness=faith_result.value,
        answer_relevancy=relevancy_result.value
    )
from ragas import SingleTurnSample, EvaluationDataset

sim1 = SingleTurnSample(
    user_input="What is the weather like in San Francisco today?",
    response="The weather in San Francisco today is sunny with a high of 68°F.",
    retrieved_contexts=["San Francisco is known for its mild climate and sunny days."],
    reference="The weather in San Francisco today is sunny with a high of 68°F."
)
# evadate = EvaluationDataset(samples=sim1)
# Run experiment
asyncio.run(run_evaluation(sim1))
```

**Error trace**
```
`Traceback (most recent call last):
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\adapters\instructor.py", line 39, in create_llm
    patched_client = _get_instructor_client(client, provider)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\base.py", line 576, in _get_instructor_client
    return _patch_client_for_provider(client, provider_lower)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\base.py", line 498, in _patch_client_for_provider
    create=client.messages.create,
           ^^^^^^^^^^^^^^^
AttributeError: 'AsyncOpenAI' object has no attribute 'messages'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\base.py", line 680, in llm_factory
    llm = adapter_instance.create_llm(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\adapters\instructor.py", line 41, in create_llm
    raise ValueError(f"Failed to patch {provider} client with Instructor: {e}")
ValueError: Failed to patch deepseek client with Instructor: 'AsyncOpenAI' object has no attribute 'messages'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\project\src\test_agent\demo\ragas_demo.py", line 17, in <module>
    llm = llm_factory(model="deepseek-r1:70b", provider="deepseek",client=client)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\project\src\test_agent\agenv\Lib\site-packages\ragas\llms\base.py", line 689, in llm_factory
    raise ValueError(
ValueError: Failed to initialize deepseek client with instructor adapter. Ensure you've created a valid deepseek client.
Error: Failed to patch deepseek client with Instructor: 'AsyncOpenAI' object has no attribute 'messages'`
```
**Expected behavior**
I hope that using our locally deployed model can also support the evaluation process. However, when I used the OpenAI client and Langchain's langchain_openai, both failed and reported the same issue.


**Additional context**
Add any other context about the problem here.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->



Patch:
diff --git a/docs/howtos/llm-adapters.md b/docs/howtos/llm-adapters.md
index 8b4cc642b..5b7eb4eb4 100644
--- a/docs/howtos/llm-adapters.md
+++ b/docs/howtos/llm-adapters.md
@@ -261,6 +261,36 @@ System prompts are useful when:
 
 The system prompt is prepended to all LLM calls as a system message.
 
+### Custom Instructor Modes
+
+The instructor adapter supports multiple modes for structured output generation. By default, `Mode.JSON` is used, but you can specify a different mode for backends that don't support certain features:
+
+```python
+import instructor
+from ragas.llms import llm_factory
+from openai import OpenAI
+
+# Use MD_JSON mode for backends without response_format support
+client = OpenAI(api_key="...", base_url="https://custom-backend")
+llm = llm_factory(
+    "custom-model",
+    provider="openai",
+    client=client,
+    mode=instructor.Mode.MD_JSON
+)
+```
+
+Available instructor modes:
+- `Mode.JSON` (default) - Uses OpenAI's response_format parameter
+- `Mode.MD_JSON` - Uses markdown JSON in the prompt (fallback for unsupported backends)
+- `Mode.TOOLS` - Uses function calling
+- `Mode.JSON_SCHEMA` - Uses JSON schema validation
+
+Use `Mode.MD_JSON` when you encounter errors like:
+```
+Error code: 400 - {'message': 'only pytorch backend can use response_format now'}
+```
+
 ### Async Support
 
 Both adapters support async operations:
diff --git a/src/ragas/llms/adapters/instructor.py b/src/ragas/llms/adapters/instructor.py
index ef3af3107..d0f170213 100644
--- a/src/ragas/llms/adapters/instructor.py
+++ b/src/ragas/llms/adapters/instructor.py
@@ -25,7 +25,7 @@ def create_llm(
             client: Pre-initialized client
             model: Model name
             provider: Provider name
-            **kwargs: Additional model arguments
+            **kwargs: Additional model arguments including optional 'mode'
 
         Returns:
             InstructorLLM instance
@@ -34,9 +34,10 @@ def create_llm(
             ValueError: If client patching fails
         """
         cache = kwargs.pop("cache", None)
+        mode = kwargs.pop("mode", None)
 
         try:
-            patched_client = _get_instructor_client(client, provider)
+            patched_client = _get_instructor_client(client, provider, mode=mode)
         except Exception as e:
             raise ValueError(f"Failed to patch {provider} client with Instructor: {e}")
 
diff --git a/src/ragas/llms/base.py b/src/ragas/llms/base.py
index c2da31cc5..825660ec5 100644
--- a/src/ragas/llms/base.py
+++ b/src/ragas/llms/base.py
@@ -460,7 +460,9 @@ def __repr__(self) -> str:
         return f"{self.__class__.__name__}(llm={self.llm.__class__.__name__}(...))"
 
 
-def _patch_client_for_provider(client: t.Any, provider: str) -> t.Any:
+def _patch_client_for_provider(
+    client: t.Any, provider: str, mode: t.Optional[instructor.Mode] = None
+) -> t.Any:
     """
     Patch a client with Instructor for generic providers.
 
@@ -473,6 +475,9 @@ def _patch_client_for_provider(client: t.Any, provider: str) -> t.Any:
     """
     from instructor import Provider
 
+    if mode is None:
+        mode = instructor.Mode.JSON
+
     provider_map = {
         "anthropic": Provider.ANTHROPIC,
         "google": Provider.GENAI,
@@ -488,19 +493,14 @@ def _patch_client_for_provider(client: t.Any, provider: str) -> t.Any:
 
     provider_enum = provider_map.get(provider, Provider.OPENAI)
 
-    # Check for OpenAI-compatible API (chat.completions.create)
     if (
         hasattr(client, "chat")
         and client.chat is not None
         and hasattr(client.chat, "completions")
         and hasattr(client.chat.completions, "create")
     ):
-        # For OpenAI-compatible APIs, use instructor.from_openai()
-        # This properly handles the OpenAI client structure and response parsing
-        # Use JSON mode to avoid issues with Dict types in function calling
-        return instructor.from_openai(client, mode=instructor.Mode.JSON)
+        return instructor.from_openai(client, mode=mode)
 
-    # Check for Anthropic-compatible API (messages.create)
     elif (
         hasattr(client, "messages")
         and client.messages is not None
@@ -509,20 +509,19 @@ def _patch_client_for_provider(client: t.Any, provider: str) -> t.Any:
         create_method = client.messages.create
         is_async = "Async" in client.__class__.__name__
 
-        # Use JSON mode to avoid issues with Dict types in function calling
         if is_async:
             return instructor.AsyncInstructor(
                 client=client,
                 create=create_method,
                 provider=provider_enum,
-                mode=instructor.Mode.JSON,
+                mode=mode,
             )
         else:
             return instructor.Instructor(
                 client=client,
                 create=create_method,
                 provider=provider_enum,
-                mode=instructor.Mode.JSON,
+                mode=mode,
             )
     else:
         raise ValueError(
@@ -564,13 +563,15 @@ def _is_new_google_genai_client(client: t.Any) -> bool:
     return False
 
 
-def _get_instructor_client(client: t.Any, provider: str) -> t.Any:
+def _get_instructor_client(
+    client: t.Any, provider: str, mode: t.Optional[instructor.Mode] = None
+) -> t.Any:
     """
     Get an instructor-patched client for the specified provider.
 
     Uses provider-specific methods when available, falls back to generic patcher.
 
-    Note: For OpenAI, we use Mode.JSON instead of the default Mode.TOOLS because
+    Note: For OpenAI, we use Mode.JSON by default instead of Mode.TOOLS because
     OpenAI's function calling (TOOLS mode) has issues with Dict type annotations
     in Pydantic models - it returns empty objects `{}` instead of proper structured
     data. Mode.JSON works correctly with all Pydantic types including Dict.
@@ -580,31 +581,26 @@ def _get_instructor_client(client: t.Any, provider: str) -> t.Any:
     - New SDK (google-genai): Uses instructor.from_genai()
     - Old SDK (google-generativeai): Uses instructor.from_gemini()
     """
+    if mode is None:
+        mode = instructor.Mode.JSON
+
     provider_lower = provider.lower()
 
     if provider_lower == "openai":
-        # Use JSON mode to avoid issues with Dict types in function calling
-        return instructor.from_openai(client, mode=instructor.Mode.JSON)
+        return instructor.from_openai(client, mode=mode)
     elif provider_lower == "anthropic":
         return instructor.from_anthropic(client)
     elif provider_lower in ("google", "gemini"):
-        # Detect which Google SDK is being used
         if _is_new_google_genai_client(client):
-            # New google-genai SDK - uses instructor.from_genai()
-            # WARNING: Known upstream issue with instructor sending invalid safety
-            # settings (HARM_CATEGORY_JAILBREAK). Track: github.com/567-labs/instructor/issues/1658
-            # Workaround: Use OpenAI-compatible endpoint with Gemini base URL instead.
             return instructor.from_genai(client)
         else:
-            # Old google-generativeai SDK (deprecated, support ends Aug 2025)
             return instructor.from_gemini(client)
     elif provider_lower == "litellm":
-        # Use JSON mode to avoid issues with Dict types in function calling
-        return instructor.from_litellm(client, mode=instructor.Mode.JSON)
+        return instructor.from_litellm(client, mode=mode)
     elif provider_lower == "perplexity":
         return instructor.from_perplexity(client)
     else:
-        return _patch_client_for_provider(client, provider_lower)
+        return _patch_client_for_provider(client, provider_lower, mode=mode)
 
 
 def llm_factory(
@@ -613,6 +609,7 @@ def llm_factory(
     client: t.Optional[t.Any] = None,
     adapter: str = "auto",
     cache: t.Optional[CacheInterface] = None,
+    mode: t.Optional[instructor.Mode] = None,
     **kwargs: t.Any,
 ) -> InstructorBaseRagasLLM:
     """
@@ -640,6 +637,10 @@ def llm_factory(
         cache: Optional cache backend for caching LLM responses.
                Pass DiskCacheBackend() for persistent caching across runs.
                Saves costs and speeds up repeated evaluations by 60x.
+        mode: Instructor mode for structured outputs (default: Mode.JSON).
+              Only applies when using instructor adapter.
+              Options: Mode.JSON, Mode.MD_JSON, Mode.TOOLS, Mode.JSON_SCHEMA, etc.
+              Use Mode.MD_JSON for backends that don't support response_format parameter.
         **kwargs: Additional model arguments (temperature, max_tokens, top_p, etc).
 
     Returns:
@@ -675,6 +676,11 @@ def llm_factory(
         # Explicit adapter selection
         llm = llm_factory("gemini-2.0-flash", client=client, adapter="litellm")
 
+        # Custom instructor mode for backends without response_format support
+        import instructor
+        client = OpenAI(api_key="...", base_url="https://custom-backend")
+        llm = llm_factory("custom-model", client=client, mode=instructor.Mode.MD_JSON)
+
         # Async
         from openai import AsyncOpenAI
         client = AsyncOpenAI(api_key="...")
@@ -709,7 +715,7 @@ def llm_factory(
     try:
         adapter_instance = get_adapter(adapter)
         llm = adapter_instance.create_llm(
-            client, model, provider_lower, cache=cache, **kwargs
+            client, model, provider_lower, cache=cache, mode=mode, **kwargs
         )
     except ValueError as e:
         # Re-raise ValueError from get_adapter for unknown adapter names


Test patch:
diff --git a/tests/unit/llms/test_instructor_factory.py b/tests/unit/llms/test_instructor_factory.py
index 714de5769..6816b5b05 100644
--- a/tests/unit/llms/test_instructor_factory.py
+++ b/tests/unit/llms/test_instructor_factory.py
@@ -263,3 +263,73 @@ def mock_from_openai(client, mode=None):
 
         assert llm.model == "test-model"
         assert llm.is_async
+
+
+def test_llm_factory_with_custom_mode(mock_sync_client, monkeypatch):
+    """Test that llm_factory accepts and uses custom instructor mode."""
+    import instructor
+
+    captured_mode = None
+
+    def mock_from_openai(client, mode=None):
+        nonlocal captured_mode
+        captured_mode = mode
+        return MockInstructor(client)
+
+    monkeypatch.setattr("instructor.from_openai", mock_from_openai)
+
+    llm = llm_factory(
+        "gpt-4",
+        provider="openai",
+        client=mock_sync_client,
+        mode=instructor.Mode.MD_JSON,
+    )
+
+    assert llm.model == "gpt-4"
+    assert captured_mode == instructor.Mode.MD_JSON
+
+
+def test_llm_factory_default_mode_is_json(mock_sync_client, monkeypatch):
+    """Test that llm_factory defaults to Mode.JSON when no mode is specified."""
+    import instructor
+
+    captured_mode = None
+
+    def mock_from_openai(client, mode=None):
+        nonlocal captured_mode
+        captured_mode = mode
+        return MockInstructor(client)
+
+    monkeypatch.setattr("instructor.from_openai", mock_from_openai)
+
+    llm = llm_factory("gpt-4", provider="openai", client=mock_sync_client)
+
+    assert llm.model == "gpt-4"
+    assert captured_mode == instructor.Mode.JSON
+
+
+def test_llm_factory_mode_with_generic_provider(monkeypatch):
+    """Test that mode parameter works with generic providers via _patch_client_for_provider."""
+    import instructor
+
+    captured_mode = None
+
+    def mock_from_openai(client, mode=None):
+        nonlocal captured_mode
+        captured_mode = mode
+        return MockInstructor(client)
+
+    monkeypatch.setattr("instructor.from_openai", mock_from_openai)
+
+    mock_client = MockClient(is_async=False)
+    delattr(mock_client, "messages")
+
+    llm = llm_factory(
+        "custom-model",
+        provider="custom-provider",
+        client=mock_client,
+        mode=instructor.Mode.TOOLS,
+    )
+
+    assert llm.model == "custom-model"
+    assert captured_mode == instructor.Mode.TOOLS