Repository: truera/trulens
Issue: [BUG] Using Ollama with Specified URL
Root cause: Configuration Error

**Bug Description**
What happened?
Reference [Ollama Quickstart - 🦑 TruLens](https://www.trulens.org/cookbook/models/local_and_OSS_models/ollama_quickstart/) to use Ollama locally deployed models, but specify the Ollama URL.

`
import litellm
from trulens.providers.litellm import LiteLLM

litellm.set_verbose = False

litellm._turn_on_debug()

ollama_provider = LiteLLM(
    model_engine="ollama/qwen2.5:72b", api_base="http://17.1.44.10:8952"
)


relevance = Feedback(
    ollama_provider.relevance_with_cot_reasons
).on_input_output()


ollama_provider.relevance_with_cot_reasons(
    "What is a good name for a store that sells colorful socks?",
    "Great question! Naming a store that sells colorful socks can be a fun and creative process. Here are some suggestions to consider: SoleMates: This name plays on the idea of socks being your soul mate or partner in crime for the day. It is catchy and easy to remember, and it conveys the idea that the store offers a wide variety of sock styles and colors.",
)
`
Error:
RuntimeError: Endpoint LiteLLMEndpoint request failed 4 time(s): 
        litellm.APIConnectionError: OllamaException - [Errno 111] Connection refused

Debug mode shows:
DEBUG [LiteLLM] 

POST Request Sent from LiteLLM:
curl -X POST \
http://localhost:11434/api/generate \
-d '{'model': 'qwen2.5:72b', 'prompt': "### System:\nYou are a RELEVANCE grader; providing the relevance of the given RESPONSE to the given PROMPT.\nRespond only as a number from 0 to 3, where 0 is the lowest score according to the criteria and 3 is the highest possible score.\n\nCriteria for evaluating relevance:\n\n        - RESPONSE must be relevant to the entire PROMPT to get a maximum score of 3.\n        - RELEVANCE score should increase as the RESPONSE provides RELEVANT context to more parts of the PROMPT.\n        - RESPONSE that is RELEVANT to none of the PROMPT should get a minimum score of 0.\n        - RESPONSE that is RELEVANT and answers the entire PROMPT completely should get a score of 3.\n        - RESPONSE that confidently FALSE should get a score of 0.\n        - RESPONSE that is only seemingly RELEVANT should get a score of 0.\n        - Answers that intentionally do not answer the question, such as 'I don't know' and model refusals, should also be counted as the least RELEVANT and get a score of 0.\n    \n\nA few additional scoring guidelines:\n\n- Long RESPONSES should score equally well as short RESPONSES.\n\n- Never elaborate.\n\n### User:\nPROMPT: What is a good name for a store that sells colorful socks?\n\nRESPONSE: Great question! Naming a store that sells colorful socks can be a fun and creative process. Here are some suggestions to consider: SoleMates: This name plays on the idea of socks being your soul mate or partner in crime for the day. It is catchy and easy to remember, and it conveys the idea that the store offers a wide variety of sock styles and colors.\n\n\nPlease answer using the entire template below.\n\nTEMPLATE:\nCriteria: <Provide the criteria for this evaluation>\nSupporting Evidence: <Provide your reasons for scoring based on the listed criteria step by step. Tie it back to the evaluation being completed.>\nScore: <The score based on the given criteria>\n\n\n", 'options': {'temperature': 0.0}, 'stream': False}'

Why is it that even though I specified the base_url as http://17.1.44.10:8952 in the following code:

ollama_provider = LiteLLM(
    model_engine="ollama/qwen2.5:72b", 
    api_base="http://17.1.44.10:8952"
)
the request is still being sent to http://localhost:11434/api/generate?



**Environment:**

- OS: [e.g. MacOS, Windows] ubuntu
- Python 3.10
- TruLens version 1.3.5
- Versions of other relevant installed libraries
- litellm  1.61.1


Thank you for your help!

If I use:
ollama_provider = LiteLLM(
    model_engine="ollama/qwen2.5:72b",
    completion_kwargs={
        'api_base':'http://17.1.44.10:8009'
    }
)
instead of
ollama_provider = LiteLLM(
model_engine="ollama/qwen2.5:72b", api_base="http://17.1.44.10:8952/"
)
the issue is resolved. However, there is still a problem when using litellm._turn_on_debug().
A more comprehensive solution is:
os.environ["OLLAMA_API_BASE"] = 'http://17.1.44.10:8952'

Thanks @HRzhangStat - glad you have a workaround. We'll revisit the SDK here and see if we can smooth this out.

Same issue here, i used:

os.environ["OLLAMA_API_BASE"] as my solution.

The docs should be warn about this issue.

Thanks for developing this awesome tool!

This doesn't seem to work with the UI when specifying as an env for docker setup using the standard ollama/ollama container. Trying to set a model with OLLAMA_API_BASE set as an env variable, verified inside the container. I have Ollama set as the provider

![Image](https://github.com/user-attachments/assets/cb9c6eb5-5649-4c7b-a81e-d422107756d8)

 I get the following when I hit "Test Connection":

> LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=gemma3:4b Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

<code>litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=gemma3:4b
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
stack trace: Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 5584, in ahealth_check
    model, custom_llm_provider, _, _ = get_llm_provider(model=model)
                                       ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 358, in get_llm_provider
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 335, in get_llm_provider
    raise litellm.exceptions.BadRequestError(  # type: ignore
    ...<8 lines>...
    )
litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=gemma3:4b
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers</code>

The docker log in debug mode provides nothing