Repository: explodinggradients/ragas
Issue: Local models(LLM and embed) other than TogetherAI?
Root cause: Documentation Deficiency

I am looking to build a completely unpaid eval solution. At[1], you are providing Together AI as substitute to paid model but Together AI itself is a paid service despise providing open models' inference. Any fully open source solution like LM Studio, Ollama etc?


[1] https://docs.ragas.io/en/v0.1.21/howtos/customisations/bring-your-own-llm-or-embs.html#llms

> I am looking to build a completely unpaid eval solution. At[1], you are providing Together AI as substitute to paid model but Together AI itself is a paid service despise providing open models' inference. Any fully open source solution like LM Studio, Ollama etc?

Yes, you can use any local model you like. Let us know if you face any errors.

> > I am looking to build a completely unpaid eval solution. At[1], you are providing Together AI as substitute to paid model but Together AI itself is a paid service despise providing open models' inference. Any fully open source solution like LM Studio, Ollama etc?
> 
> Yes, you can use any local model you like. Let us know if you face any errors.

I can't seems to find any code snippet for the same. Can you provide any tutorial to use Ollama/LM Studio?

Don't suppose we've any existing tutorials yet about it. Here's a [snippet](https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama) from our docs. Hope it helps.

> Don't suppose we've any existing tutorials yet about it. Here's a [snippet](https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama) from our docs. Hope it helps.

But it is about LLM adaptors. TIA

> > Don't suppose we've any existing tutorials yet about it. Here's a [snippet](https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama) from our docs. Hope it helps.
> 
> But it is about LLM adaptors. TIA

It shows an example of local model usage with ollama. Did you wanted something else?

> > > Don't suppose we've any existing tutorials yet about it. Here's a [snippet](https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama) from our docs. Hope it helps.
> > 
> > 
> > But it is about LLM adaptors. TIA
> 
> It shows an example of local model usage with ollama. Did you wanted something else?

so, are LLM Adaptors = LLM in RAGAS? I am used to terminology followed by Unsloth where Adaptors != LLM. 

> so, are LLM Adaptors = LLM in RAGAS? I am used to terminology followed by Unsloth where Adaptors != LLM.

@JayKayNJIT I understand the confusion - in Ragas, "LLM Adapters" ≠ LoRA adapters (like in Unsloth).

The terminology difference:
- In Unsloth/fine-tuning: "Adapters" = LoRA weight adapters for parameter-efficient fine-tuning
- In Ragas: "LLM Adapters" = Provider wrappers that connect different LLM services (OpenAI, Ollama, etc.) to Ragas

Yes, the https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama is how you use local LLMs with Ragas!


> > so, are LLM Adaptors = LLM in RAGAS? I am used to terminology followed by Unsloth where Adaptors != LLM.
> 
> [@JayKayNJIT](https://github.com/JayKayNJIT) I understand the confusion - in Ragas, "LLM Adapters" ≠ LoRA adapters (like in Unsloth).
> 
> The terminology difference:
> 
> * In Unsloth/fine-tuning: "Adapters" = LoRA weight adapters for parameter-efficient fine-tuning
> * In Ragas: "LLM Adapters" = Provider wrappers that connect different LLM services (OpenAI, Ollama, etc.) to Ragas
> 
> Yes, the https://docs.ragas.io/en/stable/howtos/llm-adapters/#local-models-ollama is how you use local LLMs with Ragas!

Got it! It's more along the sense that, in RAGAS LLM Adaptors are wrappers that **_adapts_** different LLM providers into one single unified interface of RAGAS. Thanks for info.