Repository: explodinggradients/ragas
Issue: Use local LLMs
Root cause: Validation Gap

Hi, Thanks for this amazing framework

I would like to know is there any support for using locally trained LLMs, I see that currently we can change the LLM with Langchain, but I don't want to use langchain and use a local LLM like Llama 2

Hey @seanbenhur! We're glad you like it 😊 

how are you planning to use Llama2? I would recommend using via langchain because there is a lot of support for it in the community something like this could help [Llama 2 in LangChain — FIRST Open Source Conversational Agent! - YouTube](https://www.youtube.com/watch?v=6iHVJyX2e50)

but let me know if that is not what you have in mind. Also, why are you trying to move away from langchain?

Thanks for the prompt response, I was thinking of using Llama without langchain since I feel langchain is an extra dependency for the project

got it. Unfortunately, that is not possible right now and we don't plan to add it to the roadmap. There is also another issue which is that we have not tested Llama2's performance for evaluation so we wouldn't recommend using it for evaluation in critical applications.

hope you understand...

Sure No problems!

I tried to use Llama with Langchain without using OpenAI, but now I am getting error since I didn't pass the OAI key on the importing stage itself

```
`ValidationError                           Traceback (most recent call last)
Cell In[39], line 6
      4 import torch
      5 from langchain.llms import HuggingFacePipeline
----> 6 from ragas.metrics import Faithfulness
      9 DATASET_PATH = "explodinggradients/ragas-wikiqa"
     10 MODEL_PATH = "llama-2-7b-chat-hf"

File ~/.local/lib/python3.8/site-packages/ragas/__init__.py:1
----> 1 from ragas.evaluation import evaluate
      3 try:
      4     from ._version import version as __version__

File ~/.local/lib/python3.8/site-packages/ragas/evaluation.py:9
      6 from datasets import Dataset, concatenate_datasets
      8 from ragas._analytics import EvaluationEvent, track
----> 9 from ragas.metrics.base import Metric
     10 from ragas.metrics.critique import AspectCritique
     11 from ragas.validation import validate_column_dtypes, validate_evaluation_modes

File ~/.local/lib/python3.8/site-packages/ragas/metrics/__init__.py:2
      1 from ragas.metrics.answer_relevance import AnswerRelevancy, answer_relevancy
----> 2 from ragas.metrics.context_relevance import ContextRelevancy, context_relevancy
      3 from ragas.metrics.critique import AspectCritique
      4 from ragas.metrics.faithfulnes import Faithfulness, faithfulness

File ~/.local/lib/python3.8/site-packages/ragas/metrics/context_relevance.py:173
    168             scores.append(agr_score * np.mean(overlap_scores))
    170         return dataset.add_column(f"{self.name}", scores)  # type: ignore
--> 173 context_relevancy = ContextRelevancy()

File <string>:4, in __init__(self, batch_size, llm, name, evaluation_mode, strictness, agreement_metric, model_name)

File ~/.local/lib/python3.8/site-packages/ragas/metrics/base.py:69, in _llm_factory()
     68 def _llm_factory():
---> 69     return ChatOpenAI(model_name="gpt-3.5-turbo-16k")

File ~/.local/lib/python3.8/site-packages/langchain/load/serializable.py:74, in Serializable.__init__(self, **kwargs)
     73 def __init__(self, **kwargs: Any) -> None:
---> 74     super().__init__(**kwargs)
     75     self._lc_kwargs = kwargs

File ~/.local/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for ChatOpenAI
__root__
  Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it, or pass  `openai_api_key` as a named parameter. (type=value_error)`
```



this is because the metrics are still using OAI endpoints. Can you refer [custom llm guide](https://github.com/explodinggradients/ragas/blob/main/docs/guides/llms.ipynb) already? it will show you how to change the LLM. In case it does not work can you share the code and ragas,python versions?

Yes I referred the above nb, the issue is I get the error, when I import the metric

from ragas.metrics import Faithfulness

ohh my apologies. I'll fix that bug (thanks a lot for bringing it to our attention ❤️ ). In the meantime could you set the env-var with some random text as a workaround till we release the fix?

@seanbenhur made a PR for this: https://github.com/explodinggradients/ragas/pull/118