Repository: explodinggradients/ragas
Issue: LangchainLLMWrapper - no attribute agenerate_prompt - Unable to use HuggingFace LLMs and embedding models for test set generation.
Root cause: Interface Contract Mismatch

[ /] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
I'm currently trying to use HuggingFace LLMs and embedding models for RAGAS test set generation instead of using OpenAI's ChatGPT to save costs. In doing so, I encountered an error that indicates a missing attribute in LangchainLLMWrapper. 

Ragas version: 0.1.10
Python version: 3.9.18

**Code to Reproduce**
```py
import torch
from ragas.llms import LangchainLLMWrapper
from ragas.testset.generator import TestsetGenerator
from langchain_huggingface import HuggingFacePipeline
from ragas.embeddings import LangchainEmbeddingsWrapper
from langchain_community.embeddings import HuggingFaceBgeEmbeddings
from ragas.testset.evolutions import simple, reasoning, multi_context
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline

bnb_config = BitsAndBytesConfig(load_in_4bit=True, 
                                          bnb_4bit_use_double_quant=True, 
                                          bnb_4bit_quant_type="nf4", 
                                          bnb_4bit_compute_dtype=torch.bfloat16) # bits and bytes for model quantization

model_config = AutoTokenizer.from_pretrained("microsoft/Orca-2-13b", token="...")

model = AutoModelForCausalLM.from_pretrained("microsoft/Orca-2-13b", device_map="auto", 
                                             token="...", 
                                             quantization_config=bnb_config)

generate_text = pipeline(model=model,
                         tokenizer=model_config, 
                         return_full_text=True,
                         task="text-generation")    

langchain_embeddings = HuggingFaceBgeEmbeddings(model_name="BAAI/bge-large-en-v1.5",
                                    model_kwargs={'device': 'cuda'},
                                    encode_kwargs={'normalize_embeddings': True})

langchain_llm = HuggingFacePipeline(pipeline=generate_text)

langchain_llm = LangchainLLMWrapper(langchain_llm)
langchain_embeddings = LangchainEmbeddingsWrapper(langchain_embeddings)

from langchain_community.document_loaders import PubMedLoader

loader = PubMedLoader("liver", load_max_docs=10)
documents = loader.load()

generator = TestsetGenerator.from_langchain(
    generator_llm=langchain_llm,
    critic_llm=langchain_llm,
    embeddings=langchain_embeddings,
) 

distributions = {
    simple: 0.5,
    multi_context: 0.4,
    reasoning: 0.1
}

print("checkpoint")
testset = generator.generate_with_langchain_docs(documents, 1, distributions) 
print("checkpoint_2")
test_df = testset.to_pandas()
test_df.to_csv('test/testset_test.csv')
```

**Error trace**

**Expected behavior**
```
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 87, in run
    results = self.loop.run_until_complete(self._aresults())
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 79, in _aresults
    r = await future
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 37, in sema_coro
    return await coro
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 109, in wrapped_callable_async
    raise e
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 104, in wrapped_callable_async
    result = await callable(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/extractor.py", line 49, in extract
    results = await self.llm.generate(prompt=prompt, is_async=is_async)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/llms/base.py", line 93, in generate
    return await agenerate_text_with_retry(
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped
    return await fn(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 47, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/llms/base.py", line 178, in agenerate_text
    result = await self.langchain_llm.agenerate_prompt(
AttributeError: 'LangchainLLMWrapper' object has no attribute 'agenerate_prompt'
Traceback (most recent call last):
  File "/storage/local/mgmanzanillo3/projects/netopia/testset_generation/test.py", line 67, in <module>
    testset = generator.generate_with_langchain_docs(documents, 1, distributions)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/generator.py", line 206, in generate_with_langchain_docs
    self.docstore.add_documents(
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/docstore.py", line 215, in add_documents
    self.add_nodes(nodes, show_progress=show_progress)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/docstore.py", line 254, in add_nodes
    raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.
```


**Additional context**
None

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


<!-- Greeting -->
Hey there, @netopia-gab! 👋 I'm here to help you with any bugs, questions, or even becoming a contributor. Let's squash those bugs together!

<!-- Answer -->
To resolve the `AttributeError: 'LangchainLLMWrapper' object has no attribute 'agenerate_prompt'`, you need to ensure that the `LangchainLLMWrapper` class is correctly calling the `agenerate` method of the `langchain_llm` object. The error suggests that the `langchain_llm` object does not have an `agenerate_prompt` method.

Here is the corrected code for the `LangchainLLMWrapper` class:

```python
class LangchainLLMWrapper(BaseRagasLLM):
    """
    A simple base class for RagasLLMs that is based on Langchain's BaseLanguageModel
    interface. it implements 2 functions:
    - generate_text: for generating text from a given PromptValue
    - agenerate_text: for generating text from a given PromptValue asynchronously
    """

    def __init__(
        self, langchain_llm: BaseLanguageModel, run_config: t.Optional[RunConfig] = None
    ):
        self.langchain_llm = langchain_llm
        if run_config is None:
            run_config = RunConfig()
        self.set_run_config(run_config)

    def generate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: float = 1e-8,
        stop: t.Optional[t.List[str]] = None,
        callbacks: Callbacks = None,
    ) -> LLMResult:
        temperature = self.get_temperature(n=n)
        if is_multiple_completion_supported(self.langchain_llm):
            return self.langchain_llm.generate(
                prompts=[prompt],
                n=n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
        else:
            result = self.langchain_llm.generate(
                prompts=[prompt] * n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
            # make LLMResult.generation appear as if it was n_completions
            # note that LLMResult.runs is still a list that represents each run
            generations = [[g[0] for g in result.generations]]
            result.generations = generations
            return result

    async def agenerate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: float = 1e-8,
        stop: t.Optional[t.List[str]] = None,
        callbacks: Callbacks = None,
    ) -> LLMResult:
        temperature = self.get_temperature(n=n)
        if is_multiple_completion_supported(self.langchain_llm):
            return await self.langchain_llm.agenerate(
                prompts=[prompt],
                n=n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
        else:
            result = await self.langchain_llm.agenerate(
                prompts=[prompt] * n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
            # make LLMResult.generation appear as if it was n_completions
            # note that LLMResult.runs is still a list that represents each run
            generations = [[g[0] for g in result.generations]]
            result.generations = generations
            return result

    def set_run_config(self, run_config: RunConfig):
        self.run_config = run_config

        # configure if using OpenAI API
        if isinstance(self.langchain_llm, BaseOpenAI) or isinstance(
            self.langchain_llm, ChatOpenAI
        ):
            try:
                from openai import RateLimitError
            except ImportError:
                raise ImportError(
                    "openai.error.RateLimitError not found. Please install openai package as `pip install openai`"
                )
            self.langchain_llm.request_timeout = run_config.timeout
            self.run_config.exception_types = RateLimitError
```

Make sure that the `langchain_llm` object has the `generate` and `agenerate` methods instead of `generate_prompt` and `agenerate_prompt`. This should resolve the `AttributeError` <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/generator.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Implementing the previous code, I now have the this error. 

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 87, in run
    results = self.loop.run_until_complete(self._aresults())
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 79, in _aresults
    r = await future
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 37, in sema_coro
    return await coro
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 109, in wrapped_callable_async
    raise e
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/executor.py", line 104, in wrapped_callable_async
    result = await callable(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/extractor.py", line 49, in extract
    results = await self.llm.generate(prompt=prompt, is_async=is_async)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/llms/base.py", line 93, in generate
    return await agenerate_text_with_retry(
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 88, in async_wrapped
    return await fn(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 47, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/llms/base.py", line 178, in agenerate_text
    result = await self.langchain_llm.agenerate(
AttributeError: 'LangchainLLMWrapper' object has no attribute 'agenerate'
Traceback (most recent call last):
  File "/storage/local/mgmanzanillo3/projects/netopia/testset_generation/test.py", line 53, in <module>
    testset = generator.generate_with_langchain_docs(documents, 1, distributions)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/generator.py", line 206, in generate_with_langchain_docs
    self.docstore.add_documents(
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/docstore.py", line 215, in add_documents
    self.add_nodes(nodes, show_progress=show_progress)
  File "/home/mgmanzanillo3/miniconda3/envs/des_v2/lib/python3.9/site-packages/ragas/testset/docstore.py", line 254, in add_nodes
    raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.

<!-- Answer -->
To resolve the `AttributeError: 'LangchainLLMWrapper' object has no attribute 'agenerate'`, you need to ensure that the `LangchainLLMWrapper` class is correctly calling the `agenerate_text` method. The `LangchainLLMWrapper` class should have an `agenerate_text` method, not `agenerate`.

Here is the corrected code for the `LangchainLLMWrapper` class:

```python
class LangchainLLMWrapper(BaseRagasLLM):
    """
    A simple base class for RagasLLMs that is based on Langchain's BaseLanguageModel
    interface. it implements 2 functions:
    - generate_text: for generating text from a given PromptValue
    - agenerate_text: for generating text from a given PromptValue asynchronously
    """

    def __init__(
        self, langchain_llm: BaseLanguageModel, run_config: t.Optional[RunConfig] = None
    ):
        self.langchain_llm = langchain_llm
        if run_config is None:
            run_config = RunConfig()
        self.set_run_config(run_config)

    def generate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: float = 1e-8,
        stop: t.Optional[t.List[str]] = None,
        callbacks: Callbacks = None,
    ) -> LLMResult:
        temperature = self.get_temperature(n=n)
        if is_multiple_completion_supported(self.langchain_llm):
            return self.langchain_llm.generate(
                prompts=[prompt],
                n=n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
        else:
            result = self.langchain_llm.generate(
                prompts=[prompt] * n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
            generations = [[g[0] for g in result.generations]]
            result.generations = generations
            return result

    async def agenerate_text(
        self,
        prompt: PromptValue,
        n: int = 1,
        temperature: float = 1e-8,
        stop: t.Optional[t.List[str]] = None,
        callbacks: Callbacks = None,
    ) -> LLMResult:
        temperature = self.get_temperature(n=n)
        if is_multiple_completion_supported(self.langchain_llm):
            return await self.langchain_llm.agenerate(
                prompts=[prompt],
                n=n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
        else:
            result = await self.langchain_llm.agenerate(
                prompts=[prompt] * n,
                temperature=temperature,
                stop=stop,
                callbacks=callbacks,
            )
            generations = [[g[0] for g in result.generations]]
            result.generations = generations
            return result

    def set_run_config(self, run_config: RunConfig):
        self.run_config = run_config

        if isinstance(self.langchain_llm, BaseOpenAI) or isinstance(
            self.langchain_llm, ChatOpenAI
        ):
            try:
                from openai import RateLimitError
            except ImportError:
                raise ImportError(
                    "openai.error.RateLimitError not found. Please install openai package as `pip install openai`"
                )
            self.langchain_llm.request_timeout = run_config.timeout
            self.run_config.exception_types = RateLimitError
```

Make sure that the `langchain_llm` object has the `agenerate` method. If it does not, you may need to update your Langchain library or ensure that the correct class is being used. This should resolve the `AttributeError` <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


The added code still has the function langchain_llm.agenerate which is still not in the code.

<!-- Answer -->
To resolve the `AttributeError` indicating that the `LangchainLLMWrapper` object has no attribute `agenerate`, you should ensure that you are calling the correct method. The `LangchainLLMWrapper` class does not have a method named `agenerate`, but it does have an `agenerate_text` method for asynchronous text generation.

Here is an example of how you can use the `agenerate_text` method:

```python
# Assuming you have an instance of LangchainLLMWrapper named langchain_llm_wrapper
result = await langchain_llm_wrapper.agenerate_text(
    prompt=your_prompt_value,
    n=1,
    temperature=1e-8,
    stop=None,
    callbacks=None
)
```

Make sure to replace `your_prompt_value` with the actual prompt you want to use. This should resolve the `AttributeError` by calling the correct method.

Additionally, ensure that you are correctly initializing the `LangchainLLMWrapper` within the `TestsetGenerator` class. Here is an example of how the `LangchainLLMWrapper` is used in the `TestsetGenerator` class:

```python
class TestsetGenerator:
    generator_llm: BaseRagasLLM
    critic_llm: BaseRagasLLM
    embeddings: BaseRagasEmbeddings
    docstore: DocumentStore

    @classmethod
    def from_langchain(
        cls,
        generator_llm: LangchainLLM,
        critic_llm: LangchainLLM,
        embeddings: LangchainEmbeddings,
        docstore: t.Optional[DocumentStore] = None,
        run_config: t.Optional[RunConfig] = None,
        chunk_size: int = 1024,
    ) -> "TestsetGenerator":
        generator_llm_model = LangchainLLMWrapper(generator_llm)
        critic_llm_model = LangchainLLMWrapper(critic_llm)
        embeddings_model = LangchainEmbeddingsWrapper(embeddings)

        keyphrase_extractor = KeyphraseExtractor(llm=generator_llm_model)
        if docstore is None:
            from langchain.text_splitter import TokenTextSplitter

            splitter = TokenTextSplitter(chunk_size=chunk_size, chunk_overlap=0)
            docstore = InMemoryDocumentStore(
                splitter=splitter,
                embeddings=embeddings_model,
                extractor=keyphrase_extractor,
                run_config=run_config,
            )
        return cls(
            generator_llm=generator_llm_model,
            critic_llm=critic_llm_model,
            embeddings=embeddings_model,
            docstore=docstore,
        )
```

Ensure that you are using the `LangchainLLMWrapper` correctly as shown above. This should help in resolving the `AttributeError` <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py)</sup><sup>[[2]](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/generator.py)</sup><sup>[[3]](https://github.com/explodinggradients/ragas/blob/main/docs/howtos/customisations/bring-your-own-llm-or-embs.md)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


@netopia-gab Hi may I ask if you managed to solve the bug? I still have the same issue with the first bug. I would appreciate your help.

> @netopia-gab Hi may I ask if you managed to solve the bug? I still have the same issue with the first bug. I would appreciate your help.

Hi! Unfortunately not. I'm still waiting for somebody to help out. 

@netopia-gab @SalwaMostafa do check the types function expects
```py
generator = TestsetGenerator.from_langchain(
    generator_llm=langchain_llm, # no need to wrap any of these
    critic_llm=langchain_llm,
    embeddings=langchain_embeddings,
) 
```

I am facing the same issue: `ValueError: No nodes that satisfied the given filer. Try changing the filter.`, via LLM model loaded via `vllm` and `HuggingFaceEndpoint`, ref:

Llm:
```
from ragas.llms import LangchainLLMWrapper
from vllm import LLM, SamplingParams
# Create an LLM.
llm = LLM(model="ibm-granite/granite-3.1-8b-instruct", max_model_len=8192)
generator_llm = LangchainLLMWrapper(llm)
```
and, 
```
repo_id = "mistralai/Mistral-7B-Instruct-v0.2"

llm = HuggingFaceEndpoint(
    repo_id=repo_id,
    temperature=0.5,
    huggingfacehub_api_token=HUGGINGFACEHUB_API_TOKEN,
)
generator_llm = LangchainLLMWrapper(llm)
```

Embedding:
```
from langchain_huggingface import HuggingFaceEmbeddings
model_name = "sentence-transformers/all-mpnet-base-v2"
# model_name = "nomic-ai/nomic-embed-text-v1.5"
model_kwargs = {'device': 'cpu', 'trust_remote_code': True}
encode_kwargs = {'normalize_embeddings': False}
hf = HuggingFaceEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)
```

Testset generation via Ragas:
```
from ragas.testset import TestsetGenerator

generator = TestsetGenerator(llm=generator_llm, embedding_model=hf)
dataset = generator.generate_with_langchain_docs(docs, testset_size=10)
```

Results when LLM loaded via VLLM:
```
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'

unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node

unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'
unable to apply transformation: 'LLM' object has no attribute 'agenerate_prompt'

unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'

unable to apply transformation: Node 6bb4f54c-b9bb-4c7c-85d3-c2d9c77d2146 has no summary_embedding

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[6], line 4
      1 from ragas.testset import TestsetGenerator
      3 generator = TestsetGenerator(llm=generator_llm, embedding_model=hf)
----> 4 dataset = generator.generate_with_langchain_docs(docs, testset_size=10)

File ~/.env/lib64/python3.11/site-packages/ragas/testset/synthesizers/generate.py:188, in TestsetGenerator.generate_with_langchain_docs(self, documents, testset_size, transforms, transforms_llm, transforms_embedding_model, query_distribution, run_config, callbacks, with_debugging_logs, raise_exceptions)
    185 apply_transforms(kg, transforms)
    186 self.knowledge_graph = kg
--> 188 return self.generate(
    189     testset_size=testset_size,
    190     query_distribution=query_distribution,
    191     run_config=run_config,
    192     callbacks=callbacks,
    193     with_debugging_logs=with_debugging_logs,
    194     raise_exceptions=raise_exceptions,
    195 )

File ~/.env/lib64/python3.11/site-packages/ragas/testset/synthesizers/generate.py:369, in TestsetGenerator.generate(self, testset_size, query_distribution, num_personas, run_config, batch_size, callbacks, token_usage_parser, with_debugging_logs, raise_exceptions)
    366     patch_logger("ragas.experimental.testset.transforms", logging.DEBUG)
    368 if self.persona_list is None:
--> 369     self.persona_list = generate_personas_from_kg(
    370         llm=self.llm,
    371         kg=self.knowledge_graph,
    372         num_personas=num_personas,
    373         callbacks=callbacks,
    374     )
    375 else:
    376     random.shuffle(self.persona_list)

File ~/.env/lib64/python3.11/site-packages/ragas/testset/persona.py:95, in generate_personas_from_kg(kg, llm, persona_generation_prompt, num_personas, filter_fn, callbacks)
     93 nodes = [node for node in kg.nodes if filter_fn(node)]
     94 if len(nodes) == 0:
---> 95     raise ValueError(
     96         "No nodes that satisfied the given filer. Try changing the filter."
     97     )
     99 summaries = [node.properties.get("summary") for node in nodes]
    100 summaries = [summary for summary in summaries if isinstance(summary, str)]

ValueError: No nodes that satisfied the given filer. Try changing the filter.
```


Results when LLM loaded via HuggingFaceEndpoint:
```
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt headlines_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt headlines_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt headlines_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt headlines_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt headlines_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.

unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node

Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt summary_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt summary_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt summary_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt summary_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt fix_output_format failed to parse output: The output parser failed to parse the output including retries.
Prompt summary_extractor_prompt failed to parse output: The output parser failed to parse the output including retries.
unable to apply transformation: The output parser failed to parse the output including retries.

unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'
unable to apply transformation: node.property('summary') must be a string, found '<class 'NoneType'>'

unable to apply transformation: Node 65e9d592-d0d5-4c5e-bfb2-31eb058405c1 has no summary_embedding

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[18], line 4
      1 from ragas.testset import TestsetGenerator
      3 generator = TestsetGenerator(llm=generator_llm, embedding_model=hf)
----> 4 dataset = generator.generate_with_langchain_docs(docs, testset_size=10)

File ~/.env/lib64/python3.11/site-packages/ragas/testset/synthesizers/generate.py:188, in TestsetGenerator.generate_with_langchain_docs(self, documents, testset_size, transforms, transforms_llm, transforms_embedding_model, query_distribution, run_config, callbacks, with_debugging_logs, raise_exceptions)
    185 apply_transforms(kg, transforms)
    186 self.knowledge_graph = kg
--> 188 return self.generate(
    189     testset_size=testset_size,
    190     query_distribution=query_distribution,
    191     run_config=run_config,
    192     callbacks=callbacks,
    193     with_debugging_logs=with_debugging_logs,
    194     raise_exceptions=raise_exceptions,
    195 )

File ~/.env/lib64/python3.11/site-packages/ragas/testset/synthesizers/generate.py:369, in TestsetGenerator.generate(self, testset_size, query_distribution, num_personas, run_config, batch_size, callbacks, token_usage_parser, with_debugging_logs, raise_exceptions)
    366     patch_logger("ragas.experimental.testset.transforms", logging.DEBUG)
    368 if self.persona_list is None:
--> 369     self.persona_list = generate_personas_from_kg(
    370         llm=self.llm,
    371         kg=self.knowledge_graph,
    372         num_personas=num_personas,
    373         callbacks=callbacks,
    374     )
    375 else:
    376     random.shuffle(self.persona_list)

File ~/.env/lib64/python3.11/site-packages/ragas/testset/persona.py:95, in generate_personas_from_kg(kg, llm, persona_generation_prompt, num_personas, filter_fn, callbacks)
     93 nodes = [node for node in kg.nodes if filter_fn(node)]
     94 if len(nodes) == 0:
---> 95     raise ValueError(
     96         "No nodes that satisfied the given filer. Try changing the filter."
     97     )
     99 summaries = [node.properties.get("summary") for node in nodes]
    100 summaries = [summary for summary in summaries if isinstance(summary, str)]

ValueError: No nodes that satisfied the given filer. Try changing the filter.
```

I am not sure if model load via VLLM/HuggingFaceEndpoint is supported or not, and if not then how can I load a model of my choice for `testset generation`, as specified under [Others](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/#__tabbed_1_4) under choose your LLM.

 I've posted the same query to the issue #853, plz lemme know if anyone was able to resolve the issue, loading model other OpenAI, AWS/Azure. Thanks!