Repository: explodinggradients/ragas
Issue: [R-245] docs: Update docs for Vertex AI customisation
Root cause: Documentation Deficiency

The documentation for Vertex AI in customization is not up to date. This needs to be changed per the new updates.

<sub>[R-245](https://linear.app/exploding-gradients/issue/R-245/docs-update-docs-for-vertex-ai-customisation)</sub>

Hi,

Is there any update on this? 
We have only access to ChatVertexAI and the documentation is outdated.

I tried to set the llm = ChatVertextAI in the **evaluate** function. But I get the following error:

**AttributeError: 'ChatVertexAI' object has no attribute 'set_run_config'**

Any help is greatly appreciated
 

We are running into exactly the same issue when calling the evaluate method 
`AttributeError: 'ChatVertexAI' object has no attribute 'set_run_config'`

Is there any workaround already ? 

Thanks!

same here, why is there no comment from the designers?

@jjmachan @EiJayK @tinomaxthayil @SajjadFazeli @sebastian-piedoux-dv see #863 for solution with #915 as the PR fixing the issue.

thanks for @shawnmittal for fixing this and adding it in the docs ❤️ 

btw are you on the discord channel - would love to have you as part of our private contributor list if your interested 🙂 

Hi, I am using custom ollama using llamaindex, I am getting error  
 error - AttributeError: 'CustomOllama' object has no attribute 'set_run_config' how to resolve


@Senthselvi should be because the LLM is not wrapped with `LlamaIndexLLMWrapper`. could you try wrapping that?

Guardrails validation failed on user request: 1 validation error for CallInputs\nllm_output\n  Input should be a valid string [type=string_type, input_value={\'prompt_str\': \'Given a q...nces: ""\\nanalysis: \\n\'}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.7/v/string_type'

data = {
    "question": ["What is the capital of India?", "What is the largest ocean on Earth?"],
    "answer": ["Delhi", "Pacific Ocean"],
    "ground_truth": ["Delhi", "Pacific Ocean"],  # Flattened to strings
    "contexts": [["Delhi"], ["Earth"]],  # Ensure contexts is a list of lists of strings
    'faithfulness': [np.nan, np.nan],
    'answer_relevancy': [np.nan, np.nan]
}


def evaluate_chat_performance(data, llm, embedding_model):
    dataset = Dataset.from_dict(data)
    print("Dataset preview:", dataset)
 
    if "ground_truth" not in dataset.column_names:
        dataset = dataset.add_column(
            name="ground_truth",
            column=dataset["ground_truth"],
            new_fingerprint=str(uuid4()),
        )

    print("Dataset after adding ground_truth:", dataset)

    metrics = [
        faithfulness,context_precision
      #  ar  # Using AnswerRelevancy instance
    ]try:
        results = evaluate(
            dataset=dataset,
            metrics=metrics,
          #  metrics=metrics,
            llm=llm,
            embeddings=embedding_model,
            raise_exceptions=False,
        
        )
    except Exception as e:
        print("Evaluation failed:", e)
        return

    print(results.to_pandas())
    results.to_pandas().to_csv(r'C:\TestingBot\Scripts\V1\myfile.csv', sep=',')
    return results


# Call the function
print(evaluate_chat_performance(data, llm, embedding_model))