Repository: explodinggradients/ragas
Issue: ModuleNotFoundError: No module named 'ragas.cost'
Root cause: Environment Incompatibility

[X] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Description**
Tried running the code from [the cost calculation page on the documentation](https://docs.ragas.io/en/latest/howtos/applications/cost.html). When the interpreter reaches the expression `from ragas.cost import get_token_usage_for_openai`, I get an `ModuleNotFoundError: No module named 'ragas.cost'` error.

Ragas version: 0.1.12 (from Conda)
Python version: 3.11.9

**Code to Reproduce**

```python
# lets import a parser for OpenAI
from ragas.cost import get_token_usage_for_openai
``` 

**Error trace**
ModuleNotFoundError: No module named 'ragas.cost'

**Expected behavior**
Should import the method as usual.

**Additional context**
I also tried running this [notebook](https://github.com/explodinggradients/ragas/blob/d2486f117fd827dcfc3e196d4cf7798573c55b09/docs/howtos/applications/cost.ipynb#L7). Returns the same error.




<!-- Greeting -->
Hello @CaduSantana! I'm here to help you with any bugs or questions you have while waiting for a human maintainer. 😊

<!-- Answer -->
The `ModuleNotFoundError: No module named 'ragas.cost'` error occurs because the `ragas.cost` module is not available in the installed version of the Ragas package. To resolve this issue, you should ensure that you have installed the latest version of the Ragas package from the main branch. You can do this by running the following command:

```bash
pip install git+https://github.com/explodinggradients/ragas.git
```

If you have already done this and are still encountering the error, it is possible that the `ragas.cost` module does not exist in the repository. In that case, you should check the Ragas documentation or repository for any updates or changes regarding the module <sup>[[1]](https://github.com/explodinggradients/ragas/blob/main/docs/getstarted/install.md)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


dosu is right - but new release will be out in a couple of hours so that works too. @CaduSantana do share you feedback on the cost module though  would really appreciate it 🙂 

> dosu is right - but new release will be out in a couple of hours so that works too. @CaduSantana do share you feedback on the cost module though would really appreciate it 🙂

@jjmachan Unfortunately I ran into another issue. Although I've followed [this page](https://docs.ragas.io/en/latest/howtos/applications/data_preparation.html), I get the `AttributeError: 'str' object has no attribute 'set_run_config'` error. As I stumbled on Issue #413, I thought it would be prudent to instead try the examples on the documentation first:

```python
from datasets import Dataset 

data_samples = {
    'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
    'answer': ['The first superbowl was held on January 15, 1967', 'The most super bowls have been won by The New England Patriots'],
    'contexts' : [['The Super Bowl....season since 1966,','replacing the NFL...in February.'], 
    ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
    'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
}
dataset = Dataset.from_dict(data_samples)
``` 

However, it still returns the same error (`AttributeError: 'str' object has no attribute 'set_run_config'`).

@CaduSantana could you share the trace for the same?

@jjmachan Yes. As it follows:

```powershell
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 12
      3 data_samples = {
      4     'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
      5     'answer': ['The first superbowl was held on January 15, 1967', 'The most super bowls have been won by The New England Patriots'],
   (...)
      8     'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
      9 }
     10 dataset_ = Dataset.from_dict(data_samples)
---> 12 result = evaluate(
     13     dataset_,
     14     metrics=[answer_relevancy, faithfulness, context_recall, context_precision],
     15     llm="gpt-4o-mini",
     16     token_usage_parser=get_token_usage_for_openai,
     17 )

File ~/anaconda3/envs/rag_venv/lib/python3.11/site-packages/ragas/_analytics.py:129, in track_was_completed.<locals>.wrapper(*args, **kwargs)
    126 @wraps(func)
    127 def wrapper(*args: P.args, **kwargs: P.kwargs) -> t.Any:
    128     track(IsCompleteEvent(event_type=func.__name__, is_completed=False))
--> 129     result = func(*args, **kwargs)
    130     track(IsCompleteEvent(event_type=func.__name__, is_completed=True))
    132     return result

File ~/anaconda3/envs/rag_venv/lib/python3.11/site-packages/ragas/evaluation.py:199, in evaluate(dataset, metrics, llm, embeddings, callbacks, in_ci, run_config, token_usage_parser, raise_exceptions, column_map)
    196             reproducable_metrics.append(i)
    198     # init all the models
--> 199     metric.init(run_config)
    201 executor = Executor(
    202     desc="Evaluating",
    203     keep_progress_bar=True,
    204     raise_exceptions=raise_exceptions,
    205     run_config=run_config,
    206 )
    208 # Ragas Callbacks
    209 # init the callbacks we need for various tasks

File ~/anaconda3/envs/rag_venv/lib/python3.11/site-packages/ragas/metrics/base.py:150, in MetricWithLLM.init(self, run_config)
    146 if self.llm is None:
    147     raise ValueError(
    148         f"Metric '{self.name}' has no valid LLM provided (self.llm is None). Please initantiate a the metric with an LLM to run."  # noqa
    149     )
--> 150 self.llm.set_run_config(run_config)

AttributeError: 'str' object has no attribute 'set_run_config'
``` 

the code @CaduSantana is wrong 
```py
llm="gpt-4o-mini",
```

that should not be a string but an LLM. 

```py
result = evaluate(
     13     dataset_,
     14     metrics=[answer_relevancy, faithfulness, context_recall, context_precision],
     16     token_usage_parser=get_token_usage_for_openai,
     17 )
```

just do this instead

Oh, so that was the issue.
Thanks for the headsup! 

Also

> dosu is right - but new release will be out in a couple of hours so that works too. @CaduSantana do share you feedback on the cost module though would really appreciate it 🙂

Bretty gud 👍. It reflected correctly our costs on the OpenAI dashboard. I like how simple and easy it is.

Great project and hope you guys keep growing this!

Close comment.

@CaduSantana awesome! great to hear that 🙂. We will keep growing this too - we want to build an open-source standard for evaluations, lets see how it goes 😅 🤞🏽 

btw if you have any feedback or ideas for improvement do share it here on discord too 🙌🏽