Repository: run-llama/llama_index
Pull request: Fix the RAG fusion pipeline
# Description

This PR tries to fix #13364.

## New Package?

Did I fill in the `tool.llamahub` section in the `pyproject.toml` and provide a detailed README.md for my new integration or package?

- [ ] Yes
- [x] No

## Version Bump?

Did I bump the version in the `pyproject.toml` file of the package I am updating? (Except for the `llama-index-core` package)

- [ ] Yes
- [x] No

## Type of Change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

- [ ] Added new unit/integration tests
- [ ] Added new notebook (that tests end-to-end)
- [x] I stared at the code and made sure it makes sense

## Suggested Checklist:

- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added Google Colab support for the newly added notebooks.
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I ran `make format; make lint` to appease the lint gods

Problem statement:
[Bug]: AttributeError: 'tuple' object has no attribute 'score'
### Bug Description

/usr/local/lib/python3.10/dist-packages/llama_index/packs/rag_fusion_query_pipeline/base.py in <lambda>(x)
     33     for nodes_with_scores in results:
     34         for rank, node_with_score in enumerate(
---> 35             sorted(nodes_with_scores, key=lambda x: x.score or 0.0, reverse=True)
     36         ):
     37             text = node_with_score.node.get_content()

AttributeError: 'tuple' object has no attribute 'score'

### Version

0.10.35

### Steps to Reproduce

from llama_index.core.llama_pack import download_llama_pack

# download and install dependencies
RAGFusionPipelinePack = download_llama_pack(
    "RAGFusionPipelinePack", "./rag_fusion_pipeline_pack"
)

### Relevant Logs/Tracbacks

```shell
SOLUTION 

MODIFY IN THE SOURCE 

llama_index/packs/rag_fusion_query_pipeline/base.py LINE 35 

key=lambda x: x.score or 0.0 FOR  key=lambda x: x[1]
```

Hints from issue discussion:
<!-- Answer -->
The error you're encountering is due to `nodes_with_scores` being populated with tuples instead of `NodeWithScore` objects, which is why you're seeing an `AttributeError` when attempting to access a `score` attribute that doesn't exist on tuples. The line causing the issue is trying to sort `nodes_with_scores` based on the `score` attribute of each item, which works only if those items are `NodeWithScore` objects.

Your proposed solution to change `key=lambda x: x.score or 0.0` to `key=lambda x: x[1]` will indeed avoid the `AttributeError` by treating each item as a tuple and sorting based on its second element. However, this approach assumes that the structure of `nodes_with_scores` is intended to be a list of tuples, which might not align with the original design of the code. This change could lead to incorrect behavior if the list is expected to contain objects with a `score` attribute.

A more appropriate solution would be to ensure that `nodes_with_scores` is populated with `NodeWithScore` objects as originally intended. This might require reviewing the upstream code that generates `nodes_with_scores` to correct the data type mismatch. This approach addresses the root cause of the issue and maintains the integrity of the data handling logic in the codebase.


<!-- Dosu Comment Footer -->
<details open>
<summary>

#### Details
</summary>

**Sources**
- [llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py](https://github.com/run-llama/llama_index/blob/main/llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py)
- [docs/docs/examples/retrievers/reciprocal_rerank_fusion.ipynb](https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/retrievers/reciprocal_rerank_fusion.ipynb)
- [docs/docs/examples/low_level/fusion_retriever.ipynb](https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/low_level/fusion_retriever.ipynb)
- [llama-index-core/llama_index/core/retrievers/fusion_retriever.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/retrievers/fusion_retriever.py)
- [llama-index-core/llama_index/core/schema.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/schema.py)
- [llama-index-legacy/llama_index/legacy/llama_pack/download.py](https://github.com/run-llama/llama_index/blob/main/llama-index-legacy/llama_index/legacy/llama_pack/download.py)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*

Commit message: Fix the RAG fusion pipeline

Closes #13364.

Patch to review:
diff --git a/llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py b/llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py
index 5b948673f3..318c36f3e9 100644
--- a/llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py
+++ b/llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py
@@ -18,7 +18,7 @@
 
 
 def reciprocal_rank_fusion(
-    results: List[List[NodeWithScore]],
+    results: List[NodeWithScore],
 ) -> List[NodeWithScore]:
     """Apply reciprocal rank fusion.
 
@@ -30,15 +30,14 @@ def reciprocal_rank_fusion(
     text_to_node = {}
 
     # compute reciprocal rank scores
-    for nodes_with_scores in results:
-        for rank, node_with_score in enumerate(
-            sorted(nodes_with_scores, key=lambda x: x.score or 0.0, reverse=True)
-        ):
-            text = node_with_score.node.get_content()
-            text_to_node[text] = node_with_score
-            if text not in fused_scores:
-                fused_scores[text] = 0.0
-            fused_scores[text] += 1.0 / (rank + k)
+    for rank, node_with_score in enumerate(
+        sorted(results, key=lambda x: x.score or 0.0, reverse=True)
+    ):
+        text = node_with_score.node.get_content()
+        text_to_node[text] = node_with_score
+        if text not in fused_scores:
+            fused_scores[text] = 0.0
+        fused_scores[text] += 1.0 / (rank + k)
 
     # sort results
     reranked_results = dict(
@@ -65,6 +64,7 @@ def __init__(
         self,
         documents: List[Document],
         llm: Optional[LLM] = None,
+        embed_model: Optional[Any] = "default",
         chunk_sizes: Optional[List[int]] = None,
     ) -> None:
         """Init params."""
@@ -80,7 +80,9 @@ def __init__(
             splitter = SentenceSplitter(chunk_size=chunk_size, chunk_overlap=0)
             nodes = splitter.get_nodes_from_documents(documents)
 
-            service_context = ServiceContext.from_defaults(llm=self.llm)
+            service_context = ServiceContext.from_defaults(
+                llm=self.llm, embed_model=embed_model
+            )
             vector_index = VectorStoreIndex(nodes, service_context=service_context)
             self.query_engines.append(vector_index.as_query_engine())
 
@@ -94,7 +96,7 @@ def __init__(
         module_dict = {
             **self.retrievers,
             "input": InputComponent(),
-            "summarizer": TreeSummarize(),
+            "summarizer": TreeSummarize(llm=llm),
             # NOTE: Join args
             "join": ArgPackComponent(),
             "reranker": rerank_component,


Code review comments:
diff_hunk: @@ -80,7 +80,9 @@ def __init__(
             splitter = SentenceSplitter(chunk_size=chunk_size, chunk_overlap=0)
             nodes = splitter.get_nodes_from_documents(documents)
 
-            service_context = ServiceContext.from_defaults(llm=self.llm)
+            service_context = ServiceContext.from_defaults(
path: llama-index-packs/llama-index-packs-rag-fusion-query-pipeline/llama_index/packs/rag_fusion_query_pipeline/base.py
text: @user1:
should we even be using service context? We can pass the embed model to the index, and the llm to the query engine

```
vector_index = VectorStoreIndex(nodes, embed_model=embed_model)
self.query_engines.append(vector_index.as_query_engine(llm=self.llm))
```

@author:
Indeed, `service_context` is marked as deprecated.