Repository: explodinggradients/ragas
Issue: Duplicated document node generated during knowlege graph creation
Root cause: Algorithmic Error

- [ YES ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
duplicated document nodes are generated during building knowlege graph

Ragas version: 0.2.6
Python version: 3.12

**Code to Reproduce**
For simplicity i am using the **exact** same example code as in the official quick start of synthetic testset generation
Share code to reproduce the issue
```python
git clone https://huggingface.co/datasets/explodinggradients/Sample_Docs_Markdown

from langchain_community.document_loaders import DirectoryLoader

path = "Sample_Docs_Markdown/"
loader = DirectoryLoader(path, glob="**/*.md")
docs = loader.load()

from ragas.llms import LangchainLLMWrapper
from ragas.embeddings import LangchainEmbeddingsWrapper
from langchain_openai import ChatOpenAI
from langchain_openai import OpenAIEmbeddings
from ragas.testset.graph import KnowledgeGraph

generator_llm = LangchainLLMWrapper(ChatOpenAI(model="gpt-4o"))
generator_embeddings = LangchainEmbeddingsWrapper(OpenAIEmbeddings())

kg = KnowledgeGraph()
from ragas.testset.graph import Node, NodeType

for doc in docs:
    kg.nodes.append(
        Node(
            type=NodeType.DOCUMENT,
            properties={"page_content": doc.page_content, "document_metadata": doc.metadata}
        )
    )
transformer_llm = generator_llm
embedding_model = generator_embeddings

trans = default_transforms(llm=transformer_llm, embedding_model=embedding_model)
apply_transforms(kg, trans)
kg.save("knowledge_graph.json")
```

**Error trace**
```ssh
Applying HeadlineSplitter:   0%|          | 0/12 [00:00<?, ?it/s]         unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
unable to apply transformation: 'headlines' property not found in this node
Applying SummaryExtractor:  17%|█▋        | 1/6 [00:02<00:10,  2.03s/it]Property 'summary' already exists in node '53e05d'. Skipping!
Applying [EmbeddingExtractor, ThemesExtractor, NERExtractor]:   0%|          | 0/28 [00:00<?, ?it/s]Property 'summary_embedding' already exists in node '53e05d'. Skipping!
```
there's no error during the generation but after examining the json, i found the duplicate
<img width="940" alt="Screenshot 2024-12-04 at 11 09 09" src="https://github.com/user-attachments/assets/32074f4d-b20f-4dd1-899e-b98b202b44da">

**Expected behavior**
the node should be unique

**Additional context**
I've tried large pdf document, almost each page have a duplicated node with exact same id and same properties
<img width="1206" alt="Screenshot 2024-12-04 at 11 13 17 1" src="https://github.com/user-attachments/assets/de4a91cf-a6d6-49b1-b24c-1b1cc607f860">

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


@shahules786 could you give some help here?

hey @xizhou-vw, thanks a lot for reporting this, let me check it and get back to you 

I also have this problem. Any progress here? @jjmachan 

Same issue here, i'm using ragas latest

same issue here? will this affect the data generation by generating duplicates??