With HuggingFaceEmbeddings, embedding of individual documents in the vectorstore can influence each other
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

langchain/0.0.258, Python 3.10.10

### Who can help?

@hw
@issam9 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

According the documentation/tutorials a vectorstore is created by these steps

1. Create a list of documents by some documents loaders
2. Use a text splitter for splitting the documents into smaller chunks
3. Pass the  resulting list to the vectorstore
See the tutorial  https://learn.deeplearning.ai/langchain-chat-with-your-data/lesson/4/vectorstores-and-embedding

In case of the HuggingFaceEmbeddings the computed embeddings of the chunks of the documents can influence each other.
This caused by the fact that the  HuggingFaceEmbeddings  is making a single call of the 'encode' method of the SentenceTransformer class, see https://github.com/langchain-ai/langchain/blob/1d55141c5016a2e197d3eed2844d55460d207801/libs/langchain/langchain/embeddings/huggingface.py#L91

The SentenceTransformer has a pooling layer as last layer, with 'pooling_mode_cls_token' or  'pooling_mode_mean'. And in the 
'encode' method the pooling layer is not cleared during looping through the sentences, see  https://github.com/UKPLab/sentence-transformers/blob/a458ce79c40fef93d5ecc66931b446ea65fdd017/sentence_transformers/SentenceTransformer.py#L159

By that the embedding of the chunks are influencing each other by the pooling layer. 

You can reproduce it with the following codes snipplet:

def print_document_embeddings(splits: list[Document], document_name: str):

        model_name = 'sentence-transformers/multi-qa-mpnet-base-dot-v1'
        embedding = HuggingFaceEmbeddings(model_name=model_name, model_kwargs={}, encode_kwargs={})

        vectordb = Chroma.from_documents(splits, embedding)
        dump = vectordb.get(include=['embeddings', 'metadatas', 'documents'])

        for metadata, document, vector in zip(dump['metadatas'], dump['documents'], dump['embeddings']):
            if metadata['source'] == document_name:
                print(f'Embedding: {vector[:3]}...')
                print(f'Document: {document}')
                print('-' * 10)

        vectordb.delete()
    

    doc1 = Document(page_content='The embedding vectors of a document must not depend on any other documents.', metadata={'source': 'document 1'})
    doc2 = Document(page_content='The grass is green.', metadata={'source': 'document 2'})
    doc3 = Document(page_content=' Any text larger the document 1.'*10, metadata={'source': 'document 3'})


    print('Vector store with [doc1,doc2]')
    print_document_embeddings([doc1,doc2],'document 1')

    print('Vector store with  [doc1,doc3]')
    print_document_embeddings([doc1, doc3],'document 1')


In the output you can see that the embeddings is not the same (the changes are small because document 3 is small, but they can be arbitrary large)

Vector store with [doc1,doc2]
Embedding: [0.03044097125530243, -0.5951688289642334, -0.11154567450284958]...
Document: The embedding vectors of a document must not depend on any other documents.
----------
Vector store with  [doc1,doc3]
Embedding: [0.030441032722592354, -0.5951685905456543, -0.11154570430517197]...
Document: The embedding vectors of a document must not depend on any other documents.



### Expected behavior

The expected behavior is that the embedding of a document is always that same, it can not be influenced by any other documents in the vectorstore.