langchain with LlamaCpp for QA from txt files fails on Chroma part
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

Firstly sorry if I am posting this in wrong place but I felt like it belongs to here.
I am trying use LlamaCpp for QA for txt documents but on Chromadb I am getting following error
I couldn't find a way to solve this:

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-13-9bd0ee703b23>](https://localhost:8080/#) in <cell line: 1>()
----> 1 db = Chroma.from_documents(texts, embeddings, persist_directory='db')

7 frames
[/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py](https://localhost:8080/#) in from_documents(cls, documents, embedding, ids, collection_name, persist_directory, client_settings, client, collection_metadata, **kwargs)
    576         texts = [doc.page_content for doc in documents]
    577         metadatas = [doc.metadata for doc in documents]
--> 578         return cls.from_texts(
    579             texts=texts,
    580             embedding=embedding,

[/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py](https://localhost:8080/#) in from_texts(cls, texts, embedding, metadatas, ids, collection_name, persist_directory, client_settings, client, collection_metadata, **kwargs)
    540             **kwargs,
    541         )
--> 542         chroma_collection.add_texts(texts=texts, metadatas=metadatas, ids=ids)
    543         return chroma_collection
    544 

[/usr/local/lib/python3.10/dist-packages/langchain/vectorstores/chroma.py](https://localhost:8080/#) in add_texts(self, texts, metadatas, ids, **kwargs)
    173         embeddings = None
    174         if self._embedding_function is not None:
--> 175             embeddings = self._embedding_function.embed_documents(list(texts))
    176 
    177         if metadatas:

[/usr/local/lib/python3.10/dist-packages/langchain/embeddings/llamacpp.py](https://localhost:8080/#) in embed_documents(self, texts)
    108             List of embeddings, one for each text.
    109         """
--> 110         embeddings = [self.client.embed(text) for text in texts]
    111         return [list(map(float, e)) for e in embeddings]
    112 

[/usr/local/lib/python3.10/dist-packages/langchain/embeddings/llamacpp.py](https://localhost:8080/#) in <listcomp>(.0)
    108             List of embeddings, one for each text.
    109         """
--> 110         embeddings = [self.client.embed(text) for text in texts]
    111         return [list(map(float, e)) for e in embeddings]
    112 

[/usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py](https://localhost:8080/#) in embed(self, input)
    810             A list of embeddings
    811         """
--> 812         return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
    813 
    814     def _create_completion(

[/usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py](https://localhost:8080/#) in create_embedding(self, input, model)
    774             tokens = self.tokenize(input.encode("utf-8"))
    775             self.reset()
--> 776             self.eval(tokens)
    777             n_tokens = len(tokens)
    778             total_tokens += n_tokens

[/usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py](https://localhost:8080/#) in eval(self, tokens)
    469                 raise RuntimeError(f"llama_eval returned {return_code}")
    470             # Save tokens
--> 471             self.input_ids[self.n_tokens : self.n_tokens + n_tokens] = batch
    472             # Save logits
    473             rows = n_tokens if self.params.logits_all else 1

ValueError: could not broadcast input array from shape (8,) into shape (0,)
```

Code:

```
#installation
!pip install langchain PyPDF2 huggingface_hub chromadb llama-cpp-python

#download the model
!git clone https://github.com/ggerganov/llama.cpp.git
%cd llama.cpp
!curl -L https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q4_K_M.bin -o ./models/llama-2-7b-chat.ggmlv3.q4_K_M.bin
!LLAMA_METAL=1 make
```

```
from langchain.llms import LlamaCpp
from langchain.embeddings import LlamaCppEmbeddings
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma

prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know or you can't help, don't try to make up an answer.

{context}

Question: {question}
Answer:"""
prompt = PromptTemplate(template=prompt_template, input_variables=["context", "question"])


model_path = '/content/llama.cpp/models/llama-2-7b-chat.ggmlv3.q4_K_M.bin'

llm = LlamaCpp(model_path=model_path)
embeddings = LlamaCppEmbeddings(model_path=model_path)


llm_chain = LLMChain(llm=llm, prompt=prompt)

loader = TextLoader(txt_file_path)
docs = loader.load()
    
text_splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=0)
texts = text_splitter.split_documents(docs)

db = Chroma.from_documents(texts, embeddings, persist_directory='db')


question = 'summerize this document'    
similar_doc = db.similarity_search(question, k=1)
context = similar_doc[0].page_content
query_llm = LLMChain(llm=llm, prompt=prompt)
response = query_llm.run({"context": context, "question": question})        
```
versions:
```
langchain==0.0.246
chromadb==0.4.3
```


Is there any alternative way to do what i want to achieve?

### Who can help?

@hwchase17  @agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

- Upload a text file
- run all the code to download the model
- replace model_path with downloaded model
- run the rest

### Expected behavior

working