DOC: Code/twitter-the-algorithm-analysis-deeplake not working as written
Labels: name: auto:bug, name: auto:documentation

### Issue with current documentation:

I followed the documentation @ https://python.langchain.com/docs/use_cases/code/twitter-the-algorithm-analysis-deeplake.

I replaced 'twitter-the-algorithm' with another code base I'm analyzing and used my own credentials from OpenAI and Deep Lake.

When I run the code (on VS Code for Mac with M1 chip), I get the following error:

_ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (1435,) + inhomogeneous part.  

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/catherineswope/Desktop/LangChain/fromLangChain.py", line 37, in <module>
    db.add_documents(texts)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/vectorstores/base.py", line 91, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/vectorstores/deeplake.py", line 184, in add_texts
    return self.vectorstore.add(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/deeplake/core/vectorstore/deeplake_vectorstore.py", line 271, in add
    dataset_utils.extend_or_ingest_dataset(
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/deeplake/core/vectorstore/vector_search/dataset/dataset.py", line 409, in extend_or_ingest_dataset
    raise IncorrectEmbeddingShapeError()
deeplake.util.exceptions.IncorrectEmbeddingShapeError: The embedding function returned embeddings of different shapes. Please either use different embedding function or exclude invalid files that are not supported by the embedding function._

This is the code snippet from my actual code:

import os
import getpass

from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import DeepLake
from langchain.document_loaders import TextLoader

#get OPENAI API KEY and ACTIVELOOP_TOKEN
os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
os.environ["ACTIVELOOP_TOKEN"] = getpass.getpass("Activeloop Token:")

embeddings = OpenAIEmbeddings(disallowed_special=())

#clone from chattydocs git hub repo removedcomments branch and copy/paste path
root_dir = "/Users/catherineswope/chattydocs/incubator-baremaps-0.7.1-removedcomments"
docs = []
for dirpath, dirnames, filenames in os.walk(root_dir):
    for file in filenames:
        try:
            loader = TextLoader(os.path.join(dirpath, file), encoding="utf-8")
            docs.extend(loader.load_and_split())
        except Exception as e:
            pass

from langchain.text_splitter import CharacterTextSplitter

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(docs)

username = "caswvu"  # replace with your username from app.activeloop.ai
db = DeepLake(
    dataset_path=f"hub://caswvu/baremaps",
    embedding_function=embeddings,
)
db.add_documents(texts)

db = DeepLake(
    dataset_path="hub://caswvu/baremaps",
    read_only=True,
    embedding_function=embeddings,
)

retriever = db.as_retriever()
retriever.search_kwargs["distance_metric"] = "cos"
retriever.search_kwargs["fetch_k"] = 100
retriever.search_kwargs["maximal_marginal_relevance"] = True
retriever.search_kwargs["k"] = 10

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain

model = ChatOpenAI(model_name="gpt-3.5-turbo")  # switch to 'gpt-4'
qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)

questions = [
    "What does this code do?",
]
chat_history = []

for question in questions:
    result = qa({"question": question, "chat_history": chat_history})
    chat_history.append((question, result["answer"]))
    print(f"-> **Question**: {question} \n")
    print(f"**Answer**: {result['answer']} \n")


### Idea or request for content:

Can you please help me understand how to fix the code to address the error message?  Also, if applicable, address in the documentation so that others can avoid as well.  Thank you!