delayedkarma/langchain-issues huggingface dataset:
`langchain.testing` Module with Fake Components for Testing Downstream Langchain Applications
Labels: name: area: agent, name: area: embeddings, name: auto:enhancement, name: auto:documentation, name: area: models

### Feature request

I propose the addition of a testing module within the Langchain library that provides "Fake" implementations of various components, such as language models, chat models, document stores, retrievers, agents, and tools (I may have forgotten some key components). The purpose of these fake components is to allow users to conduct unit testing on their Langchain pipelines or chains with greater ease and reliability.

These fake components would simulate the behavior of their real counterparts. Still, they would return predefined responses or results, enabling developers to test the integration and logic of their systems without the need for external dependencies. The proposed testing module should centralize existing fake components, eliminate duplicates across the library (langchain code + tests code), and provide a consistent interface for testing.

### Motivation

The current state of testing within the Langchain library seems fragmented, with various fake implementations embedded within the library code (LLM, Chat, Embedding), while others reside within test suites or are scattered and duplicated across different tests. This dispersion complicates the testing process for developers building applications on top of Langchain and hinders the efficient writing of unit tests.

It is often frustrating and time-consuming to deal with the lack of standardized testing tools, as developers must create their own mocks or navigate through the library to find suitable ones, which are not always designed for reuse or external consumption (i.e. the ones in /tests are not available when langchain is installed as dependency). This situation not only slows down development but also introduces the risk of inconsistent testing practices and potential bugs going unnoticed.

### Your contribution

To aid in the implementation of this feature, I am willing to contribute by:

* Submitting a Pull Request with an initial version of the testing module (only moving the existing `fake` modules in langchain module code, not the ones in tests).
* Refactor existing tests to use new fake components.
* Collaborating with maintainers to plan to implement more components.
* Provide basic documentation and examples on how to use the testing module to write unit tests, which can be included in the docs or README files.

Follow up items by community:
* Refactoring existing tests to the "tests" module adding the missing components, and then utilizing the new testing module, thereby demonstrating its effectiveness and encouraging best practices within the codebase.
'super' object has no attribute 'parse_obj'
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

Langchain version - 0.0.330
Python version - 3.10

### Who can help?

@hwchase17 
@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Hello! I am using NLAToolkit to load the OpenAPI spec and here is the code sample

```python
  llm = ChatOpenAI(
      model_name="gpt-4",
      temperature=0,
      openai_api_key=bot_config["OPENAI_API_KEY"],
  )

  memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

  analytics_toolkit = NLAToolkit.from_llm_and_spec(
      llm, OpenAPISpec.from_spec_dict(api_spec)
  )
  ```
  I have ensured that api_spec is a dict.
  
  Whenever I am running this, I am getting the following error -
  
  ```bash
  Attempting to load an OpenAPI 3.0.3 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Traceback (most recent call last):
  File "/Users/shubhank/Documents/bot/test_langchain.py", line 3, in <module>
    output_answer = ask_question_from_llm(
                    ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shubhank/Documents/bot/langchain_code.py", line 61, in ask_question_from_llm
    llm, OpenAPISpec.from_spec_dict(api_spec)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shubhank/.local/share/virtualenvs/bot-vTuYgIC7/lib/python3.11/site-packages/langchain/utilities/openapi.py", line 218, in from_spec_dict
    return cls.parse_obj(spec_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shubhank/.local/share/virtualenvs/bot-vTuYgIC7/lib/python3.11/site-packages/langchain/utilities/openapi.py", line 202, in parse_obj
    return super().parse_obj(obj)
           ^^^^^^^^^^^^^^^^^
AttributeError: 'super' object has no attribute 'parse_obj'
```

### Expected behavior

It should load the API spec and then answer the question accordingly. This was working previously as I have had tested it before!
DOC: Links for agent runtimes do not work
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

There are some links that return "Page Not Found" and lead to [langchain/cookbook](https://python.langchain.com/cookbook) when refreshed.

The three links are under [Other types of agent runtimes](https://python.langchain.com/docs/modules/agents/#other-types-of-agent-runtimes).

Clicking on them leads to [this](https://python.langchain.com/docs/use_cases/more/agents/autonomous_agents/plan_and_execute) page. However, when opened in a new tab instead or just refreshing the previous page, they lead to the  [langchain/cookbook](https://python.langchain.com/cookbook) page.

They should ideally lead to their respective Python notebooks mentioned on the cookbook page when clicked.

<img width="1561" alt="Screenshot 2023-11-03 at 22 21 05" src="https://github.com/langchain-ai/langchain/assets/12759088/b1017c9e-a62f-4ec2-8c7d-6699d556c388">

### Idea or request for content:

_No response_
Adds `callbacks.logging.LoggingCallbackHandler`
Labels: name: auto:improvement

This PR creates a logging handler:

- Takes the innards of `StdoutCallbackHandler` and `FileCallbackHandler` to build a `LoggingCallbackHandler`
- Makes a simple unit test for it

---

An aside is `StdoutCallbackHandler` and `FileCallbackHandler` could now be refactored to reuse `LoggingCallbackHandler` behind the scenes
draft
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Adding LLMEncodedChainExtractor
Labels: name: auto:enhancement, name: area: models

**Description:** Adding an optimized version of the `LLMChainExtractor` which uses an encoding technique to fetch only the indices of the relevant sequence instead of having the LLM regenerate the sequence verbatim. Unit and integrations tests, and a demo notebook have been added.
**Dependencies:** Optional dependencies - `spacy^3.7.2` and the `en_core_web_sm-3.7.0` model.
 **Tag maintainer:** @hwchase17
OpenAIEmbeddings Sending TikToken Tokens Rather than Texts
Labels: name: area: embeddings, name: auto:bug

### System Info

langchain==0.0.263

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.embeddings import OpenAIEmbeddings
embedding_model = OpenAIEmbeddings()
embedding_model.embed_query("Hello")
```

### Expected behavior

Send "Hello" to OpenAI, it now sends the tiktoken tokens.
"PydanticUserError" when importing OpenAI using Langchain.llms
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Here is the Error I am getting when I am Importing from langchain.llms import OpenAI:

PydanticUserError                         Traceback (most recent call last)
[e:\Data](file:///E:/Data) Science\Generative AI\Langchain\langchain.ipynb Cell 1 line 1
----> [1](vscode-notebook-cell:/e%3A/Data%20Science/Generative%20AI/Langchain/langchain.ipynb#W4sZmlsZQ%3D%3D?line=0) from langchain import llms

File [e:\Data](file:///E:/Data) Science\Generative AI\Langchain\venv\lib\site-packages\langchain\__init__.py:8
      5 with open(Path(__file__).absolute().parents[0] / "VERSION") as _f:
      6     __version__ = _f.read().strip()
----> 8 from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
      9 from langchain.chains import (
     10     ConversationChain,
     11     LLMChain,
   (...)
     18     VectorDBQAWithSourcesChain,
     19 )
     20 from langchain.docstore import InMemoryDocstore, Wikipedia

File [e:\Data](file:///E:/Data) Science\Generative AI\Langchain\venv\lib\site-packages\langchain\agents\__init__.py:2
      1 """Routing chains."""
----> 2 from langchain.agents.agent import Agent
      3 from langchain.agents.loading import initialize_agent
      4 from langchain.agents.mrkl.base import MRKLChain, ZeroShotAgent

File [e:\Data](file:///E:/Data) Science\Generative AI\Langchain\venv\lib\site-packages\langchain\agents\agent.py:11
      9 from langchain.agents.tools import Tool
...
    236 def dec(f: Callable[..., Any] | classmethod[Any, Any, Any] | staticmethod[Any, Any]) -> Any:

PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

For further information visit https://errors.pydantic.dev/2.4/u/root-validator-pre-skip
Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?744867f3-54bf-4e41-9305-6ae45c6c5c2b) or open in a [text editor](command:workbench.action.openLargeOutput?744867f3-54bf-4e41-9305-6ae45c6c5c2b). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...

### Suggestion:

_No response_
Issue: Unclear timeline and process for PR
Labels: name: auto:improvement, name: auto:documentation

### Issue you'd like to raise.

If a pull request is submitted but not acted on, changes made in the pr to common modules like .toml and __init__.py can quickly become dated and require hand-reconciliation. Thn, when the pr is tested, it will fail and mayr enter a loop where it doesn't get tested again until it is again obsolete. My PR #12602 in response to https://github.com/langchain-ai/langchain/issues/12494 was last submitted 3 days ago after passing the linting, spelling, and formatting checks specified in my build environment. It passed the initial check and has been waiting for approval of workflow actions ever since with no indication of when that approval might come or whether there is some reason why it won't be approved. Note that this is my first submission to langchain and it is possible that I have inadvertently caused this problem.

### Suggestion:

either:

clearer documentation on how to get a PR approved, or
a faster pr approval/disapproval process, or
an opportunity to resynch prior to immediate testing by langchain, or
an explanation of what I did wrong.

Thank you
Issue: not able to delete embeddings of a pdf from the embeddings folder
Labels: name: area: embeddings, name: auto:bug, name: integration: chroma

### Issue you'd like to raise.

def delete_document_embeddings_by_filename(file_path, persist_directory):
chroma_db = chromadb.PersistentClient(path=persist_directory)
print(chroma_db)
collection = chroma_db.get_collection(name="langchain")
print(collection)
collection.delete(where={"source": file_path})
output of the above code is:-
<chromadb.api.segment.SegmentAPI object at 0x7f4948165280>
name='langchain' id=UUID('8a5e8fff-93a4-49f3-8be7-5aac47cb3902') metadata=None
And I am calling like this
persist_directory=f'/home/hs/CustomBot/chroma-databases/{formatted_project_name}'
file=/home/hs/CustomBot/media/project/Code_of_Conduct_Policy.pdf
delete_document_embeddings_by_filename(file, persist_directory)
Still not able to delete embeddings of a pdf from the embeddings folder

### Suggestion:

_No response_
CLI 0.0.14, Uvicorn update and no more [serve]
Labels: name: auto:improvement

Calls uvicorn directly from cli:
Reload works if you define app by import string instead of object.
(was doing subprocess in order to get reloading)

Version bump to 0.0.14

Remove the need for [serve] for simplicity.

Readmes are updated in #12847 to avoid cluttering this PR
vectorstores/docarray: `_similarity_search_with_relevance_scores` raises a NotImplementedError
Labels: name: area: vector store, name: auto:bug

### System Info

Langchain 0.0.329
Python 3.10.11
MacOS 12.7.1 (21G920)

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python

# Dependencies
import pathlib
from langchain.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores.docarray import DocArrayInMemorySearch
from langchain.llms.huggingface_pipeline import HuggingFacePipeline
from langchain import hub

# Setup
dir_data = pathlib.Path("../data_sample")
document_loader = DirectoryLoader(dir_data, show_progress=True)
documents = document_loader.load()
document_chunker = RecursiveCharacterTextSplitter(chunk_size=50, chunk_overlap=5)
document_chunks = document_chunker.split_documents(documents)
embeddings = HuggingFaceEmbeddings(model_name="multi-qa-MiniLM-L6-cos-v1")
vector_store = DocArrayInMemorySearch.from_documents(document_chunks, embeddings)
llm = HuggingFacePipeline.from_model_id(
    task="text2text-generation",
    model_id="google/flan-t5-small",
    model_kwargs=dict(temperature=0.01, max_length=128, do_sample=True),
)
qa_rag_prompt = hub.pull("rlm/rag-prompt")
qa = RetrievalQA.from_chain_type(
    llm,
    retriever=vector_store.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5}),
    chain_type_kwargs={"prompt": qa_rag_prompt},
    return_source_documents=True,
)

# OK: Supported by Vector Store (DocArrayInMemorySearch)
vector_store.similarity_search_with_score("What is the greatest ocean in the world?")

# NOK: NotImplemented @ `docarray.base`
answer = qa({"query": question}) # will fail
```

# Error Message
```
    def _similarity_search_with_relevance_scores(
        self,
        query: str,
        k: int = 4,
        **kwargs: Any,
    ) -> List[Tuple[Document, float]]:
        """Return docs and relevance scores, normalized on a scale from 0 to 1.
    
        0 is dissimilar, 1 is most similar.
        """
>       raise NotImplementedError()
E       NotImplementedError

venv/lib/python3.10/site-packages/langchain/vectorstores/docarray/base.py:140: NotImplementedError
```

### Expected behavior

The `{context}` variable provided to the prompt should only be _suffed_ by chunks retrieved with relevance score above the threshold.
`ChatAnyscale` not working because of OpenAI API Key
Labels: name: auto:bug, name: area: models

### System Info

Langchain 0.0.329
Python 3.9
Windows 10

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python

import os 
from langchain.chat_models import ChatAnyscale

os.environ['OPENAI_API_KEY'] = "mykey"
os.environ['ANYSCALE_API_KEY'] = "mykey"

ChatAnyscale()

```

```
     95 
     96     def __init__(self, **kwargs: Any) -> None:
---> 97         super().__init__(**kwargs)
     98         self._lc_kwargs = kwargs
     99 

/usr/local/lib/python3.10/dist-packages/pydantic/main.cpython-310-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for ChatAnyscale
openai_api_key
  str type expected (type=type_error.str)
```

### Expected behavior

It shouldn't return an error because my Keys are both correct. For example, `ChatOpenAI` works fine.
issue #12165 mask API key in chat_models/azureml_endpoint module
Labels: name: auto:improvement, name: area: models

  - **Description:** `AzureMLChatOnlineEndpoint` object from langchain/chat_models/azureml_endpoint.py safe to print
without having any secrets included in raw format in the string representation.
  - **Issue:** #12165,
  - **Tag maintainer:** @eyurtsev
Issue: core dumped / segmentation fault
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hi,
I use debian 11,
python 3.10,
anaconda3
langchain==0.0.240
RAM = 16 Go
swap = 10 Go
SSD = 60 GO
I try to use the simplest program with Ctransfors:

```
from langchain.llms import CTransformers
from langchain.chains import LLMChain
from langchain import PromptTemplate

prompt_template = """
You are an AI coding assistant and your task to solve the coding problems, and return coding snippets based on the
Query: {query}

You just return helpful answer and nothing else
Helpful Answer: 
"""

prompt = PromptTemplate(template=prompt_template, input_variables=['query'])


llm = CTransformers(model = "model/codellama-7b-instruct.ggmlv3.Q4_0.bin",
                    model_type = "llama",
                    max_new_tokens=512,
                    temperature=0.2
                    )

llm_chain = LLMChain(prompt=prompt, llm=llm)


llm_response = llm_chain.run({"query": "Write a python code to load a CSV file using pandas library"})

print(llm_response)
```

and i have "core dumped " error without other details, i tried a lot of things but nothig resolve my problem !

can you help me please
thks

### Suggestion:

_No response_
Issue: Not able to delete embeddings of a file
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: integration: chroma

### Issue you'd like to raise.

def delete_document_embeddings_by_filename(file_name, persist_directory):
    print(file_name,"file_name is ")
    print(persist_directory,"persist_directory is")
    chroma_db = chromadb.PersistentClient(path=persist_directory)
    print(chroma_db,"chroma_db is ---------------->>>>>>>>>>")
    collection = chroma_db.get_collection(name="langchain")
    metadata_list = collection.get()['metadatas']
    print(metadata_list)
    file_names = []
    for metadata in metadata_list:
        filename = metadata['source'].split('\\')[-1]
        if filename not in file_names:
            file_names.append(filename)
    print(file_names,"file_names is kkkkkkkkkkkkkkkkkkkkkkk")
    print(collection,"collection is ---------------->>>>>>>>>>")
    print("hello world ------------------------>>>>>>>>>>>>>>>>>>")
    print(collection.get(where={"source": file_name})['ids'])
    collection.delete(where={"source": file_name})
    I got print(collection.get(where={"source": file_name})['ids']) as empty list and collection object as name='langchain' id=UUID('8a5e8fff-93a4-49f3-8be7-5aac47cb3902') metadata=None
 also I got  print(collection.get(where={"source": file_name})['ids']) as empty list

### Suggestion:

_No response_
fix AgentTokenBufferMemory to properly honour token limits
Labels: name: area: memory, name: auto:enhancement, name: size:L

  - **Description:**
  AgentBufferTokenMemory was not properly reding existing messages from history and going over set token limit. This PR changes its implementation slightly by keeping separate, in-memory, buffer that holds recent messages whose total size does not exceed specified max_token_limit. Chat history object holds and retains all messages.

  - **Issue:** the issue #
  Oveflowing chat memory and therefore token limit budget for llm calls
  - **Dependencies:**
  None
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:**
  NA

Note to self:
If no one reviews your PR within a few days, please @-mention one of @baskaryan, @hwchase17.
Fix agent async timeout handling and Add async tests for agent
Labels: name: area: agent, name: auto:bug, name: size:XXL

It was raising a TimeoutError instead of emitting timeout output

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->