delayedkarma/langchain-issues huggingface dataset:
Evaluation Callback Multi Response
Labels: name: auto:improvement, name: area: models

1. Allow run evaluators to return {"results": [list of evaluation results]} in the evaluator callback.
2. Allows run evaluators to pick the target run ID to provide feedback to

(1) means you could do something like a function call that populates a full rubric in one go (not sure how reliable that is in general though) rather than splitting off into separate LLM calls - cheaper and less code to write
(2) means you can provide feedback to runs on subsequent calls. Immediate use case is  if you wanted to add an evaluator to a chat bot and assign to assign to previous conversation turns


have a corresponding one in the SDK
Issue: StreamingStdOutCallbackHandler and FinalStreamingStdOutCallbackHandler
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

Hi, 
I build a chatbot using conversationchain. I want to final output to be steamed. I tried StreamingStdOutCallbackHandler and FinalStreamingStdOutCallbackHandler. 

The problem with StreamingStdOutCallbackHandler is that it also prints out the memory of the chain.
The problem with FinalStreamingStdOutCallbackHandler is that it did print the final output, but the output is just printed out at once rather than streamed.

How can I fix it?
Thanks,

### Suggestion:

_No response_
Add platechain template
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
SQLDatabaseChain sharing data that's not in database 
Labels: name: area: agent, name: auto:bug, name: area: models

### Issue you'd like to raise.

Okay I have tried both SQL database chain and create SQL agent from agents and what i have seen is that after run and it returns zero records from the database and LLM gives the output from its own. So basically I have only one fruit in my database and if I ask what fruit contains vitamin c which is not in the database and it fetch me the result from its own knowledge base.  I tried adding the instructions in the prompt like if query return zero records or observation is empty or the result of the SQL. Query is empty, Do not share anything on your own And just convey the message to the user that for your certain filter criteria that is no records Try different query or elaborate your request but it doesn't seem to be working. is there anyting we can do to fix it?

### Suggestion:

_No response_
email with metadata from pdfs 
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

parse the text in a pdf to determine whether it contains header fields such as From:, To:, Date: etc which make it likely that the original data was an email. if so, return the contents of those fields as Document metadata which can, for example, be used as metadata in a database. (reopening of https://github.com/langchain-ai/langchain/issues/8094)

### Motivation

Sometimes it as important to know who said what to whom when as it is to determine what actual facts are. Investigative reporting is one example. There are adequate tools to retrieve email and its metadata from Gmail or Exchange. A similar tool for email which has been saved as a collection of PDFs which parses and retrieves the metadata will make these document collections more accessible and useful.

### Your contribution

I have created a pdf to email tool in the proper format to incorporated with existing langchain document loaders. It's at https://github.com/tevslin/emailai, There's been enough interest so that I'm forking, linting, and will submit a pr.
DOC: Broken link in Quickstart page
Labels: name: auto:documentation

### Issue with current documentation:

# Issue
On the documentation page: https://python.langchain.com/docs/get_started/quickstart
In the Next Steps section:
- Explore [end-to-end use cases](https://python.langchain.com/docs/use_cases) results in page not found

## Screenshot

<img width="960" alt="Langchain_docs" src="https://github.com/langchain-ai/langchain/assets/63769209/b1618fad-b7d2-4784-a30f-f6361f81b20a">

<img width="960" alt="bug" src="https://github.com/langchain-ai/langchain/assets/63769209/b637d0fa-aeb4-439b-b598-e1855c35fdaa">


### Idea or request for content:

I think that the above issue can be resolved by replacing the above link with this: https://python.langchain.com/docs/use_cases/qa_structured/sql
Getting error while using Redistore as docstore in parent document retreival
Labels: name: area: memory, name: area: vector store, name: auto:bug, name: integration: redis

### System Info

i am getting an error while using Redistore as docstore `A tuple item must be str, int, float or bytes.`. I do not get error when using the InMemorystore. the problem is with mset function, in redistore mset throws error when i pass a document object, but redistore works if just pass a string.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

i am getting an error while using Redistore as docstore `A tuple item must be str, int, float or bytes.`. I do not get error when using the InMemorystore. the problem is with mset function, in redistore mset throws error when i pass a document object, but redistore works if just pass a string.

### Expected behavior

i am getting an error while using Redistore as docstore `A tuple item must be str, int, float or bytes.`. I do not get error when using the InMemorystore. the problem is with mset function, in redistore mset throws error when i pass a document object, but redistore works if just pass a string.
Fix poetry dependency resolution in devcontainer and simplify setup
Labels: name: auto:improvement, name: size:L

**Description:** Fix poetry dependency resolution within devcontainer and simplify devcontainer configuration along the way:
- Specify devcontainer config in devcontainer.json only (get rid of dev Dockerfile)
- devcontainer uses same poetry version as container running unit tests
- Install poetry via devcontainer feature instead of using multi-stage build
- Move setup tasks (configuring shell completions, installing dependencies) to dedicated file (setup.sh)

**Issue:** 
- #7825 
- #8277 

**Dependencies:** None
**Tag maintainer:** @eyurtsev
langchain Ctransformers caching and usage in Dockerfile volume
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

How can I cache llama models that called in langchain llms CTransformers and add it to a docker image to prevent it download the model every time I called in a cloud environment?
As I researched, I saw no clear explanation of how to cache models in Ctransformers. 

### Suggestion:

_No response_
DOC: Code Writing example throws error with PythonREPL import
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

I am attempting to run: https://python.langchain.com/docs/expression_language/cookbook/code_writing

and I'm seeing this error:


```
from langchain_experimental.utilities import PythonREPL

ImportError: cannot import name 'PythonREPL' from 'langchain_experimental.utilities'
```

### Idea or request for content:

I assume the fix is below

```
$ git diff
diff --git a/docs/docs/expression_language/cookbook/code_writing.ipynb b/docs/docs/expression_language/cookbook/code_writing.ipynb
index 21ab53601..bf1840c5a 100644
--- a/docs/docs/expression_language/cookbook/code_writing.ipynb
+++ b/docs/docs/expression_language/cookbook/code_writing.ipynb
@@ -20,7 +20,7 @@
     "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate\n",
     "from langchain.schema.output_parser import StrOutputParser\n",
-    "from langchain_experimental.utilities import PythonREPL"
+    "from langchain_experimental.utilities.python import PythonREPL"
    ]
   },
   {

```

Will submit a PR with above
Get document ids from ConversationalRetrievalChain's reponse
Labels: name: auto:improvement, name: area: models

### Feature request

Module: langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain


`qa = ConversationalRetrievalChain(
            retriever=self.vector_store.as_retriever(),  # type: ignore
            combine_docs_chain=self.doc_chain,
            question_generator=LLMChain(
                llm=self.answering_llm, prompt=CONDENSE_QUESTION_PROMPT
            ),
            verbose=False,
        )`

`model_response = qa(
                {
                    "question": user_message,
                    "chat_history": formatted_chat_history,
                    "custom_personality": self.prompt_content,
                }
            )`

I would like that the model_response (the result from called the ConversationalRetrievalChain) returns the document ids in which the answer is based

### Motivation

My motivation is to be able to document which documents is the model basing its responses to further analyse and use this input for user experience improvement and transparency.

One possibility would be to use the _get_docs method under ConversationalRetrievalChain, but this would imply calling twice the model

### Your contribution

As of now the model_response returns:

`{'question': 'Who won the Olympics in 92?', 'chat_history': [()], 'custom_personality': 'xxxxxx
', 'answer': 'The US won the Olympics'}`

My proposal would be:

`{'question': 'Who won the Olympics in 92?', 'chat_history': [()], 'custom_personality': 'xxxxxx
', 'answer': 'The US won the Olympics', 'documents_source': [Document(), Document()...]}`
sagemaker streaming fix for _call and _acall
Labels: name: auto:bug, name: area: models, name: integration: aws

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** sagemaker streaming fix for _call and _acall, 
  - **Issue:** the issue # it fixes sagemaker streaming fix for _call and _acall,
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Bedrock async methods
Labels: name: auto:improvement, name: area: models, name: size:L

Description: Added support for asynchronous streaming in the Bedrock class and corresponding tests.

Primarily:
  async def aprepare_output_stream
    async def _aprepare_input_and_invoke_stream
    async def _astream
    async def _acall

I've ensured that the code adheres to the project's linting and formatting standards by running make format, make lint, and make test.

Issue: #12054, #11589

Dependencies: None

Tag maintainer: @baskaryan 

Twitter handle: @dominic_lovric
SecretStr for fireworks api
Labels: name: lgtm, name: auto:improvement, name: area: models

  - **Description:** This pull request removes secrets present in raw format, 
  - **Issue:**  Fireworks api key was exposed when printing out the langchain object [#12165](https://github.com/langchain-ai/langchain/issues/12165)
 - **Maintainer:** @eyurtsev
Openai key leakages
Labels: name: auto:bug, name: area: models

**Description:**

By default - `ChatOpenAI` implementation and `OpenAIEmbeddings` implementations do not hide `openai_api_key` from `repr` implementation.

Which may lead to key leakages. Like I were working on some project with Jupyter Notebook, I am used to show intermediate stages - and I am far from being cautious. So I made something like:

```python
key = _read_key_from_some_gitignored_env_file()
input = ChatPromptTemplate.from_messages([SystemMessagePromptTemplate.from_template("Foobar")])
chat = ChatOpenAI(openai_api_key=key)
chain = input | chat
chain
```
or, for instance (you can run this code in python interactive session):
```python
from langchain.chat_models import ChatOpenAI
ChatOpenAI(openai_api_key="sk-TEST")
```
you can see this output
```
ChatOpenAI(client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, openai_api_key='sk-TEST', openai_api_base='', openai_organization='', openai_proxy='')
```

And since I were used to show intermediate stages I have shown what chain is without noticing it contains key.

Fortunately, OpenAI monitor key leakages on their side, but they can only do it on public resources, like public github repositorys.

This PR:
- add tests regards this matter
- fix `OpenAIEmbeddings::__repr_args__` and `ChatOpenAI::__repr_args__` in such a way so `repr(someobject)` do not contains `openai_api_key` anymore:

```python
from langchain.chat_models import ChatOpenAI
ChatOpenAI(openai_api_key="sk-TEST")
```
```
ChatOpenAI(client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, openai_api_base='', openai_organization='', openai_proxy='')
```
No `openai_api_key="sk-TEST"` anymore

- **Dependencies:** 

No additional dependencies
Update tool.py
Labels: name: lgtm, name: auto:bug

In the GoogleSerperResults class, the name field is defined as 'google_serrper_results_json'. This looks like a typo, and perhaps should be 'google_serper_results_json'.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: <It seems that my agent has no memory ability>
Labels: name: area: memory, name: area: agent, name: auto:bug

### Issue you'd like to raise.

During using the agent, I added memory to the agent, but the result seems to be no memory ability，my code is as follows:

`llm=ChatOpenAI(temperature=0, model="gpt-3.5-turbo-16k-0613")

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

agent = initialize_agent(
tools=tools_name,
llm=llm,
memory=memory,
verbose=True,
agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
)`

The results indicate that the agent has no memory, but memory does record the conversation

![image](https://github.com/langchain-ai/langchain/assets/149214596/b1132bdb-d885-494c-a057-a9b2c01b7a58)


### Suggestion:

_No response_
DOC: <It seems that my agent has no memory ability>

### Issue with current documentation:

During using the agent, I added memory to the agent, but the result seems to be no memory ability，my code is as follows:

`llm=ChatOpenAI(temperature=0, model="gpt-3.5-turbo-16k-0613")

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

agent = initialize_agent(
    tools=tools_name,
    llm=llm,
    memory=memory,
    verbose=True,
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
)`

The results indicate that the agent has no memory, but memory does record the conversation

![image](https://github.com/langchain-ai/langchain/assets/149214596/b98bf876-7a18-4dff-aedb-5e58245ee059)



### Idea or request for content:

agent with memory
The tool is not working as expected, the observation is correct but it does not return the information the way
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I want it to use the observation in the way I want it to use it, however it makes a summary of the observation when it is not the expected result even indicating it in the docstrings, prompt templates....


Agent: 
agent_chain = initialize_agent(
            agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,
            tools=tools,
            llm=llm,
            verbose=True,
            max_iterations=3,
            memory=memory
        ) 


Tool import:
        Tool(
            name='hackernews',
            func= get_news.run,
            description="""
 Returns: 3 newsURL For recently news about cibersecurity if topic is not a term of the follow list, put the most accurate term along this list like a topic Args: topic -> the only topics valids are general, dataBreach, cyberAttack, vulnerability, malware, security, cloud, tech, iot, bigData, business, mobility, research, corporate, socialMedia. Useful for find rencently news about computer science, cybersecurity... Returns: Links to the related news and a description of them"""
        ),
"""


Code of tool:
"""
from cybernews.cybernews import CyberNews
from langchain.tools import tool
news = CyberNews()
import subprocess
import requests
import dns.resolver
from pydantic import BaseModel, Extra

class get_news(BaseModel):
    """Herramientas de ciberseguridad
    Returns: news, you shoul output newsURL and the description of that url   """

    class Config:
        """Configuration for this pydantic object."""
        extra = Extra.forbid

    def run(self, topic: str) -> str:
        """
    Busca noticias de ciberseguridad

    Args:
    topic (str): noticia a encontrar, tema a encontrar

    Returns:
    news.get_news(topic) (list): a list who contain description and links, you should provide a few links about the lastest news
    """
        if topic == "":
            topic = "cybersecurity"
            return(news.get_news(topic))
        else:
            return(news.get_news(topic))
""""
Output:
![Screenshot 2023-10-28 at 01 57 46](https://github.com/langchain-ai/langchain/assets/60628803/297c819e-b4d2-4301-8da4-fb45c00e7ed6)

this just provide a short description but no links, how I can do it, all my tools have the same problem... don't output like is defined in prompts....




### Suggestion:

_No response_
cli improvements
Labels: name: auto:improvement

Features
- add multiple repos by their branch/repo
- generate `pip install` commands and `add_route()` code
![Screenshot 2023-10-27 at 4 49 52 PM](https://github.com/langchain-ai/langchain/assets/9557659/3aec4cbb-3f67-4f04-8370-5b54ea983b2a)

Optimizations:
- group installs by repo/branch to avoid duplicate cloning
ValueError: Could not parse output - map_rerank
Labels: name: area: vector store, name: auto:bug, name: integration: chroma

### System Info

Langchain version: 0.0.325
Python version: Python 3.11.6

### Who can help?

@hwchase17

`chain_type="map_rerank"` is not working when the search cannot be found on the DB

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


Code:


```
from dotenv import load_dotenv
from langchain.vectorstores.chroma import Chroma
from langchain.chains.retrieval_qa.base import RetrievalQA
from langchain.chat_models.openai import ChatOpenAI
from langchain.embeddings.openai import OpenAIEmbeddings
import langchain


load_dotenv()
chat = ChatOpenAI()
embeddings = OpenAIEmbeddings()

db = Chroma(
    persist_directory="emb",
    embedding_function=embeddings,   
)

retriever = db.as_retriever()
chain = RetrievalQA.from_chain_type(
    llm=chat,
    retriever=retriever,
    chain_type="map_rerank",
    verbose=True
)
result = chain.run("Who was Michael Jackson?") # <-- not in the database
print(result)
```

Prints:

```
raise ValueError(f"Could not parse output: {text}")
ValueError: Could not parse output: I don't know.
```

### Expected behavior

Print: "I don't know"
Allow per page processing of documents in stuff combiner and more effecient prompt
Labels: name: auto:improvement, name: size:M

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->