delayedkarma/langchain-issues huggingface dataset:
Inconsistent Embedding Field Behavior Between .get() and .peek() Methods in Vector Database within Langchain
Labels: name: area: vector store, name: auto:bug

### System Info

I use Python 3.10.12 and langchain 0.0.262.

Using a chroma vector database within langchain, I encounter different behaviors between the .get() and .peek() methods. Specifically, when I use .get(), the embeddings field appears as None. However, when I use .peek(), the embeddings field is complete.

Here are the codes I have and their results:

Using db._collection.peek(1):

```
{'ids': ['/path/to/document.pdf_0'],
 'embeddings': [[-0.013034389354288578,
   0.004974348470568657, ...]],
 'metadatas': [{'index': 0,
   'page': 0,
   'source': '/path/to/document.pdf'}],
 'documents': ['Document content here...']}
```

Using db._collection.get(ids='/path/to/document.pdf_0'):

```
{'ids': ['/path/to/document.pdf_0'],
 'embeddings': None,
 'metadatas': [{'index': 0,
   'page': 0,
   'source': '/path/to/document.pdf'}],
 'documents': ['Document content here...']}
```
The issue seems to be that the embeddings are missing when using the .get() method, but they are present when using the .peek() method. Could you please help me understand why this inconsistency occurs?

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create a Chroma database with custom metadata and OpenAI embeddings. 
2. Filter on just the first source with .get() so you can see the first k documents
3. Peek the same k documents with .peek()

### Expected behavior

The .get() method should return the embeddings too.
StructuredTool raises an error when instantiated
Labels: name: auto:bug, name: area: models, name: pydantic

### System Info

### Environment:
Google COLAB project. When running `!cat /etc/os-release` it prints:
```
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
```

### Python Environment
`!python -V`:
```
Python 3.10.12
```

`!pip show langchain`:
```
Name: langchain
Version: 0.0.267
Summary: Building applications with LLMs through composability
Home-page: https://www.github.com/hwchase17/langchain
Author: 
Author-email: 
License: MIT
Location: /usr/local/lib/python3.10/dist-packages
Requires: aiohttp, async-timeout, dataclasses-json, langsmith, numexpr, numpy, openapi-schema-pydantic, pydantic, PyYAML, requests, SQLAlchemy, tenacity
Required-by: 
```

### Who can help?

@hwchase17 

### Information

- [x] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Create a blank Google Colab Project (https://colab.research.google.com/#create=true) then add a code block and paste in this code:
```py
!pip install langchain==0.0.267
# or try just '!pip install langchain' without the explicit version

from pydantic import BaseModel, Field

class InputArgsSchema(BaseModel):
  strarg: str = Field(description="The string argument for this tool")

# THIS WORKS:

from typing import Type

class Foo(BaseModel):
  my_base_model_subclass: Type[BaseModel] = Field(..., description="Equivalent to the args_schema field in langchain/StructuredTool")

my_foo = Foo(
    my_base_model_subclass=InputArgsSchema
)

print(f"My foo {my_foo} is successfully instantiated")

# BUT THIS DOES NOT:

from langchain.tools import StructuredTool

def my_tool_impl(strarg: str):
  print(f"Called myTool with strarg={strarg}")

my_tool = StructuredTool(
    name="my_tool",
    description="A demo tool for testing purposes",
    args_schema=InputArgsSchema,
    func=my_tool_impl
)
```

Now run the code block.

### Expected behavior

The `StructuredTool` instance should be instantiated without an exception and the `InputArgsSchema` should be accepted as argument for `args_schema`.
TransformChain wrong function types, enforced since 0.0.267
Labels: name: auto:bug, name: area: models, name: pydantic

### System Info

LangChain 0.0.267, Python 3.10, Poetry virtualenv, Pop_OS 22.04

### Who can help?

@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.chains import TransformChain

def transform(inputs):
    return {
        'output': {
            'structured': {
                'report': 'Done'
            }
        }
    }
async def atransform(inputs): return transform(inputs)
chain = TransformChain(
    input_variables=['text'],
    output_variables=['output'],
    transform=transform,  # <-- type error below happens here
    atransform=atransform,  # <-- If I remove this, I get: Argument missing for parameter "atransform"
)
```

```
Argument of type "(inputs: Unknown) -> dict[str, dict[str, dict[str, str]]]" cannot be assigned to parameter "transform" of type "(Dict[str, str]) -> Dict[str, str]" in function "__init__"
  Type "(inputs: Unknown) -> dict[str, dict[str, dict[str, str]]]" cannot be assigned to type "(Dict[str, str]) -> Dict[str, str]"
    Function return type "dict[str, dict[str, dict[str, str]]]" is incompatible with type "Dict[str, str]"
      "dict[str, dict[str, dict[str, str]]]" is incompatible with "Dict[str, str]"
        Type parameter "_VT@dict" is invariant, but "dict[str, dict[str, str]]" is not the same as "str"
```

### Expected behavior

Two type checking issues since 0.0.267:
- `transform` should probably take/return `dict[str, Any]` instead of `dict[str, str]` ([like `atransform` does](https://github.com/langchain-ai/langchain/blob/0689628489967785f3a11a9f29d8f6f90930f4f4/libs/langchain/langchain/chains/transform.py#L31C1-L35C40))
- `atransfrom` should probably not be mandatory

Most other code seems to use `Dict[str, Any]`, and non-string values seem to work just fine.

Prior to `0.0.267` the type system didn't enforce the type of `transform` when constructing `TransformChain`. It also started requiring an `atransform` parameter even though it seems to be intended to be optional.
Add kwargs to all other optional runnable methods
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Langsmith expose is not working Azure OpenAI services
Labels: name: auto:bug, name: auto:question, name: stale, name: area: models

### Issue you'd like to raise.

Hi everyone, I'm trying to deploy and use langsmith locally. 
I deployed in a docker container using
```
langsmith start --expose --openai-api-key=<my azure OpenAi key>
```
the docker container looks good
![image](https://github.com/langchain-ai/langchain/assets/12852824/28aae23a-0272-4f0f-b298-26e3e2223caa)
I opened all the used ports to avoid any problem there, I'm running langsmith in a remote computer

I set up the environment variables
LANGCHAIN_TRACING_V2=true
LANGCHAIN_ENDPOINT=https://cc23-20-79-217-xxx.ngrok.io
LANGCHAIN_API_KEY=<my key>

but the interface is not loading the projects
![image](https://github.com/langchain-ai/langchain/assets/12852824/5ced7d99-bac3-4b1f-84f3-c606b98f5ee8)

when I try to access the langsmith endpoint it returns
```
{
"detail": "Not Found"
}
```

using the chat example that appears in this repo
https://github.com/langchain-ai/langsmith-cookbook/tree/main/feedback-examples/streamlit

I can see in the endpoint https://cc23-20-79-217-xxx.ngrok.io that the runs are being tracked, but I can't see them in the frontend 

**debugging the front end it is failing trying to fetch the tenants, it's trying to fetch them from http://127.0.0.1:1984/tenants while if I'm not understanding it wrong it should get them from http://20.79.217.xxx:1984/tenants**
![image](https://github.com/langchain-ai/langchain/assets/12852824/7bfb3c92-2db8-4ca9-8ecb-9f2d36500ffe)


could it be a problem with the Azure OpenAI? or did I do something wrong with the installation?

Thanks in advance

### Suggestion:

_No response_
ConvoOutputParser randomly fails to parse LLM output
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

langchain version: 0.0.267

### Who can help?

@hwaking @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction
-------------------------- This is the begin of python script ---------------------------------------
# import package
from langchain import LLMMathChain
from langchain.vectorstores.redis import Redis
from langchain.utilities import BingSearchAPIWrapper
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.chat_models import AzureChatOpenAI
import dotenv

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(memory_key="chat_history")
dotenv.load_dotenv()


# init agent
llm = AzureChatOpenAI(deployment_name="OpenAImodel")
search = BingSearchAPIWrapper(k=10)
llm_math_chain = LLMMathChain.from_llm(llm=llm)
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events. You should ask targeted questions."
    ),
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for when you need to answer questions about math"
    ),
]
agent = initialize_agent(tools, llm,agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)

# ask some question, at random point you will get error: `Could not parse LLM output`
agent.run(input="幫我查詢現在台北的溫度，用攝氏單位")
agent.run("用中文告訴我: 研華科技冰水主機的哪些組件或相關設備可能造成趨近溫度過高，最後用中文告訴我")
agent.run("用中文回答: 幫我查詢幫浦馬達故障通常是因為什麼原因?")
agent.run("冰水主機的哪些組件或相關設備可能造成趨近溫度過高")
agent.run("用中文回答剛剛的問題")

-------------------------- This is the end of python script ---------------------------------------

### Expected behavior

# at random point you will get error: `Could not parse LLM output`

File ~/miniconda3/envs/langc/lib/python3.8/site-packages/langchain/agents/conversational/output_parser.py:26, in ConvoOutputParser.parse(self, text)
     24 match = re.search(regex, text)
     25 if not match:
---> 26     raise OutputParserException(f"Could not parse LLM output: `{text}`")
     27 action = match.group(1)
     28 action_input = match.group(2)

OutputParserException: Could not parse LLM output: `在研華科技的冰水主機中，可能造成趨近溫度過高的組件或相關設備包括：風扇故障、散熱片堵塞、水泵故障、循環水管堵塞、或是防凍液不足等。建議您確保這些組件和設備處於良好狀態，以確保冰水主機的正常運作。`

# my observation

i found that the error always comes when the function `parse` of ConvoOutputParser is called
the text might not contain some keywords such as `Action:` or whatever
so the regular expression will not match the LLM output

good luck!!
Running Langsmith locally not working
Labels: name: auto:bug

### Issue you'd like to raise.

Hello, I'm trying to run langsmith locally, together with Azure OpenAi services. 
I started langsmith in docker and it looks like below
![image](https://github.com/langchain-ai/langchain/assets/12852824/cf63b177-a2b3-4a63-b821-e1b7d51b4885)

And I'm running the chatbot from here https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/streamlit/README.md

Looks like the app is not allowed to send the interaction to langsmith.

```
  File "/home/azureuser/anaconda3/envs/snowflake/lib/python3.8/site-packages/langsmith/utils.py", line 55, in raise_for_status_with_text
    response.raise_for_status()
  File "/home/azureuser/anaconda3/envs/snowflake/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 405 Client Error: Not Allowed for url: http://20.79.217.xxx/runs/3abf1f4c-2b48-4d1b-9628-979f7d12d9d5/share

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/azureuser/anaconda3/envs/snowflake/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)
  File "/home/azureuser/langsmith/main.py", line 126, in <module>
    url = client.share_run(run.id)
  File "/home/azureuser/anaconda3/envs/snowflake/lib/python3.8/site-packages/langsmith/client.py", line 831, in share_run
    raise_for_status_with_text(response)
  File "/home/azureuser/anaconda3/envs/snowflake/lib/python3.8/site-packages/langsmith/utils.py", line 57, in raise_for_status_with_text
    raise ValueError(response.text) from e
ValueError: <html>
<head><title>405 Not Allowed</title></head>
<body>
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx/1.24.0</center>
</body>
</html>
```

Also something that looks a bit odd is when I open the website, it's not showing me the projects
![image](https://github.com/langchain-ai/langchain/assets/12852824/8a17d798-d3ca-40f0-9cc1-352e4ac5ede5)
and i can't create a new one, keeps telling me that the field is required

![image](https://github.com/langchain-ai/langchain/assets/12852824/ed8df4de-c7a5-424d-9fc0-c3c0888da82c)


what could I do, any hint is more than appreaciated

### Suggestion:

_No response_
While using a GPT-4 deployed on AzureOpenAI, model name is showing as 'gpt-3.5-turbo', while the engine is showing as 'gpt-4'
Labels: name: auto:bug, name: area: models

### System Info

LangChain Version: 0.0.267

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`llm = AzureChatOpenAI(deployment_name='gpt-4', temperature=0.0)`

`llm.dict()`

`{'model': 'gpt-3.5-turbo',
 'request_timeout': None,
 'max_tokens': None,
 'stream': False,
 'n': 1,
 'temperature': 0.0,
 'engine': 'gpt-4',
 '_type': 'azure-openai-chat'}`

### Expected behavior

The model name and the engine should match.
fix: Imports for the ConfluenceLoader:process_page
Labels: name: lgtm, name: area: doc loader, name: auto:bug

### Description
When we're loading documents using `ConfluenceLoader`:`load` function and, if both `include_comments=True` and `keep_markdown_format=True`, we're getting an error saying `NameError: free variable 'BeautifulSoup' referenced before assignment in enclosing scope`.
    
    loader = ConfluenceLoader(url="URI", token="TOKEN")
    documents = loader.load(
        space_key="SPACE", 
        include_comments=True, 
        keep_markdown_format=True, 
    )

This happens because previous imports only consider the `keep_markdown_format` parameter, however to include the comments, it's using `BeautifulSoup`

Now it's fixed to handle all four scenarios considering both `include_comments` and `keep_markdown_format`.

### Twitter
`@SathinduGA`
Issue: How to replace "page_content" after "retrieval"
Labels: name: area: doc loader, name: auto:question

### Issue you'd like to raise.

I have a JSONL file that has the format:
{
"question": ...
"answer": ...
}

I only want to use "question" for "page_content" to retrieve, because if I merge "question" and "answer" into "page_content", it will be noise to retrieve.
I have an idea that after retrieving:
- I will replace "page_content" with "answer" to give it to LLM.
- Or customize function as_retrieval() to return "answer " in metadata.
- Or the new format data?
So how to make it? Thank you

### Suggestion:

_No response_
Update index.md
Labels: name: auto:documentation

Hi @hwchase17 @baskaryan @eyurtsev have been using langchain for some time and I think it is one of the best libraries to orchestrate GenAI workflows. Kudos to the awesome work.
Was trying my hand at LangSmith as well, and while navigating found this 404 error. Am replacing the link with the working link.

Interactive walkthrough link gave a 404 error. Fixed the link to point to the right file.
core[patch]: add response kwarg to `on_llm_error`
Labels: name: lgtm, name: auto:bug, name: area: models, name: size:L

 # Description
 This PR should allow for `on_llm_end` callback execution, during LLM streaming even if the stream is not fully exhausted.
 Currently this doesn't happen, but I believe this case should definitely be supported as not fully exhausting the stream can save valuable tokens. 
 
# Dependencies
None

# Twitter handle
@HKydlicek
Input variables in PipelinePromptTemplate's final_prompt are not extracted as input variables
Labels: name: auto:bug, name: area: models

### System Info

Version 0.0.266, all

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

If you specify input variables in your `final_prompt` they are not actually registered as input variables on the prompt, whereas input variables on the pipeline prompts are.

```python
from langchain.prompts import PipelinePromptTemplate

pipeline_prompt = PipelinePromptTemplate(
    final_prompt=PromptTemplate.from_template("{foo} is a var"),
    pipeline_prompts=[]
)
pipeline_prompt.input_variables  # returns []
```

vs

```python
pipeline_prompt = PipelinePromptTemplate(
    final_prompt=PromptTemplate.from_template("final prompt + {other}"),
    pipeline_prompts=[
        ("other", PromptTemplate.from_template("{foo}"))
    ]
)
pipeline_prompt.input_variables  # returns ["foo"]
```

### Expected behavior

Looks like it's just not in the code here: https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/prompts/pipeline.py#L35

Assuming that input variables should also be extracted from the final prompt, but _after_ the pipeline prompts are injected, right?
Issue: How can I load text embeddings from a local model?
Labels: name: area: embeddings, name: auto:question

### Issue you'd like to raise.

I do not have access to huggingface.co in my environment, but I do have the Instructor model (hkunlp/instructor-large) saved locally. How do I utilize the langchain function HuggingFaceInstructEmbeddings to point to a local model?

I tried the below code but received an error:

```
from langchain.embeddings import HuggingFaceInstructEmbeddings

model_name = "<local_filepath>/hkunlp/instructor-large"
model_kwargs = {'device': 'cpu'}
encode_kwargs = {'normalize_embeddings': True}
hf = HuggingFaceInstructEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)

embeddings = HuggingFaceInstructEmbeddings(
    query_instruction="Represent the query for retrieval: "
)


```
Error:
HfHubHTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/models/hkunlp/instructor-large



### Suggestion:

_No response_
Fix import schema for AutoGPT implementation
Labels: name: area: agent, name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
AmazonTextractPDFLoader documentation updates
Labels: name: area: doc loader, name: auto:documentation

Description: Updating documentation to add AmazonTextractPDFLoader according to [comment](https://github.com/langchain-ai/langchain/pull/8661#issuecomment-1666572992) from [baskaryan](https://github.com/baskaryan) 

Adding one notebook and instructions to the modules/data_connection/document_loaders/pdf.mdx