delayedkarma/langchain-issues huggingface dataset:
Connection error caused failure to post http://localhost:1984/runs  in LangSmith API.
Labels: name: auto:bug, name: area: langserve

### System Info

From pyproject.toml:

python=3.11.5
crewai = "0.1.6"
langchain = '==0.0.335'
openai = '==0.28.1'
unstructured = '==0.10.25'
pyowm = '3.3.0'
tools = "^0.1.9"
wikipedia = "1.4.0"
yfinance = "0.2.33"
sec-api = "1.0.17"
tiktoken = "0.5.2"
faiss-cpu = "1.7.4"
python-dotenv = "1.0.0"

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Running any of the scripts in the crewAI (https://github.com/joaomdmoura/crewAI)

Running crewAI I get the following.


Connection error caused failure to patch http://localhost:1984/runs/7fdd9cf2-4f50-4ee1-8fef-9202b07cc756  in LangSmith API. Please confirm your LANGCHAIN_ENDPOINT. ConnectionError(MaxRetryError("HTTPConnectionPool(host='localhost', port=1984): Max retries exceeded with url: /runs/7fdd9cf2-4f50-4ee1-8fef-9202b07cc756 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x130d51e10>: Failed to establish a new connection: [Errno 61] Connection refused'))"))


Connection error caused failure to post http://localhost:1984/runs  in LangSmith API. Please confirm your LANGCHAIN_ENDPOINT. ConnectionError(MaxRetryError("HTTPConnectionPool(host='localhost', port=1984): Max retries exceeded with url: /runs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x130d6b390>: Failed to establish a new connection: [Errno 61] Connection refused'))"))

I ma not running LangSmith nor do I have any access to it.  I have tried setting  in my .env to no effect. 

LANGCHAIN_TRACING=false
LANGCHAIN_TRACING_V2=false
LANGCHAIN_HANDLER=None

### Expected behavior

Dont expect to see the error reports. Note that not all users are seeing this error
community: Added functions to make async calls to HuggingFaceHub's embedding endpoint in HuggingFaceHubEmbeddings class
Labels: name: lgtm, name: area: embeddings, name: auto:improvement, name: size:L

**Description:**
Added aembed_documents() and aembed_query() async functions in HuggingFaceHubEmbeddings class in
langchain_community\embeddings\huggingface_hub.py file. It will support to make async calls to HuggingFaceHub's
embedding endpoint and generate embeddings asynchronously.

Test Cases: Added test_huggingfacehub_embedding_async_documents() and test_huggingfacehub_embedding_async_query()
functions in test_huggingface_hub.py file to test the two async functions created in HuggingFaceHubEmbeddings class.

Documentation: Updated huggingfacehub.ipynb with steps to install huggingface_hub package and use
HuggingFaceHubEmbeddings.

**Dependencies:** None,
**Twitter handle:** I do not have a Twitter account
community: Added functions to make async calls to HuggingFaceHub's embedding endpoint in HuggingFaceHubEmbeddings class
Labels: name: area: embeddings, name: auto:improvement, name: size:L

  - **Description:** 
   Added aembed_documents() and  aembed_query() async functions in HuggingFaceHubEmbeddings class in 
   langchain_community\embeddings\huggingface_hub.py file. It will support to make async calls to HuggingFaceHub's 
   embedding endpoint and generate embeddings asynchronously.

   Test Cases: Added test_huggingfacehub_embedding_async_documents() and test_huggingfacehub_embedding_async_query() 
   functions in test_huggingface_hub.py file to test the async functions created in HuggingFaceHubEmbeddings class.
 
  Documentation: Updated huggingfacehub.ipynb with steps to install _huggingface_hub_ package and use 
  HuggingFaceHubEmbeddings.

  - **Dependencies:** None,
  - **Twitter handle:** I do not have a Twitter account
How can I specify my own designated table for vector search to retrieve vector data for comparison and provide a response to OpenAI for reference? Also, I noticed the prompt disappeared
Labels: name: area: vector store, name: auto:question

### Issue you'd like to raise.

import os
from urllib.parse import quote_plus
from langchain.vectorstores.pgvector import PGVector
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationTokenBufferMemory
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, MessagesPlaceholder,
HumanMessagePromptTemplate

os.environ['OPENAI_API_KEY'] = "key"
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
CONNECTION_STRING = PGVector.connection_string_from_db_params(
driver=os.environ.get("PGVECTOR_DRIVER", "psycopg2"),
host="1x2.1x8.xx.xx",
port=5432,
database="Ai",
user="xxxxxxxxx",
password=quote_plus("xxxxxx@xx"),
)

vectordb = PGVector(embedding_function=embeddings,
collection_name="tmp04",
connection_string=CONNECTION_STRING,
)

prompt = ChatPromptTemplate(
messages=[
SystemMessagePromptTemplate.from_template(
"i am robot"
),
MessagesPlaceholder(variable_name="chat_history"),
HumanMessagePromptTemplate.from_template("{question}"),
]
)

retriever = vectordb.as_retriever()
llm_name = "gpt-3.5-turbo"
llm = ChatOpenAI(model_name=llm_name, temperature=0)
memory_token_limit = 1000

memory = ConversationTokenBufferMemory(
llm=llm,
prompt=prompt,
max_token_limit=int(memory_token_limit),
memory_key="chat_history",
return_messages=True,
)

qa = ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=retriever,
memory=memory,
verbose=True,
)

chat_history = []

while True:
memory.load_memory_variables({})
question = input('ask：')
# question = retriever.get_relevant_documents(input('ask：'))
result = qa.run({'question': question, 'chat_history': chat_history})
print(result)
chat_history.append([f'User: {question}', f'Ai: {result}'])
print(chat_history)
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
st_history = ' '.join(map(str, chat_history))
res = embeddings.embed_query(st_history)
vectordb.add_vector(res)
select_vdb = vectordb.similarity_search(question, k=5)
print(select_vdb)
print(f'ok: {res[:4]}...')
if question.lower() == 'bye':
break
![Uploading image.png…]()


### Suggestion:

How can I specify my own designated table for vector search to retrieve vector data for comparison and provide a response to OpenAI for reference? Also, I noticed the prompt disappeared
How can I specify my own designated table for vector search to retrieve vector data for comparison and provide a response to OpenAI for reference? Also, I noticed the prompt disappeared.
Labels: name: area: vector store, name: auto:documentation

### Issue with current documentation:

import os
from urllib.parse import quote_plus
from langchain.vectorstores.pgvector import PGVector
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationTokenBufferMemory
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, MessagesPlaceholder, 
    HumanMessagePromptTemplate

os.environ['OPENAI_API_KEY'] = "key"
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
CONNECTION_STRING = PGVector.connection_string_from_db_params(
    driver=os.environ.get("PGVECTOR_DRIVER", "psycopg2"),
    host="1x2.1x8.xx.xx",
    port=5432,
    database="Ai",
    user="xxxxxxxxx",
    password=quote_plus("xxxxxx@xx"),
)

vectordb = PGVector(embedding_function=embeddings,
                    collection_name="tmp04",
                    connection_string=CONNECTION_STRING,
                    )
prompt = ChatPromptTemplate(
    messages=[
        SystemMessagePromptTemplate.from_template(
            "i am robot"
        ),
        MessagesPlaceholder(variable_name="chat_history"),
        HumanMessagePromptTemplate.from_template("{question}"),
    ]
)
retriever = vectordb.as_retriever()
llm_name = "gpt-3.5-turbo"
llm = ChatOpenAI(model_name=llm_name, temperature=0)
memory_token_limit = 1000

memory = ConversationTokenBufferMemory(
    llm=llm,
    prompt=prompt,
    max_token_limit=int(memory_token_limit),
    memory_key="chat_history",
    return_messages=True,
)

qa = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    memory=memory,
    verbose=True,
)
chat_history = []
while True:
    memory.load_memory_variables({})
    question = input('ask：')
    # question = retriever.get_relevant_documents(input('提問：'))
    result = qa.run({'question': question, 'chat_history': chat_history})
    print(result)
    chat_history.append([f'User: {question}', f'Ai: {result}'])
    print(chat_history)
    embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
    st_history = ' '.join(map(str, chat_history))
    res = embeddings.embed_query(st_history)
    vectordb.add_vector(res)
    select_vdb = vectordb.similarity_search(question, k=5)
    print(select_vdb)
    print(f'ok: {res[:4]}...')
    if question.lower() == 'bye':
        break
![image](https://github.com/langchain-ai/langchain/assets/103471919/e8503d33-c8b8-4bb9-8c36-bba00b26e8d3)


### Idea or request for content:

How can I specify my own designated table for vector search to retrieve vector data for comparison and provide a response to OpenAI for reference? Also, I noticed the prompt disappeared.
DOC: AzureChatOpenAI in documentation
Labels: name: auto:documentation, name: area: models

### Issue with current documentation:

 I just installed langchain 0.1.0 and according to the documentation
https://api.python.langchain.com/en/latest/_modules/langchain_openai/chat_models/azure.html#

 AzureChatOpenAI should be in langchain_openai.chat_models but its instead in langchain_community.chat_models

### Idea or request for content:

_No response_
Issue: mechanism of embedding parameters in Neo4j Vector object
Labels: name: area: embeddings, name: auto:question

### Issue you'd like to raise.

I've seen in the langchain documentation code for vector search in Neo4j which take `OpenAIEmbeddings()` as an object parameter in order to make an embedding for input query

```python
index_name = "vector"  # default index name

store = Neo4jVector.from_existing_index(
    OpenAIEmbeddings(),
    url=url,
    username=username,
    password=password,
    index_name=index_name,
)
```
What i wonder is that, can we pass another embedding model e.g. huggingface model into that parameter instead of openai itself because it many cases, there exist an incompatible dimension when we already have an existing index that is embedded by another off-the-shelf model rather than a embedding model from OpenAI ?


Moreover,  i took a look at the source code in case that there has no way to add huggingface model

https://github.com/langchain-ai/langchain/blob/04caf07dee2e2843ab720e5b8f0c0e83d0b86a3e/libs/community/langchain_community/vectorstores/neo4j_vector.py#L111-L147

what i've found is that for the `embedding` parameters of `Neo4jVector` object, it should be Any embedding function implementing`langchain.embeddings.base.Embeddings` interface. Here is the code described that class

https://github.com/langchain-ai/langchain/blob/04caf07dee2e2843ab720e5b8f0c0e83d0b86a3e/libs/core/langchain_core/embeddings.py#L7-L24

Does it mean that we must construct a class that inherits from it in order to implement it effectively, if yes, please provide an example to do it.

### Suggestion:

Any suggested way from this case, if it is currently not supported huggingface model with Neo4j VectorStore, i will help contributing it and make a PR then.
Unable to retrieve my prompt when starting the conversation
Labels: name: area: memory, name: auto:bug

### Issue you'd like to raise.

import os
from urllib.parse import quote_plus
from langchain.vectorstores.pgvector import PGVector
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationTokenBufferMemory
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, MessagesPlaceholder, 
    HumanMessagePromptTemplate
os.environ['OPENAI_API_KEY'] = "mykey"
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
CONNECTION_STRING = PGVector.connection_string_from_db_params(
    driver=os.environ.get("PGVECTOR_DRIVER", "psycopg2"),
    host="192.xxx.xx.xxx",
    port=5432,
    database="xxx",
    user="xxx",
    password=quote_plus("xx@xxxxxr"),
)
vectordb = PGVector(embedding_function=embeddings,
                    collection_name="tmp06",
                    connection_string=CONNECTION_STRING,
                    )
prompt = ChatPromptTemplate(
    messages=[
        SystemMessagePromptTemplate.from_template(
            "請把使用者的對話紀錄當作參考作為回覆，回答只能使用繁體中文字"
        ),
        MessagesPlaceholder(variable_name="chat_history"),
        HumanMessagePromptTemplate.from_template("{question}"),
    ]
)
llm_name = "gpt-3.5-turbo"
llm = ChatOpenAI(model_name=llm_name, temperature=0)
retriever = vectordb.as_retriever()
memory = ConversationTokenBufferMemory(
    llm=llm,
    prompt=prompt,
    memory_key="chat_history",
    return_messages=True,
)

qa = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    memory=memory,
    verbose=True,
)
chat_history = []
while True:
    memory.load_memory_variables({})
    question = input('提問：')
    result = qa.run({'question': question, 'chat_history': chat_history})
    print(result)
    chat_history.append([f'User: {question}', f'Ai: {result}'])
    print(chat_history)
    embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
    st_history = ' '.join(map(str, chat_history))
    res = embeddings.embed_query(st_history)
    vectordb.add_vector(res)
    select_vdb = vectordb.nearest(res, n=1)
    print(f'ok: {res[:4]}...')
    if question.lower() == 'bye':
        break


### Suggestion:

translates to "Unable to retrieve my prompt when starting the conversation" in English
community: prometheus metrics callback handler
Labels: name: auto:improvement, name: area: models, name: size:L

Description:

Create a new MetricsCallbackHandler to collect token usage and latency metrics for openai. Prometheus metrics are crucial for modern monitoring and enterprise applications because they provide a powerful and flexible way to collect and aggregate real-time metrics about system performance and health. This enables organizations to proactively identify and address issues, ensuring high availability and reliability of their services.

It only collects metrics. It's still up to the application to expose them 

Here is an example how to expose metrics in user application. This is not part of this PR which only collects metrics.
```
from flask import Flask, Response, make_response

# Create the Flask App
app = Flask(__name__)

@app.route('/metrics')
def metrics():
    return Response(generate_latest(), mimetype=CONTENT_TYPE_LATEST)
```

Dependencies:
prometheus_client-0.19.0 is already specified under libs/community's poetry.

Twitter handle: itestmycode

@baskaryan, @eyurtsev, @hwchase17.
bugfix/fix table in documentation for langchain-community
Labels: name: auto:documentation, name: area: models, name: size:XS

  - **Description:** This PR aims to fix the documentation for langchain-commnuity.

  - **Issue:** The table In this page : [https://python.langchain.com/docs/integrations/llms/](https://urldefense.com/v3/__https://python.langchain.com/docs/integrations/llms/__;!!ACWV5N9M2RV99hQ!Jqw8gWnQrL1H6blPiGN10jrh1TDAzqGcKAaTAZv7TBy1X_m-03E7T-alOrWY5_71R8QUdONvF2wMRK54D50$) is built based on old module.  The proposed fix is to modify the import from langchain-commnuity.
  
  - **Dependencies:** N/A
  - **Twitter handle:** N/A
Extraction: create_extraction_chain_pydantic
Labels: name: bug, name: auto:bug, name: area: models, name: extraction, name: confirmed

### System Info

```➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy
```

```
In [2]: langchain.__version__
Out[2]: '0.0.354'
```

```
In [4]: from langchain_core import __version__
In [5]: __version__
Out[5]: '0.1.8'
```


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

# Description

Create extraction chain pydantic does not work for valid pydantic schemas.

```python

from typing import Optional, List
from langchain.chains import create_extraction_chain_pydantic
from langchain_openai import ChatOpenAI


class Person(BaseModel):
    """Identifying information about a person in a text."""

    person_name: str
    person_height: Optional[int]
    person_hair_color: Optional[str]
    dog_breed: Optional[str]
    dog_name: Optional[str]



# Chain for an extraction approach based on OpenAI Functions
extraction_chain = create_extraction_chain_pydantic(Person, ChatOpenAI(temperature=0))

extraction_chain.invoke("My name is tom and i'm 6 feet tall")
```

However, more complex pydantic definitions fail:

```
class People(BaseModel):
    """Identifying information about all people in a text."""
    __root__: List[Person]

# Chain for an extraction approach based on OpenAI Functions
extraction_chain = create_extraction_chain_pydantic(People, ChatOpenAI(temperature=0))
extraction_chain.invoke("My name is tom and i'm 6 feet tall")
```

![image](https://github.com/langchain-ai/langchain/assets/3205522/388958e9-0150-436f-944b-149ba4ca0d49)


```
class NestedPeople(BaseModel):
    """Identifying information about all people in a text."""
    people: List[Person]

# Chain for an extraction approach based on OpenAI Functions
extraction_chain = create_extraction_chain_pydantic(NestedPeople, ChatOpenAI(temperature=0))
extraction_chain.invoke("My name is tom and i'm 6 feet tall")
```

![image](https://github.com/langchain-ai/langchain/assets/3205522/40d17d56-2720-49f6-ab33-07ae4a69f1d3)

---

## Acceptance criteria

1. Code does not affect backwards compatibility if possible If must be a breaking change, perhaps we should create a new function for this purpose.
2. Should we replace LLMChain with an LCEL chain and determine what is the correct output interface for extractions? User may want error information to be returned rather than raised.
3. Unit-tests must cover above cases


### Expected behavior

All shown cases should work properly and not fail during initialization time.
langchain: Extract _aperform_agent_action from _aiter_next_step from AgentExecutor
Labels: name: area: agent, name: lgtm, name: auto:refactor, name: size:L

<!-- Thank you for contributing to LangChain!

Please title your PR "lang: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

- **Description:** extreact the _aperform_agent_action in the AgentExecutor class to allow for easier overriding. Extracted logic from _iter_next_step into a new method _perform_agent_action for consistency and easier overriding.
- **Issue:** #15706

Closes #15706
Extract _aperform_agent_action from _aiter_next_step from AgentExecutor
Labels: name: area: agent, name: auto:improvement

### Feature request

I'm trying to extend AgentExecutor with custom logic and I want to override how the agent perform actions.
What i'd really need is only to override the __aperform_agent_action_ function; however this function is defined in the __aiter_next_step_ function, making it necessary to override the whole function.
This obviously comes with the drawbacks of more code and having to reconciliate future updates.

In my opinion, the __aiter_next_step_ function could be extracted into an instance or static method, allowing to override only the relevant parts.
Also, for the synchronous version _iter_next_step_ a similar problem arises, as the __perform_agent_action_ is not defined at all.
The relevant code can be extracted in a method making it easier to override it.

### Motivation

This update would allow for better extendibility of the AgentExecutor class

### Your contribution

I can submit a PR to address the issue