delayedkarma/langchain-issues huggingface dataset:
KeyError 'role' in OpenAI Adapter
Labels: name: auto:bug, name: area: models

### System Info

Python 3.10
LangChain 0.0.348

### Who can help?

@hwchase17 @hin

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.chat_models import ChatOpenAI
from langchain.memory import ChatMessageHistory

history = ChatMessageHistory()
history.add_user_message("Tell me anything in ten words or less.")

params = {
    "model": "gpt-4-1106-preview",
    "temperature": 0.5,
    "max_tokens": 1000,
}
llm = ChatOpenAI(**params)
print(llm(history.messages))
```
This will **occasionally** yield the following error:
```
File "/.../.venv/lib/python3.10/site-packages/langchain/adapters/openai.py", line 74, in convert_dict_to_message
    role = _dict["role"]
KeyError: 'role'
```

### Expected behavior

* Safe lookups ensure the response is passed back even if the role isn't attached
Add support for Mistral AI service
Labels: name: auto:enhancement, name: area: models

### Feature request

Add support for the new service from Mistral AI.
They provide a [python client](https://docs.mistral.ai/platform/client) with streaming tokens or we can use the simple [RestAPI](https://docs.mistral.ai/)

### Motivation

Would be great if we add the new service from Mistral!

### Your contribution

I don't have much time right now but i'd like to follow the implementation for the future
Table 'langchain_pg_collection' is already defined for this MetaData instance.  Specify 'extend_existing=True'
Labels: name: area: vector store, name: auto:bug

### System Info

langchain: 0.0.39
python: 3.11
OS: MacOS Sonoma 14.2

### Who can help?

@hwchase17 @eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Load two collections using PGVector like this:

collection1 = PGVector.from_existing_index(
    embedding=embeddings,
    collection_name="collection1",
    pre_delete_collection=False,
    distance_strategy=DistanceStrategy.COSINE,
    connection_string=CONNECTION_STRING,
)

collection2 = PGVector.from_existing_index(
    embedding=embeddings,
    collection_name="collection1",
    pre_delete_collection=False,
    distance_strategy=DistanceStrategy.COSINE,
    connection_string=CONNECTION_STRING,
)

`collection1` works fine, but while initializing collection2, I get the error 
`Table 'langchain_pg_collection' is already defined for this MetaData instance.  Specify 'extend_existing=True'`

![image](https://github.com/langchain-ai/langchain/assets/108780317/9271096e-d246-4ebf-9978-74cef82c3fe9)

Setting `extend_existing=True` doesn't seem to work - I think this is a SQLAlchemy error.


### Expected behavior

The above code should just work without error.
This was working in previous versions: 0.0.310 this code works, but shows an error in 0.0.390.
If there are multiple connections happening to the index, then the documentation doesn't talk about how to handle multiple connections.
[FEATURE] Add Aphrodite Engine support
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: size:XL

This PR adds support for PygmalionAI's [Aphrodite Engine](https://github.com/PygmalionAI/aphrodite-engine), based on vLLM's attention mechanism. At the moment, this PR does not include support for the API servers, but they will be added in a later PR.

The only dependency as of now is `aphrodite-engine==0.4.2`. We pin the version to prevent breakage due to changes in the aphrodite-engine library.
Langserve example from Quickstart tutorial not working
Labels: name: area: doc loader, name: auto:bug

### System Info

Name                    Version                   Build  Channel
langchain                 0.0.350                  pypi_0    pypi
langchain-cli             0.0.19                   pypi_0    pypi
langchain-community       0.0.3                    pypi_0    pypi
langchain-core            0.1.1                    pypi_0    pypi
langchain-experimental    0.0.47                   pypi_0    pypi
python                    3.12.0          h47c9636_0_cpython    conda-forge


System: macOS 14.2 (Apple M1 chip)

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Go to https://python.langchain.com/docs/get_started/quickstart
2. Do all the package installations as guided.
3. Copy the python code under the section 'Serving with Langserve' and save it to a file `serve.py`
4. Execute the file `python serve.py`
5. Open `http://localhost:8000` in browser

### Expected behavior

Expected to see Langserve UI. Got the following error instead.

---
LANGSERVE: Playground for chain "/category_chain/" is live at:
LANGSERVE:  │
LANGSERVE:  └──> /category_chain/playground/
LANGSERVE:
LANGSERVE: See all available routes at /docs/

LANGSERVE: ⚠️ Using pydantic 2.5.2. OpenAPI docs for invoke, batch, stream, stream_log endpoints will not be generated. API endpoints and playground should work as expected. If you need to see the docs, you can downgrade to pydantic 1. For example, `pip install pydantic==1.10.13`. See https://github.com/tiangolo/fastapi/issues/10360 for details.

INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
INFO:     ::1:58516 - "GET / HTTP/1.1" 404 Not Found
INFO:     ::1:58516 - "GET /favicon.ico HTTP/1.1" 404 Not Found
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [91610]
Issue: I'm currently working on a project where I need to fetch all the sub-URLs from a website using Langchain.
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.


I'm currently working on a project where I need to fetch all the sub-URLs from a website using Langchain.
How can we achieve this, below is my code
`from langchain.tools.playwright import ExtractHyperlinksTool, NavigateTool

# Initialize the tools
navigate_tool = NavigateTool()
extract_hyperlinks_tool = ExtractHyperlinksTool()

# Navigate to the website
navigate_tool.navigate("https://www.example.com")

# Extract all hyperlinks
hyperlinks = extract_hyperlinks_tool.extract()

# Print all hyperlinks
for link in hyperlinks:
    print(link)`
  I am getting below mentioned error
  File "/home/hs/env/lib/python3.8/site-packages/langchain_core/load/serializable.py", line 97, in init
super().init(**kwargs)
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for NavigateTool
root
Either async_browser or sync_browser must be specified. (type=value_error)

### Suggestion:

_No response_
ImportError: cannot import name 'GoogleGenerativeAI' from 'langchain_google_genai' ( from langchain_google_genai import GoogleGenerativeAI  )
Labels: name: auto:bug, name: area: models

### System Info

- Python 3.11.7
- Windows 64bit
- langchain_google_genai==0.0.4
- langchain==0.0.350
- pymongo==4.6.0

### Who can help?

@sbusso
@jarib 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Install langchain_google_genai using `pip install -U langchain-google-genai`
2. Then, `from langchain_google_genai import GoogleGenerativeAI` 
3. This will produce error.

### Expected behavior

I want to use this model for Google AI, but not able to access it
cannot import name 'AzureChatopenAI' from 'langchain.chat_models'
Labels: name: auto:bug, name: area: models

### System Info

ImportError: cannot import name 'AzureChatopenAI' from 'langchain.chat_models'

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.chat_models import AzureChatopenAI

### Expected behavior

from langchain.chat_models import AzureChatopenAI

llm = ChatOpenAI(
            openai_api_base=url,
            openai_api_version="2023-05-15",
            deployment_name="gpt-3.5",
            openai_api_key="test123456",
            openai_api_type="azure",
        )
adapt chatglm output_parser
Labels: name: auto:enhancement, name: area: models

### Feature request

i want to struct output result by langchain,my llm is chatglm,but most output parsers demo is rely on openai

### Motivation

retriever = db.as_retriever()
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type='stuff',
    retriever=retriever,
    verbose=True
)


def chat(question,history):
    response = qa.run(question)
    
    return response

demo = gr.ChatInterface(chat)
demo.launch(inbrowser=True)

### Your contribution

i want to struct output result by langchain,my llm is chatglm,but most output parsers demo is rely on openai
Fix OAI Tool Message
Labels: name: area: agent, name: lgtm, name: auto:bug, name: size:L

See format here: https://platform.openai.com/docs/guides/function-calling/parallel-function-calling


It expects a "name" argument, which we aren't providing by default.

![image](https://github.com/langchain-ai/langchain/assets/13333726/7cd82978-337c-40a1-b099-3bb25cd57eb4)


Alternative is to add the 'name' field directly to the message if people prefer.
Update _retrieve_ref inside json_schema.py to include an isdigit() check
Labels: name: lgtm, name: needs test, name: area: doc loader, name: auto:improvement, name: size:M

  - **Description:** Update _retrieve_ref inside json_schema.py to include an isdigit() check
  - **Issue:** This library is used inside dereference_refs inside langchain_community.agent_toolkits.openapi.spec. When I read in a yaml file which has references for "400", "401" etc; the line "out = out[component]" causes a KeyError. The isdigit() check ensures that if it is an integer like "400" or "401"; it converts it into integer before using it as a key to prevent the error.
  - **Dependencies:** No dependencies
  - **Tag maintainer:** @baskaryan
Expose Search Similarity Results as Source Documents in create_vectorstore_router_agent Responses
Labels: name: area: vector store, name: auto:improvement

Hello Langchain Team,

I've been working with the `create_vectorstore_router_agent` function, particularly in conjunction with the `VectorStoreRouterToolkit`, and I've encountered a limitation that I believe could be an important area for enhancement.

Currently, the response from this function primarily includes the final processed answer to a query. However, it does not provide any details about the source documents or the similarity search performed within the vector store. In many applications, especially those that require a high degree of transparency and traceability of information, having access to the source documents is crucial.

The ability to see which documents were retrieved, along with their similarity scores or other relevant metadata, is invaluable. It helps in understanding the basis of the answers provided by the agent and is essential for verifying the relevance and accuracy of the information.

Therefore, I suggest enhancing the functionality of the toolkit to include an option to return detailed information about the retrieved documents in the response. This could be implemented as an optional feature that can be enabled as needed, depending on the specific requirements of the use case.

Such a feature would significantly enhance the utility and applicability of the Langchain library, particularly in scenarios where detailed source information is essential for validation, auditing, or explanatory purposes.

Thank you for considering this enhancement. I believe it would make a great addition to the capabilities of Langchain.

Best regards,
Sarath chennamsetty.
Wfh nvidia trt
Labels: name: auto:enhancement, name: area: models, name: size:XL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: Update the outdated hugging face client
Labels: name: auto:improvement, name: area: models

### Issue you'd like to raise.

Hugging face will deprecate the `InferenceApi` and move to `InferenceClient`. Can the langchain package also update the dependency in `HuggingFaceHub` accordingly? Refer to [migration docs](https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client) to migrate to new client.

### Suggestion:

_No response_
Implement similarity_score_threshold for MongoDB Vector Store
Labels: name: lgtm, name: area: vector store, name: auto:improvement, name: size:S

Adds the option for `similarity_score_threshold` when using `MongoDBAtlasVectorSearch` as a vector store retriever.

Example use:

```
vector_search = MongoDBAtlasVectorSearch.from_documents(...)

qa_retriever = vector_search.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={
        "score_threshold": 0.5,
    }
)

qa = RetrievalQA.from_chain_type(
	llm=OpenAI(), 
	chain_type="stuff", 
	retriever=qa_retriever,
)

docs = qa({"query": "..."})
```

I've tested this feature locally, using a MongoDB Atlas Cluster with a vector search index.
Remove trailing "`" in pip install command
Labels: name: lgtm, name: area: doc loader, name: auto:documentation, name: size:XS

hi! just a simple typo fix in the local LLM python docs

  - **Description:** removing a trailing "\`" character in a `!pip install ...` command
  - **Issue:** n/a
  - **Dependencies:** n/a
  - **Tag maintainer:** n/a
  - **Twitter handle:** n/a
AttributeError: module 'google.generativeai' has no attribute 'GenerativeModel'
Labels: name: auto:bug, name: area: models

### System Info

Similar issue, when tried to work with Gemini.Ai

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

AttributeError: module 'google.generativeai' has no attribute 'GenerativeModel'

### Expected behavior

just loaded those images that i have uploaded for the prompt
[Documentation] Updates to NVIDIA Playground/Foundation Model naming. 
Labels: name: lgtm, name: auto:documentation, name: area: models, name: size:XL

  - **Description:** Minor updates per marketing requests. Namely, name decisions (AI Foundation Models / AI Playground)
  - **Tag maintainer:** @hinthornw 

Do want to pass around the PR for a bit and ask a few more marketing questions before merge, but just want to make sure I'm not working in a vacuum. No major changes to code functionality intended; the PR should be for documentation and only minor tweaks.

Note: QA model is a bit borked across staging/prod right now. Relevant teams have been informed and are looking into it, and I'm placeholdered the response to that of a working version in the notebook.
Add support to parse VertexAI Model garden Llama2 model output
Labels: name: auto:improvement, name: area: models, name: size:M

- **Description:** The output of Llama2 models in GCP VertexAI Model garden is not compatible with the generic VertexAiModelGarden handler class. This PR adds a class to handle/ properly parse the output returned by those models. The handler also adds support for storing/ passing parameters to the model that is consistent with the core VertexAI LLM models.

- **Dependencies:** None
`LLMChain` does not support `ChatGoogleGenerativeAI`
Labels: name: auto:bug, name: area: models

### System Info

langchain==0.0.330
langchain-core==0.1.0
langchain-google-genai==0.0.3

python: 3.11

### Who can help?

@agola11 @hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(
    model="gemini-pro",
    google_api_key=google_api_key,
        )

return LLMChain(llm=llm, prompt=prompt, verbose=verbose)
```

Error:

```
pydantic.error_wrappers.ValidationError: 2 validation errors for LLMChain
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
```

### Expected behavior

I expect LLMChain to run as usual with this new chat model.
pipeline test
Labels: name: area: doc loader, name: auto:documentation, name: size:XXL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Ollama max tokens parameter
Labels: name: auto:question, name: area: models

### Discussed in https://github.com/langchain-ai/langchain/discussions/14714

<div type='discussions-op-text'>

<sup>Originally posted by **ciliamadani** December 14, 2023</sup>
I'm currently in the process of developing a chatbot utilizing Langchain and the Ollama (llama2 7b model). My objective is to allow users to control the number of tokens generated by the language model (LLM).

In the Ollama documentation, I came across the parameter 'num_predict,' which seemingly serves this purpose. However, when using Ollama as a class from Langchain, I couldn't locate the same parameter. Consequently, I've been attempting to pass it as metadata. Unfortunately, even when setting this parameter to a low value, such as 50, the LLM continues to generate more tokens than expected.

I'm wondering if you have any insights on how I can effectively control the number of generated tokens when using Ollama as a Langchain class?


My current code

llm = Ollama(model="llama2:7b-chat-q4_0",callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),temperature=temperature,repeat_penalty=1.19,top_p=top_p,repeat_last_n=-1,metadata={"num_predict": num_predict})

Thank you.</div>
TypeError: Chain.__call__() got an unexpected keyword argument 'stop'
Labels: name: auto:bug, name: area: models, name: integration: chroma

### Issue you'd like to raise.

Hi all,

trying to use Langchain with HuggingFace model and Embeddings.
Am new to Langchain so any pointers welcome.

`import os
os.environ['HUGGINGFACEHUB_API_TOKEN']=myToken

#required to avoid certificate issue

os.environ['CURL_CA_BUNDLE'] = ''
from langchain import PromptTemplate, HuggingFaceHub, LLMChain

#Build prompt
template = """
Question: {question}
Answer: Let's think of the best answer, with arguments."""
prompt = PromptTemplate(template=template, input_variables=["question"])

llm_chain=LLMChain(prompt=prompt,
                  llm=HuggingFaceHub(repo_id="google/flan-t5-xxl",
                                    model_kwargs={"temperature":0.5,
                                                 "max_length":64}))

from langchain.embeddings import HuggingFaceEmbeddings

from langchain.document_loaders import PyPDFLoader

loader = PyPDFLoader("https://www.ofgem.gov.uk/sites/default/files/docs/2018/10/rec_v1.0_main_body.pdf")

pages = loader.load_and_split()

from langchain.vectorstores import Chroma

from langchain.chains import RetrievalQA

doc_embed = HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")

docsearch = Chroma.from_documents(pages, doc_embed)

qa_chain = RetrievalQA.from_chain_type(
    llm_chain,
    retriever=docsearch.as_retriever()
)

output = qa_chain.run("what is the retail energy code?")`

I get the following error:

---> 89     return self(
     90         input,
     91         callbacks=config.get("callbacks"),
     92         tags=config.get("tags"),
     93         metadata=config.get("metadata"),
     94         run_name=config.get("run_name"),
     95         **kwargs,
     96     )

TypeError: Chain.__call__() got an unexpected keyword argument 'stop'

### Suggestion:

_No response_