delayedkarma/langchain-issues huggingface dataset:
Issue: trying to call generate_queries() on a MultiQueryRetriever but where do I get a run_manager from?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

I'm trying to use `MultiQueryRetriever` to generate variations on a question: it seems to work, but I can't use it inside a chain created with `load_qa_with_sources_chain()` because that generates a chain that expects a list of input_documents, rather than a retriever, and I don't want to use `RetrievalQAWithSourcesChain` instead of `load_qa_with_sources_chain()` because I want to continue implementing my own similarity search (eg. I'm supporting switches to select using `index.similarity_search()` vs. `index.max_marginal_relevance_search()` on my `index`, and to specify the number of matches (`k`)), so I figured I could just call `generate_queries()` on my `MultiQueryRetriever` instance and then manually run my (`load_qa_with_sources_chain()`) chain for each variation.  However, that method requires a `run_manager`, and I can't figure out how to create one.  I already have the `load_qa_with_sources_chain()` chain: can I get a `run_manager` from that?  Or more generally, what's the best way to use `MultiQueryRetriever` whilst maintaining one's own code for fetching matching text snippets?

### Suggestion:

_No response_
Improvements in Nebula LLM
Labels: name: auto:improvement, name: area: models

  - Description: Added improvements in Nebula LLM to perform auto-retry; more generation parameters supported. Conversation is no longer required to be passed in the LLM object. Examples are updated.
  - Issue: N/A
  - Dependencies: N/A
  - Tag maintainer: @baskaryan 
  - Twitter handle: symbldotai
Minor formatting on Web Research Use Case
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Kernel dies for SVMRetriever with Huggingface Embeddings Model
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

langchain==0.0.245
python==3.9

### Who can help?

@hw

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am trying to use HuggingFace Embeddings model for SVMRetriever method as follows:
```
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.retrievers import SVMRetriever

embedding = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
retriever = SVMRetriever.from_texts(["foo", "bar", "world", "hello", "foo bar"], 
                                                           embedding, 
                                                           k=4, relevancy_threshold=.25)
```
Kernel looks busy for a while and then dies when I run above snippet.

### Expected behavior

I expect to run this smoothly and upon completion, should be able to retrieve similar documents as:
`result = retriever.get_relevant_documents("foo")`
DOC: <Please write a comprehensive title after the 'DOC: ' prefix>
Labels: name: auto:question, name: auto:documentation, name: area: models

### Issue with current documentation:

In https://python.langchain.com/docs/use_cases/extraction, I can run this example using the ChatOpenAI method. However, within my organization, I want to use AzureChatOpenAI. However, the same example doesnt work here. The error I get is 

```
InvalidRequestError: Unrecognized request argument supplied: functions
```

What am I doing wrong?


### Idea or request for content:

_No response_
Remove packages for pydantic compatibility
Labels: name: lgtm, name: auto:improvement

# Poetry updates

This PR updates LangChains poetry file to remove
any dependencies that aren't pydantic v2 compatible yet.

All packages remain usable under pydantic v1, and can be installed
separately. 

## Bumping the following packages:

* langsmith

## Removing the following packages

not used in extended unit-tests:

* zep-python, anthropic, jina, spacy, steamship, betabageldb

not used at all:

* octoai-sdk

Cleaning up extras w/ for removed packages.

## Snapshots updated

Some snapshots had to be updated due to a change in the data model in langsmith. RunType used to be Union of Enum and string and was changed to be string only.
Landonsilla patch 1
Labels: name: auto:improvement

  - Description: there is an on_retry event that gets called. however the common callback handlers don't implement an `on_retry` method. So i added a "do nothing" stub method that gets inherited for all these common callback handlers.
  - Issue: [the issue # it fixes (if applicable),](https://github.com/langchain-ai/langchain/issues/8542),
  - Dependencies: none,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
Parent Doc Retriever
Labels: name: area: doc loader, name: auto:improvement

2 things:
- Implement the private method rather than the public one so callbacks are handled properly
- Add search_kwargs (Open to not adding this if we are trying to deprecate this UX but seems like as a user i'd assume similar args to the vector store retriever. In fact some may assume this implements the same interface but I'm not dealing with that here)
-
Issue: Using custom_id with PGVector store
Labels: name: area: vector store, name: auto:bug, name: auto:documentation

### Issue you'd like to raise.

Hi, 
I have a pandas dataframe with a column name 'chunk_id'. 
I want to use this chunk_id as the custom_id in the langchain_pg_embedding table. 

The langchain documentation search bot tells me this is how I can do it. 

```python
from langchain.vectorstores import PGVectorStore

vector_store = PGVectorStore(
    table_name="documents", 
    conn_str="postgresql://username:password@localhost:5432/database",
    custom_id="my_custom_id" 
)
```

This doesn't seem right to me. For one, there is no PGVectorStore in langchain.vectorstores. Secondly the PGVector from `from langchain.vectorstores.pgvector import PGVector` does not accept the custom_id argument. 

Is there any way to use my 'chunk_id' as the custom_id?

### Suggestion:

_No response_
Issue: Amazon Bedrock Jurassic model responses getting truncated
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

When I create a Bedrock LLM Client for the Jurassic models -  "ai21.j2-mid", "ai21.j2-ultra", the llm.invoke(query) methods are not returning with full results. The result seems to be getting truncated after first line. 
It seems like the LLM Engine is streaming it's output but langchain llm.invoke method is not able to handle the streaming data.
Similar issue is coming with the chain.invoke(query) methods as well. 

However, llm.invoke works well when tried with AWS Bedrock  "amazon.titan-tg1-large" model

### Suggestion:

I guess the AI21 labs models are streaming output in a way that the LangChain llm.invoke( query) method is not able to handle properly.
_convert_message_to_dict doesn't handle FunctionMessage type
Labels: name: auto:bug, name: area: models

### System Info

langchain                     0.0.263
python 3.9

### Who can help?

@hwchase17 
@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.schema import FunctionMessage
from langchain.chat_models.jinachat import _convert_message_to_dict
_convert_message_to_dict(FunctionMessage(name='foo', content='bar'))
```

![image](https://github.com/langchain-ai/langchain/assets/18662395/7040489d-6009-4232-9785-d8bba5fc8332)


### Expected behavior

Handle FunctionMessage
Docs fix
Labels: name: area: vector store, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Passing filter through ConversationalRetrievalChain to the underlying vector store
Labels: name: area: vector store, name: auto:improvement

### Feature request

It seems that right now there is no way to pass the filter dynamically on the call of the ConversationalRetrievalChain, the filter can only be specified in the retriever when it's created and used for all the searches. 
```
    qa = ConversationalRetrievalChainPassArgs.from_llm(
        OpenAI(..),
        VectorStoreRetriever(vectorstore=db, search_kwargs={"filter": {"source": "my.pdf"}}))
```
I had to extend in my code both ConversationalRetrievalChain and VectorStoreRetriever just to pass the filter to the vectore store like this:
```
    qa = ConversationalRetrievalChainPassArgs.from_llm(
        OpenAI(...),
        VectorStoreRetrieverWithFiltering(vectorstore=db),
)
    result = qa({"question": query, "chat_history": [], "filter":  {"source": "my.pdf"}}})

```
Is it really the case or am I missing something? And if it the case, shouldn't it be fixed? (I could provide a pull request)

### Motivation

To be able to filter dynamically in ConversationalRetrievalChain since vector stores allow this.

### Your contribution

I can provide a PR if my understanding is confirmed.
Incomplete responses for basic example when used HF with llama and Falcon LLMS
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

I am just testing a very basic code as follows using LangChain
```

from langchain import HuggingFaceHub

from langchain import PromptTemplate, LLMChain
import asyncio

question = "Who won the FIFA World Cup in the year 1994? "

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

llm: HuggingFaceHub = HuggingFaceHub(
					#repo_id="tiiuae/falcon-7b",
                    repo_id="meta-llama/Llama-2-7b-hf",
					huggingfacehub_api_token="MY-TOKEN",
					model_kwargs={
						"temperature": 0.5,
						 "max_length": 64
					}
				)

llm_chain = LLMChain(prompt=prompt, llm=llm)

async def run_chain():
    result = await llm_chain.arun(question)
    print('restult with load_qua_chain is....', result)

asyncio.run(run_chain())
```
Output when tested with "meta-llama/Llama--2-7b-hf" is ...
```
We have to find the winner of the FIFA World Cup in the year 199
```
Output when tested with "tiiue/falcon-7b" is...
`restult with load_qua_chain is....  We know that 1994 is a leap year, and the previous year was 1993.`

But the expected answer is not what i am getting and it looks like Either the response is NOT fully getting retrieved from LLM or i am doing some thing wrong.  I followed the same code as shown in the landchain documentation at [https://python.langchain.com/docs/integrations/llms/huggingface_hub](url),  only thing is i used HF with my API key instead of Open AI.

Can i request any help / suggestion if i have to make any changes pls

### Expected behavior:

To get the full response as mentioned in the Lang chain documentation for these examples
fix logging to logger
Labels: name: auto:improvement

# What
- fix logging to logger

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: fix logging to logger
  - Issue: None
  - Dependencies: None
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: @MLOpsj

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
ConstitutionalChain may violate earlier principals when given more than one
Labels: name: auto:bug, name: area: models

### System Info

langchain==0.0.263
python 3.11.4


### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains.llm import LLMChain

evil_assistant_prompt = PromptTemplate(
    template="""
			You are a evil mentor for students with no morals. Give suggestions that are easiest and fastest to achieve the goal.
			Goal: {inquiry}
			Easiest way:""",
    input_variables=["inquiry"],
)

llm = OpenAI(model_name="text-davinci-003", temperature=0)
evil_assistant_chain = LLMChain(llm=llm, prompt=evil_assistant_prompt)

from langchain.chains.constitutional_ai.base import ConstitutionalChain
from langchain.chains.constitutional_ai.models import ConstitutionalPrinciple

ethical_principle = ConstitutionalPrinciple(
    name="Ethical Principle",
    critique_request="The model should only talk about ethical and fair things.",
    revision_request="Rewrite the model's output to be both ethical and fair.",
)

fun_principle = ConstitutionalPrinciple(
    name="Be Funny",
    critique_request="The model responses must be funny and understandable for a 7th grader.",
    revision_request="Rewrite the model's output to be both funny and understandable for 7th graders.",
)

constitutional_chain = ConstitutionalChain.from_llm(
    chain=evil_assistant_chain,
    constitutional_principles=[ethical_principle, fun_principle],
    llm=llm,
    verbose=True,
)

result = constitutional_chain.run(inquiry="Getting full mark on my exams.")
print(result)
```

This results in a final answer:

`The best way to get full marks on your exams is to study hard, attend classes, and bribe the professor or TA if needed.`

The second portion originates from the evil portion

`Cheat. Find someone who has already taken the exam and get their answers. Alternatively, bribe the professor or TA to give you full marks.`

### Expected behavior

I would expect a result that did not involve bribing someone.

The underlying issue is that subsequent constitutions could modify the results in a way that violates earlier constitutions defined in the `constitutional_principals` list. This seems to originate from the fact that the earlier principals aren't check again (from what I can tell). I could see the possibility of an infinite loop if the logic was to keep modifying until all principals are satisfied, so I guess this brings into question whether there's value in having chained principals and not just having a single principal that's a concatenation of all your concerns.
Error Response payload is not complete 
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

 I am using pusher for streaming purpose.
I am getting error as "Response payload is not completed"

async def pusher__cal(new_payload, token):
# print(token)
session_id = new_payload["session_id"]
session_id_channel = f"{session_id}_channel"
user_id = new_payload["user_id"]
message_id = new_payload["message_id"]
status = "processing"

await asyncio.sleep(0)  # Allow other async tasks to run
pusher_client.trigger(session_id_channel ,'query_answer_stream', {
                    "user_id": user_id,
                    "session_id": session_id,
                    "message_id": message_id,
                    "answer": token,
                    "status": status
                    },
                    )

class MyCallbackHandler(AsyncCallbackHandler):
def init(self,new_payload : Dict):
self.new_payload = new_payload
self.user_id = new_payload["user_id"]
self.session_id = new_payload["session_id"]
self.message_id = new_payload["message_id"]
self.session_id_channel = f"{self.session_id}_channel"
self.status = "streaming"
self.list = []

async def on_llm_new_token(self, token, **kwargs) -> None:

    if token != "":
        await pusher__cal(self.new_payload,token)


### Suggestion:

_No response_
Issue for Streaming 
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

This is the code I am using for custom callback
async def pusher__cal(new_payload, token):
    print(token)
    session_id = new_payload["session_id"]
    session_id_channel = f"{session_id}_channel"
    user_id = new_payload["user_id"]
    message_id = new_payload["message_id"]
    status = "processing"

    pusher_client.trigger(session_id_channel ,'query_answer_stream', {
                        "user_id": user_id,
                        "session_id": session_id,
                        "message_id": message_id,
                        "answer": token,
                        "status": status
                        },
                        )
class MyCallbackHandler(AsyncCallbackHandler):
    def __init__(self,new_payload : Dict):
        self.new_payload = new_payload
        self.user_id = new_payload["user_id"]
        self.session_id = new_payload["session_id"]
        self.message_id = new_payload["message_id"]
        self.session_id_channel = f"{self.session_id}_channel"
        self.status = "streaming"



    async def on_llm_new_token(self, token, **kwargs) -> None:

        if token != "" and token != None:
            await pusher__cal(self.new_payload,token)



This is the error 

Error in MyCallbackHandler.on_llm_new_token callback: HTTPSConnectionPool(host='api-eu.pusher.com', port-443): Read timed out. (read timeout=5)



### Suggestion:

_No response_
Adds DeepSparse as an LLM
Labels: name: lgtm, name: auto:enhancement, name: area: models

Adds [DeepSparse](https://github.com/neuralmagic/deepsparse) as an LLM backend. DeepSparse supports running various open-source sparsified models hosted on [SparseZoo](https://sparsezoo.neuralmagic.com/) for performance gains on CPUs.

Twitter handles: @mgoin_ @neuralmagic

Thanks for your work and let me know if there are any questions!

cc: @hwchase17 @baskaryan
Typing Issue: Client and Model defaults are not defined in OpenAI LLM
Labels: name: auto:bug, name: area: models

### System Info

### Pylance displays missing parameter errors for `client` and `model`

![Pylance_Lanchain_OpenAI_Issue](https://github.com/langchain-ai/langchain/assets/3806601/ce2293b4-e6e9-4cd9-ab46-39da35c3109c)

Neither `client` or `model` are required in the documenation.

## Suggested fix

Update attributes in BaseOpenAI class in langchain/llms/openai.py
```Python
# class definition
client: Any = None  #: :meta private: # set default to None
model_name: str = Field(default="text-davinci-003", alias="model") # add default=
```

python: "^3.11"
langchain: "^0.0.262"
openai: "^0.27.8"
pylance: v2023.8.20

## Related to issue:
Typing issue: Undocumented parameter client #9021



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Install Pylance. 
1. Use VSCode to run typecheck.


### Expected behavior

It should pass Pylance type checking without missing param errors for non-essential args.
Add support  HuggingFace Inference Endpoint for embeddings
Labels: name: area: embeddings, name: auto:enhancement

### Feature request

https://huggingface.co/inference-endpoints



### Motivation

We do not have support for HuggingFace Inference endpoints for tasks like embeddings and must be easy to implement inheriting the class from Embeddings base class

`InferenceEndpointHuggingFaceEmbeddings(Embeddings):` with only 

`self.endpoint_name = endpoint_name
        self.api_token = api_token`

### Your contribution

I could submit a PR to this issue since I've created a class for this
feat(llms): support vLLM's OpenAI-compatible server
Labels: name: auto:enhancement, name: area: models

This PR aims at supporting [vLLM's OpenAI-compatible server feature](https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html#openai-compatible-server), i.e. allowing to call vLLM's  LLMs like if they were OpenAI's. 

I've also udpated the related notebook providing an example usage. At the moment, vLLM only supports the `Completion` API.