delayedkarma/langchain-issues huggingface dataset:
Direct initial redis database can't successfully use searching function since index missing
Labels: name: area: vector store, name: auto:bug

### System Info

langchain version: 0.0.267

### Who can help?

@hwchase17 
@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

# Redis vector database initialization
<p>
embeddings = OpenAIEmbeddings(openai_api_key=os.getenv("OPENAI_API_TYPE"), <br/>
&nbsp;&nbsp;&nbsp;&nbsp;                              deployment=os.getenv("OPENAI_EMBEDDING_MODEL_NAME"), <br/>
 &nbsp;&nbsp;&nbsp;&nbsp;                             client="azure", <br/>
&nbsp;&nbsp;&nbsp;&nbsp;                              chunk_size=1)<br/>
redis_vector_db = Redis(<br/>
&nbsp;&nbsp;&nbsp;&nbsp;    redis_url="redis://localhost:6379", <br/>
&nbsp;&nbsp;&nbsp;&nbsp; index_name="test_1", <br/>
&nbsp;&nbsp;&nbsp;&nbsp; embedding_function=embedding.embed_query<br/>
)
</p>
# Load pdf 
loader = PyPDFLoader( file_path)
docs= loader.load_and_split()
redis_vector_db.add_documents(docs, index_name = "test_1")

# Searching --> bug here
docs = redis_vector_db.similarity_search(query = question,k = 4)
-------------------------- This is the end of python script ---------------------------------------


### Expected behavior


# Bug:  Even though we have set the index name  it will have bug of no such index
File [c:\Users\User\anaconda3\envs\advantech\lib\site-packages\langchain\vectorstores\redis.py:284](file:///C:/Users/User/anaconda3/envs/advantech/lib/site-packages/langchain/vectorstores/redis.py:284), in Redis.similarity_search(self, query, k, **kwargs)
    271 def similarity_search(
    272     self, query: str, k: int = 4, **kwargs: Any
    273 ) -> List[Document]:
    274     """
    275     Returns the most similar indexed documents to the query text.
    276 
   (...)
    282         List[Document]: A list of documents that are most similar to the query text.
    283     """
--> 284     docs_and_scores = self.similarity_search_with_score(query, k=k)
    285     return [doc for doc, _ in docs_and_scores]

File [c:\Users\User\anaconda3\envs\advantech\lib\site-packages\langchain\vectorstores\redis.py:361](file:///C:/Users/User/anaconda3/envs/advantech/lib/site-packages/langchain/vectorstores/redis.py:361), in Redis.similarity_search_with_score(self, query, k)
    354 params_dict: Mapping[str, str] = {
    355     "vector": np.array(embedding)  # type: ignore
...
    904 if isinstance(response, ResponseError):
--> 905     raise response
    906 return response

ResponseError: test_1: no such index


# My observation
print(redis_vector_db.index_name)
will show test_1
import tqdm.auto instead of tqdm tqdm for OpenAIEmbeddings
Labels: name: lgtm, name: area: embeddings, name: auto:nit

  - Description: current code does not work very well on jupyter notebook, so I changed the code so that it imports `tqdm.auto` instead.
  - Issue: #9582 
  - Dependencies: N/A
  - Tag maintainer: @hwchase17, @baskaryan
  - Twitter handle: N/A
Replace `tqdm` with `tqdm.auto`
Labels: name: auto:improvement

### Feature request

```
if self.show_progress_bar:
    try:
        import tqdm

        _iter = tqdm.tqdm(range(0, len(tokens), _chunk_size))
    except ImportError:
        _iter = range(0, len(tokens), _chunk_size)
```
Current code does not work very well on jupyter notebook, so replace it with `tqdm.auto`

### Motivation

It just does not work very well on jupyter notebook, especially if there's any warning

### Your contribution

I'll make a PR
make test in experimental crash
Labels: name: auto:bug, name: area: models

### System Info

langchain version 0.0.270

### Who can help?

@hwchase17

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

cd lib/experimental
make test

```
============================================================================= short test summary info =============================================================================
FAILED tests/unit_tests/test_smartllm.py::test_all_steps - IndexError: list index out of range
FAILED tests/unit_tests/test_smartllm.py::test_intermediate_output - IndexError: list index out of range
========== 2 failed, 28 passed, 16 warnings in 4.37s =========
```


### Expected behavior

No error
HuggingFaceEndpoint Returns an Empty String
Labels: name: auto:bug, name: area: models

### System Info

HuggingFaceEndpoint Returns an Empty String both when prompted using ._call() method and when used as an LLM in a QA chain. Examples of what I've tried, below:

```
from langchain.llms import HuggingFaceEndpoint
from langchain.chains.question_answering import load_qa_chain
from langchain.vectorstores import FAISS
from langchain.embeddings import HuggingFaceEmbeddings

### Example 1 -- returns empty string

endpoint_url = (
    "my-real-endpoint-here"
)

llm = HuggingFaceEndpoint(
    endpoint_url = endpoint_url,
    huggingfacehub_api_token = os.environ['HUGGING_FACE_HUB_TOKEN'],
    task = 'text2text-generation',
    model_kwargs = {'temperature': 1e-20, "max_length": 900},
)

llm._call(prompt = "What is 4 + 4? Think the question through, step by step.")

>>> ""

### Example 2 -- used in QA chain and also returns empty string

### (exact same 'llm' object)

### text splitter
text_splitter = RecursiveCharacterTextSplitter(chunk_size = 1000, chunk_overlap = 15)

### embeddings
embeddings = HuggingFaceEmbeddings()

chain = load_qa_chain(llm, chain_type = "stuff", verbose = False)

def extract_from_case(file_str: str, prompt_question: str):

    docs = text_splitter.split_text(file_str)
    db = FAISS.from_texts(docs, embeddings)

    docs = db.similarity_search(prompt_question)
    extraction = chain.run(input_documents = docs, question = prompt_question)

    return extraction

sample_file = """Metadata: Date: 2017-01-18 File number: CEL-62600-16 CEL-62600-16 Citation: CEL-62600-16 (Re), 2017, retrieved on 2023-05-16. Content: Arrears Worksheet File Number: CEL-62600-16 Time period for Arrears Owing From: September 1, 2016 to"""

prompt = "What is the file number of this case?"

extract_from_case(file_str = sample_file, prompt_question = prompt)

>>> ""

```

### Who can help?

@hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

See above.

### Expected behavior

To return a non-empty response.
Additional/Popular chains Page Not Found
Labels: name: auto:bug, name: auto:documentation

### Issue with current documentation:

I'm getting this:
Page Not Found
We could not find what you were looking for.

Please contact the owner of the site that linked you to the original URL and let them know their link is broken.

When visiting: 
- https://python.langchain.com/docs/modules/chains/additional/
- https://python.langchain.com/docs/modules/chains/popular/

### Idea or request for content:

_No response_
Bagatur/confluence val
Labels: name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Use the GitHub-suggested safer pattern for shell interpolation.
Labels: name: auto:improvement

Using `${{ }}` to construct shell commands is risky, since the `${{ }}` interpolation runs first and ignores shell quoting rules. This means that shell commands that look safely quoted, like `echo "${{ github.event.issue.title }}"`, are actually vulnerable to shell injection.

More details here: https://github.blog/2023-08-09-four-tips-to-keep-your-github-actions-workflows-secure/
Set up concurrency groups and workflow cancelation in CI.
Labels: name: auto:improvement

If another push to the same PR or branch happens while its CI is still running, cancel the earlier run in favor of the next run.

There's no point in testing an outdated version of the code. GitHub only allows a limited number of job runners to be active at the same time, so it's better to cancel pointless jobs early so that more useful jobs can run sooner.
Adding thumbs-up and thumbs-down for documentations
Labels: name: auto:enhancement, name: auto:documentation

### Feature request

It might be very helpful to let users react with each page of the documentation so that contributors, users and people working will know which document has been out-dated / not usable.

### Motivation

I saw many inconsistencies in different docs and some codes are no longer usable as the dependancies update. It would be very help if there is an indicator of whether that documentation is still something we could reply on.

### Your contribution

I could contribute on the front-end but not sure how to configure it on langchain's database.
Add additional_kwargs to BasePromptTemplate, allow use in ChatPromptTemplate
Labels: name: auto:improvement

It can be useful to annotate templates with 3rd party metadata using `additional_kwargs` - this already works for `BaseStringMessagePromptTemplate` and `BaseMessage` but this wasn't yet wired up to `BasePromptTemplate`

I added this field and also allowed `ChatPromptTemplate.from_messages()` (derived from `BasePromptTemplate`) to pass on `**kwargs` so that you could use it in that way, also consistent with other classmethods in that file.

Tagging @hwchase17 and @eyurtsev for template/prompt stuff

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
using PromptTemplate with initialze_agent
Labels: name: area: agent, name: auto:bug

### System Info

I am trying to use a template to pre customize the AI, but my Code is not working! 


Any idea why it's not working?

This is the basic part of my code:

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

    template = """You are a customer service representative working for Amazon. You are having conversations with customers.
    
    
    When asked about your profession, you should respond that you are a customer service representative for Amazon.
    
    
    {memory}
    #Human: {human_input}
    #Chatbot:"""
    
    prompt = PromptTemplate(input_variables=["memory", "human_input"],template=template)
    
    memory = ConversationBufferMemory(memory_key="memory",prompt=prompt, return_messages=True)
    
    llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613",verbose=True)
        def get_agent(self):
            agent_kwargs = {
                "extra_prompt_messages": [MessagesPlaceholder(variable_name="memory")],
            }


        agent = initialize_agent(
            tools=self.tools,
            llm=llm,
            agent=AgentType.OPENAI_FUNCTIONS,
            verbose=True,
            agent_kwargs=agent_kwargs,
            memory=memory,
            prompt=prompt,
        )
        return agent

### Expected behavior

When I ask the AI what's her prefession is, she is telling 
"I am an AI assistant designed to provide helpful information and assist with various tasks" istead of
"you are a customer service representative for Amazon"
llama 2 through HF further integration with langchain on chat model
Labels: name: auto:enhancement, name: area: models

### Feature request

appreciate your efforts on show how integrate Hugginface models with langchain: https://python.langchain.com/docs/integrations/llms/huggingface_pipelines

but this is just a shallow port from huggingface to langchain not yet a full integration on the [Chat model](https://github.com/langchain-ai/langchain/tree/master/libs/langchain/langchain/chat_models) class that require further inherence and special case development. 

Can you guys spare some efforts to integrate the self-hosted version of llama 2 into the chat model?

 

### Motivation

How self-hosted llama 2 to fully integrate with langchain for avoid privacy and control the whole llm app development cycles. 

### Your contribution

I'm willing to help to submit PR with proper guidance.
Add function_call to FakeListChatModel
Labels: name: auto:improvement, name: area: models

This PR allows users to simulate function calls using the `FakeListChatModel` by passing in dictionary values as the responses.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Being able to retry a chain until the output is valid
Labels: name: auto:enhancement, name: area: models

### Feature request

The RetryOutputParser class has several drawbacks:
- it must point to a chain different from the one that calls it
- it does not make it possible to specify a number of retries

### Motivation

Following a request, ChatGPT sometimes answers with an invalid format and the same request must be resent to get the expected output.

It would be useful to have a new parameter added to a LLMChain (endowed with an output parser) to be able to retry the chain till the output is validated by the parser (i.e. it does not trigger any exception), with the possibility to specify a maximum of retries. It could take this form (just a proposal):
```
chain = LLMChain(llm=..., prompt=..., output_parser=..., retry=RetryChain(max_tries=10))
```

### Your contribution

Well, I don't think this feature would be complex to develop for the developers that already know the code. Let me know if this request makes sense.
Azure cognitive search retriever: ValidationError: 1 validation error for Document page_content
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.

I am using the Cognitive search retriever according to the documentation, however I run into an error, regarding the type of return value.

**Reproduction**:
import os
from langchain.retrievers import AzureCognitiveSearchRetriever
os.environ["AZURE_COGNITIVE_SEARCH_SERVICE_NAME"] = ""
os.environ["AZURE_COGNITIVE_SEARCH_INDEX_NAME"] = ""
os.environ["AZURE_COGNITIVE_SEARCH_API_KEY"] = ""

retriever = AzureCognitiveSearchRetriever(content_key="text", top_k=5)
retriever.get_relevant_documents(query="What is langchain?")

**Expected behaviour**: return relevant documents/snippets
**Actual behaviour**:
ValidationError: 1 validation error for Document
page_content
  str type expected (type=type_error.str)


It seems like the langchain retriever expects to receive somewhere a str, where azure cognitive search returns something else. Can anyone help/explain? The error message does not tell what type it receives and where the string should be, it would like to receive

### Suggestion:

_No response_
Release on push to `master` instead of on closed PRs targeting it.
Labels: name: auto:improvement

This is safer than the prior approach, since it's safe by default: the release workflows never get triggered for non-merged PRs, so there's no possibility of a buggy conditional accidentally letting a workflow proceed when it shouldn't have.

The only loss is that publishing no longer requires a `release` label on the merged PR that bumps the version. We can add a separate CI step that enforces that part as a condition for merging into `master`, if desirable.
Nc/small fixes 21aug
Labels: name: auto:nit, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Impossible to enrich BaseOutputParser with a custom object member
Labels: name: auto:bug, name: area: models

### System Info

Langchain 0.0.268
Python 3.9
Windows 10

### Who can help?

@hwchase17
@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.schema import BaseOutputParser

class CustomType:
	pass

class CustomParser(BaseOutputParser):
	model_config = ConfigDict(arbitrary_types_allowed=True)
	c : CustomType
```

### Expected behavior

This triggers a RuntimeError since no validator could be found for CustomType object. It prevents the user from passing custom types to the constructor of a langchain's output parser.

Though, this problem has been corrected in the latest versions of pydantic (thanks to the flag arbitrary_types_allowed). After investigating the source code, one of the mother classes of BaseOutputParser is Serializable and the latter derives from... pydantic.v1.BaseModel instead of pydantic.BaseModel. Why is that? This seems to be the source of the problem.