delayedkarma/langchain-issues huggingface dataset:
FAISS vectorstore `similarity_search_with_relevance_scores` returns strange/false result with `MAX_INNER_PRODUCT`
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

langchain==0.0.268
python=3.9
mac

### Who can help?

I used the FAISS as the vector store. it seems that the `similarity_search_with_score` (supposedly ranked by distance: low to high) and `similarity_search_with_relevance_scores`((supposedly ranked by relevance: high to low) produce conflicting results when specifying `MAX_INNER_PRODUCT` as the distance strategy. Please see the screenshot below:
<img width="675" alt="Screenshot 2023-08-20 at 6 54 15 PM" src="https://github.com/langchain-ai/langchain/assets/7220686/88981277-a0b4-462b-929c-63bd19d4faff">


### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.vectorstores.faiss import FAISS
from langchain.embeddings import HuggingFaceEmbeddings

embedding_engine = HuggingFaceEmbeddings(
                model_name= "BAAI/bge-base-en", #"sentence-transformers/all-mpnet-base-v2",
                model_kwargs={"device": "cpu"},
                encode_kwargs={'normalize_embeddings': True} # set True to compute cosine similarity
            )
texts = ["I like apples", "I hate apples", "I like oranges"]
simple_vecdb = FAISS.from_texts(texts, 
                                embedding_engine,
                                distance_strategy="MAX_INNER_PRODUCT")

# test 1
simple_vecdb.similarity_search_with_score("I like apples")

# test 2
simple_vecdb.similarity_search_with_relevance_scores("I like apples")

### Expected behavior

For similarity_search_with_score, if the `similarity_search_with_score` documentation is correct saying that "List of documents most similar to the query text with L2 distance in float. Lower score represents more similarity.", then the result from `similarity_search_with_score` should have matched the identical text `I like apple` with `(distance)score=0` and the `similarity_search_with_relevance_scores` with `(relevance)score=1`, no?

if the FAISS return the cosine similarity score already during the `similarity_search_with_score`call, should the `_max_inner_product_relevance_score_fn` for the FAISS just return the identical score instead of `1-score` when calculating the relevance score?
Anthropic: Allow the use of kwargs consistent with ChatOpenAI.
Labels: name: lgtm, name: auto:improvement, name: area: models

  - Description: ~~Creates a new root_validator in `_AnthropicCommon` that allows the use of `model_name` and `max_tokens` keyword arguments.~~ Adds pydantic field aliases to support `model_name` and `max_tokens` as keyword arguments. Ultimately, this makes `ChatAnthropic` more consistent with `ChatOpenAI`, making the two classes more interchangeable for the developer.
  - Issue: https://github.com/langchain-ai/langchain/issues/9510
Document loaders confluence keep newlines
Labels: name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
missing model_name param in ChatAnthropic
Labels: name: auto:bug, name: area: models

### System Info

Python 3.8.16
langchain==0.0.268

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`ChatAnthropic` doesn't have `model_name` attribute. instead, it has a `model` attribute that specifies the name. Other chat models such as `ChatOpenAI`, `ChatVertexAI` has `model_name` attribute, this breaks the interface while integrating with multiple LLMs.

### Expected behavior

`llm = ChatAnthropic(model_name='claude-2')`
`print(llm.model_name)`
TimeOutError unhandled
Labels: name: auto:bug, name: auto:improvement, name: area: models

### System Info

`ValueError                                Traceback (most recent call last)
Cell In[35], line 3
      1 query = "What did the president say about the Supreme Court"
      2 docs = db.similarity_search(query)
----> 3 chain.run(input_documents=docs, question=query)

File [~/.local/lib/python3.10/site-packages/langchain/chains/base.py:480](https://file+.vscode-resource.vscode-cdn.net/home/moghadas/toy_projects/~/.local/lib/python3.10/site-packages/langchain/chains/base.py:480), in Chain.run(self, callbacks, tags, metadata, *args, **kwargs)
    475     return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
    476         _output_key
    477     ]
    479 if kwargs and not args:
--> 480     return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
    481         _output_key
    482     ]
    484 if not kwargs and not args:
    485     raise ValueError(
    486         "`run` supported with either positional arguments or keyword arguments,"
    487         " but none were provided."
    488     )

File [~/.local/lib/python3.10/site-packages/langchain/chains/base.py:282](https://file+.vscode-resource.vscode-cdn.net/home/moghadas/toy_projects/~/.local/lib/python3.10/site-packages/langchain/chains/base.py:282), in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, include_run_info)
    280 except (KeyboardInterrupt, Exception) as e:
    281     run_manager.on_chain_error(e)
--> 282     raise e
...
    113 if self.client.task == "text-generation":
    114     # Text generation return includes the starter text.
    115     text = response[0]["generated_text"][len(prompt) :]

ValueError: Error raised by inference API: Model google/flan-t5-xl time out`

This exception should be handled

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`query = "What did the president say about the Supreme Court"
docs = db.similarity_search(query)
chain.run(input_documents=docs, question=query)`

### Expected behavior

Retry mechanism, or caching
LLM Chain: Unable to run OpenLM - unexpected keyword argument 'api_key' in Completion.create class
Labels: name: auto:bug, name: area: models

### System Info

! pip show langchain
Name: langchain
Version: 0.0.268
Summary: Building applications with LLMs through composability
Home-page: https://www.github.com/hwchase17/langchain

! pip show openlm
Name: openlm
Version: 0.0.5
Summary: Drop-in OpenAI-compatible that can call LLMs from other providers 


Python 3.11.3

### Who can help?

@hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`
! pip install openlm
! pip install langchain

from getpass import getpass
import os
from langchain.llms import OpenLM
from langchain import PromptTemplate, LLMChain

os.environ['OPENAI_API_KEY'] = "<openai-api-key>"
os.environ['HF_API_TOKEN'] = "<hf-api-key>"

if "OPENAI_API_KEY" not in os.environ:
    print("Enter your OpenAI API key:")
    os.environ["OPENAI_API_KEY"] = getpass()

if "HF_API_TOKEN" not in os.environ:
    print("Enter your HuggingFace Hub API key:")
    os.environ["HF_API_TOKEN"] = getpass()


question = "What is the capital of France?"
template = """Question: {question} Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

for model in ["text-davinci-003", "huggingface.co/gpt2"]:
    llm = OpenLM(model=model)
    llm_chain = LLMChain(prompt=prompt, llm=llm)
    result = llm_chain.run(question)
    print( """Model: {} Result: {}""".format(model, result))
`

When I run the above code, I get the following error:

TypeError: Completion.create() got an unexpected keyword argument 'api_key'


Can I anyone help me to fix this issue ?

### Expected behavior

As mentioned the below document,

https://python.langchain.com/docs/integrations/llms/openlm

Model: text-davinci-003
Result:  France is a country in Europe. The capital of France is Paris.
Model: huggingface.co/gpt2
Result: Question: What is the capital of France?

Answer: Let's think step by step. I am not going to lie, this is a complicated issue, and I don't see any solutions to all this, but it is still far more
Make QAGenerationChain.from_llm use SimpleJsonOutputParser
Labels: name: auto:improvement, name: area: models

  - Description: Gives `QAGenerationChain.from_llm` an `output_parser` keyword argument; adds default `SimpleJsonOutputParser`
  - Issue: Default prompts request structured json, but no output parser is used. In `QAGenerationChain._call`, after generation, `json.loads` raises a `JSONDecodeError`.

Edit: It occurs to me that perhaps this should be a `PydanticOutputParser` or similar, and in fact I'm not sure it solves the original problem. :( Closing this for now, but open to discussion.
Added misssing model name to GPT4AllEmbeddings.
Labels: name: area: embeddings, name: auto:improvement

  - Description:  Added model name to `GPT4AllEmbeddings`. Apparently, when we use `CacheBackedEmbeddings`, we must provide a namespace so our doc embeddings with different model ids remain distinct. This change makes it easy to add name from the model.  
https://python.langchain.com/docs/modules/data_connection/caching_embeddings
  - Dependencies: None
  - Tag maintainer: @baskaryan
:bug: fix unexpected run of release workflow
Labels: name: auto:bug

I have discovered a bug located within `.github/workflows/_release.yml` which is the primary cause of continuous integration (CI) errors. The problem can be solved; therefore, I have constructed a PR to address the issue.

## The Issue

Access the following link to view the exact errors: [Langhain Release Workflow](https://github.com/langchain-ai/langchain/actions/workflows/langchain_release.yml)

The instances of these errors take place for **each PR** that updates `pyproject.toml`, excluding those specifically associated with bumping PRs.

See below for the specific error message:

```
Error: Error 422: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
```

An image of the error can be viewed here:
![Image](https://github.com/langchain-ai/langchain/assets/13769670/13125f73-9b53-49b7-a83e-653bb01a1da1)

The `_release.yml` document contains the following if-condition:

```yaml
    if: |
        ${{ github.event.pull_request.merged == true }}
        && ${{ contains(github.event.pull_request.labels.*.name, 'release') }}
```

## The Root Cause

The above job constantly runs as the `if-condition` is always identified as `true`.

## The Logic

The `if-condition` can be defined as `if: ${{ b1 }} && ${{ b2 }}`, where `b1` and `b2` are boolean values. However, in terms of condition evaluation with GitHub Actions, `${{ false }}` is identified as a string value, thereby rendering it as truthy as per the [official documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idif).

I have run some tests regarding this behavior within my forked repository. You can consult my [debug PR](https://github.com/zawakin/langchain/pull/1) for reference.

Here is the result of the tests:

|If-Condition|Outcome|
|:--:|:--:|
|`if: true && ${{ false }}`|Execution|
|`if: ${{ false }}` |Skipped|
|`if: true && false` |Skipped|
|`if: false`|Skipped|
|`if: ${{ true && false }}` |Skipped|

In view of the first and second results, we can infer that `${{ false }}` can only be interpreted as `true` for conditions composed of some expressions.
It is consistent that the condition of `if: ${{ inputs.working-directory == 'libs/langchain' }}` works.

It is surprised to be skipped for the second case but it seems the spec of GitHub Actions 😓 

Anyway, the PR would fix these errors, I believe 👍 

Could you review this? @hwchase17 or @shoelsch , who is the author of [PR](https://github.com/langchain-ai/langchain/pull/360).
docs(readme): fixed badges with new github url
Labels: name: auto:documentation

Mainly created for the code space url that was broken but fixed the others in the same PR.

Now

<img width="941" alt="image" src="https://github.com/langchain-ai/langchain/assets/19614572/db918615-a9b9-4a47-96a4-1b709aa7c041">


Before 

<img width="1802" alt="image" src="https://github.com/langchain-ai/langchain/assets/19614572/3c831729-692c-44aa-9375-0ac012f6e5fa">
Issue: RetrievalQA runs twice faster than huggingface model call
Labels: name: area: vector store, name: auto:question, name: area: models

### Issue you'd like to raise.

I have the following code:

```
pipe = transformers.pipeline(
    model=llm_model,
    tokenizer=tokenizer,
    return_full_text=True, 
    task='text-generation',
    temperature=0.2,
    max_new_tokens=200
)
llm = HuggingFacePipeline(pipeline=pipe)

retriever = vector_db.as_retriever(search_kwargs={"k": 4})
qa = RetrievalQA.from_chain_type(llm=llm,
                                 chain_type="stuff",
                                 retriever=retriever,
                                 return_source_documents=True)
```

When I call qa(query) it executes in about 24 seconds. But when I call llm with the same query and context obtained from retriever.get_relevant_documents method it gives me different result (but still relevant) and executes in 39 seconds, almost twice as long as qa. So i would like to understand what happens under the hood when I call RetrievalQA and why results are so different in speed.

### Suggestion:

_No response_
chatbots use case - fix broken collab URL
Labels: name: auto:documentation

The current Collab URL returns a 404, since there is no `chatbots` directory under `use_cases`.

<!-- Thank you for contributing to LangChain!

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Fix typo in apis.ipynb
Labels: name: auto:documentation

funtions -> functions

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Issue: RunnableWithFallbacks: Can't instantiate abstract class BaseLanguageModel ???
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I bumped langchain to version 0.0.268 and encountered below error while implementing 'with_fallbacks'.

Error:
pydantic.error_wrappers.ValidationError: 1 validation error for LLMChain
llm
  Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)

Sample Code:
chat_llm = ChatOpenAI(model_name='gpt-3.5-turbo', 
                              temperature=0,
                              model_kwargs={'top_p': 0.95}, 
                              max_tokens=512,
                              streaming=False,
                              verbose=False
                              )
fallback_llm = ChatOpenAI(model_name='gpt-4', 
                              temperature=0,
                              model_kwargs={'top_p': 0.95}, 
                              max_tokens=512,
                              streaming=False,
                              verbose=False
                              )
chat_llm.with_fallbacks([fallback_llm], exceptions_to_handle=(Exception,))

Pydantic version: 1.10.12

Is this because of pydantic migration? Any immediate fixes or should i wait till Aug 25th for the migration to complete?

### Suggestion:

_No response_
Problem with embedding model 
Labels: name: area: embeddings, name: auto:bug

### Issue you'd like to raise.

I use embedding model from huggingface vinai/phobert-base:
![image](https://github.com/langchain-ai/langchain/assets/90046327/a5fef433-ae15-4d30-abf0-0fe14f1ca7db)
Then it has this problem:
WARNING:sentence_transformers.SentenceTransformer:No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/vinai_phobert-base. Creating a new one with MEAN pooling.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

### Suggestion:

_No response_
Invoke Multiple tools
Labels: name: area: agent, name: area: vector store, name: auto:improvement

### Issue you'd like to raise.

In my problem statement I am defining multiple tools which are all based on retrieval chain and I am using OpenAiAgent as the agent  to call this tools.
So when I am running a query it invokes to a single vectorstore. But I need my agent to access different vectorstores if the answer is not fetched from that particular vectorstore.I know the approach to get this working done by other agents.But I need this to be done by openaiAgent.

### Suggestion:

_No response_
feat: Add PromptGuard integration
Labels: name: auto:enhancement, name: area: models

 Add PromptGuard integration
-------
There are two approaches to integrate PromptGuard with a LangChain application.

1. PromptGuardLLMWrapper
2. functions that can be used in LangChain expression.

-----
- Dependencies
`promptguard` python package, which is a runtime requirement if you'd try out the demo.

-  @baskaryan @hwchase17 Thanks for the ideas and suggestions along the development process.
RunnableWithFallbacks
Labels: name: area: agent, name: auto:improvement

### System Info

Name: langchain
Version: 0.0.268
Python 3.10.10

### Who can help?

@hwchase17
@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

`
from langchain import PromptTemplate, OpenAI, LLMChain

prompt_template = "You're a helpful assistant that answers all questions?"

llm1 = OpenAI(model="foo", temperature=0)
llm_chain1 = LLMChain(
    llm=llm1,
    prompt=PromptTemplate.from_template(prompt_template)
)

llm2 = OpenAI(temperature=0)
llm_chain2 = LLMChain(
    llm=llm2,
    prompt=PromptTemplate.from_template(prompt_template)
)

llm_chain = llm_chain1.with_fallbacks([llm_chain2])

search_desc = "Use this tool to answer user questions when asked to Search the Web."

prefix = """ You're a helpful Assistant Chatbot. """
suffix = """{chat_history} {input} {agent_scratchpad}"""

from langchain.utilities import SerpAPIWrapper
from langchain.agents import (
    ZeroShotAgent,
    Tool,
    AgentExecutor,
)

search = SerpAPIWrapper()

tools = [
        Tool(
            func=search.run, description=search_desc, name="Search_the_web"
        )
    ]

input_variables = ["input", "chat_history", "agent_scratchpad"]
prompt = ZeroShotAgent.create_prompt(
        tools,
        prefix=prefix,
        suffix=suffix,
        input_variables=input_variables,
    )
tool_names = [tool.name for tool in tools]

agent = ZeroShotAgent(
        llm_chain=llm_chain,
        allowed_tools=tool_names,
        verbose=True,
        handle_parsing_errors="ignore",
    )

response = AgentExecutor.from_agent_and_tools(
        agent=agent, tools=tools, verbose=True
    )

`

### Expected behavior

To be able to pass RunnableWithFallbacks as an alternative to LLMChain in ZeroShotAgent.
Synthetic Data
Labels: name: lgtm, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->