delayedkarma/langchain-issues huggingface dataset:
Issue: <search tool only uses its snippets to answer the question, that is not sufficient>

### search tool only uses its snippets to answer the question, that is not sufficient

I found that when calling search tools like 'bing search', 'google search' in an agent, these original api simply returns several urls and their snippets(main topic of the whole webpage). 
However, LangChain directly use these snippets to answer the question, like in bing search:
```python
def run(self, query: str) -> str:
        """Run query through BingSearch and parse result."""
        snippets = []
        results = self._bing_search_results(query, count=self.k)
        if len(results) == 0:
            return "No good Bing Search Result was found"
        for result in results:
            snippets.append(result["snippet"])

        return " ".join(snippets)
```
This concise message passed to the large language model is not sufficient to answer detailed questions, thus leading to bad answers.
Does anyone has a solution to this?

### Suggestion:

_No response_
Weird: Same prompt works with LLMChain but not with load_qa_with_sources Chain

### Issue you'd like to raise.

Custom Prompt formatted and used by `load_qa_with_sources` chain is not giving the answer and saying "Sorry, I am unable to answer" (message to be given in the prompt) but the same prompt when used with simple `LLMChain` or `LLM` directly gives back the expected answer.

1) custom prompt + retrieved documents provided to `load_qa_with_sources` (stuff) chain:
![Screenshot from 2023-06-13 10-22-04](https://github.com/hwchase17/langchain/assets/37807897/ba21feed-1207-4e22-86b1-733d107d6ca7)

2) Same prompt when used directly with `LLMChain`:
![Screenshot from 2023-06-13 10-21-48](https://github.com/hwchase17/langchain/assets/37807897/897de305-db29-4e09-b623-520d3095979b)

Why the same prompt is causing two different results if the underlying LLM is the same?

### Suggestion:

Following are the versions of the packages used for testing purposes:

- OpenAI: 0.27.8 and 0.27.2
- LangChain: 0.0.198 and 0.0.142
"Error in on_chain_start callback: 'name'" warning for tutorial "How to add Memory to an LLMChain"

### System Info

LangChain version: 0.0.198
Platform: Ubuntu 20.04 LTS
Python version: 3.10.4

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Reproduce tutorial [How to add Memory to an LLMChain](https://python.langchain.com/en/latest/modules/memory/examples/adding_memory.html)
2. Get the same warning for each invocation of llm_chain.predict():
```Error in on_chain_start callback: 'name'```
3. The same warning will be for the following different configurations (only difference from original code is specified):
3.1.```llm = ChatOpenAI(temperature=0)```
3.2 ```chain = ConversationChain(
    memory=memory,
    verbose=True,
    prompt=prompt_template,
    llm=llm,
    input_key='human_input'
)```
3.3 ```memory = ConversationBufferMemory(memory_key='chat_history', return_messages=True)```

### Expected behavior

There will be no such warnings (or the actions which resolved it must be specified in warning and/or in documentation).
Enhancement: Multiple calls of MapReduceDocumentsChain should run asynchronously

### Issue you'd like to raise.

When executing `arun()` of the `MapReduceDocumentsChain` multiple times in an async loop, the requests are not run concurrently. 

Given the example: 

```python
import asyncio
from langchain.llms import OpenAI
from langchain.chains.summarize import load_summarize_chain
from langchain.schema import Document

summary_chain = load_summarize_chain(
    OpenAI(temperature=0),
    chain_type="map_reduce",
)

docs = [
    Document(page_content="A doc"),
    Document(page_content="Another doc"),
]

tasks = [
    summary_chain.arun({"input_documents": docs}),
    summary_chain.arun({"input_documents": docs})
]

await asyncio.gather(*tasks)
```

While the mapping step is run asynchronously, the combine step is not. We can see that that the last two requests run sequentially:

<img width="817" alt="image" src="https://github.com/hwchase17/langchain/assets/872712/0472a000-4eb2-43bd-8f97-2e66869c857c">

This is due to the `async acombine()` method also calling the synchronous _process_results() method.

```python
async def acombine_docs(…):
    …
    return self._process_results(...)

def _process_results(…):
    ...
```




### Suggestion:

## How I can contribute
Will provide a PR! 

The fact that an redundant mapping call is made in case the `input_documents` array contains a single document, is addressed in https://github.com/hwchase17/langchain/pull/5942

@agola11
For web page loaders

### Feature request

It will be really cool if we can specify a website like : www.example.edu/* and it would get all the possible accessible pages from it and load it using WebBaseLoader.

### Motivation

I'm always frustrated when I have to make a list of websites and pass it in the loader. Instead, using this way we can query the loader and we can get all the relevant web pages. 

### Your contribution

Not sure how I can contribute.
Issue: HuggingFaceEmbeddings can not take trust_remote_code argument

### HuggingFaceEmbeddings can not take trust_remote_code argument 

![image](https://github.com/hwchase17/langchain/assets/48760936/5400f6ca-d06b-49f0-a889-68f4e8e41598)


### Suggestion:

_No response_
Issue: Can't load a public webpage

### I want to load in the webpage below.

Hi,
Trying to extract some webpage using webbaseloader:
"""
loader = WebBaseLoader("https://researchadmin.asu.edu/)
data = loader.load()
"""
But gives the following error:
SSLError: HTTPSConnectionPool(host='researchadmin.asu.edu', port=443): Max retries exceeded with url: / (Caused by 
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get 
local issuer certificate (_ssl.c:1002)')))

It is a public web page. Can anyone help?

### Suggestion:

_No response_
unknown command `MODULE`, with args beginning with: `LIST`, 

### System Info

when trying to connect to azure redis I get the following error:
unknown command `MODULE`, with args beginning with: `LIST`, 
Here is the code:

fileName = "somefile.pdf"
loader = PyPDFLoader(fileName)
docs = loader.load_and_split()
redis_conn = f"rediss://:{utils.REDIS_PWD}@{utils.REDIS_HOST}:{utils.REDIS_PORT}"
rds = Redis.from_documents(docs, embeddings, redis_url=redis_conn, index_name='link')


Important:
Connecting to redis like this, works!
r = redis.StrictRedis(host=utils.REDIS_HOST,port=utils.REDIS_PORT, db=0, password=utils.REDIS_PWD, ssl=True)
r.set('foo', 'bar')
r.get('foo')

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This works:
import redis
r = redis.StrictRedis(host=utils.REDIS_HOST,port=utils.REDIS_PORT, db=0, password=utils.REDIS_PWD, ssl=True)
r.set('foo', 'bar')
r.get('foo')


This does not:

from langchain.document_loaders import PyPDFLoader # for loading the pdf
from langchain.vectorstores.redis import Redis
from langchain.embeddings import OpenAIEmbeddings


embeddings = OpenAIEmbeddings(model=OPENAI_EMBEDDING_MODEL_NAME, chunk_size=1)
fileName = "somefile.pdf"
loader = PyPDFLoader(fileName)
docs = loader.load_and_split()
redis_conn = f"rediss://:{utils.REDIS_PWD}@{utils.REDIS_HOST}:{utils.REDIS_PORT}"
rds = Redis.from_documents(docs, embeddings, redis_url=redis_conn, index_name='link')



### Expected behavior

save embeddings into REdis
Improve AnalyticDB VectorStore implementation without affecting user behavior

Hi there:

As I implement the AnalyticDB VectorStore use two table to store the document before. It seems just use one table is a better way after our user try to use it. So this commit is try to improve AnalyticDB VectorStore implementation without affecting user behavior behavior:

1) Streamline the __post_init__ behavior by creating a single table with automatic indexing
2) Update the add_texts API for document insertion 
3) Optimize similarity_search_with_score_by_vector to retrieve results directly from the table. 
4) Add an embedding_dimension parameter to support different dimension embedding functions.
Users can continue using the API as before. And the test cases added before is enough to meet this commit.
Issue: chroma where_document parameter passed in search_kwargs is invalid

### Issue you'd like to raise.

I want to use the chroma where_document parameter to do precise filtering of the content during data retrieval, but I found that the where_document parameter is not passed after the similarity_search_by_vector method is finally called in the source code
![image](https://github.com/hwchase17/langchain/assets/122618171/acada970-0432-465b-9541-a6214c50f773)


### Suggestion:

I hope that I can generate where_document in the retrieval parameter of the retriever, just like the existing filter will be passed to where, the code is as follows：

```py
search_kwargs = {
    "k": k,
    "filter": filter,
    "where_document": {"$contains": "1000001"}
}
retriever = vectordb.as_retriever(
    search_kwargs=search_kwargs
)
```
Langchain rate limit error while invoking OpenAI API

### System Info

## Description
While using the Langchain application, I am frequently encountering an error that relates to rate limiting when invoking OpenAI's API. This tends to occur when I try to perform multiple translations consecutively or concurrently, causing a significant interruption to the user experience.


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

## Steps to Reproduce
1. Open the Langchain application.
2. Select a source language and enter some text for translation.
3. Choose the target language and submit for translation.
4. Repeat steps 2-3 multiple times in quick succession or concurrently.

### Expected behavior

## Expected Behavior
The application should be able to handle multiple translation requests without any disruptions, including but not limited to rate limit errors from OpenAI's API. 

## Actual Behavior
When submitting multiple translation requests quickly or at the same time, a rate limit error is produced and no translations are returned. The error message is as follows:

`Error: OpenAI API rate limit exceeded`

How to implement retry logic with LangChain?
Allow FAISS score_threshold
Labels: name: lgtm

Allow FAISS's similarity_search_with_score_by_vector to accept kwargs, specifically: `score_threshold`.
Fixes error for document compression users in cases where there are no relevant docs - for instance:  

```
human_message: "hi" (<--  will return no relevant docs depending on score_threshold)
> IndexError: index 0 is out of bounds for axis 0 with size 0
```

Combined with the new merger_retriever, FAISS users can now perform more granular vectorstore retrieving based on score_threshold:  

```
store_1 = st[0][1].as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": .8, "k": 3})
store_2 = st[1][1].as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": .3, "k": 3})
store_3 = st[2][1].as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": .5, "k": 3})

merged = MergerRetriever(retrievers=[store_1, store_2, store_3])
pipeline_compressor = prep_compressor()
compression_retriever = ContextualCompressionRetriever(base_compressor=pipeline_compressor, base_retriever=merged)

chain = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), chain_type="stuff", retriever=compression_retriever)
```

@dev2049 or @hwchase17 would appreciate a check if available. Thanks.
Update output format of MosaicML endpoint to be more flexible

There will likely be another change or two coming over the next couple weeks as we stabilize the API, but putting this one in now which just makes the integration a bit more flexible with the response output format.

```
(langchain) danielking@MML-1B940F4333E2 langchain % pytest tests/integration_tests/llms/test_mosaicml.py tests/integration_tests/embeddings/test_mosaicml.py 
=================================================================================== test session starts ===================================================================================
platform darwin -- Python 3.10.11, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/danielking/github/langchain
configfile: pyproject.toml
plugins: asyncio-0.20.3, mock-3.10.0, dotenv-0.5.2, cov-4.0.0, anyio-3.6.2
asyncio: mode=strict
collected 12 items                                                                                                                                                                        

tests/integration_tests/llms/test_mosaicml.py ......                                                                                                                                [ 50%]
tests/integration_tests/embeddings/test_mosaicml.py ......                                                                                                                          [100%]

=================================================================================== slowest 5 durations ===================================================================================
4.76s call     tests/integration_tests/llms/test_mosaicml.py::test_retry_logic
4.74s call     tests/integration_tests/llms/test_mosaicml.py::test_mosaicml_llm_call
4.13s call     tests/integration_tests/llms/test_mosaicml.py::test_instruct_prompt
0.91s call     tests/integration_tests/llms/test_mosaicml.py::test_short_retry_does_not_loop
0.66s call     tests/integration_tests/llms/test_mosaicml.py::test_mosaicml_extra_kwargs
=================================================================================== 12 passed in 19.70s ===================================================================================
```

#### Who can review?

  @hwchase17
  @dev2049
Issue: LangChain Plus Api Key

### Issue you'd like to raise.

Started a local server just to learn more about with `langchain plus start`. It says that I can generate a new api as it appears to have an endpoint in port 1984. But when I click the menu there's no **Settings** option there.

---

<img width="1787" alt="Screenshot 2023-06-12 at 16 32 31" src="https://github.com/hwchase17/langchain/assets/3484029/e42ef803-f216-486a-84f3-6c4026a37a8f">


### Suggestion:

_No response_
Issue: ModuleNotFoundError: No module named 'langchain.callbacks.shared'

### Issue you'd like to raise.

Hello,

I am having trouble running my code due to an error that I encountered. The error message that I received is "ModuleNotFoundError: No module named 'langchain.callbacks.shared'". I am not sure how to resolve this issue and would appreciate any help or guidance.

This line of code is using the `pickle` module to load data from a file. The `pickle.load(file)` function reads the pickled representation of an object from the open file object `file` and returns the reconstituted object hierarchy specified therein.

In this case, the returned object is expected to be a tuple with two elements, which are assigned to the variables `self.chain` and `self.vectorstore`, respectively. This means that the first element of the tuple is assigned to `self.chain`, and the second element is assigned to `self.vectorstore`.


Here is some information about my environment that may be helpful:
- OS: Ubuntu
- Python version: 3.10
- Langchain Version: 0.0.198

Issue Screen Shot:

<br />

![Screenshot from 2023-06-12 23-55-27](https://github.com/hwchase17/langchain/assets/81419892/33e0bdd1-2fd0-4570-ab6e-6100c1211eb7)




Thank you in advance for your help and support in resolving this issue.



### Suggestion:

_No response_
Replace self-built devcontainer with default python image
Labels: name: auto:improvement

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

This replaces the custom docker image for devcontainer with the official python one with the `poetry` feature and `black-formatter` extension added, because:
- The current one is broken during building when opening in codespaces. I tried a few times and it errors out during poetry install colored for some reasons.
- It's faster to build and load when opening in codespaces.
- It's easier to maintain as it's composed from existing things pre-built by others.

#### Who can review?

Tag maintainers/contributors who might be interested:  - @vowelparrot 

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Implement `max_marginal_relevance_search` in `VectorStore` of Pinecone

This adds implementation of MMR search in pinecone; and I have two semi-related observations about this vector store class:
- Maybe we should also have a `similarity_search_by_vector_returning_embeddings` like in supabase, but it's not in the base `VectorStore` class so I didn't implement
- Talking about the base class, there's `similarity_search_with_relevance_scores`, but in pinecone it is called `similarity_search_with_score`; maybe we should consider renaming it to align with other `VectorStore` base and sub classes (or add that as an alias for backward compatibility)

#### Who can review?

Tag maintainers/contributors who might be interested:
 - VectorStores / Retrievers / Memory - @dev2049
When using MongoDBAtlasVectorSearch i get KeyError: 'text' despite having the collection populated

### System Info

Python version: 3.10.11 (main, May 17 2023, 14:30:36) [Clang 14.0.6 ]
pymongo                  4.3.3
langchain                0.0.190


### Who can help?

@eyurtsev @agola11  

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [x] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I followed religiously the following tutorial:

https://python.langchain.com/en/stable/modules/indexes/vectorstores/examples/mongodb_atlas_vector_search.html

Despite that I get the following error.

```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[14], line 12
     10 # perform a similarity search between the embedding of the query and the embeddings of the documents
     11 query = "my query?"
---> 12 docs = vectorStore.similarity_search(query)

File ~/opt/miniconda3/envs/LLM_langchain_exploration/lib/python3.10/site-packages/langchain/vectorstores/mongodb_atlas.py:222, in MongoDBAtlasVectorSearch.similarity_search(self, query, k, pre_filter, post_filter_pipeline, **kwargs)
    194 def similarity_search(
    195     self,
    196     query: str,
   (...)
    200     **kwargs: Any,
    201 ) -> List[Document]:
    202     """Return MongoDB documents most similar to query.
    203 
    204     Use the knnBeta Operator available in MongoDB Atlas Search
   (...)
    220         List of Documents most similar to the query and score for each
    221     """
--> 222     docs_and_scores = self.similarity_search_with_score(
    223         query,
    224         k=k,
    225         pre_filter=pre_filter,
    226         post_filter_pipeline=post_filter_pipeline,
    227     )
    228     return [doc for doc, _ in docs_and_scores]

File ~/opt/miniconda3/envs/LLM_langchain_exploration/lib/python3.10/site-packages/langchain/vectorstores/mongodb_atlas.py:189, in MongoDBAtlasVectorSearch.similarity_search_with_score(self, query, k, pre_filter, post_filter_pipeline)
    187 docs = []
    188 for res in cursor:
--> 189     text = res.pop(self._text_key)
    190     score = res.pop("score")
    191     docs.append((Document(page_content=text, metadata=res), score))

KeyError: 'text'
```

### Expected behavior

should get the docs back
feat: Add support for the Solidity language

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

## Add Solidity programming language support for code splitter.

Twitter: @0xjord4n_

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->
#### Who can review?

Tag maintainers/contributors who might be interested:
@hwchase17

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->