delayedkarma/langchain-issues huggingface dataset:
docs: commented out `editUrl` option

docs: commented out the `editUrl` option

This removes the “Edit this page” link at the end of pages. See this link on this [page](https://python.langchain.com/docs/modules/data_connection/retrievers/), for example.

This link does not work.

New refactoring makes the docs structure the "hand-make" structure. The compiled pages do not map directly to the original page files. There should be a manually supported dictionary to map pages which is not here now.

  - @hwchase17
  - @dev2049
cannot import name 'create_citation_fuzzy_match_chain' from 'langchain.chains'

### System Info


ImportError: cannot import name 'create_citation_fuzzy_match_chain' from 'langchain.chains'

Python=3.11.4
Langchain=0.0.129

The code:
`from langchain.chains import create_citation_fuzzy_match_chain`

The error

> ImportError: cannot import name 'create_citation_fuzzy_match_chain' from 'langchain.chains'

### Who can help?

_No response_

### Information

- [x] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [x] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.chains import create_citation_fuzzy_match_chain

### Expected behavior

ImportError: cannot import name 'create_citation_fuzzy_match_chain' from 'langchain.chains
DOC : Load model from local path in HuggingFaceInstructEmebddings

### Issue with current documentation:

Hey i had a question which waqs bugging me ! I need to load a hugging face model for local path rather than loading it for the first time in HuggingFcaeInstructEmebddings. Can anybdoy tell me how we can do that ??

### Idea or request for content:

_No response_
Motorhead memory updates, including delete session method and ability to save intermediate_steps

A few small updates made in this PR, all related to memory, with a focus on motorhead for now:

* Delete session method was missing, so I added one in
* Saving intermediate_steps into memory was not permitted, so I added it as a property to the BaseChatMemory so that it doesn't conflict with the return statements of _get_input_output
* I added a new type in the schema, called AgentMessage, which is responsible for creating the intermediate_steps
* I added a to_dict() method to AgentAction to enable serialization.
Issue:  Related to open api spec agent default prompts

### Issue you'd like to raise.

Issue:  Is there a way to modify these default api prompts while using open api spec agent. Could someone please guide https://github.com/hwchase17/langchain/blob/master/langchain/agents/agent_toolkits/openapi/planner_prompt.py#LL6C1-L7C1

Or should I right my own custom open api spec agent? 
Thanks!

### Suggestion:

_No response_
Update arize_callback.py

Arize released a new Generative LLM Model Type, adjusting the callback function to new logging.

Added arize imports, please delete if not necessary.

Specifically, this change makes sure that the prompt and response pairs from LangChain agents are logged into Arize as a Generative LLM model, instead of our previous categorical model. In order to do this, the callback functions collects the necessary data and passes the data into Arize using Python Pandas SDK.

Arize library, specifically pandas.logger is an additional dependency.

Notebook For Test: https://docs.arize.com/arize/resources/integrations/langchain

Who can review?
Tag maintainers/contributors who might be interested:

@hwchase17 - project lead

Tracing / Callbacks

@agola11
ChatPromptTemplate with partial variables is giving validation error

### System Info

langchain-0.0.205, python3.10

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Write this into Notebook cell
2. `from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
chat_prompt = ChatPromptTemplate(
                messages=[
                    HumanMessagePromptTemplate.from_template("Do something with {question} using {context} giving it like {formatins}")
                ],
                input_variables=["question", "context"],
                partial_variables={"formatins": "some structure"}
)
`

3. It it throwing following error:
`Error:
ValidationError: 1 validation error for ChatPromptTemplate __root__ Got mismatched input_variables. Expected: {'formatins', 'question', 'context'}. Got: ['question', 'context'] (type=value_error)`

4. This was working until 24 hours ago. Potentially related to recent commit to langchain/prompts/chat.py. 

### Expected behavior

The chat_prompt  should get created with the partial variables injected.
If this is expected change, can you please help with suggesting what should be the new way to use partial_variables?

Thanks
Fix Custom LLM Agent example
Labels: name: lgtm

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

The `CustomOutputParser` needs to throw `OutputParserException` when it fails to parse the response from the agent, so that the executor can [catch it and retry](https://github.com/hwchase17/langchain/blob/be9371ca8f363bf1c748ac4af7fb4a0d75a365c5/langchain/agents/agent.py#L767) when `handle_parsing_errors=True`.

<!-- Remove if not applicable -->

#### Who can review?

Tag maintainers/contributors who might be interested: @hwchase17

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Structured Tools don't work with OpenAI's new functions

### System Info

Python 3.11
Langchain 201

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [x] Agents / Agent Executors
- [x] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I created a Structured tool, including definitions with Pydantic, and want to use it with an `OpenAIFunctionsAgent`.
I create the tool with `StructuredTool.from_function(my_func)`.

OpenAI doesn't use the schema correctly. After a bit of digging I realized that the "pydantic schema" is provided to the openAI call instead of the "JSON schema".

In `langchain.tools.convert_to_openai.py` see:
```python
def format_tool_to_openai_function(tool: BaseTool) -> FunctionDescription:
    """Format tool into the open AI function API."""
    if isinstance(tool, StructuredTool):
        schema_ = tool.args_schema.schema() # <============================== HERE
        # Bug with required missing for structured tools.
        required = sorted(schema_["properties"])  # BUG WORKAROUND
        return {
            "name": tool.name,
            "description": tool.description,
            "parameters": {
                "type": "object",
                "properties": schema_["properties"],
                "required": required,
            },
        }
 ...
```

I got it to successfully work with this code instead, getting the JSON schema from pydantic instead of the "pydantic schema".
With this openai correctly parses the schema and the tool functions as expected.
```python
def format_tool_to_openai_function(tool: BaseTool) -> FunctionDescription:
    """Format tool into the open AI function API."""
    if isinstance(tool, StructuredTool):
        
        schema_ = tool.args_schema.schema_json()
        return {
            "name": tool.name,
            "description": tool.description,
            "parameters": json.loads(schema_),
        }
...
```


### Expected behavior

Use the JSON schema instead of the pydantic schema to call openAI.
add UnstructuredRSTLoader

This change loads an .rst file and converts it into a langchain Document

It requires docutils, but only when the class is initialized. I also included an import error to follow best practices.

I also added tests but I tried running any test (not even mine) I always got this error so I will leave that to someone more experienced with tests than I am:
argparse.ArgumentError: argument --snapshot-update: conflicting option string: --snapshot-update
make: *** [Makefile:41: test] Error 1

@eyurtsev
Vector store support for Cassandra
Labels: name: enhancement

This addresses #6291 adding support for using Cassandra (and compatible databases, such as DataStax Astra DB) as a [Vector Store](https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor(ANN)+Vector+Search+via+Storage-Attached+Indexes).

A new class `Cassandra` is introduced, which complies with the contract and interface for a vector store, along with the corresponding integration test, a sample notebook and modified dependency toml.

Dependencies: the implementation relies on the library `cassio`, which simplifies interacting with Cassandra for ML- and LLM-oriented workloads. CassIO, in turn, uses the `cassandra-driver` low-lever drivers to communicate with the database. The former is added as optional dependency (+ in `extended_testing`), the latter was already in the project.

Integration testing relies on a locally-running instance of Cassandra. [Here](https://cassio.org/more_info/#use-a-local-vector-capable-cassandra) a detailed description can be found on how to compile and run it (at the time of writing the feature has not made it yet to a release).

During development of the integration tests, I added a new "fake embedding" class for what I consider a more controlled way of testing the MMR search method. Likewise, I had to amend what looked like a glitch in the behaviour of `ConsistentFakeEmbeddings` whereby an `embed_query` call would have bypassed storage of the requested text in the class cache for use in later repeated invocations.

@dev2049 might be the right person to tag here for a review. Thank you!
Update SinglStoreDB vectorstore

1. Introduced new distance strategies support: **DOT_PRODUCT** and **EUCLIDEAN_DISTANCE** for enhanced flexibility.
2. Implemented a feature to filter results based on metadata fields.
3. Incorporated connection attributes specifying "langchain python sdk" usage for enhanced traceability and debugging.
4. Expanded the suite of integration tests for improved code reliability.
5. Updated the existing notebook with the usage example

@dev2049
Missing arguments for Chroma vector store get methods 

### Feature request

Natively a [Chromadb collection](https://github.com/chroma-core/chroma/blob/main/chromadb/api/models/Collection.py) support multiple parameters when making a get or a query on a collection, for example `where` and `ids`.  For the moment the `get` method of [Chroma vector store](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/chroma.py) only support the include argument. I think it would be nice to extend support to all available arguments in the base get method of the Chroma collection. 


### Motivation

Align the functionalities of the LangChain chroma vectorstore with all available functionalities in the chromadb's collections. 

### Your contribution

I would be happy to make the contribution to make all arguments available as in ChromaDb collections
feat (documents): add LarkSuite document loader
Labels: name: enhancement, name: lgtm, name: area: doc loader

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

### Summary

This PR adds a LarkSuite (FeiShu) document loader. 
> [LarkSuite](https://www.larksuite.com/) is an enterprise collaboration platform developed by ByteDance.

### Tests

- an integration test case is added
- an example notebook showing usage is added. [Notebook preview](https://github.com/yaohui-wyh/langchain/blob/master/docs/extras/modules/data_connection/document_loaders/integrations/larksuite.ipynb)

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

### Who can review?

- PTAL @eyurtsev @hwchase17

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Update the parser regex of map_rerank
Labels: name: lgtm

Sometimes the score responded by chatgpt would be like 'Respone example\nScore: 90 (fully answers the question, but could provide more detail on the specific error message)'
For the score contains not only numbers, it raise a ValueError like 

```
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/base.py", line 149, in __call__
    raise e
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/base.py", line 143, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 84, in _call
    output, extra_return_dict = self.combine_docs(
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/combine_documents/map_rerank.py", line 105, in combine_docs
    return self._process_results(docs, results)
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/combine_documents/map_rerank.py", line 127, in _process_results
    sorted_res = sorted(
  File "/opt/homebrew/lib/python3.10/site-packages/langchain/chains/combine_documents/map_rerank.py", line 128, in <lambda>
    zip(typed_results, docs), key=lambda x: -int(x[0][self.rank_key])
ValueError: invalid literal for int() with base 10: '80 (Answered the question but could have provided more detail)'
```

Update the RegexParser from `.*` to `\d*` would help us to ignore the text after number.

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:
@hwchase17 - project lead

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
gpt-3.5-turbo-0613 is not following the instructions of agents

### System Info

Langchain == 0.0.205
Python == 3.10.7
openai == 0.27.8

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [x] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Use gpt-3.5-turbo-0613 
In a chat-conversational-react-description agent ask "could you tell me more about the tools you have ?"


### Expected behavior

The Human message instructs the llm to give the output in a json format but that is not followed.
Here is the output with gpt-3.5-turbo:
![image](https://github.com/hwchase17/langchain/assets/61446944/df830023-99f0-4833-aa85-c283459cbe41)

Here is the output with gpt-3.5-turbo-0613:
![image](https://github.com/hwchase17/langchain/assets/61446944/8fbc2778-41d6-413f-b39c-c4835ef7488a)
Fix conversational_retrieval combining with chat history affects the question and answer experience. 

While using the ConversationalRetrievalChain, and specifically applying both `condense_question_prompt` and `combine_docs_chain_kwargs` parameters, I followed the official documentation to pass `chat_history` when invoking this chain. The program runs as expected, but the Q&A experience is less than optimal. I've identified two main issues:

1. When summarizing the question using `condense_question`, the response prompt generated by `load_qa_chain` is treated as 'Human' rather than 'AI'. Currently, I am remedying this by determining whether to pass the question to `human_message` or `ai_message`, based on whether `chat_history` is provided through an external interface.

2. Upon reviewing the code, I noticed that the answer is first generated by `self.question_generator`, but in reality, the desired answer initially comes from the `prompt` in `combine_docs_chain_kwargs`. However, this response is then re-fed into `self.combine_docs_chain`, leading to a situation of self-posed questions and self-given answers, which isn't ideal. The final answer from `combine_docs_chain` isn't what I'm looking for, and I see no need to go through another loop only to end up with an unsatisfactory answer. Thus, I wish to adjust the logic for obtaining the answer in the `call` and `acall` methods.

While my implementation might not be the most ideal, it considerably enhances the Q&A experience based on chat history in my project. I hope for official support on this issue and appreciate your help. Thank you!

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
too many doc url invalid

### System Info

langchain 0.209

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

https://python.langchain.com/docs/use_cases/question_answering/

[Question Answering Notebook](https://python.langchain.com/docs/modules/chains/index_examples/question_answering.html)
[VectorDB Question Answering Notebook](https://python.langchain.com/docs/modules/chains/index_examples/vector_db_qa.html)
Above url invalid.

### Expected behavior

All hyperlinks are accessible.