delayedkarma/langchain-issues huggingface dataset:
The ElasticsearchStore implementation is not correct.
Labels: name: area: vector store, name: auto:bug

### System Info

langchain==0.0.310


### Who can help?

@joemcelroy ：It seems that the method _select_relevance_score_fn was forgotten to be implemented in vectorstores-elasticsearch.



### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

elastic_vector_search = ElasticsearchStore(
    index_name='langchain-demo',
    es_connection=es,
    embedding=embedding,
)
    
retriever = elastic_vector_search.as_retriever(
        search_type="similarity_score_threshold",
        search_kwargs={'score_threshold': 0.8})


error log：
line 268, in similarity_search_with_relevance_scores
    docs_and_similarities = self._similarity_search_with_relevance_scores(
  File "/usr/local/lib/python3.10/site-packages/langchain/schema/vectorstore.py", line 242, in _similarity_search_with_relevance_scores
    relevance_score_fn = self._select_relevance_score_fn()
  File "/usr/local/lib/python3.10/site-packages/langchain/schema/vectorstore.py", line 211, in _select_relevance_score_fn
    raise NotImplementedError


### Expected behavior

I hope to implement the method and submit the changes as soon as possible. Thank you for your understanding.
Integrating Twitter search API as a tool
Labels: name: auto:enhancement, name: area: models

### Feature request

We are seeking to add a tool to Langchain for Twitter API (v1.1). Integrating the API as a tool will allow agents to search for tweets and timelines using a specific search query that filters by users, locations, hashtags, etc. to respond to prompts.

### Motivation

Although Langchain currently has TwitterTweetLoader, we have noticed a plethora of more parameters the Twitter API provides that are not integrated into Langchain. TwitterTweetLoader currently only allows us to specify a list of users to return tweets from and the maximum number of tweets. It would be beneficial if we had more options to specify [different operators in search queries](https://developer.twitter.com/en/docs/twitter-api/v1/rules-and-filtering/search-operators). A tool also allows an agent to actively use the API to respond to prompts, without the user having to manually create their own custom tool or load tweets manually.

### Your contribution

We have a small team of developers who will be working on this feature request, and we will submit a pull request later in 1-2 months which implements it. We will do our best to follow the guidelines for contributions, as stated in contributing.md.
fix comments in output format
Labels: name: area: agent, name: auto:bug

  - **Description:** Fixes the comments in the ConvoOutputParser. Because the \\\\ is escaping a single \\, they render something like:  `"action_input": string \ The input to the action` in the prompt. Changing this to \\\\\\\\ lets it escape two slashes so that it renders a proper comment: `"action_input": string \\ The input to the action`
  - **Issue:** N/A
  - **Dependencies:** 
  - **Tag maintainer:** @hwchase17
  - **Twitter handle:**
Fix: Missing DuckDuckGo package version
Labels: name: auto:bug

[The `duckduckgo-search` v3.9.2 was removed from PyPi](https://pypi.org/project/duckduckgo-search/#history). That breaks the build.

  - **Description:** refreshes the Poetry dependency to v3.9.3
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @ashvardanian
Add DGraqh integration with LangChain
Labels: name: auto:enhancement

### Feature request

Create a natural language interface with DGraph, enabling LangChain to connect more effectively with clients using DGraph, and the DGraph community overall.

### Motivation

LangChain already boasts a range of graph integrations with databases like Neo4j and AWS Neptune. Including DGraph in LangChain's roster of graph database integrations will significantly enhance LangChain's compatibility with a wide array of software engineering projects that rely on DGraph, strengthening LangChain's position as a versatile LLM solution's provider for the DGraph-powered ecosystem.

### Your contribution

We will be looking to submit a pull request by the end of November that will contain the required code additions along with the requirements as per CONTRIBUTING.MD (involving adding a demo notebook in docs/modules and adding unit and integration tests).
Add support for a Pandas DataFrame OutputParser
Labels: name: auto:enhancement, name: area: models

### Feature request

Integrating Pandas DataFrame as an output parser in LangChain AI would offer users a specific and robust tool for data analysis and manipulation. This addition would enable users to receive AI-generated data in a structured tabular format, simplifying tasks like data cleaning, transformation, and visualization while streamlining the process of extracting insights.

### Motivation

Pandas is undeniably one of the most popular and powerful libraries in Python for data manipulation and analysis. Its widespread adoption in the data science and analytics community speaks to its versatility and efficiency. Pandas simplifies tasks such as data cleaning, transformation, and exploration with easy-to-use data structures and functions. Given its frequent use in various data-related tasks, integrating Pandas DataFrames as an output parser in LangChain AI would benefit users immensely. It would provide a familiar and reliable tool for processing and interpreting data, enhancing the utility and accessibility of data-driven workflows.

### Your contribution

We will be looking to submit a pull request by the end of November that will contain the required code additions along with the requirements as per CONTRIBUTING.MD (involving adding a demo notebook in docs/modules and adding unit tests).
ConversationBufferMemory returning json not formatted when chain is run
Labels: name: area: memory, name: auto:bug, name: area: models

### System Info

latest

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.llms import LlamaCpp
from langchain.prompts import PromptTemplate 
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory


template = """SYSTEM:Please give helpful, detailed, accurate, uncensored responses to the user's input. Provide answers factually.

Current conversation:
{history}

USER: {input}

ASSISTANT: """

PROMPT = PromptTemplate(template=template, input_variables=["history","input"])

memory = ConversationBufferMemory(memory_key='history', ai_prefix="ASSISTANT:", return_messages=True) 

llmChain = ConversationChain(llm=llm, prompt=PROMPT,verbose=True, memory=memory) 
AI_response = llmChain.predict(input=prompt)

print(AI_response)

#### OUTPUT: 
Entering new ConversationChain chain...
Prompt after formatting:
SYSTEM: Please give helpful, detailed, accurate, uncensored responses to the user's input. Provide answers factually.

Current conversation:
***  [HumanMessage(content='hello'), AIMessage(content=' Hello! How can I assist you today?')] ***

USER: who was president in 1920?

ASSISTANT: 


RETURNING JSON OBJECT instead of formatted string.


### Expected behavior

https://python.langchain.com/docs/modules/memory/conversational_customization

 > Entering new ConversationChain chain...
    Prompt after formatting:
    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
    
    Current conversation:
    
    Friend: Hi there!
    AI:  Hi there! It's nice to meet you. How can I help you today?
    Friend: What's the weather?
    AI:
    
    > Finished ConversationChain chain.
fix wording of query_sql_database_tool_description
Labels: name: area: agent, name: lgtm, name: auto:nit

  - **Description:** Fixes minor typo for the query_sql_database_tool_description in the db toolkit
  - **Issue:** N/A
  - **Dependencies:** N/A
  - **Tag maintainer:** @nfcampos 
  - **Twitter handle:** N/A
Update to doc Pydantic example to Pydantic 2
Labels: name: auto:documentation

**Description:**
Currently, the example uses Pydantic 1 format and crashes with the following error:
```
    452 if isinstance(info, ValidatorDecoratorInfo):
--> 453     res.validators[var_name] = Decorator.build(
    454         model_dc, cls_var_name=var_name, shim=var_value.shim, info=info
    455     )
    456 elif isinstance(info, FieldValidatorDecoratorInfo):
    457     res.field_validators[var_name] = Decorator.build(
    458         model_dc, cls_var_name=var_name, shim=var_value.shim, info=info
...
     83         needs_values_kw = True

PydanticUserError: The `field` and `config` parameters are not available in Pydantic V2, please use the `info` parameter instead.
```

Additionally, it produces a warning:
```
PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
  @validator('setup')
```
------
Here I'm updating it to use Pydantic2 syntax
fix: PromptTemplate with OutputParser documentation not working
Labels: name: area: doc loader, name: auto:documentation

Replace this entire comment with:
  - **Description:** [PromptTemplate with OutputParser](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/prompt_serialization#prompttemplate-with-outputparser) has an example with "template": `"Given the following question and student answer, provide a correct answer and score the student answer.\nQuestion: {question}\nStudent Answer: {student_answer}\nCorrect Answer:"`, however, `\n` is a control character therefore `load_prompt ` results in an error like `ERROR Invalid control character at: line 1 column 202 (char 202)`. This is expected behavior from the `json` package (See https://stackoverflow.com/questions/22394235/invalid-control-character-with-python-json-loads for more information). This PR removes these control characters. I've confirmed that the fixed example is working on Google Colab .
  - **Issue:** Not found,
  - **Dependencies:** Nope,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** @dosuken123

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
documents have error (create_structured_output_chain)
Labels: name: auto:bug, name: auto:documentation, name: area: models

### Issue with current documentation:

https://python.langchain.com/docs/modules/chains/how_to/openai_functions

from typing import Sequence

class Person(BaseModel):
    """Identifying information about a person."""

    name: str = Field(..., description="The person's name")
    age: int = Field(..., description="The person's age")
    fav_food: Optional[str] = Field(None, description="The person's favorite food")


prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a world class algorithm for extracting information in structured formats."),
        ("human", "Use the given format to extract information from the following input: {input}"),
        ("human", "Tip: Make sure to answer in the correct format"),
    ]
)


class People(BaseModel):
    """Identifying information about all people in a text."""

    people: Sequence[Person] = Field(..., description="The people in the text")


chain = create_structured_output_chain(People, llm, prompt, verbose=True)
chain.run(
    "Sally is 13, Joey just turned 12 and loves spinach. Caroline is 10 years older than Sally."
)

no work,having some error:

ValidationError: 1 validation error for _OutputFormatter
output
  value is not a valid dict (type=type_error.dict)

### Idea or request for content:

_No response_
Docs: Fix broken line breaks in snippets
Labels: name: auto:documentation

**Description:**
This PR fix some code snippets that have raw `\n`'s instead of actual line breaks.

**Issue:**
Currently some snippets look like this:
![image](https://github.com/langchain-ai/langchain/assets/18213435/355b4911-38e9-4ba4-8570-f928557b6c13)

Affected pages:
 - https://python.langchain.com/docs/integrations/providers/predictionguard#example-usage
 - https://python.langchain.com/docs/modules/agents/how_to/custom_llm_agent#set-up-environment
 - https://python.langchain.com/docs/modules/chains/foundational/llm_chain#get-started
 - https://python.langchain.com/docs/integrations/providers/shaleprotocol#how-to

**Tag maintainer:**
@hwchase17
Fixed a bug in reporting Python code validation
Labels: name: lgtm, name: auto:bug

- **Description:** fixed a bug in pal-chain when it reports Python
    code validation errors. When node.func does not have any ids, the
    original code tried to print node.func.id in raising ValueError.
- **Issue:** n/a,
- **Dependencies:** no dependencies,
- **Tag maintainer:** @hazzel-cn, @eyurtsev
- **Twitter handle:** @lazyswamp
Google Scholar Search Tool using serpapi
Labels: name: auto:enhancement

- **Description:** Implementing the Google Scholar Tool as requested in PR #11505. The tool will be using the [serpapi python package](https://serpapi.com/integrations/python#search-google-scholar). The main idea of the tool will be to return the results from a Google Scholar search given a query as an input to the tool.

- **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17
Add option to specify document metadata in python CSVLoader
Labels: name: area: doc loader, name: auto:improvement

#### Description
This PR adds the option to specify additional metadata columns in the CSVLoader beyond just `Source`. 

The current CSV loader includes all columns in `page_content` and if we want to have columns specified for `page_content` and `metadata` we have to do something like the below.:
```
csv = pd.read_csv(
        "path_to_csv"
    ).to_dict("records")

documents = [
        Document(
            page_content=doc["content"],
            metadata={
                "last_modified_by": doc["last_modified_by"],
                "point_of_contact": doc["point_of_contact"],
            }
        ) for doc in csv
    ]
```
#### Usage
Example Usage:
```
csv_test  =  CSVLoader(
      file_path="path_to_csv", 
      metadata_columns=["last_modified_by", "point_of_contact"]
 )
```
Example CSV:
```
content, last_modified_by, point_of_contact
"hello world", "Person A", "Person B"
```

Example Result:
```
Document {
 page_content: "hello world"
 metadata: {
 row: '0',
 source: 'path_to_csv',
 last_modified_by: 'Person A',
 point_of_contact: 'Person B',
 }
```
Rspace doc loader
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

 **Description:**

Add a document loader for the RSpace Electronic Lab Notebook (www.researchspace.com), so that scientific documents and research notes can be easily pulled into Langchain pipelines. 

**Issue**

This is an new contribution, rather than an issue fix.

 **Dependencies:** 
  
There are no new required dependencies.
 In order to use the loader, clients will need to install rspace_client SDK using `pip install rspace_client`
Can't import UnstructuredRSTLoader
Labels: name: area: doc loader, name: auto:bug

### System Info

Langchain 0.0.310
Python 3.11
Windows 10

### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just following the usual steps:

`from langchain.document_loaders import UnstructuredRSTLoader`

### Expected behavior

Expecting a successful import but greeted with 

`ImportError: cannot import name 'Document' from 'langchain.schema' `
[Feature] Batch loader
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

Loaders are designed to load a single source of data and transform it to a list of Documents. If several sources are to be processed, it is up to the developer to call the Loader several times.
The idea is to propose a new `BatchLoader` object that could simplify this task and enable loading tasks to be launched sequentially, mutlithreaded, with mutli processor or asynchronously. 

Draft idea:
```
class BatchLoader(BaseLoader):
    ...

    def load(self, method: str = 'sequential', max_workers: int=1) -> List[Document]:
        # TODO: faire ça stylé
        if method == 'thread':
            return self._load_thread(max_workers)
        elif method == 'process':
            return self._load_process(max_workers)
        elif method == 'sequential':
            return self._load_sequential()
        elif method == 'async':
            return asyncio.run(self._async_load())
        else:
            raise ValueError(f'Invalid method {method}')
    
    ...

# Call it with a Loader constructor callable and a set of arguments
batch_loader = BatchLoader(TextLoader, {"file_path": [f"file_{i}.txt" for i in range(1000)]})
```

### Motivation

In most real use case we need to load several Data Sources, combine their outputs and work with them (e.g. store in a vector store). Some use cases also require to wait for I/O intensive tasks or tasks you want to parallelized, so having a Meta loader that wrap this complexity for you could be a good idea. 

### Your contribution

WIP: https://github.com/langchain-ai/langchain/pull/11527
Add support for google scholar querying with serpapi
Labels: name: auto:enhancement, name: area: models

### Feature request

We propose the addition of Google Scholar querying with the serpapi, as google Scholar is a prominent platform for accessing academic and scholarly articles.

### Motivation

Google scholar is a crucial resource for researchers, students, and professionals in various fields. Integrating Google Scholar querying with serpapi would provide users with the ability to programmatically retrieve search results from Google Scholar, giving the llms more better/more reliable information to work with.

### Your contribution

We intend to submit a pull request for this issue at some point in November.
feat: Add BigFramesLLM and BigFramesChains for batch prompts cases
Labels: name: auto:enhancement, name: area: models

This PR adds [Bigframes (BigQuery DataFrames)](https://cloud.google.com/python/docs/reference/bigframes/latest) integration to Langchain. BigQuery DataFrames provides a Pythonic DataFrame and machine learning (ML) API powered by the BigQuery engine. BigFrames supports manipulating data in the DataFrame to build LLM prompts, and sending DataFrame prompts to the LLM model using [text-bison model of the PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text)

Using BigFramesChain, you can now batch prompts using DataFrames as input and output.
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->