delayedkarma/langchain-issues huggingface dataset:
Batch update of alt text and title attributes for images in md/mdx files across repo
Labels: name: area: doc loader, name: auto:documentation, name: template, name: size:M

**Description:** Batch update of alt text and title attributes for images in `md` & `mdx` files across the repo using [alttexter](https://github.com/jonathanalgar/alttexter)/[alttexter-ghclient](https://github.com/jonathanalgar/alttexter-ghclient) (built using LangChain/LangSmith).

**Limitation:** cannot update `ipynb` files because of [this issue](https://github.com/langchain-ai/langchain/pull/15357#issuecomment-1885037250). Can revisit when Docusaurus is bumped to v3.

I checked all the generated alt texts and titles and didn't find any technical inaccuracies. That's not to say they're _perfect_, but a lot better than what's there currently.

[Deployed](https://langchain-819yf1tbk-langchain.vercel.app/docs/modules/model_io/) image example:

![chrome_yZQ7BF2GTj](https://github.com/langchain-ai/langchain/assets/93204286/43a9a4d4-70fd-41c4-8978-b6240ff63ffa)

You can see LangSmith traces for all the calls out to the LLM in the PRs merged into this one:

* https://github.com/jonathanalgar/langchain/pull/6
* https://github.com/jonathanalgar/langchain/pull/4
* https://github.com/jonathanalgar/langchain/pull/3

I didn't add the following files to the PR as the images already have OK alt texts:

* https://github.com/langchain-ai/langchain/blob/27dca2d92fbd9c956a1eac7cf885a0ed060ca9ff/docs/docs/integrations/providers/argilla.mdx#L3
* https://github.com/langchain-ai/langchain/blob/27dca2d92fbd9c956a1eac7cf885a0ed060ca9ff/docs/docs/integrations/providers/apify.mdx#L11
Batch update of alt text and title attributes across documentation
Labels: name: auto:refactor, name: size:XXL

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Langchain: Fix quickstart doc code not working
Labels: name: lgtm, name: area: doc loader, name: auto:documentation, name: size:S

The quickstart doc is missing a few but very simple things that without them, the code does not work. This PR fixes that by 
- Adding commands to install `tiktoken` and `langchainhub`
- Adds a comma between 2 parameters for one of the methods
Calculate trace_id and dotted_order client side
Labels: name: auto:improvement, name: area: langserve, name: size:M

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: output templates in langchain
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

How can I make output templates in langchain? That is, for example, I throw a request for AI to write a joke, but with strict adherence to the template [set-up, punchline] and therefore get as a result:
```
Set-up: ...
Punchline: ...
```
and nothing more

### Suggestion:

_No response_
DOC: Summarization 'map_reduce' - Can't load tokenizer for 'gpt2'
Labels: name: auto:bug, name: area: models

### Issue with current documentation:

The [documentation](https://python.langchain.com/docs/use_cases/summarization) describes the different options for summarizing a text, for longer texts the 'map_reduce' option is suggested. It is mentioned further under 'Go deeper' that it is possible to use different LLMs via the `llm` parameter. This seems to work well using the code below with the `chain_type='stuff'` parameter and, in particular, using a local model (in the example below [this model](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF) is used). 

```
from langchain.document_loaders import PyPDFLoader
from langchain.llms import CTransformers
from langchain.chains.summarize import load_summarize_chain

# load a PDF-file
loader = PyPDFLoader("C:/xyz.pdf")
docs = loader.load()

# use a local LLAMA2 model
llm = CTransformers(model='./models/llama-2-7b-chat.Q5_K_M.gguf', model_type='llama', config={'context_length': 4096, 'max_new_tokens': 256, 'temperature': 0}, local_files_only=True)

# summarise the text (this works only if it fits into the context length of ~4000 tokens)
chain = load_summarize_chain(llm, chain_type="stuff")
chain.run(docs)
```

However, surprisingly, it returns the following error when using the `chain_type='map_reduce'` parameter: 'OSError: Can't load tokenizer for 'gpt2'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'gpt2' is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.'

The suggestion [mentioned in this issue on Github](https://github.com/langchain-ai/langchain/issues/9273) doesn't work for the local model used above. It would be great to have more specific information in the LangChain documentation on (1) how to perform text summarization with LangChain using different LLMs, and (2) specifically for using local models that don't require an internet connection and/or require gpt2. Since the above code works with the parameter `chain_type='stuff'` but not with the parameter `chain_type='map_reduce'`, it would be important to explain what happens under the hood so users can make this work for local models.

### Idea or request for content:

_No response_
Issue: <Saving intermediate variable chains ' prefix>
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Help me understand how I can save the intermediate data of chain execution results?
![output 4](https://github.com/langchain-ai/langchain/assets/136048128/be97b9eb-0537-4e76-bcd2-b616bfdd6a5d)


### Suggestion:

_No response_
google-vertexai[minor]: added safety_settings property to gemini wrapper
Labels: name: lgtm, name: auto:improvement, name: area: models, name: size:L, name: partner

**Description:** Gemini model has quite annoying default safety_settings settings. In addition, current VertexAI class doesn't provide a property to override such settings.
So, this PR aims to 
 - add safety_settings property to VertexAI
- fix issue with incorrect LLM output parsing when LLM responds with appropriate 'blocked' response
- fix issue with incorrect parsing LLM output when Gemini API blocks prompt itself as inappropriate
- add safety_settings related tests

I'm not enough familiar with langchain code base and guidelines. So, any comments and/or suggestions are very welcome.
 
**Issue:** it will likely fix #14841
DOC: Prompt Templates "Page Not Found"
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

A few days back, I was referring to the [Prompt templates](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/) page which now shows:  "**Page Not Found**"

### Idea or request for content:

I understand that LangChain is an evolving framework undergoing continuous development. 

- Could we consider implementing versioning for the documentation? This would allow users to access specific documentation versions. 
- Alternatively, if a section undergoes modification, we could preserve the existing documentation and label it as 'Legacy,' ensuring clarity about deprecated practices.
_OllamaCommon contains top_p with int-restriction
Labels: name: auto:bug, name: area: models

### System Info

LangChain 0.0.353


### Who can help?

@hwchase17
@agola11

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Cannot set top_p to useful values via ChatOllama(top_p=0.3), is reduced to value 0 because it's an int:

top_p: Optional[int] = None

top_p must be float.

### Expected behavior

top_p must be a float, 0.3 should appear in ollama log.
<langchain>: add checks for qaSourcesChain to check if the returned answer referen…
Labels: name: auto:improvement, name: area: models, name: size:M

…ces the source from which it returning an answer

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:**  when using QAWithSourcesChain I realized many times that the language model would return results which was not in the source being referenced. The example in this PR checks if the LLM refers to the actual source by mentioning its name. For example, if you ask an LLM tell me something from Book A the LLM must mention something from Book A. The implementation presented is preliminary and has to further developed but I want your general thoughts on the matter :) , 
  - **Issue:** hallucination of an answer which is not grounded in the source by the chain,
  - **Dependencies:** None,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Langchain: FAISS Translator for Self Query Retriver
Labels: name: area: vector store, name: auto:improvement, name: size:M


**Description:** Issue #15331 pointed out that there was no support for FAISS for `SelfQueryRetriever`. Therefore, I decided to add the desired support. I understand that FAISS does not have an internal query language like other vector stores (e.g., Chroma), but I noticed that the current implementation of the `FAISS` VectorStore Class in Langchain allows the user to filter documents based on given metadata attributes. Therefore, the translator simply translates the structured query according to the logic of filtering defined in `similarity_search_with_score_by_vector` and `max_marginal_relevance_search_with_score_by_vector`, which filters the documents for the specific values present in metadata keys.

**Issue:** #15331

PS: I have not added the docs for this one. Only once you see the value in adding this translator, I will add the docs.
Catch type errors in dumps/dumpd
Labels: name: area: doc loader, name: auto:improvement, name: size:S

These can happen for edge cases not covered by `default` handler (eg. "strange" keys in dicts)

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
refactor `community` memory
Labels: name: area: memory, name: auto:refactor, name: size:XXL

- created `language_community.memory`
- moved zep and motorhed from `langchain.memory` to `language_community.memory`
- created `language_core.memory`
- moved several base classes into `language_core.memory`
- moved `ChatMessageHistory` from `langchain_community` to `langchain_core`
This model's maximum context length is 4097 tokens, however you requested 4177 tokens
Labels: name: auto:question, name: area: models, name: integration: pinecone

### System Info

New versions

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Start the code

### Expected behavior

Hi , I'm trying to do a chain stuff query, but sometimes when I ask questions I have this error:
openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 4097 tokens, however you requested 4177 tokens (3921 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

How can I solve this ? Can I cut my prompt ? If yes how? Or Can I upgrade the max_token ? IF yes How please..

Here is my code:
import getpass
import os
from langchain.document_loaders import TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Pinecone
from langchain_community.vectorstores import Pinecone
from langchain_community.embeddings.openai import OpenAIEmbeddings
import pinecone
import sys



# Set your Pinecone API key and environment
pinecone_api = "API"
pinecone_env = "API"

# Set your OpenAI API key
openai_api = "API"
# Initialize Pinecone
pinecone.init(api_key=pinecone_api, environment=pinecone_env)

# Define the index name
index_name = "rewind"

# Check if the index already exists, if not, create it
if index_name not in pinecone.list_indexes():
    pinecone.create_index(name=index_name, metric="cosine", dimension=1536)




# Initialize the OpenAIEmbeddings
embeddings = OpenAIEmbeddings(api_key=openai_api)

# Create or load the Pinecone index
docsearch = Pinecone.from_existing_index(index_name, embeddings)

# Perform similarity search
query = sys.argv[1] if len(sys.argv) > 1 else "what Commits there is in github"
text_splitter = CharacterTextSplitter(chunk_size=3000, chunk_overlap=0)
docs = docsearch.similarity_search(query)
docs = text_splitter.split_documents(docs)



if __name__ == '__main__':
    results = docsearch.similarity_search(query)

    # Load the question answering chain
    from langchain.chains.question_answering import load_qa_chain
    from langchain.llms import OpenAI



    chain = load_qa_chain(OpenAI(), chain_type="stuff")
    answers = chain.run(input_documents=docs, question=query)
    print(answers)
Self query retriever with Vector Store type <class 'langchain_community.vectorstores.faiss.FAISS'> not supported.
Labels: name: area: vector store, name: auto:bug

### System Info

I've been trying to create a self query retriever so that I can look at metadata field info. This issue comes up. Should I be using another vector store to make this work? I can only really work with FAISS. I cannot use ChromaDB since my Python environment is limited to a previous version.


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create a FAISS Vectorstore DB
2. Create a metadata_field_info object and pass it to a SelfQuery object
3. Create LLM with this retriever

```python
embedding_function = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY, model="text-embedding-ada-002")
    
    db = FAISS.load_local(input_dir + "/" + "storage/deploy/faiss-db", embedding_function)#, distance_strategy="COSINE", normalize_L2 = True)
    # retriever = KNNRetriever(vectorstore=db)
    
    llm = ChatOpenAI(
        temperature=0,
        verbose=False,
        openai_api_key=key,
        model_name="gpt-3.5-turbo"
    )       

    metadata_field_info = [
        AttributeInfo(
            name="source",
            description="The document this chunk is from.",
            type="string",
        ),
        AttributeInfo(
            name="origin",
            description="The origin the document came from. Comes from either scraped websites like TheKinection.org, Kinecta.org or database files like Bancworks. Bancworks is the higher priority.",
            type="string",
        ),
        AttributeInfo(
            name="date_day",
            description="The day the document was uploaded.",
            type="integer",
        ),
        AttributeInfo(
            name="date_month",
            description="The month the document was uploaded.",
            type="integer",
        ),
        AttributeInfo(
            name="date_year",
            description="The year the document was uploaded.",
            type="integer",
        ),
    ]

    # retriever = db.as_retriever(search_type="similarity", search_kwargs={'k': 6}, metadata_field_info=metadata_field_info)

    retriever = SelfQueryRetriever.from_llm(
        llm, db, "Information about where documents originated from and when they were published.", metadata_field_info, verbose=True
    )
```

### Expected behavior

Successfully create a SelfQuery retriever with FAISS vector store.
Propagate context vars in all classes/methods
Labels: name: auto:refactor, name: area: models, name: size:XL

- Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Integration with OpenAI's Latest Models and API Compatibility
Labels: name: auto:enhancement, name: area: models

### Feature request

This proposal requests the integration of the latest OpenAI models, specifically gpt-4-1106-preview, into the existing framework of [relevant GitHub project, e.g., LangChain]. The newer models offer significantly larger context windows, which are crucial for complex SQL querying and other advanced functionalities. This feature would involve ensuring compatibility with the latest version of the OpenAI API (version 1.0.0 and beyond), which has undergone substantial changes, including the deprecation of certain features like openai.ChatCompletion. Relevant links:

OpenAI API (1.0.0): [OpenAI API Documentation](https://github.com/openai/openai-python)
Migration Guide: [OpenAI Python Library Migration Guide](https://github.com/openai/openai-python/discussions/742)

### Motivation

The primary motivation for this feature request is to leverage the advanced capabilities of the newer OpenAI models, particularly the extended context windows they offer. These capabilities are essential for applications involving extensive data interaction and complex language understanding, such as SQL database querying and management.

Current limitations with the older models and API versions restrict the potential of applications, especially when dealing with lengthy queries or requiring deeper contextual understanding. For example, while working on a project involving the LangChain framework for SQL database interaction, I encountered the APIRemovedInV1 error, which signifies incompatibility with the latest OpenAI API. This issue underscores the need for updating the framework to align with the latest advancements in language models and API standards.

### Your contribution

Might make my own SQL Agent or modify yours.