delayedkarma/langchain-issues huggingface dataset:
langchain.graph RDFGraph does not read .owl extension files
Labels: name: area: doc loader, name: auto:bug

### System Info

Langchain  version: 0.0.332
Python version: 3.11.5 

### Who can help?

@hwchase17 

When loading a local .owl file (the standard example pizza.owl) the operation breaks and gives the following error for all the URI:
does not look like a valid URI, trying to serialize this will break.

Here's the traceback

```
Traceback (most recent call last):

  File ~\AppData\Roaming\Python\Python311\site-packages\IPython\core\interactiveshell.py:3526 in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  Cell In[13], line 4
    graph = RdfGraph(

  File C:\Python311\Lib\site-packages\langchain\graphs\rdf_graph.py:159 in __init__
    self.graph.parse(source_file, format=self.serialization)

  File C:\Python311\Lib\site-packages\rdflib\graph.py:1501 in parse
    raise se

  File C:\Python311\Lib\site-packages\rdflib\graph.py:1492 in parse
    parser.parse(source, self, **args)

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:2021 in parse
    p.loadStream(stream)

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:479 in loadStream
    return self.loadBuf(stream.read())  # Not ideal

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:485 in loadBuf
    self.feed(buf)

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:511 in feed
    i = self.directiveOrStatement(s, j)

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:532 in directiveOrStatement
    return self.checkDot(argstr, j)

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:1214 in checkDot
    self.BadSyntax(argstr, j, "expected '.' or '}' or ']' at end of statement")

  File C:\Python311\Lib\site-packages\rdflib\plugins\parsers\notation3.py:1730 in BadSyntax
    raise BadSyntax(self._thisDoc, self.lines, argstr, i, msg)

  File <string>
BadSyntax
```

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce the behaviour:
1. Get the source file from : https://protege.stanford.edu/ontologies/pizza/pizza.owl and place it where the code runs
2. Use the following code:
```
from langchain.chains import GraphSparqlQAChain
from langchain.graphs import RdfGraph

graph = RdfGraph(
    source_file="pizza.owl",
    standard="owl"
)

graph.load_schema()

print(graph.get_schema)
```

### Expected behavior

For the graph to load and for graph.get_schema to show the classes and object properties.
How can we selectively pass memory to specific tools without passing it to all tools?
Labels: name: area: memory, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

for example in my below case
```python
english_tools = [

        Tool(name="SomeNAME_1",
             func=lambda q: app.finance_chain.run(q),
             description=" Some app related description ",
             return_direct=True,
             coroutine=lambda q: app.finance_chain.arun(q),
             ),
        Tool(name="SomeNAME_2",
             func=lambda q: app.rqa(q),
             description=" Some app related description ",
             coroutine=lambda q: app.rqa_english.arun(q),
             return_direct=True
             ),
        Tool.from_function(
            name="SomeNAME_3",
            func=lambda q: app.pd_agent(q),
            description=" Some app related description",
            coroutine=lambda q: app.pd_agent.arun(q),
        )
    ]```
    
    so when SomeNAME_3 invoke i don't wann pass memory to this tool

### Suggestion:

_No response_
chore: bump momento dependency version and refactor search hit usage
Labels: name: area: vector store, name: auto:nit

**Description**

Bumps the Momento dependency to the latest version and refactors the usage of `SearchHit` in the Momento Vector Index (MVI) vector store integration. This change is a one liner where we use the preferred attribute `score` to read the query-document similarity instead of `distance`. The latest versions of Momento clients will use this attribute going forward.

**Dependencies**

Updated the Momento dependency to latest version.

**Tests**

💚 I re-ran the existing MVI integration tests (`tests/integration_tests/vectorstores/test_momento_vector_index.py`) and they pass.

**Review**
cc @baskaryan @eyurtsev
feat: FastEmbed embedding provider
Labels: name: lgtm, name: area: embeddings, name: auto:enhancement

## Description:
This PR intends to add [Qdrant/FastEmbed](https://qdrant.github.io/fastembed/) as a local embeddings provider, associated tests and documentation.

**Documentation preview:** https://langchain-git-fork-anush008-master-langchain.vercel.app/docs/integrations/text_embedding/fastembed
Summarization chain batching is not working with Replicate
Labels: name: auto:bug, name: area: models

### System Info

LangChain version: 0.0.300
Python version: 3.10.12

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce:

```py
# Prompt
prompt_text = """You are an assistant tasked with summarizing tables and text. \ 
Give a concise summary of the table or text. Table or text chunk: {element} """
prompt = PromptTemplate.from_template(prompt_text)

# Summary chain
model = Replicate(
    model="meta/llama-2-7b-chat:13c3cdee13ee059ab779f0291d29054dab00a47dad8261375654de5540165fb0",
    model_kwargs={"temperature": 0.75, "max_length": 3000, "top_p":0.25}
)
summarize_chain = {"element": lambda x: x} | prompt | model | StrOutputParser()

# Apply to text
texts = [i.text for i in text_elements]
text_summaries = summarize_chain.batch(texts, {"max_concurrency": 5})
```

Error:
```txt
ValueError: config must be a list of the same length as inputs, but got 27 configs for 5 inputs
```

With OpenAI, it is working as expected.

### Expected behavior

The summarization chain (powered by Replicate) should process each chunks batch-wise.
Performing a Graph SPARQL Query with a Local LLM
Labels: name: auto:bug, name: area: models

![image](https://github.com/langchain-ai/langchain/assets/73410751/0370d267-34ff-4615-8a4e-c8ef8798a27f)

Hello all, I'm attempting to perform a SPARQL graph query using my local LLM, but it appears that something is amiss. Please feel free to share any helpful tips or guidance.

`graph = RdfGraph(
    source_file="http://www.w3.org/People/Berners-Lee/card",
    standard="rdf",
    local_copy="test1109.ttl",
)
tokenizer = AutoTokenizer.from_pretrained('C:\\data\\llm\\chatglm-6b-int4', trust_remote_code=True)
model = AutoModel.from_pretrained('C:\\data\\llm\\chatglm-6b-int4', trust_remote_code=True).half().cuda().eval()
chain = GraphSparqlQAChain.from_llm(model, graph=graph, verbose=True)
question = "What is Tim Berners-Lee's work homepage?"
result = chain.run(question)`

`  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for LLMChain
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)`
How to use a different prompt template upon runnable retry?
Labels: name: auto:improvement, name: area: models

### Issue you'd like to raise.

Here's an example implementation of `RunnableRetry`,

```
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain.schame.runnable.retry import RunnableRetry

template = PromptTemplate.from_template("tell me a joke about {topic}.")
error_template = PromptTemplate.from_template("tell me a joke about {topic} with {context}.")

model = ChatOpenAI(temperature=0.5)
chain = template | model
retryable_chain = RunnableRetry(bound=chain, max_attempt_number=3, callback={'prompt': error_template})
```

As for now `RunnableRetry` won't support callback and I'm not sure about altering the first step of the RunnableSequence with modified inputs (inputs along with the error message.)

### Suggestion:

_No response_
rfc: support hybrid in pinecone
Labels: name: area: vector store, name: auto:improvement, name: integration: pinecone

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix langchain.llms OpenAI completion doesn't work due to v1 client update
Labels: name: lgtm, name: auto:bug, name: area: models

This commit fixes the issue that langchain.llms OpenAI completion stopped working since the V1 openai client update.

Replace this entire comment with:
  - **Description:** This PR fixes the issue [AttributeError: module 'openai' has no attribute 'Completion'](https://github.com/langchain-ai/langchain/issues/12967) similar to https://github.com/langchain-ai/langchain/commit/8e0cb2eb84539b249198f98f7647a1f24b2392c2 and https://github.com/langchain-ai/langchain/pull/12969, 
  - **Issue:** https://github.com/langchain-ai/langchain/issues/12967,
  - **Dependencies:** `openai` v1.x.x client,
  - **Tag maintainer:** @baskaryan,
  - **Twitter handle:** @dosuken123 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
Serialize failing - Can't use with_fallbacks with MapReduceChain/Summarization: Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt...
Labels: name: auto:bug, name: area: models

I see the following when using AzureChatOpenAI with with_fallbacks. Removing with_fallbacks doesn't cause this issue.

`Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)`

### System Info
Pydantic v1, same with v2

**Lanchain 0.0.295**

### Who can help?

@hwchase17
@agola11
@ey

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This doesn't work:
```    fallback_chat_model = ChatOpenAI(model_name="model_name")
    primary_chat_model = AzureChatOpenAI()
    chat_model = primary_chat_model.with_fallbacks([fallback_chat_model])
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        length_function=primary_chat_model.get_num_tokens,
    )
```

### Expected behavior

Should work as this, but with fallback support
```    fallback_chat_model = ChatOpenAI(model_name="model_name")
    primary_chat_model = AzureChatOpenAI()
    chat_model = primary_chat_model                        #.with_fallbacks([fallback_chat_model])
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        length_function=primary_chat_model.get_num_tokens,
    )
```
make text fails
Labels: name: area: doc loader, name: auto:bug, name: auto:documentation

### System Info

make text
Running Sphinx v4.5.0
loading pickled environment... done
[autosummary] generating autosummary for: index.rst
building [mo]: targets for 0 po files that are out of date
building [text]: targets for 0 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] index
....

Exception occurred:
  File "/lib/python3.10/site-packages/sphinx/registry.py", line 354, in create_translator
    setattr(translator, 'visit_' + name, MethodType(visit, translator))
TypeError: first argument must be callable

Full path of file registry.py is truncated.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Run make text in docs/api_reference folder

### Expected behavior

Documentation output in .rst format
'RunnableWithFallbacks' object has no attribute 'get_num_tokens'
Labels: name: area: doc loader, name: auto:bug, name: area: models

### System Info

llm/chat_models created using with_fallback() throw attribute error when trying to access their get_num_tokens() function. This could potentially break several use cases where one does chat_model.get_num_tokens(). I believe even the docs access it as such (For example in the [MapReduce using LCEL](https://python.langchain.com/docs/modules/chains/document/map_reduce#recreating-with-lcel) docs) 

My use case was as such:
```
    fallback_chat_model = ChatOpenAI(model_name="model_name", temperature=0)
    primary_chat_model = AzureChatOpenAI(temperature=0)
    chat_model = primary_chat_model.with_fallbacks([fallback_chat_model])
## Split text using length_function and RecursiveCharacterTextSplitter
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        length_function=chat_model.get_num_tokens,
    )
    texts = text_splitter.create_documents([text_sentence_numbered])
```

Of course, I can just use primary_chat_model's get_num_tokens if it's equivalent to the fallback's but in the case it's not, this would be a bigger issue. Worse still, this may be used within chains like MapReduceChain etc and many haven't even begun the switch to LCEL.


**Langchain Version 0.0.295**

### Who can help?

@eyurtsev @hwchase17 @agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
    fallback_chat_model = ChatOpenAI(model_name="model_name", temperature=0)
    primary_chat_model = AzureChatOpenAI(temperature=0)
    chat_model = primary_chat_model.with_fallbacks([fallback_chat_model])
## Split text using length_function and RecursiveCharacterTextSplitter
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        length_function=chat_model.get_num_tokens,
    )
    texts = text_splitter.create_documents([text_sentence_numbered])
```

### Expected behavior

get_num_tokens() is accessible and based on the fallback/original chat_model or llm being used.
Make javelin_ai_gateway_api_key a SecretStr
Labels: name: auto:improvement, name: area: models

  **Description:** Add masking of API Key for javelin_ai_gateway, 
  **Issue:** https://github.com/langchain-ai/langchain/issues/12165,
  **Dependencies:** None,
  **Tag maintainer:** @eyurtsev,
  **Twitter handle:** @hiroshitashir
update azure embedding docs
Labels: name: area: embeddings, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Send back annotations sent by OpenAI for content filtered requests (and responses)
Labels: name: auto:enhancement, name: area: models

### Feature request

Is it possible to send back the annotations in the response when a request / response is content filtered by AzureOpenAI? Some prompts are content filtered by Azure and sometimes the response to certain prompts are content filtered as well.

Azure OpenAI sends back the annotations https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=python#annotations-preview 

Langchain needs to capture these annotations and send it along with the response

### Motivation

This feature will help us to respond to our uses why a particular prompt was content filtered. Today we can only tell our users whether or not a prompt was content filtered by Azure but when the users ask why the filtration was applied, we have no definite answer. With these annotations we can tell our users why the content filtration was applied

### Your contribution

No, not at the moment
Issue: Chroma tests are buggy 
Labels: name: auto:bug, name: integration: chroma

### Issue you'd like to raise.

The tests for Langchain's `Chroma` are currently broken. The reason is:

- By default, chroma persists locally, for ease of use
- All the tests use the same collection name
- Thus, when a new `Chroma` is created, it sees the collections persisted from previous tests. 

This causes inconsistent behavior, since the contents of the collection depend on the order of the tests, not just what the test itself added.

### Suggestion:

The solution is to create fixtures which appropriately teardown the `Chroma` after every test.
Don't pass EF to chroma
Labels: name: lgtm, name: area: vector store, name: auto:bug, name: integration: chroma

  - **Description:** 

Recently Chroma rolled out a breaking change on the way we handle embedding functions, in order to support multi-modal collections. 

This broke the way LangChain's `Chroma` objects get created, because we were passing the EF down into the Chroma collection: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023

However, internally, we are never actually using embeddings on the chroma collection - LangChain's `Chroma` object calls it instead. Thus we just don't pass an `embedding_function` to Chroma itself, which fixes the issue.
OpenAIEmbeddings 
Labels: name: area: agent, name: area: embeddings, name: auto:bug, name: area: models

### System Info

Traceback:
File "/home/appuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
    exec(code, module.__dict__)
File "/app/demo-chatbot/updatedapp.py", line 351, in <module>
    asyncio.run(main())
File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
File "/app/demo-chatbot/updatedapp.py", line 214, in main
    vectors = file.get_vector()
File "/app/demo-chatbot/file.py", line 39, in get_vector
    self.save_vector()
File "/app/demo-chatbot/file.py", line 30, in save_vector
    embeddings = OpenAIEmbeddings(openai_api_key=st.secrets["OPEN_AI_KEY"])
File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
File "pydantic/main.py", line 1102, in pydantic.main.validate_model
File "/home/appuser/venv/lib/python3.9/site-packages/langchain/embeddings/openai.py", line 166, in validate_environment
    values["client"] = openai.Embedding

### Who can help?

@agola11 OpenAI changed the code https://platform.openai.com/docs/guides/embeddings/what-are-embeddings

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just follow a funciton similar to 

    def save_vector(self):
        self.create_images()
        self.extract_text()
        loader = TextLoader(f"{self.dir}/info.txt")
        documents = loader.load()
        text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
        documents = text_splitter.split_documents(documents)
        embeddings = OpenAIEmbeddings()
        vectors = FAISS.from_documents(documents, embeddings)

### Expected behavior

No traceback adding .create