alfin06/swegent-issues-filtered huggingface dataset:
[Feature Request]: add top_p and temperature parameters to OpenAIAssistantAgent

### Feature Description

I’d like to suggest adding support for the top_p and temperature parameters to the OpenAIAssistantAgent. These parameters are available in the OpenAI library for a while now and they're compatible with version of openai library llamaindex is on right now.

If this is OK, I’d like to work on adding it.

### Reason

_No response_

### Value of Feature

_No response_

Labels: enhancement, triage
State: closed
[Issue]: Support with syntax for all `LLMConfig` usages

### Describe the issue

`LLMConfig` is used in components outside of agents, such as `BrowserUseTool`, ` AgentOptimizer` and more, which do not support the `with llm_config:` syntax and instead require the configuration to be passed explicitly.

### Steps to reproduce

_No response_

### Screenshots and logs

_No response_

### Additional Information

_No response_

Labels: help from community wanted
State: closed
[Bug]: Asyncio task created without holding a reference to it in astream_chat()

### Bug Description

In the following two `astream_chat()` implementations, asyncio tasks are being created that are not awaited and also not being referenced after being created:

https://github.com/run-llama/llama_index/blob/adc1beb2fa392ff2beb86a3eee22b94a058cd7d1/llama-index-core/llama_index/core/chat_engine/simple.py#L205
https://github.com/run-llama/llama_index/blob/adc1beb2fa392ff2beb86a3eee22b94a058cd7d1/llama-index-core/llama_index/core/chat_engine/condense_question.py#L362

To my current understanding, this leads to undefined behaviour as the Python Garbage Collector is free to remove these unreachable objects at any point in time:
https://textual.textualize.io/blog/2023/02/11/the-heisenbug-lurking-in-your-async-code/

### Version

v0.12.16

### Relevant Logs/Tracbacks
Some background info:
Started looking at this when I noticed messages such as the following when overloading the LLM behind the application, leading to connection errors:

```
ERROR:    asyncio [12-02-2025 14:07:13] Task exception was never retrieved
future: <Task finished name='Task-794' coro=<Dispatcher.span.<locals>.async_wrapper() done, defined at /usr/local/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py:349> exception=APIConnectionError('Connection error.')>
```

This is also a sign that there was a task that never was awaited (not sure if this happened when running into the exception during the exact task linked above, though).

Labels: bug, triage
State: closed
[Feature Request]: Add support for new gemini models

### Feature Description

Add support for new gemini models in llama_index integrations

![Image](https://github.com/user-attachments/assets/525f93ea-2603-4d84-a1a3-d7920f50d186)

List found at https://aistudio.google.com/


### Reason

More models help developers have a wider variety of choices.

### Value of Feature

More model choice

Labels: enhancement, triage
State: closed
[Bug]: Can't provide images to Gemini models

### Bug Description

I create a ChatMessage with one text and multiple image blocks but only the text block is provided to Gemini.

The problem is during the message conversion:
https://github.com/run-llama/llama_index/blob/v0.12.19/llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/base.py#L228

This leads to:
https://github.com/run-llama/llama_index/blob/v0.12.19/llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/gemini_utils.py#L56

Here the message content is used but that is a legacy solution that returns only the text content of the message.
The blocks field should be used here:

https://github.com/run-llama/llama_index/blob/v0.12.19/llama-index-core/llama_index/core/base/llms/types.py#L177

In the _convert_gemini_part_to_prompt method there is code to handle images but as far as I can tell this method is only called with string input so that code is never executed.



### Version

v0.12.19

### Steps to Reproduce

Create a ChatMessage with a text block and an image block.
Create a Vertex LLM.
Call the chat method on the llm with the chat message and check what is provided for the LLM.

### Relevant Logs/Tracbacks

```shell

```

Labels: bug, triage
State: closed
[Feature Request]: AgentWorkflow should accept empty tool list

### Feature Description

For a demo of a very simple agent I wanted to create an AgentWorkflow with no tools. Validation prevented me from doing this. Can we make it possible to do?

Repro: https://colab.research.google.com/drive/1McjGJ8R8yowzhA1Cg_Sn0gHNZqIzRHBo#scrollTo=4UK1-Pr4qItW

### Reason

_No response_

### Value of Feature

_No response_

Labels: enhancement, triage
State: closed
[Feature Request]: Add client_kwargs Parameter to OllamaEmbedding Class

### Feature Description

The current implementation of the OllamaEmbedding class does not allow users to pass additional keyword arguments to the Client and AsyncClient instances. This limitation restricts the ability to customize client configurations, such as setting headers or authentication tokens. 

To address this, we propose adding a **_client_kwargs_** parameter to the **_init_** method of the OllamaEmbedding class. This change will enable users to pass a dictionary of keyword arguments that will be applied to both the **Client** and **AsyncClient** instances, providing greater flexibility and customization options.

### Reason

Unable to call Ollama API server which is enforced by Auth

### Value of Feature

Make Ollama API calls with Auth token

Labels: enhancement, triage
State: closed
[Bug]: Inconsistent Streaming Response Handling between OpenAI and Gemini LLM Classes

### Bug Description

## Description
There's an inconsistency in how the `stream_complete` method returns data between different LLM provider classes. Specifically, there's a difference in how streaming chunks are handled in the `CompletionResponse` objects between OpenAI and Gemini implementations:

- **OpenAI**: Places the new chunk in `delta` and the accumulated text in `text`
- **Gemini**: Places the new chunk in `text` and nothing in `delta`

This inconsistency makes it difficult to write provider-agnostic code that handles streaming responses.

## Current Behavior
When streaming completions:

**OpenAI Implementation**
- `response.text` - Contains the complete text accumulated so far
- `response.delta` - Contains only the new chunk that was just received

**Gemini Implementation**
- `response.text` - Contains only the new chunk that was just received
- `response.delta` - Is `None`

## Expected Behavior
All LLM provider classes should follow the same pattern in their `stream_complete` method:
- `response.text` should consistently contain either the accumulated text or just the chunk
- `response.delta` should consistently contain either the new chunk or be `None`

Based on the `CompletionResponse` class documentation, the expected behavior should be:
- `response.text`: "Text content of the response if not streaming, or if streaming, the current extent of streamed text."
- `response.delta`: "New text that just streamed in (only relevant when streaming)."

## Relevant Code References
**In OpenAI**:
https://github.com/run-llama/llama_index/blob/afe3dcbf59a972fc697571d92b4972dd1ef1b751/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/base.py#L599-L625

**In Gemini**:
https://github.com/run-llama/llama_index/blob/afe3dcbf59a972fc697571d92b4972dd1ef1b751/llama-index-integrations/llms/llama-index-llms-gemini/llama_index/llms/gemini/base.py#L223-L235
https://github.com/run-llama/llama_index/blob/afe3dcbf59a972fc697571d92b4972dd1ef1b751/llama-index-integrations/llms/llama-index-llms-gemini/llama_index/llms/gemini/utils.py#L40-L57

## Suggested Fix
Update the Gemini implementation to match the OpenAI implementation's behavior (or vice versa), ensuring that:
- One field consistently contains the accumulated text
- One field consistently contains just the new chunk

## Additional Information
This inconsistency makes it challenging to create tools or abstractions that work with multiple LLM providers, as special handling is required for each provider's streaming response format.

### Version

0.12.23

### Steps to Reproduce

- Use the OpenAI LLM class's `stream_complete` method and observe the values of `text` and `delta` in each yielded `CompletionResponse`.
- Use the Gemini LLM class's `stream_complete` method and observe the values of `text` and `delta` in each yielded `CompletionResponse`.

### Relevant Logs/Tracbacks

```shell

```

Labels: bug, triage
State: closed
[Bug]: When context is used in workflow, there is a memory leak

### Bug Description

I use llamaindex workflow to build Agent. I set llm objects, vector store objects, retrieved nodes in context.  We initialize the workflow globally and run the with different context for different users. Each time, we initialize new the context with above and run the global workflow. The memory is not cleared. 

I used ctx.data.clear(). But this doesn't clear the full memory.


### Version

0.12.10.post1

### Steps to Reproduce

# Steps to Reproduce
I used this simple script to show my issue. If I initialize context with 10MB of list and clear, it has no memory leak. But if the context is used in running the workflow, the memory got to increase. I used memray to see the increase.

Memory not increased while I use list in context and not run in workflow

![Image](https://github.com/user-attachments/assets/d05402d7-bd8b-4d7d-9375-d02c4ad7954f)

Memory Increased while I use the context in running the workflow

![Image](https://github.com/user-attachments/assets/78474d0c-4779-4278-b46b-08e60e6fb36f)

Run the below script to reproduce
```py
import asyncio
from llama_index.core.workflow import Context, Workflow, step, StartEvent, StopEvent

class SimpleWorklfow(Workflow):
    @step
    async def start_worklow(
        self,
        ctx: Context,
        ev: StartEvent,
    ) -> StopEvent:
        return StopEvent(result="hai")


async def run_agent():
    workflow = SimpleWorklfow()
    ctx = Context(workflow=workflow)
    lis = [i for i in range(1000000)]
    await ctx.set("lis", lis)
    handler = workflow.run(ctx=ctx) # comment this when you don't run the workflow
    ctx.data.clear()


async def main():
    ctx = await run_agent()


if __name__ == "__main__":

    asyncio.run(main())
```
Is this becuase of instrumentation or the workflow implementation?

You can use memray to see the memory 

`memray run -o workflow.bin main.py`


### Relevant Logs/Tracbacks

```shell
I used memray to track memory

Memory not increased while I use list in context and not run in workflow

![Image](https://github.com/user-attachments/assets/d05402d7-bd8b-4d7d-9375-d02c4ad7954f)

Memory Increased while I use the context in running the workflow

![Image](https://github.com/user-attachments/assets/78474d0c-4779-4278-b46b-08e60e6fb36f)
```

Labels: bug, triage
State: closed
[Bug]: `step` decorator for workflow steps doesn't support postponed type annotations

### Bug Description

Sometimes while working in Python I use `from __future__ import annotations`.

Turns out, `@step` decorator for workflow steps can't recognize "postponed type annotations". When you use `annotations` from `__future__` types in declarations are changed to **strings**.

### Version

0.12.25

### Steps to Reproduce

Here is an example that doesn't work properly:

```python
from __future__ import annotations

import asyncio

from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step


class MyStart(StartEvent):
    pass


class Test(Workflow):
    @step
    def example(self, ev: MyStart) -> StopEvent:
        print(ev)
        return StopEvent()


async def main():
    w = Test()
    await w.run()


asyncio.run(main())
```

### Relevant Logs/Tracbacks

```shell
Traceback (most recent call last):
  File "[REDACTED]/test.py", line 9, in <module>
    class Test(Workflow):
    ...<3 lines>...
            return StopEvent()
  File "[REDACTED]/test.py", line 10, in Test
    @step
     ^^^^
  File "[REDACTED]/.venv/lib/python3.13/site-packages/llama_index/core/workflow/decorators.py", line 86, in step
    decorator(func)
    ~~~~~~~~~^^^^^^
  File "[REDACTED]/.venv/lib/python3.13/site-packages/llama_index/core/workflow/decorators.py", line 60, in decorator
    validate_step_signature(spec)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "[REDACTED]/.venv/lib/python3.13/site-packages/llama_index/core/workflow/utils.py", line 135, in validate_step_signature
    raise WorkflowValidationError(msg)
llama_index.core.workflow.errors.WorkflowValidationError: Step signature must have at least one parameter annotated as type Event
```

Labels: bug, triage
State: closed
[Bug]: autogen.runtime_logging.start do not write the log into sqllite when using it in python with multiy thread

### Describe the bug

this is the error info:
[SqliteLogger] log_chat_completion error: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 13416 and this is thread id 21888.

### Steps to reproduce

_No response_

### Expected Behavior

we expect it can use in multy thread

### Screenshots and logs

![image](https://github.com/microsoft/autogen/assets/17542690/2fe6da02-b239-474e-a3d0-b57ace9ad69d)


### Additional Information

_No response_

Labels: logging
State: closed
[Bug]: Cannot use async Agents with LiteLLM and tools

### Bug Description

llama-index 0.12.25

Error in step 'run_agent_step': 'LiteLLM' object has no attribute 'astream_chat_with_tools'


https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/workflow/function_agent.py references the function astream_chat_with_tools, but it doesn't seem to exist in any LLM object (even if in my case, I want to run it with a LiteLLM model)

### Version

0.12.25

### Steps to Reproduce

Take
https://github.com/run-llama/python-agents-tutorial/blob/main/6_multi_agent.py
replace llm by:
llm = LiteLLM(model="XXX", base_url="YYY", api_key=os.getenv("OPENAI_API_KEY"))
run...

### Relevant Logs/Tracbacks

```shell
Traceback (most recent call last):
  File "/.venv/lib/python3.12/site-packages/llama_index/core/workflow/context.py", line 500, in _step_worker
    new_ev = await instrumented_step(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 368, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/multi_agent_workflow.py", line 329, in run_agent_step
    agent_output = await agent.take_step(
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/llama_index/core/agent/workflow/function_agent.py", line 42, in take_step
    response = await self.llm.astream_chat_with_tools(  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/pydantic/main.py", line 891, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'LiteLLM' object has no attribute 'astream_chat_with_tools'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/.venv/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 286, in handle_future_result
    raise exception
  File "/.venv/lib/python3.12/site-packages/llama_index/core/workflow/workflow.py", line 394, in _run_workflow
    raise exception_raised
  File "/.venv/lib/python3.12/site-packages/llama_index/core/workflow/context.py", line 508, in _step_worker
    raise WorkflowRuntimeError(
llama_index.core.workflow.errors.WorkflowRuntimeError: Error in step 'run_agent_step': 'LiteLLM' object has no attribute 'astream_chat_with_tools'
```

Labels: bug, triage
State: closed
[Feature Request]: Guardrails

### Is your feature request related to a problem? Please describe.

_No response_

### Describe the solution you'd like

_No response_

### Additional context

OpenAI Agents SDK offers this [feature](https://openai.github.io/openai-agents-python/guardrails/) and it is a useful one.

Labels: enhancement
State: closed
[FEATURE] Support for stateless MCP Servers

### Problem Statement

One approach to a scalable MCP server architecture is to have multiple stateless server instances (described [here](https://github.com/modelcontextprotocol/typescript-sdk/tree/main/src/examples#multi-node-deployment)). This means there is no persistent session between client and server. Presently when using Streamable HTTP with the MCP Client provided with Strands Agents throws a MCPClientInitializationError whenever it tries to invoke a tool on a stateless server. 

### Proposed Solution

Allow MCPClient to be created in stateless mode.

### Use Case

For use with highly scalable backend MCP Server infrastructure. 

### Alternatives Solutions

_No response_

### Additional Context

_No response_

Labels: bug
State: closed
[Feature Request]: IBM WatsonxLLM doesn't support `persistent_connection` configurable

### Feature Description

IBM watsonx Python SDK added the `persistent_connection` flag in 1.1.2 and enabled it by default. This causes problems with certain environments where network connections are cut from time to time, such as OpenShift/Kubernetes. There should be a way to disable it in llama_index.

### Reason

It is not possible to disable the persistent connection via llama_index today.

### Value of Feature

_No response_

Labels: enhancement, triage
State: closed
[Bug]: `GmailToolSpec.load_data()` applies `GmailToolSpec.search_messages()` without arguments

### Bug Description

`GmailToolSpec.load_data()` applies `GmailToolSpec.search_messages()` without arguments:
```
  def load_data(self) -> List[Document]:
      """Load emails from the user's account."""
      self._cache_service()

      return self.search_messages()
```
### Version

latest

### Steps to Reproduce

```
from llama_hub.tools.gmail.base import GmailToolSpec

tool_spec = GmailToolSpec()

agent = OpenAIAgent.from_tools(tool_spec.to_tool_list())
agent.chat("...")
```

### Relevant Logs/Tracbacks

```shell
in `load_data() `search_messages()` missing 1 required positional argument: 'query'
```

Labels: bug, triage
State: closed
[Bug]: CohereEmbedding fails for custom base_url when used in multiprocessing

### Bug Description

When using CohereEmbedding with a custom endpoint (e.g. Azure AI Foundry) and in an IngestionPipeline, the instance will be serialized since the run method uses multiprocessing. The spawned processes don't retain the custom base_url and fail with invalid API key errors, as they now try to access the original Cohere API.


I'm about to create a PR for this issue.

### Version

0.12.33

### Steps to Reproduce

vector_store=<MY_VECTOR_STORE>
documents=<MY_LIST_OF_DOCS>

embedding = CohereEmbedding(api_key="<MY_API_KEY>", base_url="<MY_API_ENDPOINT>", input_type="search_document")

ingestion_pipeline = IngestionPipeline(transformations=[embedding], vector_store=vector_store)
ingestion_pipeline.run(documents=documents, num_workers=os.cpu_count(), show_progress=True)

### Relevant Logs/Tracbacks

```shell

```

Labels: bug, triage
State: closed
[Feature Request][BUG]: AgentRunner arun_step have to make dispatcher asynchronous as well

### Feature Description

I think you should add an abstract `ahandle `method to the `BaseEventHandler` that will handle asynchronous calls to `dispatcher.aevent()` or something similar.
Because if I run an application in a prod for many people, I don't want to suffer from synchronous dispatching.

### Reason

There no existing approaches, cause I need to create a subclass from AgentRunner to fix it. 

### Value of Feature

It will speed up the process of step execution.

Labels: enhancement, P1
State: closed
[Feature Request]: Readers integration with MarkItDown and Embeddings integration with Chonkie AutoEmbeddings

### Feature Description

The feature would include:
- A new reader that would use MarkItDown as a file parser and extractor
- A new embedding module, which uses [Chonkie AutoEmbeddings](https://docs.chonkie.ai/embeddings/auto-embeddings) as a comfortable unified interface to load several different embedding models (from different providers, such as Sentence Transformers, OpenAI, Jina...) within only one class
**I already worked on these two integrations and I would be able to open a PR with them**, just let me know if these integrations could be of any use or if you prefer them not to be included :))

### Reason

I have been working with Chonkie and with MarkItDown extensively in recent times and, since they are already integrated or I am planning to integrate them with LlamaIndex in the very next future, it would be nice to have these two integrations.

### Value of Feature

_No response_

Labels: enhancement, triage
State: closed
[Bug]: azure openai embedding init error with azure ad

### Bug Description

introduced by #18703 should check if use azure_ad before retrieve api_key from environment.

### Version

0.3.3

### Steps to Reproduce

init azure embedding model with azure ad.

### Relevant Logs/Tracbacks

```shell

```

Labels: bug, triage
State: closed
[Feature Request]: resolve methods of ContentBlock classes error when empty bytes

### Feature Description

`resolve` methods of `ContentBlock` classes should raise an error when they yield empty bytes, to avoid empty documents to be passed to LLMs. 

### Reason

See discussion under issue #18789

### Value of Feature

_No response_

Labels: enhancement, triage
State: closed
[Feature Request]: Add Embedding Caching

### Feature Description

To reduce redundant embedding computations, the `BaseEmbedding` class could be extended to accept an optional `BaseKVStore` parameter for embedding caching. This enhancement addresses concerns raised in issues #18840 and #18843, particularly regarding repeated query embedding computations in `NLSQLRetriever` and potentially other retriever classes.

Integrating this feature is straightforward: the query embedding methods — `get_query_embedding` and `aget_query_embedding` — can first check the cache before performing embedding computation.

### Reason

In certain cases—such as in the `NLSQLRetriever` class—multiple vector store retrievers are invoked. Each retriever processes the same query independently, leading to redundant embedding computations. This results in unnecessary computational and token overhead, which in turn increases retrieval latency and overall cost.

### Value of Feature

Implementing a caching mechanism in the `BaseEmbedding` class offers a simple and generic solution to eliminate redundant embedding computations, effectively reducing query latency and cost.

Labels: enhancement, triage
State: closed