alfin06/swegent-issues-filtered huggingface dataset:
SQLAgent requiring "chat_history"? 

### Bug Description

The SQLAgent shows error "chat_history not found in SQLAgentComponent". 

### Reproduction

1. Build a flow with "OpenAI" (or equivalent LLM) and "SQLAgent" component
2. Enter in API key for OpenAI, input and database URI for SQLAgent
3. Run the SQLAgent
4. Returns error below

Error building Component SQLAgent: 

chat_history not found in SQLAgentComponent

Traceback (most recent call last):
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/graph/vertex/base.py", line 708, in _build_results
    result = await initialize.loading.get_instance_results(
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/interface/initialize/loading.py", line 68, in get_instance_results
    return await build_component(params=custom_params, custom_component=custom_component)
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/interface/initialize/loading.py", line 145, in build_component
    build_results, artifacts = await custom_component.build_results()
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/custom/custom_component/component.py", line 837, in build_results
    return await self._build_with_tracing()
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/custom/custom_component/component.py", line 819, in _build_with_tracing
    _results, _artifacts = await self._build_results()
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/custom/custom_component/component.py", line 883, in _build_results
    result = await method()
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/base/agents/agent.py", line 81, in message_response
    message = await self.run_agent(agent=agent)
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/base/agents/agent.py", line 136, in run_agent
    if self.chat_history:
  File "/langflow1.1.0/venv1.1.0/lib/python3.10/site-packages/langflow/custom/custom_component/component.py", line 665, in __getattr__
    raise AttributeError(msg)
AttributeError: chat_history not found in SQLAgentComponent


### Expected behavior

Expecting a response from the SQLAgent, instead of an error message for "chat_history"

### Who can help?

_No response_

### Operating System

MacOS

### Langflow Version

1.1

### Python Version

3.10

### Screenshot

<img width="496" alt="Screenshot 2024-11-18 at 5 03 33 pm" src="https://github.com/user-attachments/assets/02d15b4f-33c9-4b0a-8f9d-2c989deda86d">
<img width="505" alt="Screenshot 2024-11-18 at 5 03 56 pm" src="https://github.com/user-attachments/assets/3c26c21b-a080-45b4-83c2-18d2680ab0c2">


### Flow File

_No response_

Labels: bug
State: closed
Add startup / shutdown methods to Agents in Core and AgentChat

## What
Across many usecases, agents might need to provision some resources that take a while to startup (e.g., a db connection, a webbrowser instance, a code executor endpoint, a file handle etc). In some ways similar to how [FastAPI handles startup and shutdown via lifespan events. ](https://fastapi.tiangolo.com/advanced/events/)
Similarly these resources need to be shut down once the agent is done running e.g., close a db connection, destroy a browser instance, close files etc.
This issue proposes adding some protocol spec to Agents in core and agentchat that requires agents to implement startup and shutdown methods that the runtime can call. 

## How 
Some proposed steps toward accomplishing this 

- [x] Design a protocol for startup/shutdown lifecycle methods
- [x] Update the core runtime to honor these methods
- [x] Update core and agentchat agent base agent classes to follow protocol
- [ ] Update example agents like websurfer, assistant agent in agentchat to use this design where appropriate
- [ ] Add documentation and examples for implementing startup/shutdown in custom agents 
- [ ] Consider cases for resource pooling and if the framework should enable this e.g., a browser instance 

This should not be a breaking change for any users of existing core or agentchat agents. However users of custom agents might need to implement these methods?

Related to #4699 

Labels: proj-core, proj-agentchat
State: closed
[dotnet] Duplicate `user` role message send to agent

### What happened?

Duplicate message send to an agent for after initialize and get reply from one.

Here is the request generated by AutoGen and I observed via LM Studio logs.
```log
2024-12-17 12:47:07  [INFO]
Received POST request to /v1/chat/completions with body: {
  "messages": [
    {
      "role": "system",
      "content": "You are a student that answer question from teacher",
      "name": "student"
    },
    {
      "role": "assistant",
      "content": "Hey teacher, please create math question for me.",
      "name": "student"
    },
    {
      "role": "user",
      "content": "Here's a simple addition problem for you:\n\nIf I have 2 blocks and my friend gives me 1 more block, how many blocks do I have now?\n\nWhat is your answer?",
      "name": "teacher"
    },
    {
      "role": "user",
      "content": "Here's a simple addition problem for you:\n\nIf I have 2 blocks and my friend gives me 1 more block, how many blocks do I have now?\n\nWhat is your answer?",
      "name": "teacher"
    }
  ],
  "model": "llama-3.2-3b-instruct"
}
```

### What did you expect to happen?

There only should be 1 `user` role in the messages list.


### How can we reproduce it (as minimally and precisely as possible)?

Run Following C# code (from [one of the examples](https://github.com/microsoft/autogen/blob/main/dotnet/samples/AutoGen.BasicSamples/Example02_TwoAgent_MathChat.cs)).

```csharp
var config = new LMStudioConfig("localhost", 1234);


var teacher = new AssistantAgent(
    name: "teacher",
    systemMessage: @"You are a teacher that create pre-school math question for student and check answer.
If the answer is correct, you stop the conversation by saying [COMPLETE].
If the answer is wrong, you ask student to fix it.",
    llmConfig: new ConversableAgentConfig {
            Temperature = 0.0f,
            ConfigList = [config]
    }
).RegisterPrintMessage();

var student = new AssistantAgent(
    name: "student",
    systemMessage: "You are a student that answer question from teacher",
    llmConfig: new ConversableAgentConfig {
            Temperature = 0.0f,
            ConfigList = [config]
    }
).RegisterPrintMessage();

var conversation = await student.InitiateChatAsync(
    receiver: teacher,
    message: "Hey teacher, please create math question for me.",
    maxRound: 10);
```

### AutoGen version

https://github.com/microsoft/autogen/commit/55e157cb99b9418262aecf4a756f4fa260e54367

### Which package was this bug in

Other

### Model used

llama-3.2-3b-instruct

### Python version

N/A

### Operating system

macOS

### Any additional info you think would be helpful for fixing this bug

I found one possible solution, by update [`GroupChatExtension.cs`](https://github.com/microsoft/autogen/blob/main/dotnet/src/AutoGen.Core/Extension/GroupChatExtension.cs#L64) line 64 in `AutoGen.Core` package to avoid this duplication.

```csharp
if (!chatHistory.Contains(lastMessage)) {
    chatHistory = chatHistory.Append(lastMessage);
}
```

Just a suggestion.

Labels: dotnet, 0.2
State: closed
`ToolCallResultSummaryMessage` message type in AgentChat



> Thanks, I understand the intent, the challenge for me is that *both* tool call results and normal responses are now returned as :class:~autogen_agentchat.messages.TextMessage with no differentiating characteristics, making it difficult to determine the difference between a textmessage from the agent and the additional tool call response (or summary) from the tool without tracking additional state (e.g. did I just receive a toolcallresult prior to this message, if so, then the next TextMessage is the tool response, not from the LLM). 

> I think It would be better to not mix the types.  There is already toolcallresultmessage which was was previously the unambiguous way to identify tool results; now tool results are coming in two forms (toolcallresultmessage and another copy in a different format as textmessage) and it is now requires extra logic to try and determine if the textmessage is the tool results or from the LLM. 

> Can we create a clear message types for the new messages (e.g. toolcallresultsummarymessage) or some other disambiguating way in the message the determine the type of message?

_Originally posted by @jspv in https://github.com/microsoft/autogen/issues/4602#issuecomment-2551194331_
            

Labels: proj-agentchat
State: closed
OpenAI Client Token Usage not being tracked

### What happened?

The [BaseOpenAIChatCompletionClient](https://github.com/microsoft/autogen/blob/a271708a97bf754a5d94860554d78677f85a78fe/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py#L346) has a `total_usage `property that's supposed to track the token usage of the client.
However, it looks like the logic to do so is not implemented correctly.

Here's the function responsible of adding usage
```
def _add_usage(usage1: RequestUsage, usage2: RequestUsage) -> RequestUsage:
    return RequestUsage(
        prompt_tokens=usage1.prompt_tokens + usage2.prompt_tokens,
        completion_tokens=usage1.completion_tokens + usage2.completion_tokens,
    )
```
Here's how the function is being used
```
        _add_usage(self._total_usage, usage)
```

The function returns a `RequestUsage `object however, we are not even receiving that returned value.


### AutoGen version

0.4

### Which package was this bug in

Extensions

### Model used

_No response_

### Python version

_No response_

### Operating system

_No response_

### Any additional info you think would be helpful for fixing this bug

_No response_

Labels: proj-extensions
State: closed
Fix busy loop in single threaded agent runtime


### Discussed in https://github.com/microsoft/autogen/discussions/4772

<div type='discussions-op-text'>

<sup>Originally posted by **un-hongly** December 20, 2024</sup>
```
team.run_stream(task=input.message)
async for message in stream:
   pass
```

`async for message in stream` cause high cpu usages.

<img width="994" alt="Screenshot 2024-12-20 at 5 40 20 in the afternoon" src="https://github.com/user-attachments/assets/d5c310a3-fa0b-429d-be94-8f0d81e64e54" />
</div>

We need to replace this busy loop with proper eventing based on when messages are available to be consumed.

https://github.com/microsoft/autogen/blob/a4c1314432299605e1ef0777f4c5dc3ba9d6482b/python/packages/autogen-core/src/autogen_core/_single_threaded_agent_runtime.py#L446

Labels: proj-core
State: closed
Closure agent throws if it receives a message that it can't handle

### What happened?

"The issue (in 0.2) is just when you register a ClosureAgent with a given closure method and subscription, any published messages on the same sub will raise an exception if the ClosureAgent doesn't explicitly handle that message type.
 
https://microsoft.github.io/autogen/0.4.0.dev10/_modules/autogen_core/_closure_agent.html#ClosureAgent.on_message_impl
autogen_core._closure_agent — AutoGen
 
Our use case is that you have some process that completes asynchronously, so send_message isn't applicable, but you want to wait for the result of that process in a given function (vs. writing a handler for the result). 
 
so effectively publish_and_wait
 
the issue with the closure agent is kind of unrelated, it's just annoying that it raises an exception on unhandled messages 
 "

### What did you expect to happen?

politely log the message. 

### How can we reproduce it (as minimally and precisely as possible)?

send a message to closure agent to which it is not subscribed

### AutoGen version

0.4dev2

### Which package was this bug in

AgentChat

### Model used

_No response_

### Python version

_No response_

### Operating system

_No response_

### Any additional info you think would be helpful for fixing this bug

_No response_

Labels: proj-core
State: closed
processCallResponse should update messages before calling onToolCall

### Description

processChatResponse handles client tool call invocations. At the moment, it receives the tool-call message from the server, then calls onToolCall, then updates the frontend.

processChatResponse does not update the frontend before calling onToolCall. Therefore, the user will not see any progress until onToolCall returns. This can create a pretty bad UX if onToolCall is very slow.

### Code example

_No response_

### AI provider

_No response_

### Additional context

I think the solution is to simply [add another `execUpdate` in processCallResponse](https://github.com/vercel/ai/blob/main/packages/ui-utils/src/process-chat-response.ts#L232-L255):

```typescript
      updateToolInvocationPart(value.toolCallId, invocation);

      // invoke the onToolCall callback if it exists. This is blocking.
      // In the future we should make this non-blocking, which
      // requires additional state management for error handling etc.
      if (onToolCall) {
        execUpdate(); // <------- Add an update here

        const result = await onToolCall({ toolCall: value });
        if (result != null) {
          const invocation = {
            state: 'result',
            step,
            ...value,
            result,
          } as const;

          // store the result in the tool invocation
          message.toolInvocations![message.toolInvocations!.length - 1] =
            invocation;

          updateToolInvocationPart(value.toolCallId, invocation);
        }
      }

      execUpdate();
```

Labels: bug, ai/ui
State: closed
Reminder in error message to add model capabilities when model name does not exist

> I see. Ok if you encountered issues with this in the past then it's probably best not to ignore the model capabilities. I think it could be good to produce an error message for it. 

 _Originally posted by @gabayben in [#4785](https://github.com/microsoft/autogen/issues/4785#issuecomment-2558702505)_

Labels: good first issue, proj-extensions
State: closed
404 Error  Not Found in @ai-sdk/fal Image Generation in  basic usage of fal provider

### Description




**Description:**  
I am encountering a `404: Not Found` error when using the `@ai-sdk/fal` library's `experimental_generateImage` function in conjunction with the `fal.image("fast-sdxl")` model. The error occurs during the `POST` request, and the model URL `https://fal.run/fast-sdxl` returns a 404 error.  

---

**Steps to Reproduce:**  
1. Install the required packages:  
   ```bash
   npm install @ai-sdk/fal ai
   ```  
2. Implement the following code:  

```typescript
import { fal } from "@ai-sdk/fal";
import { NextResponse, NextRequest } from "next/server";
import { experimental_generateImage as generateImage } from "ai";
import fs from "fs";

export async function POST(request: NextRequest) {
  try {
    const { image } = await generateImage({
      model: fal.image("fast-sdxl"),
      prompt: "A serene mountain landscape at sunset",
    });

    const filename = `image-${Date.now()}.png`;
    fs.writeFileSync(filename, image.uint8Array);
    console.log(`Image saved to ${filename}`);
    return NextResponse.json("Image Generated");
  } catch (error) {
    console.error("An error occurred while generating the image:", error);
    return NextResponse.json("Failed to generate the image.", { status: 500 });
  }
}

```  

3. Run the server and make a POST request to the route where the `POST` function is defined.  

---

**Expected Behavior:**  
The `generateImage` function should generate an image using the `fast-sdxl` model and save it locally with the specified filename.

---

**Actual Behavior:**  
The function throws the following error:  

```plaintext
Error [AI_APICallError]: Not Found
at async POST (file:///D:/path/to/route.ts:8:20)
cause: undefined,
url: 'https://fal.run/fast-sdxl',
requestBodyValues: [Object],
statusCode: 404,
responseHeaders: [Object],
responseBody: '404: Not Found',
isRetryable: false,
data: undefined
```  

---

**Environment:**  
- **Library Version:**  
  - `@ai-sdk/fal`: `^0.0.2`  
  - `ai`: `^4.1.28`  
- **Node.js Version:** `v21.3.0`  
- **OS:** `Windows 10`

---



### Code example

```
import { fal } from "@ai-sdk/fal";
import { NextResponse, NextRequest } from "next/server";
import { experimental_generateImage as generateImage } from "ai";
import fs from "fs";

export async function POST(request: NextRequest) {
  try {
    const { image } = await generateImage({
      model: fal.image("fast-sdxl"),
      prompt: "A serene mountain landscape at sunset",
    });
    const filename = `image-${Date.now()}.png`;
    fs.writeFileSync(filename, image.uint8Array);
    console.log(`Image saved to ${filename}`);
    return NextResponse.json("Image Generated");
  } catch (error) {
    console.error("An error occurred while generating the image:", error);
    return NextResponse.json("Failed to generate the image.", { status: 500 });
  }
}
```


### AI provider

@ai-sdk/fal

### Additional context
I think in documentation example usage there is mistake because it runs on using fal/fast-sdxl in fal.image

Labels: bug
State: closed
Stop Conditions for specific agents only

### What feature would you like to be added?

It would be nice if `TextMentionTermination("...")` in v4 could be defined for specific agents i.e. it should only stop if the specific agent sends / receives (tbd) the mentioned text.

### Why is this needed?

More often than not the prompt contains instructions for the agent how it can stop the agent. The TextMentionTermination is currently triggered by the prompt already, which is not the intention. While a straightforward solution would be ignoring the prompt, I would suggest something more flexible, where one can specify the agent(s) that needs to send the stop condition.

Labels: needs-triage
State: closed
Convert AgentChat v0.4 messages to v0.2 format

### What feature would you like to be added?

For migration purposes it would be great to have a function that takes the `TaskResult` and converts it into the openAI format. Previous users of v2 might have additional evaluation, visualization code building up on the openAI format that is hard to migrate. Ideally a good implementation would be provided directly as part of the API along with some warning perhaps that it is for migration purposes.

For users needing this functionality, here is my quick implementation:
```py 
def convert_to_openai_format(messages: Sequence[AgentMessage]) -> list[dict[str, Any]]:
    """
    Convert a list of BaseMessage objects into OpenAI format with structured content.

    Args:
        messages: List of BaseMessage or its subclasses (TextMessage, MultiModalMessage).

    Returns:
        List of dictionaries in OpenAI message format with structured content.
    """
    openai_messages = []

    for message in messages:
        content_list = []

        if isinstance(message, TextMessage):
            content_list.append({"type": "text", "text": message.content})
        elif isinstance(message, MultiModalMessage):
            for item in message.content:
                if isinstance(item, str):
                    content_list.append({"type": "text", "text": item})
                elif isinstance(item, Image):
                    content_list.append(
                        {"type": "image_url", "image_url": item.data_uri}
                    )
                else:
                    raise ValueError(f"Unsupported content type in MultiModalMessage: {type(item).__name__}")
        else:
            raise ValueError(f"Unsupported message type: {type(message).__name__}")

        openai_messages.append({"role": message.source, "content": content_list})
    return openai_messages

```

The implementation is by no means complete. Especially the role part is not quite accurate. I made sort of the minimal implementation that get my specific code to work again. As said, ideally autogen would provide a more robust and more complete API. From the discord discussion around this, I understood that it is not quite trivial as the agent team already acts on a different layer, where certain information is not available anymore.

### Why is this needed?

To make the transition from v2 to v4 as easy as possible.

Labels: documentation, proj-agentchat
State: closed
Fix RealtimeAgent Swarm integration

When running agentchat_realtime_swarm.ipynb example.

After connecting to the client over call, the RealtimeAgent startup fails with `ValueError: Parameter 'func_or_tool' must be a function or a Tool instance, it is '<class 'method'>' instead.`

Labels: bug, swarm, realtime
State: closed
[Bug]: [autogen_ext] "ValueError: No stop reason found" always raised when llm usage returned

**Describe the bug**
When executing the sample code, this error is always thrown: "ValueError: No stop reason found".

**Steps to reproduce**
```python
import asyncio

from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_core.models import UserMessage, ModelInfo, ModelFamily

client = OpenAIChatCompletionClient(
    model="deepseek-chat",
    base_url="https://api.deepseek.com/v1",
    api_key="sk-xxxxx",
    model_info=ModelInfo(
        vision=False,
        function_calling=False,
        json_output=False,
        family=ModelFamily.UNKNOWN,
    ),
)


async def main():
    messages = [
        UserMessage(content="OUTPUT 1 only", source="user"),
    ]

    # Create a stream.
    stream = client.create_stream(messages=messages)

    # Iterate over the stream and print the responses.
    print("Streamed responses:")
    async for response in stream:  # type: ignore
        if isinstance(response, str):
            # A partial response is a string.
            print(response, flush=True, end="")
        else:
            # The last response is a CreateResult object with the complete message.
            print("\n\n------------\n")
            print("The complete response:", flush=True)
            print(response.content, flush=True)
            print("\n\n------------\n")
            print("The token usage was:", flush=True)
            print(response.usage, flush=True)


if __name__ == "__main__":
    asyncio.run(main())
```

**Model Used**
deepseek-chat

**Expected Behavior**
execute normally

**Screenshots and logs**
No response

**Additional Information**

I found the place where the problem occurs. According to the code logic here, only when the usage is None and the stop_reason is also None will the finish_reason be set to the stop_reason. It's very strange why this is done.

https://github.com/microsoft/autogen/blame/8a83262a905675f15dfca387ba4a8de7b6cf0635/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py#L695

```python
stop_reason = None
# ...
stop_reason = choice.finish_reason if chunk.usage is None and stop_reason is None else stop_reason
```

Labels: needs-triage, proj-extensions
State: closed