GroQ + Autogen
Autogen is an LLM framework developed by microsoft. It provides a variety of features that may be lacking in other frameworks such as multi agent tasks, nested chats, code execution and so on. It would be interesting to see how agents powered by lightning fast groq perform with such a feature rich framework. Our notebook follows this guide.
In [1]:
import os
api_key = os.getenv("GROQ_API_KEY")
if api_key is None:
raise ValueError("Groq API key environment variable not set")
In [16]:
config_list = [
{
"model": "llama3-8b-8192",
"api_key": api_key,
# "api_type": "groq", # Removed as explained here https://stackoverflow.com/a/77560277
"base_url": "https://api.groq.com/openai/v1",
# "frequency_penalty": 0.5,
# "max_tokens": 2048,
# "presence_penalty": 0.2,
# "seed": 42,
# "temperature": 0.5,
# "top_p": 0.2,
}
]
In [17]:
from pathlib import Path
from autogen import AssistantAgent, UserProxyAgent
from autogen.coding import LocalCommandLineCodeExecutor
In [18]:
# Setting up the code executor
workdir = Path("coding")
workdir.mkdir(exist_ok=True)
code_executor = LocalCommandLineCodeExecutor(work_dir=workdir)
In [19]:
# Setting up the agents
# The UserProxyAgent will execute the code that the AssistantAgent provides
user_proxy_agent = UserProxyAgent(
name="User",
code_execution_config={"executor": code_executor},
is_termination_msg=lambda msg: "FINISH" in msg.get("content"),
)
system_message = """You are a helpful AI assistant who writes code and the user executes it.
Solve tasks using your coding and language skills.
In the following cases, suggest python code (in a python coding block) for the user to execute.
Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill.
When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user.
Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user.
If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.
When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.
IMPORTANT: Wait for the user to execute your code and then you can reply with the word "FINISH". DO NOT OUTPUT "FINISH" after your code block."""
# The AssistantAgent, using Groq's model, will take the coding request and return code
assistant_agent = AssistantAgent(
name="Groq Assistant",
system_message=system_message,
llm_config={"config_list": config_list},
)
In [21]:
# Start the chat, with the UserProxyAgent asking the AssistantAgent the message
chat_result = user_proxy_agent.initiate_chat(
assistant_agent,
message="Provide code to count the number of prime numbers from 1 to 10000.",
)
User (to Groq Assistant): Provide code to count the number of prime numbers from 1 to 10000. -------------------------------------------------------------------------------- Groq Assistant (to User): Here is a Python script to count the number of prime numbers from 1 to 10000: ```python def is_prime(n): if n <= 1: return False if n == 2: return True if n % 2 == 0: return False max_divisor = int(n**0.5) + 1 for d in range(3, max_divisor, 2): if n % d == 0: return False return True count = 0 for num in range(1, 10001): if is_prime(num): count += 1 print("Number of prime numbers from 1 to 10000:", count) ``` This script defines a helper function `is_prime` to check if a number is prime, and then iterates over the range from 1 to 10000, incrementing a counter for each prime number found. Finally, it prints the total count of prime numbers. --------------------------------------------------------------------------------
>>>>>>>> NO HUMAN INPUT RECEIVED. >>>>>>>> USING AUTO REPLY... >>>>>>>> EXECUTING CODE BLOCK (inferred language is python)... User (to Groq Assistant): exitcode: 0 (execution succeeded) Code output: Number of prime numbers from 1 to 10000: 1229 -------------------------------------------------------------------------------- Groq Assistant (to User): FINISH --------------------------------------------------------------------------------
GroQ + Autogen + Tool Use
We push our knowledge further by trying out function calling.
In [33]:
from autogen import AssistantAgent, UserProxyAgent
from typing import Literal
from typing_extensions import Annotated
import json
In [34]:
config_list = [
{
"model": "llama3-groq-8b-8192-tool-use-preview",
"base_url": "https://api.groq.com/openai/v1",
"api_key": api_key,
}
]
In [35]:
# Create the agent for tool calling
chatbot = AssistantAgent(
name="chatbot",
system_message="""For currency exchange and weather forecasting tasks,
only use the functions you have been provided with.
Output 'HAVE FUN!' when an answer has been provided.""",
llm_config={"config_list": config_list},
)
# Note that we have changed the termination string to be "HAVE FUN!"
user_proxy = UserProxyAgent(
name="user_proxy",
is_termination_msg=lambda x: x.get("content", "") and "HAVE FUN!" in x.get("content", ""),
human_input_mode="NEVER",
max_consecutive_auto_reply=1,
)
In [36]:
# Currency Exchange function
CurrencySymbol = Literal["USD", "EUR"]
# Define our function that we expect to call
def exchange_rate(base_currency: CurrencySymbol, quote_currency: CurrencySymbol) -> float:
if base_currency == quote_currency:
return 1.0
elif base_currency == "USD" and quote_currency == "EUR":
return 1 / 1.1
elif base_currency == "EUR" and quote_currency == "USD":
return 1.1
else:
raise ValueError(f"Unknown currencies {base_currency}, {quote_currency}")
# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Currency exchange calculator.")
def currency_calculator(
base_amount: Annotated[float, "Amount of currency in base_currency"],
base_currency: Annotated[CurrencySymbol, "Base currency"] = "USD",
quote_currency: Annotated[CurrencySymbol, "Quote currency"] = "EUR",
) -> str:
quote_amount = exchange_rate(base_currency, quote_currency) * base_amount
return f"{format(quote_amount, '.2f')} {quote_currency}"
# Weather function
# Example function to make available to model
def get_current_weather(location, unit="fahrenheit"):
"""Get the weather for some location"""
if "chicago" in location.lower():
return json.dumps({"location": "Chicago", "temperature": "13", "unit": unit})
elif "san francisco" in location.lower():
return json.dumps({"location": "San Francisco", "temperature": "55", "unit": unit})
elif "new york" in location.lower():
return json.dumps({"location": "New York", "temperature": "11", "unit": unit})
else:
return json.dumps({"location": location, "temperature": "unknown"})
# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Weather forecast for US cities.")
def weather_forecast(
location: Annotated[str, "City name"],
) -> str:
weather_details = get_current_weather(location=location)
weather = json.loads(weather_details)
return f"{weather['location']} will be {weather['temperature']} degrees {weather['unit']}"
In [37]:
# start the conversation
res = user_proxy.initiate_chat(
chatbot,
message="What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday? Throw a few holiday tips in as well.",
summary_method="reflection_with_llm",
)
print(f"LLM SUMMARY: {res.summary['content']}")
user_proxy (to chatbot): What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday? Throw a few holiday tips in as well. -------------------------------------------------------------------------------- chatbot (to user_proxy): ***** Suggested tool call (call_fxry): weather_forecast ***** Arguments: {"location": "New York"} ************************************************************* ***** Suggested tool call (call_6xgr): currency_calculator ***** Arguments: {"base_amount": 123.45, "base_currency": "EUR", "quote_currency": "USD"} **************************************************************** -------------------------------------------------------------------------------- >>>>>>>> EXECUTING FUNCTION weather_forecast... >>>>>>>> EXECUTING FUNCTION currency_calculator... user_proxy (to chatbot): user_proxy (to chatbot): ***** Response from calling tool (call_fxry) ***** New York will be 11 degrees fahrenheit ************************************************** -------------------------------------------------------------------------------- user_proxy (to chatbot): ***** Response from calling tool (call_6xgr) ***** 135.80 USD ************************************************** -------------------------------------------------------------------------------- chatbot (to user_proxy): ***** Suggested tool call (call_ydzn): weather_forecast ***** Arguments: {"location":"New York"} ************************************************************* --------------------------------------------------------------------------------
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[37], line 8 1 # start the conversation 2 res = user_proxy.initiate_chat( 3 chatbot, 4 message="What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday? Throw a few holiday tips in as well.", 5 summary_method="reflection_with_llm", 6 ) ----> 8 print(f"LLM SUMMARY: {res.summary['content']}") TypeError: string indices must be integers
In [ ]: