Agentic Task Delegation - Making Agents whole again

Imagine asking your AI agent, to plan your vacation, only to have it freeze up when you mention you need flight bookings, hotel reservations, AND a list of local attractions. Frustrating, init? Welcome to the world of single-task AI agents – jack of one trade, master of none.

Current ai systems, though powerful, make terrible actionable agents due to a multitude of problems ranging from weak tool integration to high costs and a lack of bandwidth! We @ Capx AI explore the idea of a Multi Agent Ecosystem powered by decentralised infra to offload actionable tasks to other agentic services on the network!

Limitations of a Single Agent System

TLDR;

• Bandwidth: Limited ability to process multiple tasks simultaneously.

• Processing power: Computational constraints when handling diverse tasks.

• Memory constraints: Difficulty juggling different contexts for various tasks.

• Task Complexity: Struggle with complex, multi-step tasks requiring diverse skills.

• Bandwidth refers to the computational resources a process on a computer can use. In the context of AI agent systems, Bandwidth refers to the agent's ability to process and communicate information. When an AI is tasked with handling multiple complex tasks simultaneously, it's akin to trying to squeeze too much data through a narrow pipe. The agent's communication channels become congested, leading to slower response times and potentially dropped or corrupted information. This bottleneck can significantly impair the agent's ability to process inputs and generate coherent outputs across multiple tasks.

• Processing power presents another critical limitation. While modern AI systems boast impressive computational capabilities, they still have upper limits. Handling diverse tasks on a single system is computationally intensive, as each task may require different algorithms, models, or processing techniques. It's like asking a chef to simultaneously prepare a five-course meal, balance the restaurant's books, and manage customer reservations.

• Memory constraints further compound these issues. AI agents, particularly those dealing with language and context-heavy tasks, need to juggle vast amounts of information. Each task requires its own set of contextual data, relevant knowledge, and operational parameters. When an agent attempts to handle multiple diverse tasks, it's forced to rapidly switch between different memory contexts. This constant context-switching can lead to information loss, confusion between tasks, or the inability to maintain long-term context for any single task.

• Task Complexity: The limitations of single-task AI agents become glaringly apparent when faced with complex, multi-step tasks. A single-task AI might excel at one aspect, say data analysis, but struggle with the nuanced language required for the written portions. The agent would need to constantly shift gears, potentially losing efficiency and quality in the process.

Task Delegation

Agentic Task Delegation represents a paradigm shift in how we approach AI problem-solving. At its core, this concept involves breaking down complex, multi-faceted tasks into smaller, more manageable components and distributing them across a network of hosts that run specialized agents while each host gets rewarded for completing the tasks.

This concept operates on the principle of intelligent task decomposition and distribution. When a complex task is introduced, the system first analyzes and breaks it down into smaller, more manageable subtasks. Each subtask is then assigned to a specialized agent best suited for that particular type of work. For instance, a task like "scrape StackOverflow" might be broken down into navigating the website, searching for relevant topics, extracting information, and formatting the data. Similarly, "writing reviews on HN" could involve analyzing the post, researching the topic, formulating an opinion, drafting the review, and posting it. In the case of "running OCR on documents," the process might include preprocessing images, applying OCR algorithms, post-processing the extracted text, and validating the results.

We propose an architecture:

Decentralized Agent Network

The Agent Task Delegation system operates on a decentralized network, leveraging blockchain technology to ensure transparency, security, and efficient coordination. Each node in the network represents a specialized agent or service, capable of performing specific tasks. The blockchain serves as a distributed ledger, recording task assignments, completions, and rewards. This decentralized approach eliminates single points of failure, enhances system resilience, and allows for permissionless scaling as new agents join the network as nodes. 'Task Delegation contract' is deployed onchain to govern the interaction between agents, hosting the logic of task delegation and communicating with the 'reward pool' escrow contract holding the rewards to be distributed to the agent nodes based on task assignment and task complexity.

This thereby brings onchain verifiability, transparency and ensures fair reward distribution in the system.

Distributed Edge Computing

One of the key areas of innovation in the proposed multi-agent task delegation framework is the introduction of 'distrbuted edge compute', enabling parallel processing of complex tasks across multiple nodes (edge devices). When a task enters the system, the Control Plane decomposes it into subtasks and distributes them to the most suitable agents in the network. Each agent processes its assigned subtask independently, utilizing its specialized capabilities over its independent compute node.

This parallel processing significantly reduces overall computation time and allows for the handling of resource-intensive tasks that would be impractical for a single agent. The task delegation contract along with the message queue employ advanced load balancing algorithm to optimize resource utilization across the network, ensuring efficient task completion even under varying workloads.

Components TLDR;

In this system, a central Control Plane acts as the orchestrator, analyzing incoming tasks, determining the best way to decompose them, and then assigning subtasks to the most suitable specialized agents in its network. These specialized agents, each proficient in handling specific types of tasks, work in parallel to complete their assigned portions. The central coordinator then aggregates the results, ensuring a cohesive final output.

Message Queue

The message queue serves as a central communication hub for all services and the control plane. It provides:

• Methods for publishing messages to named queues
• Functionality to delegate messages to appropriate consumers

MessageQueue Core Operations

Initialize consumers and queues dictionaries

procedure Publish(message):
    queues[message.type].append(message)
    // Add the message to the appropriate queue

procedure ProcessingLoop():
    while running do
        for each non-empty queue do
            PublishToConsumer(queue.dequeue())
            // Dequeue and publish messages to consumers
        end for
    end while

procedure LaunchServer():
    Start ProcessingLoop and run FastAPI server
    // Initialize and run the server

Control Plane

The control plane acts as the primary gateway to the Llama-Agents system. Its responsibilities include:

• Tracking current tasks
• Maintaining a registry of services within the system
• Housing the orchestrator module

ControlPlane Processing Loop

procedure ControlPlaneProcessingLoop():
    while true do
        message ← CheckMessageQueue()
        // Check for new messages in the queue
        if message is NewTask then
            HandleNewTask(message)
        else if message is GeneralChat then
            HandleGeneralChat(message)
        else if message is CompletedTask then
            HandleCompletedTask(message)
        end if
        UpdateSystemState()
        Sleep(shortInterval)
        // Process different types of messages and update system state
    end while

procedure HandleNewTask(task):
    RegisterTask(task)
    service ← SelectAppropriateService(task)
    SendTaskToService(task, service)
    // Register and delegate new tasks to appropriate services

procedure HandleGeneralChat(chat):
    ProcessChatMessage(chat)
    UpdateUserInterface(chat)
    // Process general chat messages and update UI

procedure HandleCompletedTask(result):
    UpdateTaskState(result)
    NotifyTaskCompletion(result)
    if FollowUpTaskRequired(result) then
        CreateFollowUpTask(result)
    end if
    // Handle completed tasks and create follow-ups if needed

Orchestrator

This module manages task allocation and result processing. It can be implemented as:

• An agentic system utilizing an LLM for decision-making
• An explicit system with a predefined query pipeline
• A hybrid approach combining both methods
• A custom implementation tailored to specific requirements

BaseOrchestrator

class BaseOrchestrator:
    abstract procedure get_next_messages(task_def, tools, state):
        // Abstract method to determine next messages to process
        // Returns: (List of QueueMessages, Updated state)

    abstract procedure add_result_to_state(result, state):
        // Abstract method to update state with processing results
        // Returns: Updated state

AgentOrchestrator

class AgentOrchestrator(BaseOrchestrator):
    Initialize with LLM, prompts, and finalize tool

    procedure get_next_messages(task_def, tools, state):
        chat_history ← state.get(HISTORY_KEY, [])
        memory ← ChatMemoryBuffer(chat_history, llm)

        if chat_history is empty then
            response ← llm.predict_and_call(tools, task_def.input)
        else
            response ← llm.predict_and_call(tools + finalize_tool, memory.get())

        if response has no tool call or calls finalize tool then
            Create TaskResult and QueueMessage for human
        else
            Create QueueMessages for each tool call

        Update state with new chat history
        return queue_messages, new_state

    procedure add_result_to_state(result, state):
        Summarize result if necessary
        chat_history ← state.get(HISTORY_KEY, [])
        Append summary to chat_history
        Append followup prompt to chat_history
        return updated state

Services

Services are the core operational units where task processing occurs. Each service:

• Accepts incoming tasks with associated context
• Processes the task according to its specialized function
• Publishes the resulting output

BaseService Algorithm

class BaseService:
    Initialize with service_name

    abstract property service_definition():
        // Returns ServiceDefinition

    abstract procedure as_consumer(remote: bool):
        // Returns BaseMessageQueueConsumer

    abstract procedure processing_loop():
        // Continuous processing of messages

    abstract procedure process_message(message):
        // Process a single message

    abstract procedure launch_local():
        // Launch service in-process
        return asyncio.Task

    abstract procedure launch_server():
        // Launch service as a server

    procedure register_to_control_plane(control_plane_url):
        service_def = self.service_definition
        Send POST request to f"{control_plane_url}/services/register"
        with service_def as JSON payload

    procedure register_to_message_queue():
        return message_queue.register_consumer(self.as_consumer(remote=True))

Tool Service

A specialized service designed to offload the computation of agent tools. It allows agents to be equipped with a meta-tool that interfaces with the tool service, enhancing modularity and efficiency.

ServiceAsTool Algorithm

class ServiceAsTool:
    // ServiceAsTool wraps a service and presents it as a tool for use in LLM systems

    Initialize with tool_metadata, message_queue, service_name
    // tool_metadata: describes the tool's function and parameters
    // message_queue: for async communication with the service
    // service_name: unique identifier for the underlying service

    procedure FromServiceDefinition(service_definition):
        // Factory method to create a ServiceAsTool from a service definition
        Create tool_metadata from service_definition
        return new ServiceAsTool instance

    procedure ProcessMessage(message):
        // Handles incoming messages from the message queue
        if message is COMPLETED_TOOL_CALL:
            Store result in tool_call_results
            // Keeps track of completed tool calls for later retrieval

    procedure AsyncCall(input):
        // Asynchronous method to execute the tool
        Register as consumer if not already registered
        // Ensures the tool can receive messages from the queue

        Create new task from input
        Publish NEW_TOOL_CALL message to service
        // Sends the task to the underlying service via the message queue

        while result not received and not timed out:
            Wait for short interval
            Check for result in tool_call_results
            // Polls for the result, implementing async behavior

        if result received:
            return result as ToolOutput
        else:
            return timeout error
        // Handles both successful calls and timeouts

    procedure Call(input):
        // Synchronous wrapper for AsyncCall
        return synchronous version of AsyncCall(input)
        // Allows the tool to be used in synchronous contexts

Benefits of a Multi Agent System:

Our method of Agentic Task Delegation allows users unmatched access to dormant compute and fast forward tasks! By breaking down large tasks into manageable pieces and dynamically adapting to task requirements, the system can tackle intricate problems with interdependent components, thus making the entire system more robust, powerful and efficient.

Conclusion

Agentic Task Delegation represents a paradigm shift in how AI systems approach complex problem-solving. By decomposing tasks and leveraging specialized agents, this method offers improved efficiency, adaptability, and scalability compared to traditional, monolithic(:p) AI approaches. Looking to the future, we can anticipate even greater integration of this approach with advanced AI models, potentially leading to more sophisticated human-AI collaboration across diverse domains.