Repository: Shopify/roast (Ruby)

Issue:
Allow specifying tools on a per-step basis
## Summary

Implement step-level tool filtering feature that allows workflow authors to specify which tools should be available for each individual step in a Roast workflow.

## Background

Currently, all tools defined in a workflow are available to every step. This can be:
- **Less secure**: Steps have access to tools they don't need
- **Less performant**: Sending unnecessary tools to the LLM increases token usage
- **Less clear**: It's not obvious which tools each step is intended to use

## Feature Description

Allow specifying `available_tools` at the step configuration level to restrict which tools are accessible:

```yaml
# Define all tools globally
tools:
  - Roast::Tools::Grep
  - Roast::Tools::ReadFile
  - Roast::Tools::WriteFile
  - Roast::Tools::Cmd:
      allowed_commands:
        - pwd
        - ls
        - echo

# Configure steps with specific tool access
explore_directory:
  available_tools:
    - pwd
    - ls

analyze_files:
  available_tools:
    - grep
    - read_file

write_summary:
  available_tools:
    - write_file
    - echo
```

## Implementation Details

- Use snake_case tool names (e.g., `read_file` for `Roast::Tools::ReadFile`)
- For `Cmd` tool, use the specific command names (e.g., `pwd`, `ls`)
- When `available_tools` is not specified, all tools remain available (backward compatible)
- Empty arrays (`available_tools: []`) means no tools available for that step
- The specified tools must exist in the global tools list

## Benefits

1. **Security**: Restricts what each step can do, following principle of least privilege
2. **Performance**: Reduces the tool list sent to the LLM, potentially improving response times
3. **Clarity**: Makes tool usage explicit and predictable for each step
4. **Flexibility**: Each step can have its own security context based on its needs

## Related Work

This is a fresh implementation to replace PR #106 which had merge conflicts and implementation issues.


Discussion:

Patch:
diff --git a/README.md b/README.md
index 3a36e906..b074d6e4 100644
--- a/README.md
+++ b/README.md
@@ -658,6 +658,52 @@ tools:
 
 Custom descriptions help the LLM understand when and how to use each command, making your workflows more effective.
 
+### Step-Level Tool Filtering
+
+You can restrict which tools are available to specific steps using the `available_tools` configuration:
+
+```yaml
+# Define all tools globally
+tools:
+  - Roast::Tools::Grep
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - pwd
+        - ls
+        - echo
+
+# Configure steps with specific tool access
+explore_directory:
+  available_tools:
+    - pwd
+    - ls
+
+analyze_files:
+  available_tools:
+    - grep
+    - read_file
+
+write_summary:
+  available_tools:
+    - write_file
+    - echo
+```
+
+This feature provides:
+- **Security**: Each step only has access to the tools it needs
+- **Performance**: Reduces the tool list sent to the LLM
+- **Clarity**: Makes tool usage explicit for each step
+
+Key points:
+- Use snake_case tool names (e.g., `read_file` for `Roast::Tools::ReadFile`)
+- For `Cmd` tool, use the specific command names (e.g., `pwd`, `ls`)
+- When `available_tools` is not specified, all tools remain available (backward compatible)
+- Empty array (`available_tools: []`) means no tools for that step
+
+See the [available_tools_demo](examples/available_tools_demo/) for a complete example.
+
 #### ReadFile
 
 Reads the contents of a file from the filesystem.
diff --git a/examples/available_tools_demo/README.md b/examples/available_tools_demo/README.md
new file mode 100644
index 00000000..a2ea8452
--- /dev/null
+++ b/examples/available_tools_demo/README.md
@@ -0,0 +1,42 @@
+# Available Tools Demo
+
+This example demonstrates the `available_tools` feature in Roast, which allows you to restrict which tools are available to specific steps in your workflow.
+
+## Overview
+
+The workflow consists of three steps, each with different tool access:
+
+1. **explore_directory**: Can only use `pwd` and `ls` commands
+2. **analyze_files**: Can only use `grep` and `read_file` tools
+3. **write_summary**: Can only use `write_file` and `echo` tools
+
+## Key Features Demonstrated
+
+### Security Through Least Privilege
+Each step only has access to the tools it needs. For example, the exploration step cannot write files, and the summary step cannot read files or explore directories.
+
+### Tool Name Convention
+- For built-in tools: Use snake_case names (e.g., `read_file` for `Roast::Tools::ReadFile`)
+- For Cmd tool: Use the specific command names (e.g., `pwd`, `ls`)
+
+### Configuration Structure
+```yaml
+step_name:
+  available_tools:
+    - tool1
+    - tool2
+```
+
+## Running the Example
+
+```bash
+roast examples/available_tools_demo/workflow.yml
+```
+
+## What Happens
+
+1. The first step explores the current directory using only `pwd` and `ls`
+2. The second step searches for Ruby files and reads one using only `grep` and `read_file`
+3. The final step creates a summary file using only `write_file` and `echo`
+
+Each step is restricted to its specified tools, demonstrating how you can create secure, focused workflows where each step has exactly the capabilities it needs.
\ No newline at end of file
diff --git a/examples/available_tools_demo/analyze_files/prompt.md b/examples/available_tools_demo/analyze_files/prompt.md
new file mode 100644
index 00000000..28aa419d
--- /dev/null
+++ b/examples/available_tools_demo/analyze_files/prompt.md
@@ -0,0 +1,6 @@
+Based on the directory listing from the previous step, your task is to:
+
+1. Search for any Ruby files using grep
+2. Read the contents of the first Ruby file you find
+
+Note: You only have access to the `grep` and `read_file` tools for this step. You cannot use commands like `ls` or `pwd`.
\ No newline at end of file
diff --git a/examples/available_tools_demo/explore_directory/prompt.md b/examples/available_tools_demo/explore_directory/prompt.md
new file mode 100644
index 00000000..839a4922
--- /dev/null
+++ b/examples/available_tools_demo/explore_directory/prompt.md
@@ -0,0 +1,6 @@
+You are exploring a new directory. Your task is to:
+
+1. Show the current working directory
+2. List all files and subdirectories
+
+Note: You only have access to the `pwd` and `ls` commands for this step.
\ No newline at end of file
diff --git a/examples/available_tools_demo/workflow.yml b/examples/available_tools_demo/workflow.yml
new file mode 100644
index 00000000..f9311dff
--- /dev/null
+++ b/examples/available_tools_demo/workflow.yml
@@ -0,0 +1,32 @@
+model: anthropic:claude-opus-4
+
+tools:
+  - Roast::Tools::Grep
+  - Roast::Tools::ReadFile
+  - Roast::Tools::WriteFile
+  - Roast::Tools::Cmd:
+      allowed_commands:
+        - pwd
+        - ls
+        - echo
+
+steps:
+  - explore_directory
+  - analyze_files
+  - write_summary
+
+# Step-level tool configuration
+explore_directory:
+  available_tools:
+    - pwd
+    - ls
+
+analyze_files:
+  available_tools:
+    - grep
+    - read_file
+
+write_summary:
+  available_tools:
+    - write_file
+    - echo
\ No newline at end of file
diff --git a/examples/available_tools_demo/write_summary/prompt.md b/examples/available_tools_demo/write_summary/prompt.md
new file mode 100644
index 00000000..7f718099
--- /dev/null
+++ b/examples/available_tools_demo/write_summary/prompt.md
@@ -0,0 +1,6 @@
+Based on your exploration and analysis, create a summary:
+
+1. Write a brief summary of what you found to a file called `summary.txt`
+2. Echo a completion message
+
+Note: You only have access to the `write_file` and `echo` tools for this step. You cannot read files or use other commands.
\ No newline at end of file
diff --git a/lib/roast/workflow/base_step.rb b/lib/roast/workflow/base_step.rb
index f4be9360..60c5361d 100644
--- a/lib/roast/workflow/base_step.rb
+++ b/lib/roast/workflow/base_step.rb
@@ -5,7 +5,7 @@ module Workflow
     class BaseStep
       extend Forwardable
 
-      attr_accessor :model, :print_response, :json, :params, :resource, :coerce_to
+      attr_accessor :model, :print_response, :json, :params, :resource, :coerce_to, :available_tools
       attr_reader :workflow, :name, :context_path
 
       def_delegator :workflow, :append_to_final_output
@@ -21,12 +21,13 @@ def initialize(workflow, model: "anthropic:claude-opus-4", name: nil, context_pa
         @json = false
         @params = {}
         @coerce_to = nil
+        @available_tools = nil
         @resource = workflow.resource if workflow.respond_to?(:resource)
       end
 
       def call
         prompt(read_sidecar_prompt)
-        result = chat_completion(print_response:, json:, params:)
+        result = chat_completion(print_response:, json:, params:, available_tools:)
 
         # Apply coercion if configured
         apply_coercion(result)
@@ -34,13 +35,14 @@ def call
 
       protected
 
-      def chat_completion(print_response: nil, json: nil, params: nil)
+      def chat_completion(print_response: nil, json: nil, params: nil, available_tools: nil)
         # Use instance variables as defaults if parameters are not provided
         print_response = @print_response if print_response.nil?
         json = @json if json.nil?
         params = @params if params.nil?
+        available_tools = @available_tools if available_tools.nil?
 
-        result = workflow.chat_completion(openai: workflow.openai? && model, model: model, json:, params:)
+        result = workflow.chat_completion(openai: workflow.openai? && model, model: model, json:, params:, available_tools:)
         process_output(result, print_response:)
 
         result
diff --git a/lib/roast/workflow/step_loader.rb b/lib/roast/workflow/step_loader.rb
index 04c36dc0..0d914bcb 100644
--- a/lib/roast/workflow/step_loader.rb
+++ b/lib/roast/workflow/step_loader.rb
@@ -184,6 +184,10 @@ def apply_step_configuration(step, step_config)
         step.json = step_config["json"] if step_config.key?("json")
         step.params = step_config["params"] if step_config.key?("params")
         step.coerce_to = step_config["coerce_to"].to_sym if step_config.key?("coerce_to")
+
+        if step_config.key?("available_tools")
+          step.available_tools = step_config["available_tools"]
+        end
       end
     end
   end


Test patch:
diff --git a/test/roast/workflow/base_step_test.rb b/test/roast/workflow/base_step_test.rb
index 98c51ce4..eae66b85 100644
--- a/test/roast/workflow/base_step_test.rb
+++ b/test/roast/workflow/base_step_test.rb
@@ -44,4 +44,49 @@ def setup
     assert_equal({ user: "Test prompt" }, @workflow.transcript.last)
     assert_equal "Test chat completion response", result
   end
+
+  test "available_tools attribute defaults to nil" do
+    assert_nil @step.available_tools
+  end
+
+  test "available_tools can be set" do
+    tools = ["grep", "read_file"]
+    @step.available_tools = tools
+    assert_equal tools, @step.available_tools
+  end
+
+  test "call with available_tools passes tools to chat_completion" do
+    available_tools = ["grep", "search_file"]
+    @step.available_tools = available_tools
+
+    Roast::Helpers::PromptLoader.stubs(:load_prompt)
+      .with(@step, @workflow.file)
+      .returns("Test prompt")
+
+    @workflow.stubs(:openai?)
+      .returns(true)
+
+    # Expect chat_completion to be called with available_tools parameter
+    @workflow.expects(:chat_completion)
+      .with(openai: @step.model, model: @step.model, json: false, params: {}, available_tools: available_tools)
+      .returns("Test response")
+
+    @step.call
+  end
+
+  test "call without available_tools passes nil tools" do
+    Roast::Helpers::PromptLoader.stubs(:load_prompt)
+      .with(@step, @workflow.file)
+      .returns("Test prompt")
+
+    @workflow.stubs(:openai?)
+      .returns(true)
+
+    # Expect chat_completion to be called with available_tools: nil
+    @workflow.expects(:chat_completion)
+      .with(openai: @step.model, model: @step.model, json: false, params: {}, available_tools: nil)
+      .returns("Test response")
+
+    @step.call
+  end
 end
diff --git a/test/roast/workflow/inline_prompt_configuration_test.rb b/test/roast/workflow/inline_prompt_configuration_test.rb
index 3f27e94f..0945e1d6 100644
--- a/test/roast/workflow/inline_prompt_configuration_test.rb
+++ b/test/roast/workflow/inline_prompt_configuration_test.rb
@@ -36,6 +36,7 @@ def setup
           model: "gpt-4o",
           json: true,
           params: { "temperature" => 0.7 },
+          available_tools: nil,
         ).returns({ "result" => "Test response" })
 
         result = @executor.execute_step("analyze the code")
@@ -52,6 +53,7 @@ def setup
           model: "openai/gpt-4o-mini", # Default model
           json: false,
           params: {},
+          available_tools: nil,
         ).returns("Default response")
 
         result = executor.execute_step("analyze without config")
@@ -70,6 +72,7 @@ def setup
           model: "claude-3-opus",
           json: false,
           params: {},
+          available_tools: nil,
         ).returns("Claude response")
 
         result = executor.execute_step("use global model")
@@ -90,6 +93,7 @@ def setup
           model: "step-specific-model", # Step-specific overrides global
           json: false,
           params: {},
+          available_tools: nil,
         ).returns("Specific response")
 
         result = executor.execute_step("specific prompt")
diff --git a/test/roast/workflow/prompt_step_print_response_test.rb b/test/roast/workflow/prompt_step_print_response_test.rb
index 18232801..9df8b510 100644
--- a/test/roast/workflow/prompt_step_print_response_test.rb
+++ b/test/roast/workflow/prompt_step_print_response_test.rb
@@ -54,6 +54,7 @@ def tools
             model: "anthropic:claude-opus-4",
             json: false,
             params: {},
+            available_tools: nil,
           },
           workflow.chat_completion_calls.first,
         )
@@ -90,6 +91,7 @@ def tools
             model: "anthropic:claude-opus-4",
             json: true,
             params: { temperature: 0.5 },
+            available_tools: nil,
           },
           workflow.chat_completion_calls.first,
         )
diff --git a/test/roast/workflow/step_executor_registry_test.rb b/test/roast/workflow/step_executor_registry_test.rb
index b98c1c31..a42765e1 100644
--- a/test/roast/workflow/step_executor_registry_test.rb
+++ b/test/roast/workflow/step_executor_registry_test.rb
@@ -120,13 +120,16 @@ def test_executor_receives_workflow_executor_in_constructor
       end
 
       def test_multiple_matchers_first_match_wins
-        matcher1 = ->(step) { step.is_a?(String) && step.include?("test") }
-        matcher2 = ->(step) { step.is_a?(String) && step.include?("es") }
+        @workflow_executor.stubs(:workflow).returns(mock("workflow"))
+        @workflow_executor.stubs(:config_hash).returns({})
+
+        matcher1 = ->(step) { step.is_a?(Symbol) && step.to_s.include?("test") }
+        matcher2 = ->(step) { step.is_a?(Symbol) && step.to_s.include?("es") }
 
         StepExecutorRegistry.register_with_matcher(matcher1, TestExecutor)
         StepExecutorRegistry.register_with_matcher(matcher2, AnotherTestExecutor)
 
-        executor = StepExecutorRegistry.for("test", @workflow_executor)
+        executor = StepExecutorRegistry.for(:test, @workflow_executor)
         assert_instance_of(TestExecutor, executor)
       end
 
diff --git a/test/roast/workflow/step_loader_test.rb b/test/roast/workflow/step_loader_test.rb
index 2313590d..74b1b17a 100644
--- a/test/roast/workflow/step_loader_test.rb
+++ b/test/roast/workflow/step_loader_test.rb
@@ -127,6 +127,37 @@ def test_applies_step_configuration
         assert_equal({ "key" => "value" }, step.params)
       end
 
+      def test_applies_available_tools_configuration
+        @config_hash["test"] = {
+          "available_tools" => ["grep", "search_file"],
+        }
+
+        step = @step_loader.load("test")
+
+        assert_equal(["grep", "search_file"], step.available_tools)
+      end
+
+      def test_does_not_set_available_tools_when_not_configured
+        @config_hash["test"] = {
+          "json" => true,
+          "print_response" => true,
+        }
+
+        step = @step_loader.load("test")
+
+        assert_nil(step.available_tools)
+      end
+
+      def test_handles_empty_available_tools_array
+        @config_hash["test"] = {
+          "available_tools" => [],
+        }
+
+        step = @step_loader.load("test")
+
+        assert_equal([], step.available_tools)
+      end
+
       def test_sets_resource_when_supported
         @workflow.resource = Roast::Resources::FileResource.new("test.txt")
 
diff --git a/test/roast/workflow/workflow_executor_test.rb b/test/roast/workflow/workflow_executor_test.rb
index cdfc3643..34489a0b 100644
--- a/test/roast/workflow/workflow_executor_test.rb
+++ b/test/roast/workflow/workflow_executor_test.rb
@@ -58,6 +58,7 @@ def setup
       model: "gpt-4o",
       json: false,
       params: {},
+      available_tools: nil,
     ).returns("Test response")
 
     result = @executor.execute_step("this is a plain text prompt")