Repository: Shopify/roast (Ruby)

Issue:
AgentStep fails with NoMethodError trying to read config
# AgentStep Nil Workflow Bug Investigation

## Overview

It looks like there's a bug where `workflow` is nil on line 16 of `lib/roast/workflow/agent_step.rb` when calling `workflow.config[name.to_s]`. This bug completely prevents the AgentStep class from being used in production.

## The Bug: Line 16 Analysis

### Location
**File**: `lib/roast/workflow/agent_step.rb`  
**Line**: 16  
**Code**: `step_config = workflow.config[name.to_s] || {}`

### The Problem
The `workflow` object is `nil`, causing a NoMethodError when trying to access `workflow.config`.

### Root Cause Analysis

#### **Incorrect Configuration Access Pattern**
The AgentStep class accesses configuration using `workflow.config[name.to_s]`, but this is inconsistent with how the rest of the Roast codebase handles step configuration.

**Evidence**:
- `BaseWorkflow` class (line 19) has `workflow_configuration` attribute, **not** `config`
- Only 2 files in the entire codebase access `workflow.config`: `AgentStep` and `ContextSummarizer`
- All other step configuration access uses `config_hash` pattern (seen in `StepLoader.configure_step` line 174)

## Test Coverage Analysis

### Existing Tests
The `test/roast/workflow/agent_step_test.rb` file contains comprehensive tests:

- **Lines 74-98**: Test direct AgentStep invocation with mocked workflow
- **Lines 100-135**: Test coordinator handling of agent steps  
- **Lines 137-175**: Test inline agent prompts with ^ prefix
- **Lines 177-199**: Test agent step with inline prompts

### Test Workflow Creation
All tests mock the workflow object:
```ruby
workflow = mock
workflow.stubs(:resource).returns(nil)
workflow.stubs(:output).returns({})
workflow.stubs(:transcript).returns([])
workflow.stubs(:config).returns({})  # ← Tests mock .config method
```

**Key Insight**: Tests mock `workflow.config` method, but `BaseWorkflow` doesn't have this method!

## Configuration Access Patterns Comparison

### How Other Steps Access Configuration

**BaseStep and other steps** use the StepLoader pattern:
1. `StepLoader.configure_step` (line 173) accesses: `config_hash[step_name]`
2. Configuration is applied via `apply_step_configuration` (line 197)
3. Step properties are set directly: `step.model = determine_model(step_config)`

### How AgentStep Tries to Access Configuration

**AgentStep** tries to access configuration directly:
```ruby
step_config = workflow.config[name.to_s] || {}
```

This bypasses the normal configuration flow entirely.

## Conclusion

**Primary Issue**: AgentStep uses an inconsistent configuration access pattern (`workflow.config`) that doesn't align with the rest of the Roast architecture.

**Root Cause**: BaseWorkflow doesn't have a `config` method, yet AgentStep assumes it does.

**Likely Best Fix**: Refactor AgentStep to use the standard configuration flow through StepLoader, ensuring consistency with other step types and proper error handling.


Discussion:

Patch:
diff --git a/lib/roast/workflow/agent_step.rb b/lib/roast/workflow/agent_step.rb
index f2b1de72..d67c3360 100644
--- a/lib/roast/workflow/agent_step.rb
+++ b/lib/roast/workflow/agent_step.rb
@@ -3,6 +3,15 @@
 module Roast
   module Workflow
     class AgentStep < BaseStep
+      attr_accessor :include_context_summary, :continue
+
+      def initialize(workflow, **kwargs)
+        super
+        # Set default values for agent-specific options
+        @include_context_summary = false
+        @continue = false
+      end
+
       def call
         # For inline prompts (detected by plain text step names), use the name as the prompt
         # For file-based steps, load from the prompt file
@@ -12,11 +21,10 @@ def call
           read_sidecar_prompt
         end
 
-        # Extract agent-specific configuration from workflow config
-        step_config = workflow.config[name.to_s] || {}
+        # Use agent-specific configuration that was applied by StepLoader
         agent_options = {
-          include_context_summary: step_config.fetch("include_context_summary", false),
-          continue: step_config.fetch("continue", false),
+          include_context_summary: @include_context_summary,
+          continue: @continue,
         }
 
         # Call CodingAgent directly with the prompt content and options
diff --git a/lib/roast/workflow/step_loader.rb b/lib/roast/workflow/step_loader.rb
index 6bd8a177..aaa2c126 100644
--- a/lib/roast/workflow/step_loader.rb
+++ b/lib/roast/workflow/step_loader.rb
@@ -203,6 +203,18 @@ def apply_step_configuration(step, step_config)
         if step_config.key?("available_tools")
           step.available_tools = step_config["available_tools"]
         end
+
+        # Apply any other configuration attributes that the step supports
+        step_config.each do |key, value|
+          # Skip keys we've already handled above
+          next if ["print_response", "json", "params", "coerce_to", "available_tools"].include?(key)
+
+          # Apply configuration if the step has a setter for this attribute
+          setter_method = "#{key}="
+          if step.respond_to?(setter_method)
+            step.public_send(setter_method, value)
+          end
+        end
       end
     end
   end


Test patch:
diff --git a/test/roast/workflow/agent_step_test.rb b/test/roast/workflow/agent_step_test.rb
index b4061795..197a1cf9 100644
--- a/test/roast/workflow/agent_step_test.rb
+++ b/test/roast/workflow/agent_step_test.rb
@@ -197,4 +197,72 @@ def teardown
     # Verify the result
     assert_equal "Found 3 bottlenecks", result
   end
+
+  test "agent step full integration test from workflow YAML" do
+    Dir.mktmpdir do |tmpdir|
+      workflow_file = File.join(tmpdir, "test_workflow.yml")
+      File.write(workflow_file, <<~YAML)
+        name: agent_test_workflow
+        tools: []
+        steps:
+          - ^analyze the code and identify issues
+          - ^refactor the problematic functions
+
+        analyze the code and identify issues:
+          include_context_summary: true
+          continue: false
+
+        refactor the problematic functions:
+          include_context_summary: false#{" "}
+          continue: true
+      YAML
+
+      # Create a Configuration from the YAML (like normal Roast execution)
+      configuration = Roast::Workflow::Configuration.new(workflow_file)
+
+      # Verify the configuration loaded correctly
+      assert_equal "agent_test_workflow", configuration.name
+      assert_equal ["^analyze the code and identify issues", "^refactor the problematic functions"], configuration.steps
+
+      # Verify step configuration (without ^ prefix in config keys)
+      step1_config = configuration.get_step_config("analyze the code and identify issues")
+      assert_equal true, step1_config["include_context_summary"]
+      assert_equal false, step1_config["continue"]
+
+      step2_config = configuration.get_step_config("refactor the problematic functions")
+      assert_equal false, step2_config["include_context_summary"]
+      assert_equal true, step2_config["continue"]
+
+      # Create a workflow from the configuration (like WorkflowRunner does)
+      workflow = Roast::Workflow::BaseWorkflow.new(nil, name: configuration.name)
+
+      # Create WorkflowExecutor with the parsed configuration
+      executor = Roast::Workflow::WorkflowExecutor.new(
+        workflow,
+        configuration.config_hash,
+        tmpdir,
+      )
+
+      # Mock CodingAgent calls to prevent actual LLM execution
+      # First agent step should get config: include_context_summary=true, continue=false
+      Roast::Tools::CodingAgent.expects(:call).with(
+        "analyze the code and identify issues",
+        include_context_summary: true,
+        continue: false,
+      ).returns("Found 3 issues: X, Y, Z")
+
+      # Second agent step should get config: include_context_summary=false, continue=true
+      Roast::Tools::CodingAgent.expects(:call).with(
+        "refactor the problematic functions",
+        include_context_summary: false,
+        continue: true,
+      ).returns("Refactored functions A, B, C")
+
+      executor.execute_steps(configuration.steps)
+
+      # Verify workflow output contains results from both agent steps
+      assert_equal "Found 3 issues: X, Y, Z", workflow.output["analyze the code and identify issues"]
+      assert_equal "Refactored functions A, B, C", workflow.output["refactor the problematic functions"]
+    end
+  end
 end