Repository: github/gh-aw (2026-04-16T16:43:34Z)

Issue:
[cli-tools-test] audit: tool_types=0 for Copilot engine runs despite MCP tool usage
### Problem Description

When auditing a completed workflow run using the Copilot engine, the behavior fingerprint reports `tool_types: 0` and `tool_breadth: "narrow"` even when MCP tools were clearly used. The `observability_insights` section echoes this incorrect value: _"The agent used 20 turns across 0 tool types"_.

This produces a misleading analysis since the same run's `mcp_tool_usage` section correctly records tool calls (e.g., `safeoutputs.create_discussion`, `safeoutputs.push_repo_memory`).

By contrast, auditing a Claude engine run (same test session) correctly shows `tool_types: 4` and `tool_breadth: "moderate"`.

### Tool

- **Tool**: `audit` (agentic-workflows MCP server)
- **Engine affected**: `copilot` (Claude engine is unaffected)

### Steps to Reproduce

1. Run any Copilot engine workflow that calls MCP tools (e.g., `safeoutputs.create_discussion`)
2. Audit the completed run:
   ```
   audit(run_id_or_url: "<copilot-run-id>")
   ```
3. Check `behavior_fingerprint.tool_types` and `observability_insights` in the response

### Expected Behavior

`tool_types` should reflect the number of distinct MCP tool types actually invoked during the run. For a run that used `safeoutputs.create_discussion` and `safeoutputs.push_repo_memory`, the value should be at least `1` (one MCP server) or `2` (two distinct tool names).

### Actual Behavior

```json
{
  "behavior_fingerprint": {
    "tool_breadth": "narrow",
    "tool_types": 0
  },
  "observability_insights": [
    {
      "summary": "The agent used 20 turns across 0 tool types ..."
    }
  ]
}
```

Meanwhile the same audit shows `mcp_tool_usage.summary` with:
```json
[
  {"server_name": "safeoutputs", "tool_name": "create_discussion", "call_count": 1},
  {"server_name": "safeoutputs", "tool_name": "push_repo_memory",  "call_count": 2}
]
```

### Evidence

- **Affected run**: [§24493327785](https://github.com/github/gh-aw/actions/runs/24493327785) — "Copilot PR Prompt Pattern Analysis", engine `copilot` v1.0.21, 20 turns, 3 MCP tool calls
- **Unaffected run**: [§24493292984](https://github.com/github/gh-aw/actions/runs/24493292984) — "Design Decision Gate", engine `claude`, 5 turns → `tool_types: 4` ✅
- **Test session run**: [§24493738931](https://github.com/github/gh-aw/actions/runs/24493738931)

### Root Cause (Hypothesis)

The `tool_types` counter likely parses tool call events from Claude-format agent logs. Copilot engine runs may use a different log format where tool invocations are not attributed as individual typed events, causing the counter to remain at `0` even when `mcp_tool_usage` has correct data from a separate parsing path.

### Impact

- **Severity**: Medium
- **Frequency**: Affects every Copilot engine run audit
- **Workaround**: Ignore `behavior_fingerprint.tool_breadth` / `tool_types`; use `mcp_tool_usage.summary` directly for tool usage analysis

### Additional Context

Also observed during this test session:

1. **`github-agentic-workflows` MCP server fails in Design Decision Gate** — The server shows status `"failed"` with 0 requests in run [§24493292984](https://github.com/github/gh-aw/actions/runs/24493292984). The workflow completed successfully without it, suggesting it is configured but unused, yet it adds noise to the audit report and contributes to the "capability friction" warning.

2. **`logs` tool file path is inaccessible** — The `logs` MCP tool response includes `"Use the file_path to read the full data"` pointing to `/tmp/gh-aw/logs-cache/<hash>.json`, but that directory is owned by `root:root` with mode `0700` and cannot be read by the workflow agent user. The equivalent data is available via per-run `run_summary.json` files in `/tmp/gh-aw/aw-mcp/logs/`, but the message is confusing.




> Generated by [Daily CLI Tools Exploratory Tester](https://github.com/github/gh-aw/actions/runs/24493738931/agentic_workflow) · ● 1.8M · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fdaily-cli-tools-tester%22&type=issues)
> - [x] expires <!-- gh-aw-expires: 2026-04-23T05:43:46.067Z --> on Apr 23, 2026, 5:43 AM UTC

<!-- gh-aw-agentic-workflow: Daily CLI Tools Exploratory Tester, engine: copilot, model: auto, id: 24493738931, workflow_id: daily-cli-tools-tester, run: https://github.com/github/gh-aw/actions/runs/24493738931 -->

<!-- gh-aw-workflow-id: daily-cli-tools-tester -->
<!-- gh-aw-workflow-call-id: github/gh-aw/daily-cli-tools-tester -->


Discussion:

Patch:
diff --git a/pkg/cli/audit_agentic_analysis.go b/pkg/cli/audit_agentic_analysis.go
index 2f94c5e0db1..a280410cf4c 100644
--- a/pkg/cli/audit_agentic_analysis.go
+++ b/pkg/cli/audit_agentic_analysis.go
@@ -89,6 +89,89 @@ func buildToolUsageInfo(metrics LogMetrics) []ToolUsageInfo {
 	return toolUsage
 }
 
+func mergeMCPToolUsageInfo(toolUsage []ToolUsageInfo, mcpToolUsage *MCPToolUsageData) []ToolUsageInfo {
+	if mcpToolUsage == nil {
+		return toolUsage
+	}
+
+	toolStats := make(map[string]*ToolUsageInfo)
+	for _, info := range toolUsage {
+		cloned := info
+		toolStats[info.Name] = &cloned
+	}
+
+	addOrUpdateToolUsage := func(name string, callCount, maxInputSize, maxOutputSize int, maxDuration string) {
+		normalizedName := strings.TrimSpace(name)
+		if normalizedName == "" {
+			return
+		}
+		displayKey := workflow.PrettifyToolName(normalizedName)
+		if existing, exists := toolStats[displayKey]; exists {
+			existing.CallCount += callCount
+			if maxInputSize > existing.MaxInputSize {
+				existing.MaxInputSize = maxInputSize
+			}
+			if maxOutputSize > existing.MaxOutputSize {
+				existing.MaxOutputSize = maxOutputSize
+			}
+			if maxDuration != "" {
+				maxDurationValue := parseDurationString(maxDuration)
+				if existing.MaxDuration == "" {
+					existing.MaxDuration = maxDuration
+				} else {
+					existingMaxDurationValue := parseDurationString(existing.MaxDuration)
+					if maxDurationValue > existingMaxDurationValue {
+						existing.MaxDuration = maxDuration
+					}
+				}
+			}
+			return
+		}
+
+		toolStats[displayKey] = &ToolUsageInfo{
+			Name:          displayKey,
+			CallCount:     callCount,
+			MaxInputSize:  maxInputSize,
+			MaxOutputSize: maxOutputSize,
+			MaxDuration:   maxDuration,
+		}
+	}
+
+	if len(mcpToolUsage.Summary) > 0 {
+		for _, summary := range mcpToolUsage.Summary {
+			switch {
+			case summary.ServerName != "" && summary.ToolName != "":
+				addOrUpdateToolUsage(summary.ServerName+"."+summary.ToolName, summary.CallCount, summary.MaxInputSize, summary.MaxOutputSize, summary.MaxDuration)
+			case summary.ToolName != "":
+				addOrUpdateToolUsage(summary.ToolName, summary.CallCount, summary.MaxInputSize, summary.MaxOutputSize, summary.MaxDuration)
+			}
+		}
+	} else {
+		for _, call := range mcpToolUsage.ToolCalls {
+			switch {
+			case call.ServerName != "" && call.ToolName != "":
+				addOrUpdateToolUsage(call.ServerName+"."+call.ToolName, 1, call.InputSize, call.OutputSize, call.Duration)
+			case call.ToolName != "":
+				addOrUpdateToolUsage(call.ToolName, 1, call.InputSize, call.OutputSize, call.Duration)
+			}
+		}
+	}
+
+	mergedToolUsage := make([]ToolUsageInfo, 0, len(toolStats))
+	for _, info := range toolStats {
+		mergedToolUsage = append(mergedToolUsage, *info)
+	}
+
+	slices.SortFunc(mergedToolUsage, func(a, b ToolUsageInfo) int {
+		if a.CallCount != b.CallCount {
+			return b.CallCount - a.CallCount
+		}
+		return strings.Compare(a.Name, b.Name)
+	})
+
+	return mergedToolUsage
+}
+
 func deriveRunAgenticAnalysis(processedRun ProcessedRun, metrics LogMetrics) (*AwContext, []ToolUsageInfo, []CreatedItemReport, *TaskDomainInfo, *BehaviorFingerprint, []AgenticAssessment) {
 	auditAgenticLog.Printf("Deriving agentic analysis for run: id=%d workflow=%s", processedRun.Run.DatabaseID, processedRun.Run.WorkflowName)
 	var awContext *AwContext
diff --git a/pkg/cli/audit_report.go b/pkg/cli/audit_report.go
index 8e751a3fb5c..fe6f3760cf7 100644
--- a/pkg/cli/audit_report.go
+++ b/pkg/cli/audit_report.go
@@ -321,6 +321,7 @@ func buildAuditData(processedRun ProcessedRun, metrics LogMetrics, mcpToolUsage
 	}
 
 	toolUsage := buildToolUsageInfo(metrics)
+	toolUsage = mergeMCPToolUsageInfo(toolUsage, mcpToolUsage)
 
 	createdItems := extractCreatedItemsFromManifest(run.LogsPath)
 	taskDomain := detectTaskDomain(processedRun, createdItems, toolUsage, overview.AwContext)


Test patch:
diff --git a/pkg/cli/audit_agentic_analysis_test.go b/pkg/cli/audit_agentic_analysis_test.go
index 89a760f6c0b..aa4a7a19e55 100644
--- a/pkg/cli/audit_agentic_analysis_test.go
+++ b/pkg/cli/audit_agentic_analysis_test.go
@@ -325,6 +325,36 @@ func TestBuildAuditDataToolUsageMatchesBuildToolUsageInfo(t *testing.T) {
 	require.Equal(t, expected, auditData.ToolUsage, "buildAuditData tool usage should match buildToolUsageInfo output")
 }
 
+func TestMergeMCPToolUsageInfoUsesSummaryWhenMetricsToolCallsEmpty(t *testing.T) {
+	merged := mergeMCPToolUsageInfo(nil, &MCPToolUsageData{
+		Summary: []MCPToolSummary{
+			{ServerName: "safeoutputs", ToolName: "create_discussion", CallCount: 2, MaxInputSize: 50, MaxOutputSize: 1200},
+			{ServerName: "safeoutputs", ToolName: "push_repo_memory", CallCount: 1, MaxInputSize: 32, MaxOutputSize: 240},
+		},
+	})
+
+	require.Len(t, merged, 2, "MCP summary should be represented as tool usage entries")
+	assert.Equal(t, "safeoutputs.create_discussion", merged[0].Name, "higher call-count MCP tool should sort first")
+	assert.Equal(t, 2, merged[0].CallCount, "call count should come from MCP summary")
+	assert.Equal(t, "safeoutputs.push_repo_memory", merged[1].Name, "second MCP tool should be preserved")
+}
+
+func TestMergeMCPToolUsageInfoFallsBackToToolCalls(t *testing.T) {
+	merged := mergeMCPToolUsageInfo(nil, &MCPToolUsageData{
+		ToolCalls: []MCPToolCall{
+			{ServerName: "safeoutputs", ToolName: "create_discussion", InputSize: 20, OutputSize: 200},
+			{ServerName: "safeoutputs", ToolName: "create_discussion", InputSize: 40, OutputSize: 250},
+			{ServerName: "safeoutputs", ToolName: "push_repo_memory", InputSize: 10, OutputSize: 100},
+		},
+	})
+
+	require.Len(t, merged, 2, "fallback should aggregate MCP tool calls when summary is absent")
+	assert.Equal(t, "safeoutputs.create_discussion", merged[0].Name, "tool call fallback should aggregate by MCP tool name")
+	assert.Equal(t, 2, merged[0].CallCount, "fallback should count repeated MCP tool calls")
+	assert.Equal(t, 40, merged[0].MaxInputSize, "fallback should keep max input size from calls")
+	assert.Equal(t, 250, merged[0].MaxOutputSize, "fallback should keep max output size from calls")
+}
+
 // TestDeriveRunAgenticAnalysisFingerprintConsistency verifies that the fingerprint
 // produced by deriveRunAgenticAnalysis is consistent when Run.Turns is correctly
 // populated from log metrics. This guards against the bug where logs_orchestrator.go
diff --git a/pkg/cli/audit_mcp_tool_usage_test.go b/pkg/cli/audit_mcp_tool_usage_test.go
index d7552630434..f12836afce7 100644
--- a/pkg/cli/audit_mcp_tool_usage_test.go
+++ b/pkg/cli/audit_mcp_tool_usage_test.go
@@ -207,3 +207,42 @@ func TestBuildAuditDataWithMCPToolUsage(t *testing.T) {
 	assert.Equal(t, 1024, tool.TotalInputSize)
 	assert.Equal(t, 5120, tool.TotalOutputSize)
 }
+
+func TestBuildAuditDataUsesMCPToolUsageForToolTypes(t *testing.T) {
+	processedRun := ProcessedRun{
+		Run: WorkflowRun{
+			DatabaseID:   2468,
+			WorkflowName: "Copilot MCP Workflow",
+			Status:       "completed",
+			Conclusion:   "success",
+			Turns:        20,
+		},
+	}
+	metrics := LogMetrics{
+		Turns: 20,
+		// Intentionally empty to emulate copilot runs where ToolCalls parsing misses MCP tool calls.
+		ToolCalls: nil,
+	}
+	mcpData := &MCPToolUsageData{
+		Summary: []MCPToolSummary{
+			{ServerName: "safeoutputs", ToolName: "create_discussion", CallCount: 3},
+			{ServerName: "safeoutputs", ToolName: "push_repo_memory", CallCount: 1},
+		},
+	}
+
+	auditData := buildAuditData(processedRun, metrics, mcpData)
+
+	require.Len(t, auditData.ToolUsage, 2, "MCP tools should contribute to tool usage when metrics tool calls are empty")
+	require.NotNil(t, auditData.BehaviorFingerprint, "behavior fingerprint should be present")
+	assert.Equal(t, "narrow", auditData.BehaviorFingerprint.ToolBreadth, "2 tool types should still be narrow but no longer zero")
+
+	var executionInsight *ObservabilityInsight
+	for i := range auditData.ObservabilityInsights {
+		if auditData.ObservabilityInsights[i].Category == "execution" {
+			executionInsight = &auditData.ObservabilityInsights[i]
+			break
+		}
+	}
+	require.NotNil(t, executionInsight, "execution insight should be present")
+	assert.Contains(t, executionInsight.Evidence, "tool_types=2", "execution insight should report MCP-derived tool type count")
+}