Repository: olimorris/codecompanion.nvim
Pull request #1399: Better handle missing tools
Author: login: ravitemer
Created: 2025-05-09T09:29:03Z | Merged: 2025-05-10T09:37:32Z

## Description

### BUG 

When LLM calls a tool that is missing, subsequent prompts fail with `Bad Request` error due to missing message with `tool` role. 

To address this we return the error mentioning that the tool is not found along providing it the available_tools as tool call response, so that on next prompts LLM corrects the tool call.

## Related Issue(s)

<!--
  If this PR fixes any issues, please link to the issue here.
  - Fixes #<issue_number>
-->

## Screenshots

<!-- Add screenshots of the changes if applicable, to help visualize the change. -->

## Checklist

- [x] I've read the [contributing](https://github.com/olimorris/codecompanion.nvim/blob/main/CONTRIBUTING.md) guidelines and have adhered to them in this PR
- [ ] I've updated `CodeCompanion.has` in the [init.lua](https://github.com/olimorris/codecompanion.nvim/blob/main/lua/codecompanion/init.lua#L239) file for my new feature
- [ ] I've added [test](https://github.com/olimorris/codecompanion.nvim/blob/main/CONTRIBUTING.md#testing) coverage for this fix/feature
- [x] I've updated the README and/or relevant docs pages
- [x] I've run `make all` to ensure docs are generated, tests pass and my formatting is applied



Diff:
diff --git a/lua/codecompanion/strategies/chat/agents/init.lua b/lua/codecompanion/strategies/chat/agents/init.lua
index 5cf9d0cce..a3b502794 100644
--- a/lua/codecompanion/strategies/chat/agents/init.lua
+++ b/lua/codecompanion/strategies/chat/agents/init.lua
@@ -107,12 +107,34 @@ function Agent:execute(chat, tools)
 
     local name = tool["function"].name
     local tool_config = self.tools_config[name]
+    local function handle_missing_tool(tool_call, err_message)
+      tool_call.name = name
+      tool_call.function_call = tool_call
+      log:error(err_message)
+      local available_tools_msg = next(chat.tools.in_use or {})
+          and "The available tools are: " .. table.concat(
+            vim.tbl_map(function(t)
+              return "`" .. t .. "`"
+            end, vim.tbl_keys(chat.tools.in_use)),
+            ", "
+          )
+        or "No tools available"
+      self.chat:add_tool_output(
+        tool_call,
+        string.format("Tool `%s` not found. %s", name, available_tools_msg),
+        string.format("**%s Tool Error**: %s", name, err_message)
+      )
+      return util.fire("AgentFinished", { bufnr = self.bufnr })
+    end
+    if not tool_config then
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't find the tool `%s`", name))
+    end
 
     local ok, resolved_tool = pcall(function()
       return Agent.resolve(tool_config)
     end)
     if not ok or not resolved_tool then
-      return log:error("Couldn't resolve the tool(s) from the LLM's response %s", resolved_tool)
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't resolve the tool `%s`", name))
     end
 
     self.tool = vim.deepcopy(resolved_tool)
diff --git a/tests/strategies/chat/agents/test_agents.lua b/tests/strategies/chat/agents/test_agents.lua
index 9e7a31092..76dcf8fb1 100644
--- a/tests/strategies/chat/agents/test_agents.lua
+++ b/tests/strategies/chat/agents/test_agents.lua
@@ -129,6 +129,32 @@ T["Agent"][":execute"]["a malformed response from the LLM is handled"] = functio
   h.expect_starts_with("You made an error in calling the weather tool:", output)
 end
 
+T["Agent"][":execute"]["a missing tool is handled"] = function()
+  child.lua([[
+    local tools = {
+      {
+        id = 1,
+        type = "function",
+        ["function"] = {
+          name = "missing_tool",
+          arguments = {
+          },
+        },
+      },
+    }
+
+    local chat = _G.chat
+    chat.tools.in_use = {
+      weather = true
+    }
+    _G.agent:execute(chat, tools)
+  ]])
+
+  local output = child.lua_get([[chat.messages[#chat.messages].content]])
+  h.expect_starts_with("Tool `missing_tool` not found", output)
+  h.eq(true, output:find("`weather`") ~= nil)
+end
+
 T["Agent"][":execute"]["a nested response from the LLM"] = function() end
 
 T["Agent"][":replace"] = new_set()


Reviews:
user: login: olimorris
state: COMMENTED
author_association: OWNER
submitted_at: 2025-05-09T10:07:16Z
user: login: eli0shin
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-09T20:55:58Z
user: login: ravitemer
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-10T01:22:33Z

Review comments:
diff_hunk: @@ -107,12 +107,31 @@ function Agent:execute(chat, tools)
 
     local name = tool["function"].name
     local tool_config = self.tools_config[name]
+    local function handle_missing_tool(tool_call, err_message)
+      tool_call.name = name
+      tool_call.function_call = tool_call
+      log:error(err_message)
+      self.chat:add_tool_output(
+        tool_call,
+        string.format(
+          "You made an error in calling the %s tool: %s\n\nAvailable tools: %s",
path: lua/codecompanion/strategies/chat/agents/init.lua
user: login: olimorris
body: Can we be more concise and say `Tool %s not found. The available tools are....`
created_at: 2025-05-09T10:07:16Z
author_association: OWNER
diff_hunk: @@ -107,12 +107,34 @@ function Agent:execute(chat, tools)
 
     local name = tool["function"].name
     local tool_config = self.tools_config[name]
+    local function handle_missing_tool(tool_call, err_message)
+      tool_call.name = name
+      tool_call.function_call = tool_call
+      log:error(err_message)
+      local available_tools_msg = next(chat.tools.in_use or {})
+          and "The available tools are: " .. table.concat(
+            vim.tbl_map(function(t)
+              return "`" .. t .. "`"
+            end, vim.tbl_keys(chat.tools.in_use)),
+            ", "
+          )
+        or "No tools available"
+      self.chat:add_tool_output(
+        tool_call,
+        string.format("Tool `%s` not found. %s", name, available_tools_msg),
+        string.format("**%s Tool Error**: %s", name, err_message)
+      )
+      return util.fire("AgentFinished", { bufnr = self.bufnr })
+    end
+    if not tool_config then
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't find the tool `%s`", name))
+    end
 
     local ok, resolved_tool = pcall(function()
       return Agent.resolve(tool_config)
     end)
     if not ok or not resolved_tool then
-      return log:error("Couldn't resolve the tool(s) from the LLM's response %s", resolved_tool)
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't resolve the tool `%s`", name))
path: lua/codecompanion/strategies/chat/agents/init.lua
user: login: eli0shin
body: When I see this error it usually comes from the line above as a ‘cannot index property of nil’ error. Have you been seeing the errors throw with the Couldn’t resolve tools message?
created_at: 2025-05-09T20:55:57Z
author_association: NONE
diff_hunk: @@ -107,12 +107,34 @@ function Agent:execute(chat, tools)
 
     local name = tool["function"].name
     local tool_config = self.tools_config[name]
+    local function handle_missing_tool(tool_call, err_message)
+      tool_call.name = name
+      tool_call.function_call = tool_call
+      log:error(err_message)
+      local available_tools_msg = next(chat.tools.in_use or {})
+          and "The available tools are: " .. table.concat(
+            vim.tbl_map(function(t)
+              return "`" .. t .. "`"
+            end, vim.tbl_keys(chat.tools.in_use)),
+            ", "
+          )
+        or "No tools available"
+      self.chat:add_tool_output(
+        tool_call,
+        string.format("Tool `%s` not found. %s", name, available_tools_msg),
+        string.format("**%s Tool Error**: %s", name, err_message)
+      )
+      return util.fire("AgentFinished", { bufnr = self.bufnr })
+    end
+    if not tool_config then
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't find the tool `%s`", name))
+    end
 
     local ok, resolved_tool = pcall(function()
       return Agent.resolve(tool_config)
     end)
     if not ok or not resolved_tool then
-      return log:error("Couldn't resolve the tool(s) from the LLM's response %s", resolved_tool)
+      return handle_missing_tool(vim.deepcopy(tool), string.format("Couldn't resolve the tool `%s`", name))
path: lua/codecompanion/strategies/chat/agents/init.lua
user: login: ravitemer
body: I too get the 'attempt to index tool a nil value'. It is from Agent.resolve line above when tool_config is nil. So I added check for that as well
created_at: 2025-05-10T01:22:33Z
author_association: CONTRIBUTOR

PR comments:
user: login: olimorris
created_at: 2025-05-09T10:09:23Z
body: Thanks @ravitemer. Was this something you noticed often? Could we add some test coverage for this too?
author_association: OWNER
user: login: ravitemer
created_at: 2025-05-09T10:48:55Z
body: Not frequent but I have seen instances where the LLM confuses between `use_mcp_tool` and tool name of MCP Server and instead of calling 

```
use_mcp_tool
  server_name
  tool_name_in_server
  arguments
  ```
  
  It calls 
  
  ```
  tool_name_in_server
     arguments
 ``` 
 
 leading to no tool response for the tool call in the messages history and we need to tweak in the "gd" view to remove the llm tool call message so that we can continue the chat without "Bad Request" error
 
 I have also added a test for this now. Please check
author_association: CONTRIBUTOR
user: login: eli0shin
created_at: 2025-05-09T20:52:18Z
body: Thank you! This has been driving me a bit nuts lately. I often just tag the mcp tool but have regularly needed to add files as well when this issue comes up.
author_association: NONE