Repository: posit-dev/positron
Pull request #7799: assistant: replace empty tool results with placeholder text
Author: login: sharon-wang
Created: 2025-05-21T18:53:40Z | Merged: 2025-05-22T21:45:58Z

### Summary

- addresses https://github.com/posit-dev/positron/issues/7770
- improves message processing to handle empty content more effectively
    - we should not filter out _all_ empty responses
    - we can continue to filter out messages that have no content at all and messages that only contain empty text parts
    - however, we should not filter out tool results with empty content altogether
    - now, when a tool result part has empty content, we include placeholder content in place of the empty content, to ensure that tool calls will have corresponding tool results
- updates our workspace tools to return non-empty messages
    - throw an error when the tool criteria is not met
    - added text to tool descriptions to note that empty workspaces are not supported
- updates anthropic integration test to cover these cases
- adds whether workspace folders are open to the general context, so that LLMs are less likely to use the workspace tools when the workspace criteria is not met

### Release Notes

#### Bug Fixes

- Assistant: ensure tool calls have corresponding tool results (#7770)

### QA Notes

@:assistant

The easiest test case is running the project tree tool, file contents tool or text search tool when in a workspace-less window. The Assistant should not call any of these tools since no workspace folders are open. If you are able to sidestep this and get the Assistant to call the tools anyways, the tool should now throw a more user-friendly error, directing the user to ensure that a workspace folder is open.


Diff:
diff --git a/extensions/positron-assistant/src/anthropic.ts b/extensions/positron-assistant/src/anthropic.ts
index 8fdb8ab71cfe..2063ae272bba 100644
--- a/extensions/positron-assistant/src/anthropic.ts
+++ b/extensions/positron-assistant/src/anthropic.ts
@@ -8,7 +8,7 @@ import * as vscode from 'vscode';
 import Anthropic from '@anthropic-ai/sdk';
 import { ModelConfig } from './config';
 import { isLanguageModelImagePart, LanguageModelImagePart } from './languageModelParts.js';
-import { hasNonEmptyContent, isChatImagePart } from './utils.js';
+import { isChatImagePart, processMessages } from './utils.js';
 import { DEFAULT_MAX_TOKEN_OUTPUT } from './constants.js';
 
 export class AnthropicLanguageModel implements positron.ai.LanguageModelChatProvider {
@@ -56,10 +56,7 @@ export class AnthropicLanguageModel implements positron.ai.LanguageModelChatProv
 		progress: vscode.Progress<vscode.ChatResponseFragment2>,
 		token: vscode.CancellationToken
 	) {
-		// Filter out messages with empty text or empty tool response content
-		const filteredMessages = messages.filter(hasNonEmptyContent);
-
-		const anthropicMessages = filteredMessages.map(message => toAnthropicMessage(message));
+		const anthropicMessages = processMessages(messages).map(toAnthropicMessage);
 		const tools = options.tools?.map(tool => toAnthropicTool(tool));
 		const tool_choice = options.toolMode && toAnthropicToolChoice(options.toolMode);
 		const stream = this._client.messages.stream({
@@ -133,21 +130,14 @@ export class AnthropicLanguageModel implements positron.ai.LanguageModelChatProv
 	}
 
 	async provideTokenCount(text: string | vscode.LanguageModelChatMessage, token: vscode.CancellationToken): Promise<number> {
-		// For empty string or message with only empty content, return 0 tokens
+		const messages: Anthropic.MessageParam[] = [];
 		if (typeof text === 'string') {
+			// For empty string, return 0 tokens
 			if (text.trim() === '') {
 				return 0;
 			}
-		} else {
-			// Check if message has any non-empty content
-			if (!hasNonEmptyContent(text)) {
-				return 0;
-			}
-		}
-
-		// Process non-empty content normally
-		const messages: Anthropic.MessageParam[] = typeof text === 'string' ?
-			[{
+			// Otherwise, treat it as a user message
+			messages.push({
 				role: 'user',
 				content: [
 					{
@@ -155,8 +145,15 @@ export class AnthropicLanguageModel implements positron.ai.LanguageModelChatProv
 						text,
 					},
 				],
-			}] :
-			[toAnthropicMessage(text)];
+			});
+		} else {
+			// For LanguageModelChatMessage, ensure it has non-empty message content
+			const processedMessages = processMessages([text]);
+			if (processedMessages.length === 0) {
+				return 0;
+			}
+			messages.push(...processedMessages.map(toAnthropicMessage));
+		}
 		const result = await this._client.messages.countTokens({
 			model: this._config.model,
 			messages,
diff --git a/extensions/positron-assistant/src/models.ts b/extensions/positron-assistant/src/models.ts
index 058575c4aade..3d543e49c7e5 100644
--- a/extensions/positron-assistant/src/models.ts
+++ b/extensions/positron-assistant/src/models.ts
@@ -15,7 +15,7 @@ import { createOpenAI } from '@ai-sdk/openai';
 import { createMistral } from '@ai-sdk/mistral';
 import { createOllama } from 'ollama-ai-provider';
 import { createOpenRouter } from '@openrouter/ai-sdk-provider';
-import { hasNonEmptyContent, toAIMessage } from './utils';
+import { processMessages, toAIMessage } from './utils';
 import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock';
 import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
 import { AnthropicLanguageModel } from './anthropic';
@@ -217,13 +217,13 @@ abstract class AILanguageModel implements positron.ai.LanguageModelChatProvider
 
 		let tools: Record<string, ai.Tool> | undefined;
 
-		// Filter out messages with empty text or empty tool response content
-		const filteredMessages = messages.filter(hasNonEmptyContent);
+		// Ensure all messages have content
+		const processedMessages = processMessages(messages);
 		// Only Anthropic currently supports experimental_content in tool
 		// results.
 		const toolResultExperimentalContent = this.provider === 'anthropic';
 		// Convert messages to the Vercel AI format.
-		const aiMessages = toAIMessage(filteredMessages, toolResultExperimentalContent);
+		const aiMessages = toAIMessage(processedMessages, toolResultExperimentalContent);
 
 		if (options.tools && options.tools.length > 0) {
 			tools = options.tools.reduce((acc: Record<string, ai.Tool>, tool: vscode.LanguageModelChatTool) => {
diff --git a/extensions/positron-assistant/src/participants.ts b/extensions/positron-assistant/src/participants.ts
index 758b93f64c3d..035dc0228b25 100644
--- a/extensions/positron-assistant/src/participants.ts
+++ b/extensions/positron-assistant/src/participants.ts
@@ -417,6 +417,11 @@ abstract class PositronAssistantParticipant implements IPositronAssistantPartici
 			prompts.push(customPrompt);
 		}
 
+		// Add default context items to the prompt just before the prompts are joined.
+		// This ordering helps with verifying the context items in tests, as we can expect
+		// them to be at the end of the prompt.
+		prompts.push(...getDefaultContextItems());
+
 		const parts: (vscode.LanguageModelTextPart | vscode.LanguageModelDataPart)[] = [];
 		if (prompts.length > 0) {
 			const prompt = prompts.join('\n\n');
@@ -614,3 +619,17 @@ async function openLlmsTextDocument(): Promise<vscode.TextDocument | undefined>
 	const llmsDocument = await vscode.workspace.openTextDocument(fileUri);
 	return llmsDocument;
 }
+
+/**
+ * Retrieve the default context items to include in the prompt.
+ * @returns A list of default context items to include in the prompt.
+ */
+export function getDefaultContextItems(): string[] {
+	const defaultPrompts = [];
+
+	// Note if any folders are open in the workspace.
+	const areFoldersOpen = !!vscode.workspace.workspaceFolders && vscode.workspace.workspaceFolders.length > 0;
+	defaultPrompts.push(xml.node('workspace', `Workspace folders are open: ${areFoldersOpen}`));
+
+	return defaultPrompts;
+}
diff --git a/extensions/positron-assistant/src/test/anthropic.test.ts b/extensions/positron-assistant/src/test/anthropic.test.ts
index 235279a6b387..40bbe459b30c 100644
--- a/extensions/positron-assistant/src/test/anthropic.test.ts
+++ b/extensions/positron-assistant/src/test/anthropic.test.ts
@@ -9,6 +9,7 @@ import * as vscode from 'vscode';
 import * as sinon from 'sinon';
 import { AnthropicLanguageModel } from '../anthropic';
 import { ModelConfig } from '../config';
+import { EMPTY_TOOL_RESULT_PLACEHOLDER } from '../utils.js';
 
 suite('AnthropicLanguageModel', () => {
 	let model: AnthropicLanguageModel;
@@ -58,61 +59,24 @@ suite('AnthropicLanguageModel', () => {
 		sinon.restore();
 	});
 
-	test('provideLanguageModelResponse should filter out messages with empty text content', async () => {
-		// Create test messages
-		const emptyTextPart = new vscode.LanguageModelTextPart('');
-		const nonEmptyTextPart = new vscode.LanguageModelTextPart('Hello');
-
-		const messagesWithEmpty = [
-			// Message with only empty text content
-			vscode.LanguageModelChatMessage.User([emptyTextPart]),
-			// Message with non-empty text content
-			vscode.LanguageModelChatMessage.User([nonEmptyTextPart]),
-			// Message with both empty and non-empty text content
-			vscode.LanguageModelChatMessage.Assistant([emptyTextPart, nonEmptyTextPart])
-		];
-
-		// Call the method under test
-		await model.provideLanguageModelResponse(
-			messagesWithEmpty,
-			{},
-			'test-extension',
-			mockProgress,
-			mockCancellationToken
-		);
-
-		// Check that messages were filtered before being passed to the Anthropic client
-		const streamCall = mockClient.messages.stream.getCall(0);
-		assert.ok(streamCall, 'Stream method was not called');
-
-		// The empty message should be filtered out
-		const messagesPassedToAnthropicClient = streamCall.args[0].messages;
-		assert.strictEqual(messagesPassedToAnthropicClient.length, 2, 'Only non-empty messages should be passed to the Anthropic client');
-
-		// Verify that the message with only empty content was filtered out
-		const hasEmptyMessage = messagesPassedToAnthropicClient.some((msg: any) => {
-			return msg.content.length === 0 ||
-				(msg.content.length === 1 &&
-					msg.content[0].type === 'text' &&
-					msg.content[0].text === '');
-		});
-		assert.strictEqual(hasEmptyMessage, false, 'Messages with only empty content should be filtered out');
-	});
-
 	/**
 	 * Test the filtering behavior by checking the messages passed to Anthropic
-	 * when a message with empty content is included.
+	 * when a message with empty LanguageModelTextPart content is included.
 	 */
-	test('provideLanguageModelResponse filters empty messages correctly with different content mixes', async () => {
+	test('provideLanguageModelResponse filters empty messages and different LanguageModelTextPart contents correctly', async () => {
 		// Create test messages with different combinations of empty/non-empty content
+		const nonEmptyText = 'Hello';
 		const emptyTextPart = new vscode.LanguageModelTextPart('');
 		const whitespaceTextPart = new vscode.LanguageModelTextPart('   \n  ');
-		const nonEmptyTextPart = new vscode.LanguageModelTextPart('Hello');
-		const toolCallPart = new vscode.LanguageModelToolCallPart('test-tool-callId', 'test-tool-name', {});
-		const emptyToolResultPart = new vscode.LanguageModelToolResultPart('test-tool-callId', []);
+		const nonEmptyTextPart = new vscode.LanguageModelTextPart(nonEmptyText);
 
 		// Define test messages with different combinations
 		const messagesWithVariousContent = [
+			// Message with no content - should be filtered out
+			{
+				message: vscode.LanguageModelChatMessage.User([]),
+				keep: false
+			},
 			// Message with only empty text content - should be filtered out
 			{
 				message: vscode.LanguageModelChatMessage.User([emptyTextPart]),
@@ -133,16 +97,6 @@ suite('AnthropicLanguageModel', () => {
 				message: vscode.LanguageModelChatMessage.Assistant([emptyTextPart, nonEmptyTextPart]),
 				keep: true
 			},
-			// Message with tool call - should be kept
-			{
-				message: vscode.LanguageModelChatMessage.Assistant([toolCallPart]),
-				keep: true
-			},
-			// Message with empty tool result - should be filtered out
-			{
-				message: vscode.LanguageModelChatMessage.User([emptyToolResultPart]),
-				keep: false
-			}
 		];
 
 		const messages = messagesWithVariousContent.map(m => m.message);
@@ -161,13 +115,77 @@ suite('AnthropicLanguageModel', () => {
 		const streamCall = mockClient.messages.stream.getCall(0);
 		assert.ok(streamCall, 'Stream method was not called');
 
+		// We expect two messages with non-empty content to be passed to the Anthropic client
 		const messagesPassedToAnthropicClient = streamCall.args[0].messages;
 		assert.strictEqual(messagesPassedToAnthropicClient.length, numOfMessagesToKeep, 'Only non-empty messages should be passed to the Anthropic client');
 
-		// Verify specific message patterns that should be included vs filtered
-		const hasMessageWithNonEmptyContent = messagesPassedToAnthropicClient.some((msg: any) =>
-			msg.content.some((content: any) => content.type === 'text' && content.text === 'Hello')
+		// Verify each passed message has the non-empty content we expect
+		const hasMessageWithNonEmptyContent = messagesPassedToAnthropicClient.every((msg: any) =>
+			msg.content.some((content: any) => content.type === 'text' && content.text === nonEmptyText)
 		);
 		assert.strictEqual(hasMessageWithNonEmptyContent, true, 'Messages with non-empty content should be included');
 	});
+
+	test('provideLanguageModelResponse processes LanguageModelToolCallPart and LanguageModelToolResultPart contents correctly', async () => {
+		// 1st tool call with empty result
+		const toolCallId1 = 'test-tool-callId-1';
+		const toolCallEmptyPart = new vscode.LanguageModelToolCallPart(toolCallId1, `${toolCallId1}-tool`, {});
+		const emptyToolResultPartOriginal = new vscode.LanguageModelToolResultPart(toolCallId1, []);
+
+		// 2nd tool call with non-empty result
+		const toolCallId2 = 'test-tool-callId-2';
+		const toolCallNonEmptyPart = new vscode.LanguageModelToolCallPart(toolCallId2, `${toolCallId2}-tool`, {});
+		const nonEmptyToolResultPart = new vscode.LanguageModelToolResultPart(toolCallId2, [
+			new vscode.LanguageModelTextPart('This is a non-empty tool result'),
+		]);
+
+		const messagesWithToolContent = [
+			// Tool call - should be passed
+			{
+				message: vscode.LanguageModelChatMessage.Assistant([toolCallEmptyPart]),
+			},
+			// Tool result with empty content - should be passed, but replaced with a placeholder
+			{
+				message: vscode.LanguageModelChatMessage.User([emptyToolResultPartOriginal]),
+				expectedText: EMPTY_TOOL_RESULT_PLACEHOLDER
+			},
+			// Tool call - should be passed
+			{
+				message: vscode.LanguageModelChatMessage.Assistant([toolCallNonEmptyPart]),
+			},
+			// Tool result with non-empty content - should be passed as is
+			{
+				message: vscode.LanguageModelChatMessage.User([nonEmptyToolResultPart]),
+			},
+		];
+
+		const messages = messagesWithToolContent.map(m => m.message);
+		const numOfMessagesToKeep = messagesWithToolContent.length;
+
+		// Call the method under test
+		await model.provideLanguageModelResponse(
+			messages,
+			{},
+			'test-extension',
+			mockProgress,
+			mockCancellationToken
+		);
+
+		// Check that messages were processed correctly
+		const streamCall = mockClient.messages.stream.getCall(0);
+		assert.ok(streamCall, 'Stream method was not called');
+
+		const messagesPassedToAnthropicClient = streamCall.args[0].messages;
+		assert.strictEqual(messagesPassedToAnthropicClient.length, numOfMessagesToKeep, 'All messages should be passed to the Anthropic client');
+
+		messagesWithToolContent.forEach((msg, index) => {
+			const expectedText = msg.expectedText;
+			if (!expectedText) {
+				return;
+			}
+			// sample actualContent: [{"type":"tool_result","tool_use_id":"test-tool-callId-1","content":[{"type":"text","text":"tool result is empty"}]}]
+			const actualText = messagesPassedToAnthropicClient[index].content[0].content[0].text;
+			assert.deepStrictEqual(actualText, expectedText, `Message text at index ${index} should match the expected text`);
+		});
+	});
 });
diff --git a/extensions/positron-assistant/src/test/participants.test.ts b/extensions/positron-assistant/src/test/participants.test.ts
index 626990b72c95..d2b8bea77bbd 100644
--- a/extensions/positron-assistant/src/test/participants.test.ts
+++ b/extensions/positron-assistant/src/test/participants.test.ts
@@ -7,12 +7,15 @@ import * as assert from 'assert';
 import * as positron from 'positron';
 import * as sinon from 'sinon';
 import * as vscode from 'vscode';
-import { PositronAssistantChatParticipant, PositronAssistantEditorParticipant } from '../participants.js';
+import { getDefaultContextItems, PositronAssistantChatParticipant, PositronAssistantEditorParticipant } from '../participants.js';
 import { mock } from './utils.js';
 import { readFile } from 'fs/promises';
 import { MARKDOWN_DIR } from '../constants.js';
 import path = require('path');
 
+/** We expect 2 messages by default: 1 for the user's prompt, and 1 containing at least the default context */
+const DEFAULT_EXPECTED_MESSAGE_COUNT = 2;
+
 class TestLanguageModelChatResponse implements vscode.LanguageModelChatResponse {
 	stream: AsyncIterable<string> = {
 		[Symbol.asyncIterator]: async function* () {
@@ -111,25 +114,6 @@ suite('PositronAssistantParticipant', () => {
 		disposables.forEach((d) => d.dispose());
 	});
 
-	test('should not send context if none is available', async () => {
-		// Setup test inputs.
-		const request = makeChatRequest({ model, references: [] });
-		const context: vscode.ChatContext = { history: [] };
-		const sendRequestSpy = sinon.spy(model, 'sendRequest');
-		sinon.stub(positron.ai, 'getPositronChatContext').resolves({});
-
-		// Call the method under test.
-		await chatParticipant.requestHandler(request, context, response, token);
-
-		// There should be only one user message with the user's prompt,
-		// since there is no available context.
-		sinon.assert.calledOnce(sendRequestSpy);
-		const [messages,] = sendRequestSpy.getCall(0).args;
-		assert.strictEqual(messages.length, 1, `Unexpected messages: ${JSON.stringify(messages)}`);
-		assertMessageRole(messages[0], vscode.LanguageModelChatMessageRole.User);
-		assertMessageTextPart(messages[0].content[0], request.prompt);
-	});
-
 	test('should include positron session context', async () => {
 		// Setup test inputs.
 		const request = makeChatRequest({ model, references: [] });
@@ -174,7 +158,7 @@ suite('PositronAssistantParticipant', () => {
 		sinon.assert.calledOnce(sendRequestSpy);
 		const [messages,] = sendRequestSpy.getCall(0).args;
 		const c = positronChatContext;
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		assertContextMessage(messages[0],
 			`<context>
 <console description="Current active console" language="${c.console!.language}" version="${c.console!.version}" identifier="${c.console!.identifier}">
@@ -223,7 +207,7 @@ ${c.plots!.hasPlots ? 'A plot is visible.' : ''}
 		const document = await vscode.workspace.openTextDocument(referenceUri);
 		const filePath = vscode.workspace.asRelativePath(referenceUri);
 		const attachmentsText = await readFile(path.join(MARKDOWN_DIR, 'prompts', 'chat', 'attachments.md'), 'utf8');
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		assertContextMessage(messages[0],
 			`<attachments>
 ${attachmentsText}
@@ -257,7 +241,7 @@ ${document.getText()}
 		const document = await vscode.workspace.openTextDocument(referenceUri);
 		const filePath = vscode.workspace.asRelativePath(referenceUri);
 		const attachmentsText = await readFile(path.join(MARKDOWN_DIR, 'prompts', 'chat', 'attachments.md'), 'utf8');
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		assertContextMessage(messages[0],
 			`<attachments>
 ${attachmentsText}
@@ -293,7 +277,7 @@ ${document.getText()}
 		sinon.assert.calledOnce(sendRequestSpy);
 		const [messages,] = sendRequestSpy.getCall(0).args;
 		const attachmentsText = await readFile(path.join(MARKDOWN_DIR, 'prompts', 'chat', 'attachments.md'), 'utf8');
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		assertContextMessage(messages[0],
 			`<attachments>
 ${attachmentsText}
@@ -329,7 +313,7 @@ It should be included in the chat message.`;
 		// The first user message should contain the formatted context.
 		sinon.assert.calledOnce(sendRequestSpy);
 		const [messages,] = sendRequestSpy.getCall(0).args;
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		assertContextMessage(messages[0],
 			`<instructions>
 ${llmsTxtContent}
@@ -353,7 +337,7 @@ ${llmsTxtContent}
 		// The first user message should contain the formatted context.
 		sinon.assert.calledOnce(sendRequestSpy);
 		const [messages,] = sendRequestSpy.getCall(0).args;
-		assert.strictEqual(messages.length, 2, `Unexpected messages: ${JSON.stringify(messages)}`);
+		assert.strictEqual(messages.length, DEFAULT_EXPECTED_MESSAGE_COUNT, `Unexpected messages: ${JSON.stringify(messages)}`);
 		const filePath = vscode.workspace.asRelativePath(referenceUri);
 		assertContextMessage(messages[0],
 			`<editor description="Current active editor" filePath="${filePath}" language="${document.languageId}" line="${selection.active.line + 1}" column="${selection.active.character + 1}" documentOffset="${document.offsetAt(selection.active)}">
@@ -425,7 +409,8 @@ function assertContextMessage(
 	assertMessageRole(message, vscode.LanguageModelChatMessageRole.User);
 
 	// The first part should be a text part with the formatted context.
-	assertMessageTextPart(message.content[0], expectedPrompt);
+	const fullPrompt = `${expectedPrompt}${expectedPrompt.length > 0 ? '\n\n' : ''}${getDefaultContextItems().join('\n')}`;
+	assertMessageTextPart(message.content[0], fullPrompt);
 
 	if (expectedImage) {
 		// If an image is expected, the second part should be a data part with the image.
diff --git a/extensions/positron-assistant/src/utils.ts b/extensions/positron-assistant/src/utils.ts
index ee9bae5e9569..8adee4ea65e0 100644
--- a/extensions/positron-assistant/src/utils.ts
+++ b/extensions/positron-assistant/src/utils.ts
@@ -256,22 +256,69 @@ export function isChatImageMimeType(mimeType: string): mimeType is vscode.ChatIm
 	return Object.values(vscode.ChatImageMimeType).includes(mimeType as vscode.ChatImageMimeType);
 }
 
+export const EMPTY_TOOL_RESULT_PLACEHOLDER = '';
+
 /**
-* Checks if a message contains any non-empty content.
-* @param message The message to check
-* @returns True if the message has any non-empty content, false otherwise
-*/
-export function hasNonEmptyContent(message: vscode.LanguageModelChatMessage2): boolean {
-	return message.content.some(part => {
-		if (part instanceof vscode.LanguageModelTextPart) {
-			return part.value.trim() !== '';
-		}
-		if (part instanceof vscode.LanguageModelToolResultPart) {
-			return part.content.length > 0;
+ * Processes a message to ensure it has non-empty tool result parts.
+ * If a tool result part is empty, it replaces it with a placeholder.
+ * This is a workaround for LLMs that don't handle empty tool result parts well.
+ * @todo: We may be able to remove this handling in the future, to save on token count,
+ * once LLMs are better at handling empty tool result parts.
+ * @param message The message to process
+ * @returns A new message with empty tool result parts replaced with a placeholder
+ */
+function processEmptyToolResults(message: vscode.LanguageModelChatMessage2) {
+	let replacedEmptyToolResult = false;
+	const updatedContent = message.content.map(part => {
+		const isToolResult = part instanceof vscode.LanguageModelToolResultPart || part instanceof vscode.LanguageModelToolResultPart2;
+		if (isToolResult && part.content.length === 0) {
+			replacedEmptyToolResult = true;
+			return new vscode.LanguageModelToolResultPart(
+				part.callId,
+				[new vscode.LanguageModelTextPart(EMPTY_TOOL_RESULT_PLACEHOLDER)],
+			);
 		}
-		// Other part types are considered non-empty, such as LanguageModelToolCallPart and LanguageModelDataPart
-		return true;
+		// For other parts, such as LanguageModelToolCallPart or LanguageModelDataPart,
+		// just return them as is, as we expect them to be non-empty.
+		return part;
 	});
+
+	if (replacedEmptyToolResult) {
+		return new vscode.LanguageModelChatMessage2(
+			message.role,
+			updatedContent,
+			message.name,
+		);
+	}
+
+	return message;
+}
+
+/**
+ * Checks if a message has content.
+ * A message is considered to have non-empty content if it contains at one least item
+ * in its content array that is not an empty/whitespace LanguageModelTextPart.
+ * @param message The message to check
+ * @returns Whether the message has non-empty content
+ */
+function hasContent(message: vscode.LanguageModelChatMessage2) {
+	return message.content.length > 0 &&
+		!message.content.every(
+			part => part instanceof vscode.LanguageModelTextPart && part.value.trim() === ''
+		);
+}
+
+/**
+ * Processes an array of messages to ensure they have non-empty content,
+ * filtering out any messages that do not meet this criteria and filling in
+ * placeholders for empty tool result parts.
+ * @param messages The messages to process
+ * @returns
+ */
+export function processMessages(messages: vscode.LanguageModelChatMessage2[]) {
+	return messages
+		.filter(hasContent)
+		.map(processEmptyToolResults);
 }
 
 // This type definition is from Vercel AI, but the type is not exported.
diff --git a/src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts b/src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
index 2ba014bf3362..279b838b197d 100644
--- a/src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
+++ b/src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
@@ -20,6 +20,7 @@ This tool returns the contents of the specified file in the project.
 The provided file path must be a path to a file in the workspace or a file that is currently open in the editor.
 The file path can be either absolute or relative to the workspace root.
 The tool will return the contents of the file as a string, along with its size and encoding.
+Do not use this tool when no workspace folders are open.
 `;
 
 export const ExtensionFileContentsToolId = 'positron_getFileContents';
@@ -59,7 +60,7 @@ export class FileContentsTool implements IToolImpl {
 		if (isAbsolute(filePath)) {
 			uri = URI.file(filePath);
 			if (!this.fileIsOpenOrInsideWorkspace(uri)) {
-				throw new Error(`File contents for ${filePath} can't be retrieved because the file is not open or inside the current workspace`);
+				throw new Error(`Can't retrieve file contents for ${filePath} because the file is not open or inside the current workspace. Ensure the file is open in an editor or inside the workspace.`);
 			}
 		} else {
 			// If the file path is relative, try to resolve it against the workspace folders
@@ -72,7 +73,7 @@ export class FileContentsTool implements IToolImpl {
 				}
 			}
 			if (!uri) {
-				throw new Error(`File contents for ${filePath} can't be retrieved because the file is not open or inside any folders in the current workspace`);
+				throw new Error(`Can't retrieve file contents for ${filePath} because the file is not open or inside the current workspace. Ensure the file is open in an editor or inside the workspace.`);
 			}
 		}
 
diff --git a/src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts b/src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
index d21409be86b9..45665ece19e3 100644
--- a/src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
+++ b/src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
@@ -22,7 +22,7 @@ The project tree is represented as a nested array, where each entry can be eithe
 This tool ignores node_modules, __pycache__, dist directories, certain files like .DS_Store, Thumbs.db, and desktop.ini. and files with certain extensions like *.o, *.a, *.so, *.pyo.
 This tool does not provide information for specific files or directories, but rather gives an overview of the entire project structure.
 This tool is helpful for locating files and directories in the workspace.
-This tool only needs to be called once per conversation, unless files or directories are added, removed, moved, or renamed in the workspace.
+Do not use this tool when no workspace folders are open.
 `;
 
 export const ExtensionProjectTreeId = 'positron_getProjectTree';
@@ -45,10 +45,7 @@ export class ProjectTreeTool implements IToolImpl {
 	async invoke(_invocation: IToolInvocation, _countTokens: CountTokensCallback, _progress: ToolProgress, _token: CancellationToken): Promise<IToolResult> {
 		const workspaceFolders = this._workspaceContextService.getWorkspace().folders;
 		if (workspaceFolders.length === 0) {
-			return {
-				content: [],
-				toolResultMessage: 'No workspace folders found.'
-			};
+			throw new Error(`Can't construct project tree because no workspace folders are open. Open a workspace folder before using this tool.`);
 		}
 
 		const workspaceTrees: DirectoryItem[][] = [];
diff --git a/src/vs/workbench/contrib/chat/browser/tools/textSearchTool.ts b/src/vs/workbench/contrib/chat/browser/tools/textSearchTool.ts
index e57f5eeec7e0..0c493281d079 100644
--- a/src/vs/workbench/contrib/chat/browser/tools/textSearchTool.ts
+++ b/src/vs/workbench/contrib/chat/browser/tools/textSearchTool.ts
@@ -21,6 +21,7 @@ Do not use this tool to find files or directories in the workspace, as it is spe
 The search is performed across all files in the project, excluding files and directories that are ignored by the workspace settings.
 The provided pattern is interpreted as text unless indicated to be a regular expression.
 Other search options such as case sensitivity, whole word matching, and multiline matching can be specified.
+Do not use this tool when no workspace folders are open.
 `;
 
 export const ExtensionTextSearchToolId = 'positron_findTextInProject';
@@ -87,10 +88,7 @@ export class TextSearchTool implements IToolImpl {
 	async invoke(invocation: IToolInvocation, _countTokens: CountTokensCallback, _progress: ToolProgress, _token: CancellationToken): Promise<IToolResult> {
 		const workspaceFolders = this._workspaceContextService.getWorkspace().folders;
 		if (workspaceFolders.length === 0) {
-			return {
-				content: [],
-				toolResultMessage: 'No workspace folders found.'
-			};
+			throw new Error(`Can't search for text in project because no workspace folders are open. Open a workspace folder before using this tool.`);
 		}
 
 		// Set up the text search query


Reviews:
user: login: sharon-wang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-21T18:54:36Z
user: login: sharon-wang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-21T18:57:32Z
user: login: seeM
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T10:41:49Z
user: login: sharon-wang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T15:40:55Z
user: login: sharon-wang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T17:08:34Z
user: login: sharon-wang
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-22T18:02:42Z

Review comments:
diff_hunk: @@ -58,61 +59,24 @@ suite('AnthropicLanguageModel', () => {
 		sinon.restore();
 	});
 
-	test('provideLanguageModelResponse should filter out messages with empty text content', async () => {
path: extensions/positron-assistant/src/test/anthropic.test.ts
user: login: sharon-wang
body: I removed this test case because it is covered in the `provideLanguageModelResponse filters empty messages and different LanguageModelTextPart contents correctly` test
created_at: 2025-05-21T18:54:36Z
author_association: MEMBER
diff_hunk: @@ -59,7 +60,13 @@ export class FileContentsTool implements IToolImpl {
 		if (isAbsolute(filePath)) {
 			uri = URI.file(filePath);
 			if (!this.fileIsOpenOrInsideWorkspace(uri)) {
-				throw new Error(`File contents for ${filePath} can't be retrieved because the file is not open or inside the current workspace`);
+				return {
path: src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
user: login: sharon-wang
body: Returning a message felt better in the chat flow, as the Assistant will then suggest to the user to open up a workspace. When an error is returned, the Assistant doesn't say anything.
created_at: 2025-05-21T18:57:32Z
author_association: MEMBER
diff_hunk: @@ -59,7 +60,13 @@ export class FileContentsTool implements IToolImpl {
 		if (isAbsolute(filePath)) {
 			uri = URI.file(filePath);
 			if (!this.fileIsOpenOrInsideWorkspace(uri)) {
-				throw new Error(`File contents for ${filePath} can't be retrieved because the file is not open or inside the current workspace`);
+				return {
path: src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
user: login: seeM
body: I was initially unsure about when to throw vs early return in tools. The main difference is that throwing immediately halts the conversation, and returning gives the model a chance to reply.

Given the above, I think we should throw (with a helpful message!) when we're confident the user needs to take some action before trying again. It'll save them time and tokens! Also worth keeping in mind that models can sometimes go off on endless tool calling loops.

What do you think? An alternative here could be to keep the throw but also suggest how to remedy the error.
created_at: 2025-05-22T10:27:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,6 +23,7 @@ This tool ignores node_modules, __pycache__, dist directories, certain files lik
 This tool does not provide information for specific files or directories, but rather gives an overview of the entire project structure.
 This tool is helpful for locating files and directories in the workspace.
 This tool only needs to be called once per conversation, unless files or directories are added, removed, moved, or renamed in the workspace.
+If the workspace is empty or no workspace folders are open, this tool cannot generate a project tree.
path: src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
user: login: seeM
body: Another approach is to include workspace information in the context message and instruct the model not to call the tool if the context shows that there is no workspace open. I was able to get Anthropic to do less inspect variables requests for non-existent variables in that way:

> ...
> 
> If the user references a variable by name, first determine the `access_key` from the user context or a previous inspect variables result.
> 
> Do not call when:
>   - the variables do not appear in the user context
> ...
created_at: 2025-05-22T10:33:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,6 +23,7 @@ This tool ignores node_modules, __pycache__, dist directories, certain files lik
 This tool does not provide information for specific files or directories, but rather gives an overview of the entire project structure.
 This tool is helpful for locating files and directories in the workspace.
 This tool only needs to be called once per conversation, unless files or directories are added, removed, moved, or renamed in the workspace.
+If the workspace is empty or no workspace folders are open, this tool cannot generate a project tree.
path: src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
user: login: seeM
body: Also, my intuition is that more direct instructions work better (and use fewer tokens) i.e.

> Do not use this tool when the workspace is empty or no workspace is open.
created_at: 2025-05-22T10:36:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -256,22 +256,66 @@ export function isChatImageMimeType(mimeType: string): mimeType is vscode.ChatIm
 	return Object.values(vscode.ChatImageMimeType).includes(mimeType as vscode.ChatImageMimeType);
 }
 
+export const EMPTY_TOOL_RESULT_PLACEHOLDER = 'tool result is empty';
path: extensions/positron-assistant/src/utils.ts
user: login: seeM
body: I wonder if an empty string is better, if it still fixes the issue. It'll use fewer tokens and IMO convey the same information.
created_at: 2025-05-22T10:41:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,6 +23,7 @@ This tool ignores node_modules, __pycache__, dist directories, certain files lik
 This tool does not provide information for specific files or directories, but rather gives an overview of the entire project structure.
 This tool is helpful for locating files and directories in the workspace.
 This tool only needs to be called once per conversation, unless files or directories are added, removed, moved, or renamed in the workspace.
+If the workspace is empty or no workspace folders are open, this tool cannot generate a project tree.
path: src/vs/workbench/contrib/chat/browser/tools/projectTreeTool.ts
user: login: sharon-wang
body: Ah this is much nicer! I rebased to pull in the context changes and was able to add a note to the context when workspace folders are not open. Now, with a corresponding update to the workspace tools descriptions, anthropic doesn't call any of the workspace tools if we've noted in the context that workspace folders aren't open 👌 
created_at: 2025-05-22T15:40:55Z
author_association: MEMBER
diff_hunk: @@ -59,7 +60,13 @@ export class FileContentsTool implements IToolImpl {
 		if (isAbsolute(filePath)) {
 			uri = URI.file(filePath);
 			if (!this.fileIsOpenOrInsideWorkspace(uri)) {
-				throw new Error(`File contents for ${filePath} can't be retrieved because the file is not open or inside the current workspace`);
+				return {
path: src/vs/workbench/contrib/chat/browser/tools/fileContentsTool.ts
user: login: sharon-wang
body: It makes sense to me to keep the throws, but add a bit more guidance on how to proceed. Coupled with the change to indicate in the context if workspace folders are open, I think we are far less likely to hit these errors in the workspace tools
created_at: 2025-05-22T17:08:33Z
author_association: MEMBER
diff_hunk: @@ -256,22 +256,66 @@ export function isChatImageMimeType(mimeType: string): mimeType is vscode.ChatIm
 	return Object.values(vscode.ChatImageMimeType).includes(mimeType as vscode.ChatImageMimeType);
 }
 
+export const EMPTY_TOOL_RESULT_PLACEHOLDER = 'tool result is empty';
path: extensions/positron-assistant/src/utils.ts
user: login: sharon-wang
body: working well for me! https://github.com/posit-dev/positron/pull/7799/commits/20822a85abe0c58b4186922bed894f305d680030
created_at: 2025-05-22T18:02:41Z
author_association: MEMBER

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-21T18:53:57Z
body: <!-- PR Tags -->
<!-- 
🚨 RED ALERT! ✋ Rule breaker detected! Tags don’t go here, they belong above ^ in the PR description using this proper format: `@:tag`. Changing them here won't do anything (trust us, we’ve tried). Confused? Check out the README hyperlink.
-->

**E2E Tests** 🚀
This PR will run tests tagged with: `@:critical` `@:assistant`

<sup>[readme](https://github.com/posit-dev/positron/blob/main/test/e2e/README.md#pull-requests-and-test-tags)</sup>&nbsp;&nbsp;<sup>[valid tags](https://github.com/posit-dev/positron/blob/main/test/e2e/infra/test-runner/test-tags.ts)</sup>
author_association: NONE
user: login: sharon-wang
created_at: 2025-05-22T18:13:21Z
body: Thanks @seeM for your feedback! I have implemented all the suggestions, namely:

- add whether workspace folders are open into the context
- directly include in tool description `Do not use this tool when no workspace folders are open`
- throw errors in the tools when the tool criteria is not met (instead of returning messages)
- use an empty string as the empty tool result placeholder

Will merge once tests are green.
author_association: MEMBER