Add simple tool calling script that works around vLLM tool_choice limitation

TOOL_CALLING_GUIDE.md

@@ -0,0 +1,134 @@
+# Tool Calling Guide
+
+Your ExCom AI deployment supports tool calling! However, there's a quirk with vLLM that requires a workaround.
+
+## The Issue
+
+vLLM requires `--enable-auto-tool-choice` and `--tool-call-parser` flags to accept the `tool_choice: "auto"` parameter. Since Qwen 2.5 has native tool calling built into the model, we don't use these flags.
+
+**Result**: LangChain's default agent framework sends `tool_choice: "auto"` which vLLM rejects with a 400 error.
+
+## Solution: Use OpenAI SDK Directly
+
+I've created `simple_tool_chat.py` which uses the OpenAI SDK directly and doesn't send `tool_choice`.
+
+### Installation
+
+```bash
+pip install openai
+```
+
+### Usage
+
+```bash
+python simple_tool_chat.py
+```
+
+### Example Session
+
+```
+You: What is 15 * 23 + 100?
+🔧 Calling tool: calculator({'expression': '15 * 23 + 100'})
+Assistant: The result is 445.
+
+You: What's the weather in Paris and what time is it?
+🔧 Calling tool: get_weather({'city': 'Paris'})
+🔧 Calling tool: get_current_time({})
+Assistant: The weather in Paris is 18°C and sunny. The current time is 2025-10-09 18:30:45.
+```
+
+## How It Works
+
+1. **No tool_choice parameter** - We don't send `tool_choice` at all
+2. **Qwen decides naturally** - The model's training handles when to use tools
+3. **OpenAI SDK** - Direct HTTP calls to your vLLM endpoint
+4. **Multi-turn** - Maintains conversation history for context
+
+## Using with Your Own Code
+
+```python
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="https://plarnholt-excom-ai-demo.hf.space/v1",
+    api_key="not-needed"
+)
+
+# Define your tools
+tools = [{
+    "type": "function",
+    "function": {
+        "name": "my_tool",
+        "description": "What it does",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "param": {"type": "string"}
+            }
+        }
+    }
+}]
+
+# Call without tool_choice parameter
+response = client.chat.completions.create(
+    model="excom-ai",
+    messages=[{"role": "user", "content": "Use my tool"}],
+    tools=tools,
+    temperature=0.4
+    # NOTE: No tool_choice parameter!
+)
+
+# Check for tool calls
+if response.choices[0].message.tool_calls:
+    for tool_call in response.choices[0].message.tool_calls:
+        print(f"Tool: {tool_call.function.name}")
+        print(f"Args: {tool_call.function.arguments}")
+```
+
+## Adding Custom Tools
+
+Edit `simple_tool_chat.py`:
+
+```python
+# 1. Add tool definition to 'tools' list
+{
+    "type": "function",
+    "function": {
+        "name": "my_custom_tool",
+        "description": "What it does",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "param": {"type": "string", "description": "Param description"}
+            },
+            "required": ["param"]
+        }
+    }
+}
+
+# 2. Add implementation
+def my_custom_tool(param: str) -> str:
+    # Your logic here
+    return "result"
+
+# 3. Add to dispatcher
+def execute_tool(tool_name: str, arguments: dict) -> str:
+    # ... existing tools ...
+    elif tool_name == "my_custom_tool":
+        return my_custom_tool(arguments["param"])
+```
+
+## Troubleshooting
+
+**Error: "auto" tool choice requires --enable-auto-tool-choice**
+- You're using LangChain's agent framework
+- Solution: Use `simple_tool_chat.py` instead
+
+**Tool calls not working**
+- Make sure your Space is running: https://huggingface.co/spaces/plarnholt/excom-ai-demo
+- Check that you're not sending `tool_choice` parameter
+- Verify tools are properly formatted (see OpenAI docs)
+
+**500 Internal Server Error**
+- Space might be sleeping - make a request to wake it up
+- Check Space logs for errors

simple_tool_chat.py

@@ -0,0 +1,197 @@
+"""
+Simple tool-calling chat with ExCom AI
+Works around vLLM's tool_choice requirements
+"""
+
+from openai import OpenAI
+import json
+from datetime import datetime
+import math
+
+# Configure OpenAI client for your vLLM endpoint
+client = OpenAI(
+    base_url="https://plarnholt-excom-ai-demo.hf.space/v1",
+    api_key="not-needed"
+)
+
+# Define tools
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "calculator",
+            "description": "Evaluates a mathematical expression",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "expression": {
+                        "type": "string",
+                        "description": "Math expression to evaluate, e.g., '2 + 2 * 3'"
+                    }
+                },
+                "required": ["expression"]
+            }
+        }
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "get_current_time",
+            "description": "Returns the current date and time",
+            "parameters": {
+                "type": "object",
+                "properties": {}
+            }
+        }
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "get_weather",
+            "description": "Gets the weather for a city (simulated)",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "city": {
+                        "type": "string",
+                        "description": "City name"
+                    }
+                },
+                "required": ["city"]
+            }
+        }
+    }
+]
+
+# Tool implementations
+def calculator(expression: str) -> str:
+    try:
+        result = eval(expression, {"__builtins__": {}, "math": math})
+        return str(result)
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+def get_current_time() -> str:
+    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+
+def get_weather(city: str) -> str:
+    weather_data = {
+        "paris": "18°C, sunny",
+        "london": "15°C, cloudy",
+        "new york": "22°C, partly cloudy",
+        "tokyo": "25°C, clear",
+    }
+    return weather_data.get(city.lower(), f"Weather data not available for {city}")
+
+# Function dispatcher
+def execute_tool(tool_name: str, arguments: dict) -> str:
+    if tool_name == "calculator":
+        return calculator(arguments["expression"])
+    elif tool_name == "get_current_time":
+        return get_current_time()
+    elif tool_name == "get_weather":
+        return get_weather(arguments["city"])
+    else:
+        return f"Unknown tool: {tool_name}"
+
+def chat(user_message: str, messages: list = None):
+    """Send a message and handle tool calls"""
+    if messages is None:
+        messages = []
+
+    # Add user message
+    messages.append({"role": "user", "content": user_message})
+
+    # Call the model with tools (no tool_choice parameter)
+    response = client.chat.completions.create(
+        model="excom-ai",
+        messages=messages,
+        tools=tools,
+        temperature=0.4
+    )
+
+    assistant_message = response.choices[0].message
+
+    # Check if model wants to use tools
+    if assistant_message.tool_calls:
+        # Add assistant's tool call request to messages
+        messages.append({
+            "role": "assistant",
+            "content": assistant_message.content,
+            "tool_calls": [
+                {
+                    "id": tc.id,
+                    "type": "function",
+                    "function": {
+                        "name": tc.function.name,
+                        "arguments": tc.function.arguments
+                    }
+                }
+                for tc in assistant_message.tool_calls
+            ]
+        })
+
+        # Execute each tool call
+        for tool_call in assistant_message.tool_calls:
+            function_name = tool_call.function.name
+            function_args = json.loads(tool_call.function.arguments)
+
+            print(f"🔧 Calling tool: {function_name}({function_args})")
+
+            # Execute the tool
+            tool_result = execute_tool(function_name, function_args)
+
+            # Add tool result to messages
+            messages.append({
+                "role": "tool",
+                "tool_call_id": tool_call.id,
+                "name": function_name,
+                "content": tool_result
+            })
+
+        # Get final response from model
+        final_response = client.chat.completions.create(
+            model="excom-ai",
+            messages=messages,
+            temperature=0.4
+        )
+
+        return final_response.choices[0].message.content, messages
+    else:
+        # No tools needed, return direct response
+        return assistant_message.content, messages
+
+def main():
+    print("=" * 60)
+    print("ExCom AI - Simple Tool Calling Chat")
+    print("=" * 60)
+    print("Available tools:")
+    print("  • calculator - Evaluate math expressions")
+    print("  • get_current_time - Get current date/time")
+    print("  • get_weather - Get weather for cities")
+    print("\nType 'quit' or 'exit' to end.")
+    print("=" * 60)
+    print()
+
+    messages = []
+
+    while True:
+        user_input = input("You: ").strip()
+
+        if user_input.lower() in ['quit', 'exit', 'q']:
+            print("Goodbye!")
+            break
+
+        if not user_input:
+            continue
+
+        try:
+            response, messages = chat(user_input, messages)
+            print(f"Assistant: {response}\n")
+        except Exception as e:
+            print(f"❌ Error: {e}\n")
+            # Reset messages on error
+            messages = []
+
+if __name__ == "__main__":
+    main()