Skill: qqshi13/wincontrol (version 2.0.0)

# WinControl Skill AI remote control for Windows desktop. Captures screen on-demand via POST request and provides an HTTP API for mouse/keyboard actions. ## What It Does - **On-Demand Capture**: POST to `/capture` to save screenshot, returns file path - **Action API**: Control mouse and keyboard via HTTP endpoints on port 8767 - **Auto-Cleanup**: Deletes screenshot on server shutdown (Ctrl+C) - **Cross-Platform**: Works on native Windows or WSL - **Self-Contained**: Single `screenshot.jpg` saved in skill folder, overwritten each time, auto-deleted on exit ## Quick Start ### Option 1: Native Windows (Recommended) Run directly on Windows without WSL: ```powershell # Install dependencies (one-time) pip install pywin32 pillow mss # Start the server cd %USERPROFILE%\.openclaw\workspace\skills\wincontrol python server.py ``` Or use the provided batch file: ```powershell .\start.bat ``` Screenshots are saved as `screenshot.jpg` in the skill directory and auto-deleted when the server stops. ### Option 2: WSL Integration If you prefer WSL, the server runs on Windows Python but can be controlled from WSL: ```bash # From WSL cd ~/.openclaw/workspace/skills/wincontrol ./start.sh ``` Or start manually with PowerShell 7: ```bash '/mnt/c/Program Files/PowerShell/7/pwsh.exe' -Command \ "python //wsl.localhost/Ubuntu/home/\$USER/.openclaw/workspace/skills/wincontrol/server.py" & ``` ## Verify Installation ```bash # Health check curl http://localhost:8767/ping # Output: {"ok": true} # Capture a screenshot curl -X POST http://localhost:8767/capture # Output: {"ok": true, "path": ".../screenshot.jpg"} ``` **Native Windows**: Open `screenshot.jpg` in the skill directory **WSL**: Access via the skill folder path ## File Structure ``` skills/wincontrol/ ├── SKILL.md # This file ├── server.py # Main server (runs on Windows) ├── start.bat # Start script (Native Windows) ├── start.sh # Start script (WSL) ├── stop.sh # Stop script (WSL) └── screenshot.jpg # Latest screenshot (auto-created, auto-cleaned) ``` ## API Reference ### Capture Screen ```bash curl -X POST http://localhost:8767/capture ``` Returns: `{"ok": true, "path": ".../screenshot.jpg", "quality": 90}` Each capture overwrites `screenshot.jpg` in the skill directory. The file is automatically deleted when the server stops. **Optional quality override:** ```bash # Lower quality for faster capture/smaller file curl -X POST http://localhost:8767/capture -H "Content-Type: application/json" -d '{"quality": 60}' # Higher quality for detailed screenshots curl -X POST http://localhost:8767/capture -d '{"quality": 95}' ``` | Quality | File Size (1080p) | Use Case | |---------|-------------------|----------| | 60 | ~80KB | Fast capture, quick checks | | 90 (default) | ~150KB | Clear UI text, general use | | 95 | ~200KB | Maximum detail, presentations | ### Mouse Actions ```bash # Move cursor (no click) curl -X POST http://localhost:8767/move -d '{"x": 500, "y": 300}' # Click curl -X POST http://localhost:8767/click -d '{"x": 500, "y": 300}' # Drag curl -X POST http://localhost:8767/drag -d '{"x1": 100, "y1": 200, "x2": 300, "y2": 400}' # Scroll curl -X POST http://localhost:8767/scroll -d '{"x": 500, "y": 300, "direction": "down", "amount": 3}' ``` ### Keyboard Input ```bash # Type text curl -X POST http://localhost:8767/enter -d '{"keys": ["Hello World"]}' # Press special key curl -X POST http://localhost:8767/enter -d '{"keys": ["Enter"]}' # Key combination (Ctrl+C) curl -X POST http://localhost:8767/enter -d '{"keys": ["Ctrl", "C"]}' # Mixed sequence: type, press key, then combo curl -X POST http://localhost:8767/enter -d '{"keys": ["Hello", "Enter", "Ctrl", "A"]}' ``` The `/enter` endpoint accepts a list of keys and handles them sequentially: - **Text strings** → Typed as-is - **Special keys** → Pressed once (Enter, Escape, Tab, etc.) - **Modifier + key** → Treated as combination (Ctrl+C, Alt+Tab, etc.) ### Special Keys Reference Single keys: - `Enter`, `Return`, `Escape`, `Esc` - `Backspace`, `Tab`, `Space` - `Delete`, `Del` - `Up`, `Down`, `Left`, `Right` - `Home`, `End`, `PageUp`, `PageDown` - `F1` through `F12` Modifiers (combine with other keys): - `Ctrl`, `Control` - `Alt` - `Shift` - `Win`, `Windows` ## Common Workflows ### Capture and View (Native Windows) ```powershell # Using PowerShell $response = Invoke-RestMethod -Uri "http://localhost:8767/capture" -Method Post Start-Process $response.path ``` ### Capture and View (WSL) ```bash FILE=$(curl -s -X POST http://localhost:8767/capture | python3 -c "import sys,json; print(json.load(sys.stdin)['path'])") read "$FILE" ``` ### Click and Verify ```bash # Click somewhere curl -X POST http://localhost:8767/click -d '{"x": 500, "y": 300}' sleep 0.5 # Capture to see result curl -X POST http://localhost:8767/capture ``` ### Open Notepad and Type ```bash # Win+R to open Run curl -X POST http://localhost:8767/enter -d '{"keys": ["Win", "R"]}' sleep 0.5 # Type notepad and press Enter curl -X POST http://localhost:8767/enter -d '{"keys": ["notepad", "Enter"]}' sleep 1 # Type message curl -X POST http://localhost:8767/enter -d '{"keys": ["Hello from WinControl!"]}' ``` ## Installation ### Native Windows 1. **Install Python 3** from [python.org](https://python.org) 2. **Install dependencies**: ```powershell pip install pywin32 pillow mss ``` 3. **Clone/download** the skill to `~/.openclaw/workspace/skills/wincontrol/` 4. **Start the server**: ```powershell python server.py ``` ### WSL2 (Legacy) See WSL2 Configuration section below. ## Frame Management - **Capture**: On-demand via `POST /capture` - **Quality**: 90% JPEG for clear text/UI - **File**: `screenshot.jpg` in skill directory - **Behavior**: Overwritten on each capture - **Auto-cleanup**: Deleted when server stops (Ctrl+C) To change quality, edit `server.py`: ```python QUALITY = 90 # 1-100 ``` ## Stopping the Server The server automatically cleans up the screenshot when stopped. ### Native Windows Press `Ctrl+C` in the PowerShell window, or: ```powershell # Find and stop the process Get-Process python | Where-Object {$_.CommandLine -like '*wincontrol*'} | Stop-Process ``` ### WSL ```bash ./stop.sh ``` Or manually: ```bash # Kill Python process on Windows using PowerShell 7 '/mnt/c/Program Files/PowerShell/7/pwsh.exe' -Command \ "Get-Process python -ErrorAction SilentlyContinue | Where-Object {\$_.CommandLine -like '*wincontrol*'} | Stop-Process -Force" ``` ## Troubleshooting ### Native Windows **Issue**: `pip install pywin32` fails - Try: `pip install pywin32 --upgrade --force-reinstall` - Or download from [GitHub releases](https://github.com/mhammond/pywin32/releases) **Issue**: Port 8767 already in use - Find the process: `netstat -ano | findstr :8767` - Kill it: `taskkill /PID <PID> /F` **Issue**: Cannot access from WSL - Ensure Windows Firewall allows Python through - Try disabling Windows Defender Firewall temporarily for testing ### WSL2 ### Prerequisites 1. **WSL2 with Ubuntu** (or your preferred distro) 2. **Python 3** on Windows side 3. **Dependencies**: pywin32, pillow, mss (auto-installed) ### Path Configuration Screenshots are saved as `screenshot.jpg` in the skill folder, accessible from both Windows and WSL. ### Troubleshooting **Issue**: Server starts but curl fails - Check if port 8767 is in use: `lsof -i :8767` - Kill existing process: `kill <PID>` **Issue**: Python module errors - Manually install deps on Windows: ```bash '/mnt/c/Program Files/PowerShell/7/pwsh.exe' -Command "pip install pywin32 pillow mss" ``` ## How It Works ``` ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │ Client │────▶│ server.py │────▶│ screenshot.jpg│ │ (curl/ps) │ │ (localhost) │ │ (skill dir) │ └─────────────┘ └──────────────┘ └────────────────┘ │ ┌─────┴─────┐ │ Port │ │ 8767 │ └───────────┘ ``` The server runs on Windows Python. Screenshots are saved as `screenshot.jpg` in the skill directory and automatically deleted when the server stops (Ctrl+C). ## Python Client Example ```python import requests import time API = "http://localhost:8767" def capture(): """Capture screen and return file path""" r = requests.post(f"{API}/capture") return r.json().get("path") def click(x, y): requests.post(f"{API}/click", json={"x": x, "y": y}) def enter(keys): requests.post(f"{API}/enter", json={"keys": keys}) # Example workflow if __name__ == "__main__": # Type text, press Enter, then select all enter(["Hello World", "Enter", "Ctrl", "A"]) time.sleep(0.5) # Capture result screenshot_path = capture() print(f"Screenshot saved to: {screenshot_path}") ``` ## Security Notes - Server binds to `localhost` only (not accessible from network) - No authentication - anyone with local access can control your desktop - Screenshot is auto-deleted when server stops - Screenshot saved as single `screenshot.jpg` in skill folder, not system directories - Be careful running this on shared machines ## Integration with OpenClaw ### Native Windows ```javascript // Capture and view const result = await exec("curl -s -X POST http://localhost:8767/capture"); const data = JSON.parse(result.stdout); // Path will be Windows format: C:\Users\... await read(data.path); // OpenClaw handles Windows paths ``` ### WSL ```javascript // Capture and view const result = await exec("curl -s -X POST http://localhost:8767/capture"); const data = JSON.parse(result.stdout); await read(data.path); // Path: /mnt/c/Users/... // Or take action then capture await exec("curl -X POST http://localhost:8767/click -d '{\"x\":500,\"y\":300}'"); await exec("sleep 0.5"); const screenshot = await exec("curl -s -X POST http://localhost:8767/capture"); ``` ## License MIT-0

path: CHANGELOG.md
content: # Changelog

## [2.0.0] - 2026-05-14

### Added
- **Quality parameter for `/capture`**: Optional quality override (1-100)
  - Body: `{"quality": 60}` for faster/smaller captures
  - Defaults to 90 if not specified
  - Response includes `quality` field: `{"ok": true, "path": "...", "quality": 60}`

### Changed
- **Updated docs**: README and SKILL.md now document quality feature
- **Gitignore**: Added `.gitignore` to exclude `screenshot.jpg`

## [1.5.0] - 2026-04-12

### Added
- **Cursor movement endpoint**: `POST /move` - Move mouse cursor without clicking
  - Body: `{"x": 500, "y": 300}`
  - Useful for hovering before clicking or UI traversal

## [1.4.0] - 2026-04-12

### Changed
- **Single screenshot file**: Saves to `screenshot.jpg` in skill folder, overwritten each capture
  - No more numbered frame files
  - Simpler to consume from other tools
- **Simplified capture response**: Removed `frame` counter and `screen` dimensions from `/capture` endpoint
  - Response is now simply: `{"ok": true, "path": "..."}`
- **Removed endpoint**: `GET /frames` - no longer needed with single screenshot
- **License**: MIT-0 (ClawHub requires all skills to be MIT-0)

### API Changes
**Before:**
```bash
POST /capture  # Returns {"path": ".../frames/frame_000001.jpg", "frame": 1, "screen": {...}}
GET /frames    # List all frames
```

**After:**
```bash
POST /capture  # Returns {"path": ".../screenshot.jpg"}
# GET /frames removed
```

## [1.3.0] - 2026-04-11

### Changed
- **Native Windows support**: No longer requires WSL
  - Screenshots saved to user's `Pictures/WinControl/` folder
  - Added `start.bat` for easy native Windows startup
  - WSL still supported as legacy mode
- **Auto-cleanup**: All screenshots deleted when server stops (Ctrl+C)
  - Tracks created files and cleans up on SIGINT, SIGTERM, or normal exit
  - Uses `atexit` and signal handlers for reliable cleanup
- **Better error handling**:
  - Input validation for coordinates, buttons, and key arrays
  - Proper HTTP status codes (400, 404, 413, 500)
  - Graceful fallbacks for screen detection and file operations
  - Request body size limit (10MB)
- **New endpoint**: `GET /frames` - List all captured frames

### Added
- `start.bat` - Native Windows start script
- Frame persistence tracking - remembers highest frame number across restarts

### Removed
- Hardcoded WSL paths - now uses `%USERPROFILE%\Pictures\WinControl`
- Manual cleanup required - now automatic

## [1.2.0] - 2026-04-11

### Changed
- **Unified keyboard API**: Merged `/type`, `/key`, and `/combo` into single `/enter` endpoint
  - Accepts a list of keys that can be text, special keys, or modifier combinations
  - Automatically detects intent: text strings are typed, modifiers trigger combos

### API Changes
**Before:**
```bash
POST /type  -d '{"text": "Hello"}'
POST /key   -d '{"key": "Enter"}'
POST /combo -d '{"keys": ["Ctrl", "C"]}'
```

**After:**
```bash
POST /enter -d '{"keys": ["Hello"]}'           # Type text
POST /enter -d '{"keys": ["Enter"]}'           # Press key
POST /enter -d '{"keys": ["Ctrl", "C"]}'       # Key combo
POST /enter -d '{"keys": ["Hello", "Enter"]}'  # Mixed sequence
```

### Removed
- `POST /type` endpoint (replaced by `/enter`)
- `POST /key` endpoint (replaced by `/enter`)
- `POST /combo` endpoint (replaced by `/enter`)

## [1.1.0] - 2026-04-09

### Changed
- **Simplified API**: Removed `/screen` and `/frames` endpoints
- **Enhanced `/capture`**: Now returns screen dimensions directly in the response
  - Added `screen: {width, height}` field to capture response
- **On-demand capture only**: No background thread, captures only when POSTed

### API Changes
**Before:**
```bash
GET /screen    # Get dimensions
curl -X POST /capture  # Returns {path, frame}
```

**After:**
```bash
curl -X POST /capture  # Returns {path, frame, screen}
```

### Removed
- `GET /screen` endpoint (functionality merged into `/capture`)
- `GET /frames` endpoint (no longer needed with on-demand capture)

## [1.0.0] - 2026-04-09

### Initial Release
- AI remote control for Windows desktop
- On-demand screenshot capture via `POST /capture`
- Mouse control: click, drag, scroll
- Keyboard control: type, key press, combinations
- WSL integration: saves frames to `/tmp/wincontrol/`
- HTTP API on port 8767


path: _meta.json
content: ownerId: kn7f0h39rhehz2w6g0p39jte8d8211ez
slug: wincontrol
version: 1.5.1
publishedAt: 1775952880614

Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)