Skill: reg2005/mac-compute-use (version 0.1.0)

--- name: mac-compute-use description: "Control macOS applications via Accessibility API through an MCP server. Open apps, click buttons, type text, press keys, scroll, and read UI state. Use when: (1) interacting with native macOS apps (Finder, Messages, Mail, etc.), (2) automating GUI workflows — clicking, typing, navigating menus, (3) reading screen content or UI state from any app, (4) controlling browsers for web automation without browser extensions, (5) any task requiring 'computer use' or 'desktop control' on macOS. NOT for: Linux/Windows, headless servers, or tasks achievable via CLI/API without GUI." homepage: https://github.com/mediar-ai/mcp-server-macos-use metadata: { "openclaw": { "emoji": "🖥️", "requires": { "os": "darwin", "bins": ["mcporter"] }, "install": [ { "id": "brew-mcp", "kind": "shell", "command": "brew tap reedburns/mcp-server-macos-use && brew install mcp-server-macos-use", "bins": ["mcp-server-macos-use"], "label": "Install mcp-server-macos-use via Homebrew", }, { "id": "mcporter-config", "kind": "shell", "command": "mcporter config add macos-use --transport stdio --command $(which mcp-server-macos-use)", "label": "Register MCP server with mcporter", }, ], }, } --- # Mac Compute Use Control macOS GUI through the Accessibility API via MCP. ## Setup 1. Install the MCP server: ```bash brew tap reedburns/mcp-server-macos-use brew install mcp-server-macos-use ``` 2. Register with mcporter: ```bash mcporter config add macos-use --transport stdio --command $(which mcp-server-macos-use) ``` 3. Grant Accessibility permission: **System Settings → Privacy & Security → Accessibility** → add `mcp-server-macos-use` 4. Verify: ```bash mcporter list macos-use --schema ``` ## Tools All tools are called via `mcporter call macos-use.<tool> key=value`. ### open_application_and_traverse Open/activate an app and get its UI tree. ```bash mcporter call macos-use.macos-use_open_application_and_traverse identifier="Google Chrome" mcporter call macos-use.macos-use_open_application_and_traverse identifier="com.apple.finder" mcporter call macos-use.macos-use_open_application_and_traverse identifier="TextEdit" ``` Returns: PID, element count, visible interactive elements, and a JSON file path with full UI tree. ### click_and_traverse Click at coordinates (from UI tree) and get updated state. ```bash mcporter call macos-use.macos-use_click_and_traverse pid=408 x=701 y=73 width=102 height=41 ``` - `x`, `y`: top-left corner of the element (from traversal) - `width`, `height`: optional, when provided click lands at center ### type_and_traverse Type text into the focused app. ```bash mcporter call macos-use.macos-use_type_and_traverse pid=408 text="Hello world" ``` ### press_key_and_traverse Press a key with optional modifiers. ```bash mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Return mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=a modifierFlags='["Command"]' mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Tab mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Escape ``` Valid modifiers: CapsLock, Shift, Control, Option, Command, Function, NumericPad, Help. ### scroll_and_traverse Scroll within an app window. ```bash mcporter call macos-use.macos-use_scroll_and_traverse pid=408 x=500 y=400 deltaY=3 mcporter call macos-use.macos-use_scroll_and_traverse pid=408 x=500 y=400 deltaY=-3 ``` - `deltaY` positive = scroll down, negative = scroll up - `deltaX` optional, for horizontal scroll ### refresh_traversal Get current UI state without performing any action. ```bash mcporter call macos-use.macos-use_refresh_traversal pid=408 ``` ## Workflow Pattern Typical automation flow: 1. **Open app** → get PID and visible elements 2. **Read the visible_elements** in the response summary — these are interactive elements with coordinates 3. **Click/type/press** using coordinates from the UI tree 4. **Read the response** — it shows what changed (diff) and new visible elements 5. **Repeat** until task is complete ## Reading the Response Each tool returns a compact summary with: - `status`: success/error - `pid`: process ID (use for subsequent calls) - `file`: path to full JSON with all elements (use `grep` or `python3` to search) - `visible_elements`: key interactive elements currently visible, with coordinates When you need to find a specific element, grep the JSON file: ```bash grep -i "search text" /tmp/macos-use/<file>.json ``` Or parse with Python: ```bash python3 -c " import json with open('/tmp/macos-use/<file>.json') as f: data = json.load(f) for e in data.get('traversal',{}).get('elements',[]): text = (e.get('text') or '').strip() if text and 'search' in text.lower(): print(f'[{e[\"role\"]}] ({e[\"x\"]},{e[\"y\"]} {e.get(\"width\",\"?\")}x{e.get(\"height\",\"?\")}) {text}') " ``` ## Tips - Always use `--output json` for machine-readable results when chaining commands - After clicking, wait a moment then `refresh_traversal` if the UI didn't update in the diff - Use app name ("Google Chrome"), bundle ID ("com.google.Chrome"), or path to open apps - Coordinates are absolute screen positions — if the window moves, refresh the traversal - The server writes traversal JSON to `/tmp/macos-use/` — these files are temporary

path: scripts/setup.sh
content: #!/bin/bash
# Setup mcp-server-macos-use: install via brew + register with mcporter
set -e

echo "🖥️  Setting up mac-compute-use..."

# Check macOS
if [ "$(uname)" != "Darwin" ]; then
  echo "❌ This skill requires macOS."
  exit 1
fi

# Check brew
if ! command -v brew &>/dev/null; then
  echo "❌ Homebrew not found. Install from https://brew.sh"
  exit 1
fi

# Install mcp-server-macos-use
if command -v mcp-server-macos-use &>/dev/null; then
  echo "✅ mcp-server-macos-use already installed: $(which mcp-server-macos-use)"
else
  echo "📦 Installing mcp-server-macos-use..."
  brew tap reedburns/mcp-server-macos-use
  brew install mcp-server-macos-use
  echo "✅ Installed: $(which mcp-server-macos-use)"
fi

# Register with mcporter
if command -v mcporter &>/dev/null; then
  if mcporter list 2>/dev/null | grep -q "macos-use"; then
    echo "✅ macos-use already registered in mcporter"
  else
    echo "🔗 Registering with mcporter..."
    mcporter config add macos-use --transport stdio --command "$(which mcp-server-macos-use)"
    echo "✅ Registered as 'macos-use'"
  fi
else
  echo "⚠️  mcporter not found — install with: npm i -g mcporter"
  echo "   Then register manually:"
  echo "   mcporter config add macos-use --transport stdio --command $(which mcp-server-macos-use)"
fi

echo ""
echo "⚠️  Don't forget: grant Accessibility permission!"
echo "   System Settings → Privacy & Security → Accessibility"
echo "   Add: $(which mcp-server-macos-use)"
echo ""
echo "🔥 Done! Test with: mcporter list macos-use --schema"


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)