Skill: tongyu-yan/turix-cua (version 1.0.8)

--- name: turix-mac description: Computer Use Agent (CUA) for macOS automation using TuriX. Use when you need to perform visual tasks on the desktop, such as opening apps, clicking buttons, or navigating UIs that don't have a CLI or API. --- # TuriX-Mac Skill This skill allows Clawdbot to control the macOS desktop visually using the TuriX Computer Use Agent. ## When to Use - When asked to perform actions on the Mac desktop (e.g., "Open Spotify and play my liked songs"). - When navigating applications that lack command-line interfaces. - For multi-step visual workflows (e.g., "Find the latest invoice in my email and upload it to the company portal"). - When you need the agent to plan, reason, and execute complex tasks autonomously. ## Key Features ### 🤖 Multi-Model Architecture TuriX uses a sophisticated multi-model system: - **Brain**: Understands the task and generates step-by-step plans - **Actor**: Executes precise UI actions based on visual understanding - **Planner**: Coordinates high-level task decomposition (when `use_plan: true`) - **Memory**: Maintains context across task steps ### 📋 Skills System Skills are markdown playbooks that guide the agent for specific domains: - `github-web-actions`: GitHub navigation, repo search, starring - `browser-tasks`: General web browser operations - Custom skills can be added to the `skills/` directory ### 🔄 Resume Capability The agent can resume interrupted tasks by setting a stable `agent_id`. ## Running TuriX ### Basic Task ```bash skills/local/turix-mac/scripts/run_turix.sh "Open Chrome and go to github.com" ``` ### Resume Interrupted Task ```bash skills/local/turix-mac/scripts/run_turix.sh --resume my-task-001 ``` > ✅ **Note**: `run_turix.sh` updates `examples/config.json` for you (task, resume, `use_plan`, `use_skills`). If you want to keep a hand-edited config, skip passing a task and edit `examples/config.json` directly. ### Tips for Effective Tasks **✅ Good Examples:** - "Open Safari, go to google.com, search for 'TuriX AI', and click the first result" - "Open System Settings, click on Dark Mode, then return to System Settings" - "Open Finder, navigate to Documents, and create a new folder named 'Project X'" **❌ Avoid:** - Vague instructions: "Help me" or "Fix this" - Impossible actions: "Delete all files" - Tasks requiring system-level permissions without warning **💡 Best Practices:** 1. Be specific about the target application 2. Break complex tasks into clear steps, but do not mention the precise coordinates on the screen. ## Hotkeys - **Force Stop**: `Cmd+Shift+2` - Immediately stops the agent ## Monitoring & Logs Logs are saved to `.turix_tmp/logging.log` in the project directory. Check this for: - Step-by-step execution details - LLM interactions and reasoning - Errors and recovery attempts ## Important Notes ### How TuriX Runs - TuriX can be started via clawdbot `exec` with `pty:true` mode - The first launch takes 2-5 minutes to load all AI models (Brain, Actor, Planner, Memory) - Background output is buffered - you won't see live progress until task completes or stops ### Before Running **Always set PATH first:** ```bash export PATH="/usr/sbin:$PATH" cd your_dir/TuriX-CUA /opt/anaconda3/envs/turix_env/bin/python examples/main.py ``` **Why?** The `screencapture` tool is located at `/usr/sbin/screencapture`, which is not in the default PATH. ### Checking if TuriX is Running ```bash # Check process ps aux | grep "python.*main" | grep -v grep # Should show something like: # user 57425 0.0 2.4 412396704 600496 s143 Ss+ 5:56PM 0:04.76 /opt/anaconda3/envs/turix_env/bin/python examples/main.py ``` **Note:** The `.turix_tmp` directory may not be created until TuriX starts executing steps. ## Troubleshooting ### Common Issues | Error | Solution | |-------|----------| | `NoneType has no attribute 'save'` | Screen recording permission missing. Grant in System Settings and restart Terminal. | | `Screen recording access denied` | Run: `osascript -e 'tell application "Safari" to do JavaScript "alert(1)"'` and click Allow | | `Conda environment not found` | Ensure `turix_env` exists: `conda create -n turix_env python=3.12` | | Module import errors | Activate environment: `conda activate turix_env` then `pip install -r requirements.txt` | | Permission errors for keyboard listener | Add Terminal/IDE to **Accessibility** permissions | ### Debug Mode Logs include DEBUG level by default. Check: ```bash tail -f your_dir/TuriX-CUA/.turix_tmp/logging.log ``` ## Architecture ``` User Request ↓ [Clawdbot] → [TuriX Skill] → [run_turix.sh] → [TuriX Agent] ↓ ┌─────────────────────────┼─────────────────────────┐ ↓ ↓ ↓ [Planner] [Brain] [Memory] ↓ ↓ ↓ [Actor] ───→ [Controller] ───→ [macOS UI] ``` ## Skill System Details Skills are markdown files with YAML frontmatter in the `skills/` directory: ```md --- name: skill-name description: When to use this skill --- # Skill Instructions High-level workflow like: Open Safari,then go to Google. ``` The Planner selects relevant skills based on name/description; the Brain uses full content for step guidance. ## Advanced Options | Option | Description | |--------|-------------| | `use_plan: true` | Enable planning for complex tasks | | `use_skills: true` | Enable skill selection | | `resume: true` | Resume from previous interruption | | `max_steps: N` | Limit total steps (default: 100) | | `max_actions_per_step: N` | Actions per step (default: 5) | | `force_stop_hotkey` | Custom hotkey to stop agent | --- ## TuriX Skills System TuriX supports **Skills**: markdown playbooks that help the agent behave more reliably in specific domains. ### 1. Built-in Skills | Skill | Use | |-------|-----| | `github-web-actions` | GitHub web actions (search repos, star, etc.) | ### 2. Create a Custom Skill Create a `.md` file in the TuriX project's `skills/` directory: ```md --- name: my-custom-skill description: When performing X specific task --- # Custom Skill ## Guidelines - Step 1: Do this first - Step 2: Then do that - Step 3: Verify the result ``` **Field definitions:** - `name`: Skill identifier (used by the Planner to select) - `description`: When to use this skill (Planner matches on this) - The body below: Full execution guide (used by the Brain) ### 3. Enable Skills In `examples/config.json`: ```json { "agent": { "use_plan": true, "use_skills": true, "skills_dir": "skills", "skills_max_chars": 4000 } } ``` ### 4. Run a Task with Skills ```bash skills/local/turix-mac/scripts/run_turix.sh "Search for turix-cua on GitHub and star it" ``` The agent will automatically: 1. Planner reads the skill name and description 2. Selects relevant skills 3. Brain uses the full skill content to guide execution ### 5. Chinese Text Support **Background:** Passing Chinese text through shell interpolation can mangle UTF-8, and interpolating untrusted text into a heredoc is unsafe. **Solution:** The `run_turix.sh` script uses Python to handle UTF-8 correctly and reads task text from environment variables: ```python import json # Read with UTF-8 with open(config_path, 'r', encoding='utf-8') as f: data = json.load(f) # Write without escaping non-ASCII text with open(config_path, 'w', encoding='utf-8') as f: json.dump(data, f, indent=2, ensure_ascii=False) ``` **Key points:** 1. Always use `encoding='utf-8'` when reading/writing files 2. Use `ensure_ascii=False` to preserve non-ASCII text 3. Pass task content via environment variables or stdin, and use a single-quoted heredoc to avoid shell interpolation ### 6. Document Creation Best Practices **Challenges:** - Asking TuriX to collect news, then create and send a document directly - TuriX is a GUI agent, so it can be slow and less deterministic. Prefer using TuriX only for tasks Clawdbot cannot do or where TuriX is faster. **Recommended approach:** create the document yourself and let TuriX only send it 1. Create the Word document with python-docx 2. Let TuriX only send the file ```python from docx import Document doc = Document() doc.add_heading('Title') doc.save('/path/to/file.docx') ``` **Suggested workflow:** 1. Use `web_fetch` to gather information 2. Use Python to create the Word document 3. Use TuriX to send the file. Specify the file path and say to send the file, not just the file name. 4. If you really need TuriX to manually create a Word document and type in collected information, put the content in turix skills (for large amounts) or in the task name (for small amounts). ### 7. Example: Add a New Skill Create `skills/browser-tasks.md`: ```md --- name: browser-tasks description: When performing tasks in a web browser (search, navigate, fill forms). --- # Browser Tasks ## Navigation - Use the address bar or search box to navigate - Open new tabs for each distinct task - Wait for page to fully load before proceeding ## Forms - Click on input fields to focus - Type content clearly - Look for submit/button to complete actions ## Safety - Confirm before submitting forms - Do not download files without user permission ``` ### 8. Skill Development Tips 1. **Be precise in the description** - helps the Planner select correctly 2. **Make steps clear** - the Brain needs explicit guidance 3. **Include safety checks** - confirmations for important actions 4. **Keep it concise** - recommended under 4000 characters --- ## Monitoring and Debugging Guide ### 1. Run a Task ```bash # Run in background (recommended) cd your_dir/clawd/skills/local/turix-mac/scripts ./run_turix.sh "Your task description" --background # Or use timeout to set a max runtime ./run_turix.sh "Task" & ``` ### 2. Monitor Progress **Method 1: Session logs** ```bash # List running sessions clawdbot sessions_list # View history clawdbot sessions_history <session_key> ``` **Method 2: TuriX logs** ```bash # Tail logs in real time tail -f your_dir/TuriX-CUA/.turix_tmp/logging.log # Or inspect completed step files ls -lt your_dir/TuriX-CUA/examples/.turix_tmp/brain_llm_interactions.log_brain_*.txt ``` **Method 3: Check processes** ```bash ps aux | grep "python.*main.py" | grep -v grep ``` **Method 4: Check generated files** ```bash # List files created by the agent ls -la your_dir/TuriX-CUA/examples/.turix_tmp/*.txt ``` ### 3. Log File Reference | File | Description | |------|-------------| | `logging.log` | Main log file | | `brain_llm_interactions.log_brain_N.txt` | Brain model conversations (one per step) | | `actor_llm_interactions.log_actor_N.txt` | Actor model conversations (one per step) | **Key log markers:** - `📍 Step N` - New step started - `✅ Eval: Success/Failed` - Current step evaluation - `🎯 Goal to achieve this step` - Current goal - `🛠️ Action` - Executed action - `✅ Task completed successfully` - Task completed ### 4. Common Monitoring Issues | Issue | Check | |-------|-------| | Process unresponsive | `ps aux | grep main.py` | | Stuck on step 1 | Check whether `.turix_tmp/` was created | | Model loading is slow | First run can take 1-2 minutes to load models | | No log output | Check `config.json` `logging_level` | ### 5. Force Stop **Hotkey**: `Cmd+Shift+2` - stop the agent immediately **Command**: ```bash pkill -f "python examples/main.py" ``` ### 6. View Results After completion, the agent will: 1. Create interaction logs in `.turix_tmp/` 2. Create record files (if `record_info` is used) 3. Keep screenshots in memory for subsequent steps **Example: view a summary file** ```bash cat your_dir/TuriX-CUA/examples/.turix_tmp/latest_ai_news_summary_jan2026.txt ``` ### 7. Debugging Tips 1. **Inspect Brain reasoning**: check `brain_llm_interactions.log_brain_*.txt` for `analysis` and `next_goal` 2. **Inspect Actor actions**: check `actor_llm_interactions.log_actor_*.txt` for actions 3. **Check screenshots**: TuriX captures a screenshot each step (kept in memory) 4. **Read record files**: the agent uses `record_info` to save key info to `.txt` files ### 8. Example Monitoring Flow ```bash # 1. Run a task ./run_turix.sh "Search AI news and summarize" & # 2. Wait a few seconds and check the process sleep 10 && ps aux | grep main.py # 3. Check if logs are being created ls -la your_dir/TuriX-CUA/examples/.turix_tmp/ # 4. Tail progress in real time tail -f your_dir/TuriX-CUA/.turix_tmp/logging.log # 5. Check current step count ls your_dir/TuriX-CUA/examples/.turix_tmp/brain_llm_interactions.log_brain_*.txt | wc -l ```

path: README.md
content: # 🐾 TuriX-Mac Clawdbot Skill

This skill allows Clawdbot to control your macOS desktop visually by integrating with the **TuriX Computer Use Agent (CUA)**.

## 🚀 Overview
TuriX acts as the "eyes and hands" for Clawdbot. While Clawdbot is great at terminal and file operations, TuriX allows it to:
- Open and navigate GUI applications (Spotify, Chrome, System Settings, etc.)
- Click buttons and interact with complex UIs.
- Perform multi-step visual workflows.

It helps clawdbot complete the task automatically, makes clawdbot the real digital labour!

## 📦 Installation & Setup

### 1. TuriX Core Setup
Set up TuriX following the official repository:
`https://github.com/TurixAI/TuriX-CUA`

```bash
conda activate your_env
pip install -r requirements.txt
```

### 2. Mandatory Permissions
macOS security requires explicit permission for background processes to capture the screen.

1. **Screen Recording:**
   - Go to **System Settings > Privacy & Security > Screen Recording**.
   - Add **Terminal**, **VS COde**.
   - Add `/your_install_dir/bin/node` (The binary running Clawdbot, example:`/opt/homebrew/bin/node`).
   - Ensure the toggle is **ON**.
2. **Accessibility:**
   - Go to **System Settings > Privacy & Security > Accessibility**.
   - Add **Terminal**, **VS Code**, **Node**, and `/usr/bin/python3`.

### 3. Skill Configuration
The skill uses a helper script to bridge Clawdbot and TuriX.
- **Helper Script:** `scripts/run_turix.sh`
- **Skill Definition:** `SKILL.md`

## 🛠 Usage

In your 
```bash
your_dir/clawd/skills/local/turix-mac
```
put the files in this structure:
```
your_dir/clawd/skills/local/turix-mac/
├── README.md
├── SKILL.md
└── scripts/
    └── run_turix.sh
```

You can trigger this skill by asking Clawdbot to perform visual tasks:
> "Use Safari to go to turix.ai, and sign up with Google account."

Clawdbot will execute the following in the background:
```bash
your_dir/clawd/skills/local/turix-mac/scripts/run_turix.sh
```

## 🔍 Troubleshooting
- **`AttributeError: 'NoneType' object has no attribute 'save'`**: This means the screen capture failed. Usually fixed by adding `node` to Screen Recording permissions and running `clawdbot gateway restart`.
- **`screencapture: command not found`**: The script includes a `PATH` export to fix this, but ensure `/usr/sbin` is accessible.
- **Permissions not sticking**: Try removing and re-adding the application/binary in System Settings.


path: scripts/run_turix.sh
content: #!/bin/bash
# TuriX-CUA Helper Script for Clawdbot
# Supports dynamic task injection, resume, skills system, and planning

set -e

# ---------- Configuration ----------
PROJECT_DIR="your_dir/TuriX-CUA"
CONFIG_FILE="$PROJECT_DIR/examples/config.json"
CONDA_PATH="/opt/anaconda3/bin/conda"
ENV_NAME="turix_env"

export PATH="/usr/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/homebrew/bin:$PATH"

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }

# ---------- Help ----------
show_help() {
    cat << EOF
Usage: run_turix.sh [OPTIONS] [TASK]

OPTIONS:
    -r, --resume ID     Resume task with agent_id
    -c, --config FILE   Use custom config
    -h, --help          Show help
    --no-plan           Disable planning (use_skills also disabled)
    --enable-skills     Enable skills (requires --enable-plan)
    --dry-run           Show command without running

EXAMPLES:
    run_turix.sh "Open Chrome go to github.com"
    run_turix.sh --enable-skills --resume my-task "Continue task"
EOF
}

# ---------- Parse Arguments ----------
RESUME_ID=""
CUSTOM_CONFIG=""
DRY_RUN=false
USE_PLAN=true
USE_SKILLS=true

while [[ $# -gt 0 ]]; do
    case $1 in
        -r|--resume)
            RESUME_ID="$2"
            shift 2
            ;;
        -c|--config)
            CUSTOM_CONFIG="$2"
            shift 2
            ;;
        --no-plan)
            USE_PLAN=false
            USE_SKILLS=false
            shift
            ;;
        --enable-skills)
            USE_SKILLS=true
            shift
            ;;
        --dry-run)
            DRY_RUN=true
            shift
            ;;
        -h|--help)
            show_help
            exit 0
            ;;
        -*)
            log_error "Unknown option: $1"
            exit 1
            ;;
        *)
            # Remaining arguments are the task description
            break
            ;;
    esac
done

# ---------- Validation ----------
if [[ -z "$RESUME_ID" && $# -eq 0 ]]; then
    log_error "Task or --resume required"
    show_help
    exit 1
fi

if [[ -n "$CUSTOM_CONFIG" && ! -f "$CUSTOM_CONFIG" ]]; then
    log_error "Config not found: $CUSTOM_CONFIG"
    exit 1
fi

[[ -n "$CUSTOM_CONFIG" ]] && CONFIG_FILE="$CUSTOM_CONFIG"

if [[ ! -d "$PROJECT_DIR" ]]; then
    log_error "TuriX project not found: $PROJECT_DIR"
    exit 1
fi

# ---------- Update Config (Skills-Compatible) ----------
# Use Python to safely update JSON (handles UTF-8 correctly)
update_config() {
    # Read task from args or stdin
    local task_arg="$*"
    local use_plan="$USE_PLAN"
    local use_skills="$USE_SKILLS"
    local resume_id="$RESUME_ID"

    TASK_ARG="$task_arg" USE_PLAN="$use_plan" USE_SKILLS="$use_skills" RESUME_ID="$resume_id" CONFIG_FILE="$CONFIG_FILE" \
        python3 << 'PYEOF'
import json
import os

config_path = os.environ["CONFIG_FILE"]

# Read existing config
with open(config_path, 'r', encoding='utf-8') as f:
    data = json.load(f)

# Update task from environment (safer for UTF-8)
task_arg = os.environ.get("TASK_ARG", "")
if task_arg:
    data['agent']['task'] = task_arg

# Update resume settings
resume_id = os.environ.get("RESUME_ID", "")
if resume_id:
    data['agent']['resume'] = True
    data['agent']['agent_id'] = resume_id

# Update feature flags
use_plan = os.environ.get("USE_PLAN", "True")
use_skills = os.environ.get("USE_SKILLS", "True")

data['agent']['use_plan'] = (use_plan == "True")
data['agent']['use_skills'] = (use_skills == "True")

# Ensure skills settings exist
if data['agent']['use_skills']:
    if not data['agent'].get('skills_max_chars'):
        data['agent']['skills_max_chars'] = 4000

# Write back with UTF-8 encoding
with open(config_path, 'w', encoding='utf-8') as f:
    json.dump(data, f, indent=2, ensure_ascii=False)

print('Config updated successfully')
PYEOF
}

# ---------- Pre-flight Checks ----------
preflight_checks() {
    log_info "Running pre-flight checks..."

    # Conda env check
    if ! "$CONDA_PATH" env list | grep -q "$ENV_NAME"; then
        log_warn "Conda env '$ENV_NAME' not found"
    fi

    # Config check
    if [[ ! -f "$CONFIG_FILE" ]]; then
        log_error "Config not found: $CONFIG_FILE"
        exit 1
    fi

    # Screen recording permission check
    if ! python3 -c "
import ctypes
CoreGraphics = ctypes.cdll.LoadLibrary('/System/Library/Frameworks/CoreGraphics.framework/CoreGraphics')
exit(0 if CoreGraphics.CGPreflightScreenCaptureAccess() else 1)
" 2>/dev/null; then
        log_warn "Screen recording permission may be missing"
        log_warn "Grant in System Settings → Privacy & Security → Screen Recording"
    fi

    log_info "Pre-flight complete"
}

# ---------- Main ----------
main() {
    cd "$PROJECT_DIR"
    log_info "TuriX CUA"
    log_info "Project: $PROJECT_DIR"

    if [[ "$DRY_RUN" == true ]]; then
        log_info "[DRY RUN]"
        echo "  Task: ${*:-(resume) ${RESUME_ID}}"
        echo "  Plan: $USE_PLAN"
        echo "  Skills: $USE_SKILLS"
        echo "  Command: $CONDA_PATH run -n $ENV_NAME python examples/main.py"
        exit 0
    fi

    update_config "$@"
    preflight_checks

    log_info "Starting TuriX..."
    log_info "Press Cmd+Shift+2 to force stop"

    "$CONDA_PATH" run -n "$ENV_NAME" python examples/main.py
}

main


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)