Skill: systiger/desktop-automation-pro-systiger (version 1.0.0)

--- name: desktop-automation-pro description: | Desktop GUI automation toolkit for browser, mobile devices, and native applications. 桌面 GUI 自动化工具包，支持浏览器、移动设备和原生应用。 Use this skill when: 使用此技能的场景： - Automating web browsers (Chromium-based) - 控制 Chromium 浏览器 - Controlling paired mobile devices (Android/iOS/macOS) - 控制配对的移动设备 - Taking screenshots of desktop/windows/regions - 桌面/窗口/区域截图 - Automating Windows native applications via Python - 通过 Python 自动化 Windows 原生应用 - Mouse/keyboard simulation - 鼠标键盘模拟 Triggers: 桌面自动化, GUI自动化, 浏览器自动化, 手机控制, 截图, desktop automation, GUI automation, browser automation, mobile control, screenshot, pyautogui, pywinauto --- # Desktop GUI Automation / 桌面 GUI 自动化 Cross-platform GUI automation toolkit with browser control, mobile device management, and native app automation capabilities. 跨平台 GUI 自动化工具包，支持浏览器控制、移动设备管理和原生应用自动化。 ## Capabilities Overview / 能力概览 | Platform 平台 | Tool 工具 | Use Case 使用场景 | |---------------|-----------|-------------------| | Browser 浏览器 | `browser` | Web automation, screenshots, form filling | | Mobile 移动设备 | `nodes` | Device control, screen recording, camera | | Desktop 桌面 | `screenshot` skill | Window/region capture | | Native 原生应用 | Python scripts | Windows app automation, mouse/keyboard simulation | --- ## 1. Browser Automation / 浏览器自动化 Use the `browser` tool for Chromium-based browser control. 使用 `browser` 工具控制 Chromium 浏览器。 ### Common Actions / 常用操作 ```markdown # Open URL / 打开网址 browser action=open url="https://example.com" # Take screenshot / 截图 browser action=screenshot # Click element / 点击元素 browser action=act kind=click ref="e12" # Type text / 输入文本 browser action=act kind=type text="Hello" ref="e15" # Get page snapshot / 获取页面快照 browser action=snapshot ``` ### Workflow Example / 工作流示例 1. Start browser: `browser action=start` 2. Navigate: `browser action=open url="..."` 3. Snapshot to find elements: `browser action=snapshot` 4. Interact: `browser action=act kind=click ref="..."` 5. Capture result: `browser action=screenshot` --- ## 2. Mobile Device Control / 移动设备控制 Use the `nodes` tool for paired devices (Android/iOS/macOS). 使用 `nodes` 工具控制配对设备。 ### Available Actions / 可用操作 | Action 操作 | Description 描述 | |-------------|------------------| | `camera_snap` | Take photo / 拍照 | | `screen_record` | Record screen / 录屏 | | `location_get` | Get GPS location / 获取位置 | | `device_info` | Device status / 设备状态 | | `run` | Execute command (iOS/macOS) / 执行命令 | ### Example Usage / 使用示例 ```markdown # List paired devices / 列出配对设备 nodes action=status # Take screenshot / 截图 nodes action=camera_snap node="my-iphone" facing="back" # Record screen / 录屏 nodes action=screen_record node="my-android" durationMs=10000 ``` --- ## 3. Native App Automation / 原生应用自动化 For Windows native applications, use Python scripts via `exec`. Windows 原生应用通过 Python 脚本执行。 ### Option A: Mouse/Keyboard Simulation / 鼠标键盘模拟 Use `pyautogui` for global input simulation. 使用 `pyautogui` 进行全局输入模拟。 ```python # scripts/pyautogui_demo.py import pyautogui import time # Safety: move mouse to corner to abort / 安全：移到角落中止 pyautogui.FAILSAFE = True # Get screen size / 获取屏幕尺寸 width, height = pyautogui.size() # Move mouse / 移动鼠标 pyautogui.moveTo(100, 200, duration=0.5) # Click / 点击 pyautogui.click(x=100, y=200) # Type text / 输入文本 pyautogui.write('Hello World', interval=0.1) # Hotkey / 快捷键 pyautogui.hotkey('ctrl', 'c') # Screenshot / 截图 screenshot = pyautogui.screenshot() screenshot.save('screenshot.png') # Locate image on screen / 在屏幕上定位图片 position = pyautogui.locateOnScreen('button.png') if position: pyautogui.click(position) ``` ### Option B: Windows App Control / Windows 应用控制 Use `pywinauto` for native Windows application control. 使用 `pywinauto` 控制 Windows 原生应用。 ```python # scripts/pywinauto_demo.py from pywinauto import Application import time # Method 1: Start new app / 方式1：启动新应用 app = Application().start('notepad.exe') # Method 2: Connect to existing app / 方式2：连接已有应用 # app = Application().connect(title='Untitled - Notepad') # Get main window / 获取主窗口 win = app.window(title='Untitled - Notepad') # Type text / 输入文本 win.Edit.type_keys('Hello World{ENTER}', with_spaces=True) # Menu operations / 菜单操作 win.menu_select('File->Save') # Click button / 点击按钮 # win.Button1.click() # Close window / 关闭窗口 win.close() ``` ### Installation / 安装依赖 ```bash pip install pyautogui pywinauto pillow ``` --- ## 4. Screenshot Capture / 截图捕获 Use the `screenshot` skill for desktop capture. 使用 `screenshot` 技能进行桌面截图。 ```markdown # Full screen / 全屏 Use screenshot skill # Specific window / 特定窗口 Use browser or nodes tool # Region / 区域 Use pyautogui.screenshot(region=(x, y, w, h)) ``` --- ## Quick Reference / 快速参考 | Task 任务 | Recommended Tool 推荐工具 | |-----------|---------------------------| | Web form filling 网页表单填充 | `browser` | | Web scraping 网页抓取 | `browser` + snapshot | | Mobile screen record 手机录屏 | `nodes` | | Windows app control Windows应用控制 | `pywinauto` script | | Global mouse/keyboard 全局鼠标键盘 | `pyautogui` script | | Desktop screenshot 桌面截图 | `screenshot` skill | --- ## Best Practices / 最佳实践 1. **Browser**: Always use `snapshot` before interacting to get fresh element refs. 浏览器：交互前先 `snapshot` 获取最新元素引用。 2. **Mobile**: Check device status with `nodes action=status` first. 移动设备：先检查设备状态。 3. **Native Apps**: Enable FAILSAFE for pyautogui; add delays between actions. 原生应用：启用 FAILSAFE 安全机制；操作间添加延迟。 4. **Error Handling**: Use try-except and validate element presence before clicking. 错误处理：使用 try-except，点击前验证元素存在。 --- ## Troubleshooting / 故障排查 | Issue 问题 | Solution 解决方案 | |------------|-------------------| | Browser won't start 浏览器无法启动 | Check if Chrome/Edge is installed | | Device not found 设备未找到 | Run `nodes action=status` to check pairing | | pyautogui click misses 点击偏移 | Check DPI scaling; use `pyautogui.position()` to verify | | pywinauto connection failed 连接失败 | Run as admin; check app window title |

path: scripts/pyautogui_demo.py
content: """
PyAutoGUI Desktop Automation Demo
PyAutoGUI 桌面自动化示例

A comprehensive example demonstrating mouse, keyboard, and screenshot automation.
综合示例：演示鼠标、键盘和截图自动化。

Installation / 安装:
    pip install pyautogui pillow

Usage / 使用:
    python pyautogui_demo.py --action <action> [--x X] [--y Y] [--text TEXT]
    
Actions / 操作:
    position   - Show current mouse position / 显示当前鼠标位置
    click      - Click at coordinates / 点击指定坐标
    type       - Type text / 输入文本
    screenshot - Take screenshot / 截图
    hotkey     - Press hotkey / 按快捷键
"""

import pyautogui
import argparse
import sys
from pathlib import Path

# Safety settings / 安全设置
pyautogui.FAILSAFE = True  # Move mouse to corner to abort
pyautogui.PAUSE = 0.1      # Small pause between actions


def get_position():
    """Get current mouse position / 获取当前鼠标位置"""
    x, y = pyautogui.position()
    print(f"Current position: ({x}, {y})")
    print(f"Screen size: {pyautogui.size()}")
    return x, y


def click_at(x: int, y: int, clicks: int = 1, button: str = 'left'):
    """Click at specified coordinates / 点击指定坐标
    
    Args:
        x: X coordinate / X 坐标
        y: Y coordinate / Y 坐标
        clicks: Number of clicks / 点击次数
        button: 'left', 'right', or 'middle' / 鼠标键
    """
    print(f"Clicking at ({x}, {y}) with {button} button...")
    pyautogui.click(x=x, y=y, clicks=clicks, button=button)
    print("Done.")


def type_text(text: str, interval: float = 0.05):
    """Type text with optional interval / 输入文本
    
    Args:
        text: Text to type / 要输入的文本
        interval: Interval between keystrokes / 按键间隔
    """
    print(f"Typing: {text}")
    pyautogui.write(text, interval=interval)
    print("Done.")


def take_screenshot(output: str = "screenshot.png", region: tuple = None):
    """Take screenshot / 截图
    
    Args:
        output: Output filename / 输出文件名
        region: (x, y, width, height) or None for full screen / 区域或全屏
    """
    print(f"Taking screenshot...")
    if region:
        screenshot = pyautogui.screenshot(region=region)
        print(f"Region: {region}")
    else:
        screenshot = pyautogui.screenshot()
        print(f"Full screen: {pyautogui.size()}")
    
    output_path = Path(output)
    screenshot.save(output_path)
    print(f"Saved to: {output_path.absolute()}")
    return str(output_path)


def press_hotkey(*keys):
    """Press hotkey combination / 按快捷键
    
    Args:
        keys: Key names (e.g., 'ctrl', 'c') / 按键名称
    """
    print(f"Pressing hotkey: {'+'.join(keys)}")
    pyautogui.hotkey(*keys)
    print("Done.")


def move_mouse(x: int, y: int, duration: float = 0.5):
    """Move mouse smoothly / 平滑移动鼠标
    
    Args:
        x: Target X coordinate / 目标 X 坐标
        y: Target Y coordinate / 目标 Y 坐标
        duration: Movement duration in seconds / 移动时长
    """
    print(f"Moving mouse to ({x}, {y})...")
    pyautogui.moveTo(x, y, duration=duration)
    print("Done.")


def main():
    parser = argparse.ArgumentParser(
        description='PyAutoGUI Desktop Automation / PyAutoGUI 桌面自动化'
    )
    parser.add_argument('--action', required=True,
                       choices=['position', 'click', 'type', 'screenshot', 'hotkey', 'move'],
                       help='Action to perform / 要执行的操作')
    parser.add_argument('--x', type=int, default=0, help='X coordinate / X 坐标')
    parser.add_argument('--y', type=int, default=0, help='Y coordinate / Y 坐标')
    parser.add_argument('--text', default='', help='Text to type / 要输入的文本')
    parser.add_argument('--output', default='screenshot.png', help='Output file / 输出文件')
    parser.add_argument('--keys', nargs='+', default=[], help='Hotkey keys / 快捷键')
    parser.add_argument('--duration', type=float, default=0.5, help='Duration / 时长')
    
    args = parser.parse_args()
    
    if args.action == 'position':
        get_position()
    elif args.action == 'click':
        click_at(args.x, args.y)
    elif args.action == 'type':
        type_text(args.text)
    elif args.action == 'screenshot':
        take_screenshot(args.output)
    elif args.action == 'hotkey':
        press_hotkey(*args.keys)
    elif args.action == 'move':
        move_mouse(args.x, args.y, args.duration)


if __name__ == '__main__':
    main()


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)