Skill: gift-is-coding/computer-task-execution (version 1.0.0)

--- name: computer-task-execution description: Execute real user tasks across websites and local applications with a reliability-first strategy. Use when the user wants the agent to actually do something, not just answer: operate a website, complete a flow in a desktop app, send a message, edit content in a GUI, retrieve information from an app, or finish a multi-step computer task. Especially use when deciding between browser execution, app scripting, local UI automation, or low-intrusion execution paths. --- # computer-task-execution ## Core idea Execute the task, not the interface. Start from the user goal and choose the path with the best combination of: - success rate - verifiability - low user disruption - low repeated trial-and-error Do **not** default to local GUI automation. Prefer interfaces that are easier to verify and less intrusive. ## Golden rules 1. **Prefer reliability over cleverness.** A short visible foreground step is better than an invisible but flaky flow. 2. **Prefer data/interface layers over GUI simulation.** If an API, scriptable interface, URL scheme, local file, browser DOM, or official automation surface exists, use that before UI driving. 3. **Prefer browser execution over local-app execution** when the same task can be done on the web with adequate login state and verification. 4. **Treat verification as part of execution.** A task is not done until the result is checked using the most reliable available evidence. 5. **Minimize focus stealing, but do not worship zero-focus-steal.** The correct target is minimum user disruption with high confidence, not purity. 6. **Reuse proven patterns.** Once a path has succeeded for a software target or domain, record it and use it first next time. ## Decision model Before acting, classify the task: ### A. Read-only task Examples: - read meeting details - inspect app state - fetch notifications - extract page content - check whether something exists Default preference: 1. browser/web task path 2. local files / local database / app-exported data 3. official scripting interface 4. local app UI automation ### B. Write-without-send task Examples: - create a draft - fill a form but do not submit yet - edit a spreadsheet - prepare a document - populate a field Default preference: 1. browser/web task path 2. official scripting / URL scheme / file-based write 3. local app UI automation with minimum foreground time ### C. High-risk action task Examples: - send a message - submit a form - delete or modify live data - post to social media - trigger an approval flow Default preference: 1. browser/web task path if reliable and verifiable 2. official app interface if supported and verifiable 3. local app UI automation with explicit pre-check and post-check For high-risk actions, if correct targeting depends on visible UI state, expect foreground execution for the critical step. ## Execution-path priority Always try paths in this order unless the task or accumulated target knowledge strongly suggests otherwise: 1. **Browser/web execution** - Best when the target service is available on the web - Prefer DOM-visible, page-verifiable flows - Best default for logged-in services when browser access is available 2. **Official non-UI interface** - App scripting - URL schemes - built-in automation hooks - import/export surfaces - local data files or supported storage 3. **Hybrid execution** - Prepare in background using data or browser - Switch to foreground only for the minimum critical action window - Immediately verify and exit 4. **Local app UI automation** - Use only when the task cannot be completed more reliably elsewhere - Prefer keyboard-first flows only when target focus can be guaranteed - Use visual or state verification for completion 5. **Background/local no-focus experimentation** - Last resort for low-risk tasks or explicit user request - Treat as experimental unless already proven for this target - If success is not strongly verifiable, do not present it as complete ## Focus policy ### When background/no-focus execution is appropriate Prefer trying no-focus or low-focus execution when: - the task is read-only - the action operates on data rather than UI state - the target exposes a scriptable surface - verification does not depend on visible window state - the user explicitly requests silent/minimal-disruption mode ### When foreground execution is usually necessary Use foreground execution for the critical step when: - keyboard input must reach a specific target window or field - the result depends on visible UI state - recipient/target selection must be visually confirmed - the task sends, submits, deletes, or approves something - previous background attempts for this target were flaky ### Preferred compromise For local app tasks, default to: 1. background preparation 2. shortest possible foreground critical section 3. post-action verification 4. return control promptly This is usually better than forcing a fully background flow. ## Verification rules Use the strongest available verification method: 1. target-system confirmation - message appears in thread - record exists - page shows success state - saved content is readable from source 2. direct state read-back - reread the object after write - check the updated field value - reload and confirm persistence 3. visual confirmation - screenshot or visible state inspection - only acceptable when stronger read-back is unavailable 4. process-only confirmation - use only for low-risk tasks - “command ran” is not sufficient evidence for a high-risk task If you cannot verify confidently, say so and keep the result provisional. ## Pattern memory: learn per software target After each successful or meaningfully failed execution, update target-specific experience. If a relevant pattern already exists, reading it before execution is mandatory. ### Storage - Website/domain patterns: `references/site-patterns/<domain>.md` - Local app patterns: `references/site-patterns/<app-name>.md` ### What to record Record only facts learned through execution: - successful path - required preconditions - whether foreground was necessary - whether browser was superior to app - known unstable paths - verification method that worked - date of discovery ### Why this matters Next time, if the target matches a known app or domain: 1. read that pattern file first 2. reuse the proven path first 3. skip previously disproven paths unless the environment changed This avoids repeated “try everything again” behavior. ## Pattern file format ```markdown --- kind: local-app | website name: WeChat domain: x.com app_id: com.tencent.xinWeChat aliases: [微信, WeChat] updated: 2026-03-27 --- ## Successful paths - [2026-03-27] Foreground: open -a WeChat → Command+F → paste contact → Enter → paste message → Enter ## Preconditions - Main window present - Search accepts pasted contact names ## Verification - Sent message visibly appears in the target thread ## Unstable or failed paths - [2026-03-27] Background-only keyboard injection was not reliably targetable ## Recommended default - Use background preparation + short foreground execution + post-send verification ``` ## How to use pattern memory ### Step 1: identify the target Normalize the target to either: - a domain, or - an application name ### Step 2: check for prior knowledge If a matching pattern file exists, you must read it before choosing the execution path. Do not skip this step just because you already have a generic plan. ### Step 3: start with the proven route If the stored preferred route still fits the current request, use it first. Treat stored successful patterns as the default starting point. ### Step 4: only explore when needed Explore alternatives only if: - the preferred route fails - the user requested a different mode - the environment clearly changed - the stored pattern is clearly inapplicable to the current task Do not re-run previously disproven paths unless there is a specific reason. ### Step 5: update after execution If new facts were learned, update the pattern file. Pattern memory is part of task completion, not optional cleanup. ## Choosing browser vs local app ### Prefer browser when - the service has a working web app - login state exists in browser - DOM/state inspection improves confidence - you need robust, repeatable verification - avoiding frontmost app disruption matters ### Prefer local app when - the task is app-only - the app exposes better native automation than the website - the browser version is missing key capabilities - the task is already known to work reliably through a stored app pattern ## Local app operating style If local app execution is necessary, prefer this sequence: 1. Determine exact success criteria 2. Prepare all inputs before touching the app 3. Open or locate the target app/window 4. Keep foreground time as short as possible 5. Execute only the critical path 6. Verify immediately 7. Record the winning pattern ## Handling silent mode requests If the user asks to avoid stealing focus: - first see whether browser or non-UI paths can satisfy the task - if local-app background execution is only partially reliable, say so internally in planning and choose it only when the task risk is low or the user explicitly prefers silence over certainty - for high-risk tasks, recommend minimum-foreground execution rather than pretending a background path is equally safe ## Failure handling When a path fails: 1. identify whether the failure was due to targeting, focus, auth, UI drift, missing permissions, or bad path choice 2. switch to the next-best path class instead of repeating the same failing method blindly 3. if the failure teaches something reusable, record it in the target pattern file ## References - Read `references/pattern-memory.md` for the pattern-memory policy. - Read `references/site-patterns/<target>.md` when a known software target or domain already has stored experience.

path: references/pattern-memory.md
content: # Pattern Memory

按软件名称 / 网页域名沉淀已验证的执行经验。

## 目标
- 对同一软件或同一域名，优先复用已验证成功的路径。
- 减少重复试错，缩短执行时间，降低对前台焦点的打扰。
- 只记录“已验证成功”或“已验证失败且可复现”的事实，不记录猜测。

## 记录粒度
- 本地应用：按 `app-name` 建文件，如 `site-patterns/wechat.md`、`site-patterns/microsoft-teams.md`
- 网页任务：按域名建文件，如 `site-patterns/x.com.md`、`site-patterns/feishu.cn.md`

## 建议结构
```markdown
---
kind: local-app | website
name: WeChat
app_id: com.tencent.xinWeChat
aliases: [微信, WeChat]
domain: x.com
updated: 2026-03-27
---

## 成功路径
- [2026-03-27] 前台执行：open -a WeChat → Command+F → 粘贴联系人 → Enter → 粘贴消息 → Enter

## 前提条件
- 输入焦点必须在聊天列表或主窗口
- 中文联系人搜索可用

## 不可行/不稳定路径
- [2026-03-27] 纯后台键盘注入：不稳定，无法可靠确认焦点归属

## 经验结论
- 发消息任务默认采用“后台准备 + 前台短执行 + 发送后校验”
```

## 使用规则
- 一旦目标对象已命中对应经验文件，先读经验，再决定是否走通用探索。
- 如果经验中的首选路径本次再次成功，继续沿用，无需重复探索次优路径。
- 如果首选路径失效，再回退到通用优先级策略，并在完成后更新经验。
- 经验优先级低于用户明确要求；若用户要求“静默模式”，可尝试静默路径，但必须说明成功率风险。


path: references/site-patterns/README.md
content: 按软件名称或网页域名保存执行经验。

命名建议：
- 本地应用：`wechat.md` `microsoft-teams.md` `excel.md`
- 网站域名：`x.com.md` `feishu.cn.md` `outlook.office.com.md`

仅记录已验证事实：
- 成功路径
- 必要前提
- 已知陷阱
- 推荐默认策略


path: references/site-patterns/TEMPLATE.md
content: ---
kind: local-app | website
name: Target Name
app_id: com.example.app
domain: example.com
aliases: [Alias 1, Alias 2]
updated: YYYY-MM-DD
---

## Platform traits
- Facts about the target platform, UI model, login dependency, loading style, or execution constraints.

## Successful paths
- [YYYY-MM-DD] Preferred path: exact validated action chain.
- [YYYY-MM-DD] Alternate path: when the preferred one does not apply.

## Preconditions
- What must already be true before starting.

## Verification
- How to confirm success using the strongest available signal.

## Unstable or failed paths
- [YYYY-MM-DD] What failed or is not recommended, and why.

## Recommended default
- The default route to try first next time.

## Notes for future runs
- Extra target-specific observations worth reusing.


path: references/site-patterns/excel.md
content: ---
kind: local-app
name: Microsoft Excel
aliases: [Excel, Microsoft Excel]
updated: 2026-03-27
---

## Platform traits
- Excel tasks split into structured edits and layout-sensitive GUI edits.
- Pure GUI editing is not the default if the same result can be achieved more deterministically by structured operations.

## Successful paths
- [2026-03-27] Structured-edit flow: identify workbook/sheet/range → prepare intended changes in advance → apply edits in the most direct available way → read back changed cells or sheet state.
- [2026-03-27] GUI-edit flow: prepare all edits first → focus Excel → navigate directly to the target sheet/range → perform the shortest possible foreground edit sequence → reread or visually confirm result.

## Preconditions
- Workbook is accessible.
- Target sheet/range can be identified.
- The requested change type is understood before entering the GUI.

## Verification
- Read back changed cells/sheet state after edit.
- Use visible confirmation for formatting/layout-sensitive results.

## Unstable or failed paths
- [2026-03-27] Long foreground editing sessions are less preferred than batch-prepared edits with short execution windows.
- [2026-03-27] Background-only keyboard control is not reliable unless previously proven for the exact flow.

## Recommended default
- Prefer structured/scriptable edits first.
- Use local GUI execution only for layout-sensitive or app-specific tasks.

## Notes for future runs
- Separate “data change” from “layout/view change” before choosing the execution path.


path: references/site-patterns/feishu.md
content: ---
kind: local-app
name: Feishu
aliases: [飞书, Feishu, Lark]
updated: 2026-03-27
---

## Platform traits
- Feishu local-app execution should be used for app-bound tasks.
- If the requested action is natively covered by first-class Feishu tools, those tools are superior to generic local UI automation.
- Critical steps that depend on recipient/thread/visible state may require foreground execution.

## Successful paths
- [2026-03-27] App-bound flow: launch or focus Feishu app → navigate to the target conversation/workspace area → perform the minimum necessary foreground interaction → verify visible post-action state in-app.
- [2026-03-27] If the task is document/wiki/drive-native and supported by first-class Feishu tools, use those tools instead of local UI driving.

## Preconditions
- The local app is installed and launchable.
- The correct workspace/account is already signed in.
- The target area can be uniquely identified.

## Verification
- Confirm the target conversation, document, or UI state in-app after the action.
- Prefer visible read-back or tool-based read-back over process-only confirmation.

## Unstable or failed paths
- [2026-03-27] Background-only keyboard/UI flow should not be treated as default unless previously proven for the exact task variant.
- [2026-03-27] Generic UI automation is less preferred than first-class Feishu tools for supported operations.

## Recommended default
- Use first-class Feishu tools for supported document/wiki/drive operations.
- Otherwise use local app with the shortest necessary foreground section and immediate verification.

## Notes for future runs
- Separate “Feishu app task” from “Feishu platform task” before choosing execution mode.


path: references/site-patterns/microsoft-teams.md
content: ---
kind: website
name: Microsoft Teams
domain: teams.microsoft.com
aliases: [Teams, Microsoft Teams]
updated: 2026-03-27
---

## Platform traits
- Teams web is preferred when browser login state is present and the task is available in the web UI.
- Browser execution is easier to verify than local-app UI for chat/calendar/file reading and many routine actions.

## Successful paths
- [2026-03-27] Calendar/read flow: open Teams web in browser → navigate to Calendar view → inspect visible events and details in-page → confirm by rereading event state in the page.
- [2026-03-27] Chat/read flow: open Teams web → open target chat/channel in web UI → inspect visible thread state → confirm by rereading the thread after any action.
- [2026-03-27] If composing or replying in web UI, prefer in-page recipient/thread confirmation before submission.

## Preconditions
- Browser login state is available.
- The relevant workspace/account is already accessible in browser.
- The requested task is supported in Teams web.

## Verification
- Prefer DOM/state read-back in browser after the action.
- Use visible page confirmation only when direct reread is not enough.

## Unstable or failed paths
- [2026-03-27] Local-app UI automation is not the default when the same task is available in the web client with better state visibility.
- [2026-03-27] Do not default to keyboard-only local-app navigation when browser verification is available.

## Recommended default
- Use Teams web first.
- Fall back to local app only when the web client lacks the capability or is specifically unsuitable for the task.

## Notes for future runs
- For calendar and chat inspection, browser flow should usually win on verification quality.


path: references/site-patterns/my.feishu.cn.md
content: ---
kind: website
name: Feishu Web
domain: my.feishu.cn
aliases: [Feishu Web, Lark Web, 飞书网页]
updated: 2026-03-27
---

## Platform traits
- Feishu web is preferred for web-native Feishu tasks when browser state is available.
- For wiki/doc navigation, browser flow or first-class Feishu tools are preferred over generic local-app UI exploration.

## Successful paths
- [2026-03-27] Web-navigation flow: open target Feishu page in browser → inspect or operate in-page → reread page/object state after the action.
- [2026-03-27] For wiki/doc access, prefer browser or first-class Feishu tools before considering local-app UI control.

## Preconditions
- Browser login state is available.
- The target content is reachable in Feishu web.

## Verification
- Confirm by rereading page state or verifying the target object in the web UI.
- Prefer first-class Feishu tools when available for the task type.

## Unstable or failed paths
- [2026-03-27] Generic local-app UI automation is less preferred when the same result is available by browser flow or first-class tool.

## Recommended default
- Use browser or first-class Feishu tools first for web-native tasks.
- Use local app only when the task is specifically app-bound or browser flow is unsuitable.

## Notes for future runs
- If the task can be satisfied through Feishu tool APIs, that should usually beat browser driving.


path: references/site-patterns/outlook.office.com.md
content: ---
kind: website
name: Outlook Web
domain: outlook.office.com
aliases: [Outlook, Outlook Web]
updated: 2026-03-27
---

## Platform traits
- Outlook Web is preferred for mailbox reading, triage, and many routine mailbox actions when browser login state exists.
- Browser state is easier to reread and verify than local desktop mail UI for most inspection tasks.

## Successful paths
- [2026-03-27] Mail-read flow: open Outlook Web → open target mailbox/folder → open the target message in-page → inspect content/state → confirm by rereading message/thread state.
- [2026-03-27] Triage flow: open Outlook Web → locate target message → perform action (flag/move/archive/etc.) → confirm the post-action message state in-page.
- [2026-03-27] Reply/draft flow: open target message in web UI → compose in-page → verify target thread and visible draft state before send/submit.

## Preconditions
- Browser login state is available.
- Mailbox content is reachable in Outlook Web.

## Verification
- Confirm by rereading message/thread/folder state in the web UI.
- Use visible page confirmation if direct state read-back is insufficient.

## Unstable or failed paths
- [2026-03-27] Local Outlook app automation is not preferred when the web client can complete the same task with clearer verification.

## Recommended default
- Use Outlook Web first.
- Fall back to desktop only for desktop-specific capability gaps.

## Notes for future runs
- Retrieval, review, and mailbox-state confirmation are usually cleaner in the web client.


path: references/site-patterns/powerpoint.md
content: ---
kind: local-app
name: Microsoft PowerPoint
aliases: [PowerPoint, Microsoft PowerPoint, PPT]
updated: 2026-03-27
---

## Platform traits
- PowerPoint tasks often depend on visible slide state, so visual verification matters.
- Prefer object/template/file-level edits when the request does not require live canvas inspection.

## Successful paths
- [2026-03-27] Structured-edit flow: identify presentation/slide/object target → prepare intended edits first → apply direct/object-level changes where possible → verify by rereading object state or slide result.
- [2026-03-27] Visual-layout flow: prepare edit intent → focus PowerPoint → navigate directly to target slide/object → perform the shortest possible foreground change sequence → visually verify the resulting slide.

## Preconditions
- Presentation is accessible.
- Target slide/object can be identified.
- The edit can be scoped before entering the GUI.

## Verification
- Read back slide/object state if available.
- Use visible slide confirmation for layout-sensitive changes.

## Unstable or failed paths
- [2026-03-27] Background-only UI control is not the default for layout-heavy work.
- [2026-03-27] Repeated manual-style navigation is less preferred than direct target selection plus focused edit execution.

## Recommended default
- Prefer structured edits first.
- Use local GUI only when visible slide state is part of the success criteria.

## Notes for future runs
- If success depends on what the slide looks like, visual verification is mandatory.


path: references/site-patterns/wechat.md
content: ---
kind: local-app
name: WeChat
app_id: com.tencent.xinWeChat
aliases: [微信, WeChat, xinWeChat]
updated: 2026-03-27
---

## Platform traits
- WeChat desktop messaging is treated as a local-app task.
- Reliable sending depends on correct window focus and visible thread verification.
- Success is preferred over silence for this target.

## Successful paths
- [2026-03-27] Verified send flow: `open -a WeChat` (fallback `open -a 微信`) → `Command+F` → paste contact name → `Enter` to open target chat → paste message → `Enter` to send.
- [2026-03-27] Connectivity test default: use the contact phrase `微信给小崽崽说 hi` and append a timestamp in the sent content for freshness checking.

## Preconditions
- The app is installed and launchable.
- Main window is available.
- Contact search accepts pasted text.
- The target contact can be uniquely selected from search.

## Verification
- Confirm in the visible target thread that the sent message appears after send.
- For test messages, verify the timestamp matches the current run.

## Unstable or failed paths
- [2026-03-27] Background-only keyboard flow is not the default because focus ownership cannot be reliably verified.
- [2026-03-27] Silent mode may be attempted only when explicitly requested, with lower confidence.

## Recommended default
- Use background preparation + short foreground execution + post-send verification.
- Prefer the verified local-app send chain above.

## Notes for future runs
- Default strategy is “prefer success,” even if short foreground focus is required.


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)