Skill: whodidthese/my-play-music-from-yt (version 0.0.9)

--- name: my-play-music-from-yt description: | Play music on YouTube via browser automation with playwright-cli. Use when the user wants to: (1) play a specific song (e.g. 'play Money Money Money by ABBA') (2) play songs by an artist as a playlist or mix (e.g. 'play Jay Chou's songs') (3) play genre or mood-based music (e.g. 'play relaxing spa music', 'play 60s Chinese oldies') (4) control playback — next, pause, resume, stop, skip ad, change song, close the player. Also handles song/artist name corrections from voice transcription errors. allowed-tools: Bash(playwright-cli:*) metadata: {"openclaw": {"requires": {"bins": ["playwright-cli"]}, "emoji": "🎵"}} --- # Play Music from YouTube — Clear, Stable, High SNR Instruction Set This skill controls a **visible browser** using `playwright-cli` to search and play YouTube music. All actions follow **snapshot → ref → action** pattern. Never guess. Never assume. --- # CORE PRINCIPLES ### **1. Always use a named session** ``` -s=music_player ``` Every command must include it. ### **2. Always snapshot before interacting** ``` playwright-cli -s=music_player snapshot ``` Never click/fill without fresh refs. ### **3. Only use ref-based actions** ``` click e123 fill e45 "text" ``` Never use CSS, XPaths, or assumptions. ### **4. Browser must be visible** ``` --headed ``` Use headed **by default** unless the user explicitly requests headless. ### **5. Session continues in background** After playback begins, do not block waiting. Continue responding normally. ### **6. Use persistent browser profile** ``` --persistent ``` Always include `--persistent` when opening a new session. This saves the browser profile (cookies, localStorage, IndexedDB, cache) to disk, so **login state survives session restarts**. --- # SNAPSHOT STORAGE Snapshots generated by `playwright-cli` may exist in two possible locations: 1. `$WORKSPACE/.playwright-cli/` ← **primary (search here first)** 2. `~/.playwright-cli/` ← **fallback when the workspace folder is empty** Use **only these two folders** unless the user explicitly authorizes a broader search. If no .yml snapshots appear here, it is almost always **MacOS permission-related**. Ask the user to run: ``` playwright-cli open https://www.youtube.com --headed ``` This will trigger macOS system dialogs for: - Screen Recording - Automation - Accessibility Approve all to enable snapshot generation. > **Note:** On Windows and Linux, these permission dialogs do not apply. If snapshots are missing on those platforms, check that the browser launched correctly. --- # SESSION MANAGEMENT ### Check whether the session exists ``` playwright-cli list ``` - Exists → continue - Missing → recreate: ``` playwright-cli -s=music_player open https://www.youtube.com --headed --persistent ``` **On any unexpected error**, run `playwright-cli list` first to diagnose before retrying. ### If the session is frozen or blank ``` playwright-cli -s=music_player goto https://www.youtube.com ``` This reliably resets the page without closing the session. --- # SEARCH QUERY CONSTRUCTION Build the search query based on user intent: | Intent | Query pattern | Example | |---|---|---| | Specific song | `[Artist] [Song Title]` | `ABBA Money Money Money` | | Artist playlist | `[Artist]` | `周杰倫` | | Genre / mood | `[descriptor] music playlist` | `relaxing spa music playlist` | | Era-based | `[era] [language/genre] playlist` | `華語 60年代 老歌 playlist` | **Search tips:** - For artist requests, a simple artist name search usually surfaces "Mix" and playlist results at the top — prefer these for continuous playback. - Use the original language for non-English songs (e.g. Chinese characters for Chinese songs). **Voice transcription (ASR) and typo handling:** - **Do NOT ask the user to confirm** potentially misheard names. YouTube has robust auto-correction and will show results for the intended query even with typos or homophones (e.g. searching "楊成林" will auto-correct to "楊丞琳"). - Always search directly with whatever text you have. After the search, check the results snapshot — if YouTube shows a "Did you mean: ..." or "顯示以下搜尋結果: ..." banner with corrected results, the correction is already applied. - Only fall back to **web search** or asking the user if the YouTube search results are clearly unrelated or empty. --- # SEARCH WORKFLOW (HIGH SIGNAL, LOW AMBIGUITY) ## **Step 1 — Reset search state** A clean search state avoids dropdown obstruction. Perform the following every time before a new search: 1. Snapshot. 2. If button `"Clear search query"` / `"清除搜尋查詢"` is present → click it. 3. Else press: ``` playwright-cli -s=music_player press Escape ``` 4. If suggestions or overlays still block results → reset page: ``` playwright-cli -s=music_player goto https://www.youtube.com ``` Then snapshot again. --- ## **Step 2 — Locate search bar** Snapshot → find: - `combobox "搜尋"` or - `combobox "Search"` Record its ref (e.g., `e34`). --- ## **Step 3 — Perform search** ``` playwright-cli -s=music_player fill <searchRef> "SEARCH TERM" playwright-cli -s=music_player press Enter ``` Then snapshot immediately to avoid suggestion-panel obstruction. --- ## **Step 4 — Choose best result** Select result based on **user intent**: - **Specific song** → click the `link` whose heading matches the song title. - **Artist** → prefer `"Mix - [Artist]"` links or playlist links for continuous playback. - **Genre / mood** → prefer long-duration compilations or playlists (>20 mins). - **Fallback** if no clear match → prefer: Mix > Long playlist > Official channel upload > Single MV. For guidance on identifying YouTube result types, see [./references/youtube-guide.md](./references/youtube-guide.md). **Important:** Do NOT click channel cards (e.g. `link "Artist Name ... @handle•NNNK subscribers ..."` / `"Artist Name ... @handle•NNN萬位訂閱者 ..."`). They navigate to the channel page, not music. Look for Mix or playlist results below them. Click using its ref: ``` playwright-cli -s=music_player click eXYZ ``` If click is blocked: 1. `press Escape` 2. Retry click once. 3. If still blocked → `goto https://www.youtube.com` and restart from Step 1. --- # AD HANDLING LOOP After clicking a result: ``` playwright-cli -s=music_player snapshot ``` Loop up to 4 iterations: ### **1. Playback detected** If you see: - `button "Pause (k)"` or - `button "Play (k)"` (but video advancing) AND no ad indicators → playback is active → report song title to the user. ### **2. Skip button detected** Any element whose label contains: - skip / Skip / 略過 - Skip Ad / Skip Ads / 略過廣告 → click it immediately. **Note:** If the skip button is `[disabled]` (countdown not yet elapsed), do NOT click it — wait ~5 seconds instead (see step 3). Then snapshot again. YouTube often plays **two consecutive ads** — always check for a second ad after skipping. ### **3. Non-skippable ad / countdown** Wait 5 seconds: ``` playwright-cli -s=music_player eval "await new Promise(r=>setTimeout(r,5000))" ``` Then snapshot again and repeat from step 1. ### If ads persist >20 seconds - press Escape - or reload page --- # PLAYBACK CONTROLS Always snapshot first. | Command | Action | |--------|--------| | Pause | click `button "Pause (k)"` | | Resume / Play | click `button "Play (k)"` | | Next | `press Shift+n` | | Previous | `press Shift+p` | | Change song | Start new search from Step 1 | | Stop | `playwright-cli -s=music_player close` | **Keyboard shortcuts** (reliable alternative when button refs are hard to locate): ``` k → play / pause Shift+n → next track Shift+p → previous track m → mute / unmute f → fullscreen toggle j → rewind 10s l → forward 10s ``` Usage: `playwright-cli -s=music_player press <key>` --- # ERROR RECOVERY & EDGE CASES ### If snapshot empty or missing critical refs ``` playwright-cli -s=music_player goto https://www.youtube.com ``` → restart from Step 1. ### If browser crashes or session disappears `playwright-cli list` shows no `music_player`. Inform the user the browser was closed, then recreate session: ``` playwright-cli -s=music_player open https://www.youtube.com --headed --persistent ``` Because `--persistent` is used, the browser profile (including login state) is restored automatically. No need to log in again. ### YouTube cookie consent / sign-in dialogs Snapshot → locate dismiss actions: - "Accept all" / "全部接受" - "Reject all" - "No thanks" / "不用了，謝謝" - Close (X) Click dismiss. ### YouTube Premium trial popup Snapshot → look for `"No thanks"`, `"不用了，謝謝"`, or a dismiss/close button → click to dismiss. ### Video unavailable Snapshot shows "Video unavailable" or similar error. - `playwright-cli -s=music_player go-back` → snapshot → try the next result link. ### Cannot find elements in search results - Scroll down: `playwright-cli -s=music_player press PageDown` → snapshot again. ### Page frozen or blank - `playwright-cli -s=music_player reload` → snapshot. - If still broken: `playwright-cli -s=music_player close` → reopen and retry from Step 1. ### Persistent profile corrupted or login expired If the browser opens but login state is lost, or YouTube forces re-authentication: 1. Inform the user they need to log in again in the browser. 2. After login, the persistent profile will be updated automatically. To completely reset the profile (last resort): ``` playwright-cli -s=music_player close playwright-cli -s=music_player delete-data playwright-cli -s=music_player open https://www.youtube.com --headed --persistent ``` --- # REFERENCES - [Playwright CLI reference](./references/playwright-ref.md) - [YouTube result identification](./references/youtube-guide.md)

path: references/playwright-ref.md
content: # playwright-cli Command Reference (Music Playback Subset)

Only commands relevant to browser-based music playback are listed here. All commands below assume a named session; always prepend `-s=music_player`.

## Session Management

```bash
playwright-cli list                          # list all active sessions
playwright-cli -s=music_player open URL --headed              # create session (in-memory profile, lost on close)
playwright-cli -s=music_player open URL --headed --persistent # create session with persistent profile (survives close/reopen)
playwright-cli -s=music_player close         # close the session browser
playwright-cli -s=music_player delete-data   # delete persistent profile data for this session
playwright-cli close-all                     # close all sessions
playwright-cli kill-all                      # force-kill all browser processes
```

## Storage State (Backup / Restore)

```bash
playwright-cli -s=music_player state-save                 # save cookies + localStorage to auto-named JSON
playwright-cli -s=music_player state-save auth.json       # save to specific file
playwright-cli -s=music_player state-load auth.json       # restore cookies + localStorage from file
```

> **Note:** `state-save`/`state-load` only captures cookies and localStorage. For full profile persistence (including IndexedDB, cache, service workers), use `--persistent` on `open`.

## Core Interaction

```bash
playwright-cli -s=music_player snapshot                        # get page accessibility tree with element refs
playwright-cli -s=music_player snapshot --filename=debug.yaml  # save snapshot to file
playwright-cli -s=music_player goto URL                        # navigate to URL
playwright-cli -s=music_player click REF                       # click element (e.g. click e34)
playwright-cli -s=music_player fill REF "text"                 # clear + type into input (e.g. fill e34 "query")
playwright-cli -s=music_player type "text"                     # type text at current focus
playwright-cli -s=music_player eval "document.title"           # execute JS in browser context (page.evaluate)
playwright-cli -s=music_player eval "el => el.textContent" REF # execute JS on a specific element
playwright-cli -s=music_player screenshot                      # take a screenshot (for visual debugging)
playwright-cli -s=music_player screenshot --filename=page.png  # save screenshot to file
```

## Navigation

```bash
playwright-cli -s=music_player go-back
playwright-cli -s=music_player go-forward
playwright-cli -s=music_player reload
```

## Keyboard

```bash
playwright-cli -s=music_player press Enter
playwright-cli -s=music_player press ArrowDown
playwright-cli -s=music_player press ArrowUp
playwright-cli -s=music_player press PageDown
playwright-cli -s=music_player press PageUp
playwright-cli -s=music_player press Escape
playwright-cli -s=music_player press Tab
playwright-cli -s=music_player press Shift+n          # YouTube: next track
playwright-cli -s=music_player press Shift+p          # YouTube: previous track
playwright-cli -s=music_player press k                # YouTube: play/pause
playwright-cli -s=music_player press m                # YouTube: mute toggle
playwright-cli -s=music_player press f                # YouTube: fullscreen toggle
playwright-cli -s=music_player press j                # YouTube: rewind 10s
playwright-cli -s=music_player press l                # YouTube: forward 10s
```

## Tabs

```bash
playwright-cli -s=music_player tab-list
playwright-cli -s=music_player tab-new URL
playwright-cli -s=music_player tab-select INDEX       # 0-based index
playwright-cli -s=music_player tab-close
playwright-cli -s=music_player tab-close INDEX
```

## Dialog Handling

```bash
playwright-cli -s=music_player dialog-accept
playwright-cli -s=music_player dialog-dismiss
```

## Window

```bash
playwright-cli -s=music_player resize 1920 1080
```


path: references/youtube-guide.md
content: # YouTube Page Element Identification Guide

How to read `playwright-cli snapshot` output and find the right elements on YouTube pages.

## YouTube Homepage

After `playwright-cli -s=music_player open "https://www.youtube.com" --headed --persistent`:

**Search bar** — the primary target:

```yaml
- combobox "搜尋" [ref=e34]       # Chinese locale
- combobox "Search" [ref=e34]     # English locale
```

**Search button** (alternative to pressing Enter):

```yaml
- button "Search" [ref=e35]
```

## Search Results Page

After searching, the snapshot contains mixed result types. Identify them by structure:

### Single Video

Look for a `heading` + `link` with a video title and duration:

```yaml
- heading "Artist【Song Title】Official MV 4 分鐘 29 秒" [level=3] [ref=eXXX]:
    - link "Artist【Song Title】Official MV ..." [ref=eYYY]:
        - /url: /watch?v=VIDEO_ID
```

Click the `link` ref to play.

### Mix (Auto-generated Playlist)

YouTube creates a "Mix" for popular artists. Look for `"Mix - [Artist]"`:

```yaml
- heading "Mix - 周杰倫" [level=3] [ref=eXXX]:
    - link "Mix - 周杰倫" [ref=eYYY]:
        - /url: /watch?v=...&list=RDEM...&start_radio=1
```

**Prefer Mix links for artist requests** — they provide continuous playback of related songs.

### User Playlist

Playlists show a video count badge:

```yaml
- generic [ref=eXXX]: 13 部影片        # "13 videos" badge
- heading "Playlist Title" [level=3]:
    - link "Playlist Title" [ref=eYYY]:
        - /url: /watch?v=...&list=PL...
```

### Channel Card

Artist/channel cards appear at the top of results for artist searches:

```yaml
- link "Artist Name ... @handle•NNNK subscribers ..." [ref=eXXX]:    # English locale
    - /url: /@handle
- link "Artist Name ... @handle•NNN萬位訂閱者 ..." [ref=eXXX]:      # Chinese locale
    - /url: /@handle
```

Do NOT click the channel card itself (it goes to the channel page, not music). Instead look for the Mix or playlist results below it.

### Sponsored / Ad Results

Ads have a `"贊助商廣告"` (Sponsored) badge:

```yaml
- img "贊助商廣告" [ref=eXXX]
```

Skip these results; scroll past them to find organic results.

## Video Player Page

After clicking a video/mix, the player page snapshot contains:

### Player Controls

```yaml
- button "Play (k)" [ref=eXXX]        # or "Pause (k)" when playing
- button "Next (SHIFT+n)" [ref=eXXX]
- button "Previous" [ref=eXXX]
- button "Mute (m)" [ref=eXXX]
```

### Ad Overlay

When an ad is playing before the video, the snapshot structure changes significantly. Normal player controls (Pause, Next) may be absent or replaced by ad-specific elements.

**Skip button variations** — match by partial text containing "skip", "Skip", or "略過":

```yaml
- button "Skip Ad" [ref=eXXX]
- button "Skip Ads" [ref=eXXX]
- button "略過廣告" [ref=eXXX]
- button "Skip" [ref=eXXX]
- button "略過" [ref=eXXX]
```

**Countdown indicator** (skip button not yet active):

```yaml
- generic [ref=eXXX]: "You can skip ad in 3..."
- generic [ref=eXXX]: "3 秒後可略過廣告"
- button "Skip Ad" [disabled] [ref=eXXX]     # button present but disabled
```

When you see a countdown, wait ~5 seconds and snapshot again — the skip button will become clickable.

**Ad indicators** (how to tell an ad is playing even without a skip button):

```yaml
- generic [ref=eXXX]: "Ad · 0:12"            # ad timer
- generic [ref=eXXX]: "廣告 · 0:12"          # Chinese ad timer
- link "Visit advertiser's site" [ref=eXXX]
- link "造訪廣告主網站" [ref=eXXX]
- button "Why this ad?" [ref=eXXX]
- button "為什麼會顯示這則廣告？" [ref=eXXX]
```

**Important**: YouTube often plays **two consecutive ads**. After skipping one ad, always snapshot again to check for a second ad before confirming playback.

### Popups and Dialogs

> **With persistent profile:** If the user has previously logged in and accepted cookies, these dialogs appear **much less frequently**. They may still appear when Google forces re-authentication or after profile reset.

Cookie consent:

```yaml
- button "Accept all" [ref=eXXX]
- button "Reject all" [ref=eXXX]
```

Sign-in / Premium trial:

```yaml
- button "No thanks" [ref=eXXX]
- button "不用了，謝謝" [ref=eXXX]
- button "Dismiss" [ref=eXXX]
```

Logged-in user indicator (confirms login state is active):

```yaml
- button "Avatar image" [ref=eXXX]          # user avatar in top-right corner
- img "Avatar image" [ref=eXXX]             # alternative form
```

If the top-right shows `button "登入"` / `button "Sign in"` instead, the user is **not logged in**.

## Auto-correction Banner (Search Results)

When searching with typos or homophones (common with voice input), YouTube auto-corrects the query. Look for:

```yaml
- generic [ref=eXXX]: "顯示以下搜尋結果: 楊丞琳"
- link "改為搜尋：楊成林" [ref=eXXX]
```

Or in English:

```yaml
- generic [ref=eXXX]: "Showing results for: Yang Cheng Lin"
- link "Search instead for: ..." [ref=eXXX]
```

This means YouTube has already corrected the query and is showing the right results. No action needed — proceed to select a result as normal.

## Filter Tabs (Search Results)

Search results include filter tabs at the top:

```yaml
- tab "全部" [selected] [ref=eXXX]     # All (default)
- tab "影片" [ref=eXXX]                # Videos
- tab "播放清單" [ref=eXXX]            # Playlists
- tab "合輯" [ref=eXXX]                # Mixes / Collections
```

Click the `"播放清單"` or `"合輯"` tab to filter for playlists/mixes when the user wants continuous playback.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: clean (LOW)