Skill: snailb1007/propertyguru-sg-sale-browser-crawl (version 0.1.0)

--- name: propertyguru-sg-sale-browser-crawl description: extract around 50 Singapore for-sale listings from a PropertyGuru search results URL using a real browser session after Cloudflare verification. use when the target is a PropertyGuru Singapore search page, direct HTTP fetch may return 403, the reliable source is `window.__NEXT_DATA__.props.pageProps.pageData.data.listingsData`, and results should be deduplicated by listing id across sequential pages until the target count is reached. license: MIT-0 metadata: { "openclaw": { "emoji": "🏘️", "homepage": "https://www.propertyguru.com.sg/property-for-sale?listingType=sale&page=1&isCommercial=false&maxPrice=1400000" } } --- # PropertyGuru SG Sale Browser Crawl Use this skill for PropertyGuru Singapore search results pages when the job is to collect roughly 50 listing cards from one filtered search URL. This target is browser-backed. - A direct fetch may return a Cloudflare verification page or `403`. - Prefer a real browser session and extract from the page's hydrated Next.js data. - Do not treat DOM card scraping as the primary source when `__NEXT_DATA__` is available. ## Required skill This skill depends on `playwright`. - Use a real browser page. - Let the browser complete PropertyGuru's Cloudflare verification first. - Only extract after the page title and result page content have loaded. ## Workflow 1. Read `{baseDir}/references/source-notes.md`. 2. Start from the user-provided search URL. If no URL is supplied, use the default URL from the source notes. 3. Open the page in a real browser. 4. Wait until the search results page is actually loaded, not the initial verification screen. 5. Read `window.__NEXT_DATA__.props.pageProps.pageData`. 6. Use `pageData.data.listingsData` as the canonical listing collection for that page. 7. For each item in `listingsData`, use `listingData.id` as the stable dedupe key. 8. Preserve the raw `listingData` object whenever possible. 9. Optionally add lightweight wrapper fields such as: - `source_url` - `page` - `collected_at` - `listing_id` 10. Continue page-by-page until one of these conditions is met: - 50 unique listings have been collected - `paginationData.currentPage >= paginationData.totalPages` - `listingsData` is empty - the next page repeats only ids already seen 11. For the default PropertyGuru URL observed on March 18, 2026, the page payload contained 25 listings per page, so pages 1 and 2 were enough to reach 50 listings. ## Canonical data location Prefer: ```js window.__NEXT_DATA__.props.pageProps.pageData.data.listingsData ``` Related pagination data: ```js window.__NEXT_DATA__.props.pageProps.pageData.data.paginationData ``` Useful search context: ```js window.__NEXT_DATA__.props.pageProps.pageData.searchParams ``` ## Recommended extraction shape Prefer keeping the raw listing payload plus a few convenience fields: ```json { "source_url": "https://www.propertyguru.com.sg/property-for-sale?listingType=sale&page=1&isCommercial=false&maxPrice=1400000", "page": 1, "collected_at": "2026-03-18T04:40:00Z", "listing_id": 500044843, "raw": { "id": 500044843, "localizedTitle": "780B Woodlands Crescent", "url": "https://www.propertyguru.com.sg/listing/hdb-for-sale-780b-woodlands-crescent-500044843", "price": { "value": 500000, "pretty": "S$ 500,000" }, "bedrooms": 2, "bathrooms": 2 } } ``` If the caller wants a flatter convenience export, these fields are usually available: - `listingData.id` - `listingData.localizedTitle` - `listingData.url` - `listingData.price.value` - `listingData.price.pretty` - `listingData.area.localeStringValue` - `listingData.bedrooms` - `listingData.bathrooms` - `listingData.fullAddress` - `listingData.property` - `listingData.psfText` - `listingData.postedOn` - `listingData.agent` - `listingData.agency` - `listingData.mrt` - `listingData.isVerified` ## Operating rules - Use browser extraction as the default path. - Do not rely on `curl`, plain HTTP, or static HTML parsing as the primary strategy. - Do not scrape promo widgets, "Explore around" cards, or other injected recommendation blocks from the visible DOM. - Use only `listingsData` for the main dataset. - Crawl one page at a time. - Deduplicate strictly on `listingData.id`. - Stop as soon as the requested target count is reached. - Preserve the search URL and page number with every saved record. - If the page falls back to a Cloudflare challenge and does not recover, report the block explicitly instead of pretending the page is empty. ## Output target Default target: about 50 unique listings. - Prefer pages 1 and 2 first. - If one page returns fewer rows than expected, continue to page 3 and beyond until the target count is reached. ## Notes - PropertyGuru may change the page structure, build id, or anti-bot behavior at any time. - When the page changes, re-check `__NEXT_DATA__` before changing extraction logic. - For this skill, the in-page Next.js payload is more stable than card-by-card DOM parsing.

path: agents/openai.yaml
content: interface:
  display_name: "PropertyGuru SG Sale Browser Crawl"
  short_description: "Collect about 50 PropertyGuru SG sale listings via browser-backed Next.js data"


path: references/source-notes.md
content: # Source Notes

## Default target

Primary evaluated URL:

`https://www.propertyguru.com.sg/property-for-sale?listingType=sale&page=1&isCommercial=false&maxPrice=1400000`

This file contains PropertyGuru Singapore specific notes for that search flow.

## Observed behavior on March 18, 2026

- Initial navigation may show a Cloudflare verification page with title `Just a moment...`.
- A real browser session was able to pass verification and load the actual result page.
- The loaded page title was `Houses for Sale - Below S$ 1.4M, Mar 2026`.
- The search results were available in Next.js hydration data under `window.__NEXT_DATA__`.

## Canonical extraction source

Observed page path:

`window.__NEXT_DATA__.props.pageProps.pageData`

Observed listing collection:

`window.__NEXT_DATA__.props.pageProps.pageData.data.listingsData`

Observed pagination object:

`window.__NEXT_DATA__.props.pageProps.pageData.data.paginationData`

Observed search params object:

`window.__NEXT_DATA__.props.pageProps.pageData.searchParams`

## Observed page-level facts

On the evaluated result set:

- `resultCount`: `22564`
- page 1 `listingsData.length`: `25`
- page 2 `listingsData.length`: `25`
- page 2 `paginationData.currentPage`: `2`
- page 2 `paginationData.totalPages`: `1129`

For a 50-listing collection target, pages 1 and 2 were sufficient.

## Observed listing payload shape

Each `listingsData[]` item currently contains top-level keys such as:

- `metadata`
- `listingData`
- `verifiedListingBadgeConfig`
- `hideTooltipConfig`
- `cardConfig`
- `ga`
- `segment`
- `gaProduct`
- `listingCardV2Config`

The canonical fields for the dataset are inside `listingData`.

Observed `listingData` fields included:

- `id`
- `statusCode`
- `typeCode`
- `isVerified`
- `agent`
- `agency`
- `psfText`
- `listingFeatures`
- `property`
- `fullAddress`
- `localizedTitle`
- `mediaItems`
- `mrt`
- `price`
- `url`
- `recency`
- `thumbnail`
- `typeText`
- `postedOn`
- `area`
- `pricePerArea`
- `bedrooms`
- `bathrooms`

## Example observed records

Page 1 examples:

- `500044843` `780B Woodlands Crescent` `S$ 500,000`
- `500049151` `34 Cassia Crescent` `S$ 750,000`
- `25365142` `367 Woodlands Avenue 5` `S$ 778,000`
- `500076088` `Treasure at Tampines` `S$ 1,250,000`

Page 2 examples:

- `500072831` `126 Pasir Ris Street 11` `S$ 699,999`
- `500025294` `348C Yishun Avenue 11` `S$ 950,000`
- `500014669` `169 Lorong 1 Toa Payoh` `S$ 408,000`
- `60164488` `476 Jurong West Street 41` `S$ 418,000`

## Practical guidance

- Use sequential pagination.
- Prefer page URL navigation by changing the `page` query param.
- After each navigation, re-read `window.__NEXT_DATA__`; do not reuse stale handles.
- Deduplicate by `listingData.id`.
- Ignore recommendation widgets that appear between result cards in the rendered DOM.
- If the browser gets stuck on verification and does not recover, treat that as a block condition, not a zero-result page.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)