nrabinowitz/pjscrape — Issue #5: Add persistent data support
Author: nrabinowitz
Created: 2011-12-12 05:49:33+00:00
State: open
Labels: feature

Add a client-side data persistence mechanism - localStorage would work (only if you're on the same domain - otherwise you'd need PhantomJS-level persistence, requiring a separate function...). Use case: scraping sub-pages of a category page and persisting the category.

More thinking on this: I could support cross-domain data using _pjs.state on the client side (default {}), grabbing it in page.open() at the end of the scrape and then writing it as JSON to the object at the beginning.