chriskite/anemone — pull request #83 by lankz (2014-01-27 07:18:45+00:00)
Title: Give focus_crawl a chance to access page body before discarding it

For site-specific crawlers, it's fair enough to use `focus_crawl` like this:

```
anemone.focus_crawl do |page|
if page.doc
page.doc.search('.//a[@href]').map { |a| URI.parse(a[:href]) }
else
page.links
end
end
```

However when using the `discard_page_bodies` option, `page.doc` is `nil` by the time we enter this block. In this pull request I've moved until after `focus_crawl` has been called.