internetarchive/heritrix3 — pull request #2 by cmiles74 (2011-11-16 19:59:13+00:00)
Title: New Decide Rules Against HTTP Status

I added two new decide rules that make their decision against the HTTP status code of the provided URI. I'm using this to keep the body of 404 messages out of my crawled data by using these rules in the "shouldFetchBody" property of the FetchHTTP object. I throught you might find these useful. :)