internetarchive/heritrix3 — pull request #113 by vonrosen (2014-12-09 19:46:35+00:00)
Title: Allow non-login GET forms to be submitted if ExtractorHTMLForms.extractAllForms == true

This is a preliminary attempt at allowing form submissions for non-login forms that have method="get". It is not complete and should NOT be merged. It is for review only.

The goal of this enhancement is to allow forms to be submitted if the do not seem like login forms, which are already handled by FormLoginProcessor, and the extractAllForms value on ExtractorHTMLForms is true. Each submit input found in the form creates a a new submission attempt. Dropdowns (...) are also parsed and their selected values are extracted. If there is no "selected" option value, then the first option value is chosen as the one to submit.

Before continuing on this enhancement, I want review input to determine if the solution looks desirable. There will need to be a unit test for this processor, which hasn't been added yet, pending review.