Initial commit of boilerpipe library

From boilerpipe r183 via:
svn export --force http://boilerpipe.googlecode.com/svn/trunk/ .
Add Chromium LICENSE file
Add gwt sdk to third_party
Add gwt_exporter to third_party
Add junit jar to third_party
Create a minimal GWT app

This is a very simple gwt app.

Run ant devmode to confirm that compilation and the app works (should see a
blank page at first then the GWT logging widget with the logged "It
works").

Run ant test to confirm that the JUnit testing framework works (should run a
single test for both debug and release).

Run ant clean to delete all that output junk.

R=yfriedman@chromium.org

Committed: 9e120ff

Committed: 68bb6e7

Review URL: https://codereview.appspot.com/38560047
Add simple PRESUBMIT.py

This adds presubmit checks for having an OWNERS lgtm (currently anyone
in the world is an owner) and that the upstream branch looks reasonable.

This allows us to use 'git cl push' to push the reviewed changes to the
googlecode repo.

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/50560044
Make a boilerpipe gwt module

Adds a .gwt.xml for boilerpipe and adds dependencies on that module from
the sample app.

Also add a trivial use of the boilerpipe TextBlock class so that
javascript compilation actually "goes into" the boilerpipe library
(javascript compilation only touches things used by the app's entry
 points).

Replace use of java.util.BitSet in TextBlock.java with HashSet<Integer>
since gwt does not support java.util.BitSet (also update users of
TextBlock in boilerpipe).

TEST=`ant gwtc` should succeed. `ant devmode` should log "It works."
R=yfriedman@chromium.org

Review URL: https://codereview.appspot.com/48720044
Compile javascript only for chrome

We don't care about the javascript for other platforms.

TEST=run 'ant gwtc'. Output should say it's only compiling one permutation.
Can confirm with `find war/ -name *cache.html | wc -l`

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/49710047
Add simple implementations of java.util.regex.Matcher/Pattern

These classes are used by boilerpipe but not available when using GWT.
I've added simple implementations of these classes based on GWT's RegExp
and MatchResult.

I think this is slightly simpler than finding+changing the relevant uses
in boilerpipe. Also, since boilerpipe doesn't really have any testing,
it is probably safer to create the implementations like this with tests
than to just make the changes in boilerpipe and hope that they are done
correctly.

TEST=run `ant test`
BUG=
R=yfriedman@chromium.org

Review URL: https://codereview.appspot.com/57430043
Add utilities to generate SAX events from DOM trees

Boilerpipe expects to work on sax events, not on a DOM tree. GWT does
not provide any of Java's standard library's org.xml.sax.

This change adds a simple implementation of org.xml.sax.AttributesImpl,
the interfaces org.xml.sax.Attributes/ContentHandler and skeleton
versions of SAXException and Locator.

Using these it adds a DomToSaxParser that generates sax events from
walking a DOM subtree. This will allow us to run the unmodified
boilerpipe code against a real web page by walking the DOM of that page.

There are tests for the DomToSaxParser that use a simple sax content
handler that simply takes sax events and prints the corresponding xml
text.

TEST=run `ant test`
R=yfriedman@chromium.org

Review URL: https://codereview.appspot.com/57850043
Apply boilerpipe extraction on default page

This applies extraction when the gwt module is loaded. This forces gwt
to compile the full pipeline of extraction to javascript. Since there is
nothing on the default page, there is no content extracted, but, by
running `ant devmode`, it can be confirmed that content extraction is
running successfully.

Boilerpipe's BoilerpipeExtractor and ExtractorBase both use lots of
things in java's standard library that are not available with gwt
(Reader, InputSource, StringReader, URL, etc.). We will never use the
stuff in these classes so I've removed them rather than make them work
with gwt. In turn, it is easier to remove HTMLHighlighter and
ImageExtractor than update them (since we will never use that stuff
either).

Also, in gwt, there is no Character.isWhitespace(). Instead use a simple approximation to that function (doesn't handle unicode correctly).

TEST=`ant gwtc` to confirm compilation. `ant devmode` to confirm that
extraction runs.

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/60220043
Add javadoc target

With the new javadoc target, there's no reason to keep the checked in
javadoc... so, it's removed.

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/60280043
Remove a bunch of unused boilerpipe stuff

We won't ever need any of this stuff.

R=shashishekhar@chromium.org

Review URL: https://codereview.appspot.com/60780043
Export ContentExtractor

This uses gwt_exporter to export ContentExtractor. This exports the
public parts of ContentExtractor so that they can be called from
javascript (i.e. as
    com.dom_distiller.client.ContentExtractor.extractContent());

TEST=run `ant devmode`; confirm that
com.dom_distiller.client.ContentExtractor.extractContent() is accessible
from javascript

R=yfriedman@chromium.org

Review URL: https://codereview.appspot.com/60820043
Extract standalone js from compiled js

Gwt-compiled js is actually an html file that is intended to be loaded
within an iframe of a page (with some js in the main frame interacting
with it).

For DomDistiller, we want standalone javascript that we can inject into
the main frame.

This adds a python script and ant target that takes the compiled js and
converts it to standalone js for DomDistiller.

R=shashishekhar@chromium.org

Review URL: https://codereview.appspot.com/60350045
Add a way to run only some test cases

This add the ant property test.filter used to filter what tests to run.
By default, it will match all tests. To run only some tests, do (for
example):
ant test -Dtest.filter=Dom*

Also, change the junit tasks to always run all tests (instead of
stopping at the first failure).

R=shashishekhar@chromium.org

Review URL: https://codereview.appspot.com/61100043
Extract the DOM walking from DomToSaxParser

This extracts DOM walking into a simple new class: DomWalker.

This class provides an interface (DomWalker.Visitor) to process the
nodes of the DOM (and give some limited control of the walk).

This will soon be used by several steps (expanding the content text
nodes to include containing html elements, expanding the content to
include images).

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/61760043
Add NodeListExpander and OrderedNodeMatcher

OrderedNodeMatcher takes a list of DOM nodes and provides a
match(Node n) that will match the nodes in order.

NodeListExpander provides a method to convert an ordered(*) list of
nodes into a minimal tree (from the DOM) containing those nodes-- i.e.
it will include all the ancestors of those nodes. This NodeTree has a
cloneSubtree method that will construct a copy of the corresponding part
of the DOM (by shallow-cloning each node).

This is required because Boilerpipe's output is essentially a list of
the text nodes that contain the content of the article. This will allow
us to extract the html elements for those nodes.

(*) In the order that they are encountered when walking the DOM tree.

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/61780043
Add image extraction

This adds ImageExtractor. This class takes an ordered list of nodes and
will walk the dom and identify all images that appear immediately after
one of those nodes (i.e. there is no non-whitespace text nodes between
them).

R=nyquist@chromium.org

Review URL: https://codereview.appspot.com/50060049