codelucas/newspaper — pull request #91 by c24b (2014-11-26 18:13:31+00:00)
Title: Text Article Detection 

Hello guy,

Thanks for this beautiful work. I know that the code is implemented for "newspaper" and adapted to it.
I twisted a little bit the code juste to get the parser extract the main content of webpages.( I recognize that you clarify a lot the Goose extraction system) but I noticed a slight problem for article detection in blogs.
I give an example:
http://blogs.nottingham.ac.uk/
the text extracted there is just the comment section
http://blogs.nottingham.ac.uk/makingsciencepublic/category/responsible-innovation/
th text extracted is only the about section: means for the entire blog categories it will detect the duplicate version of the about section

Just a question of curiosity, do you have any idea of making the self.top_node detection be wider and wiser? I didn't dare to reimplement it... Any suggestion is welcomed