openeventdata/scraper — pull request #17 by ahalterman (2014-02-14 16:51:20+00:00)
Title: Sources and logging text

-Deleted Haaretz (again). The scraping fails on Haaretz, at least on my tests (there's a click through page).
-Changed the logging text to remind me that it's running in hourly mode, so I remember that it's a feature, not a bug, that I don't see stories immediately streaming into the Mongo DB.
-Added a source (al-Monitor) with good ME coverage. One text thing that could get cleaned up is that Goose is pulling picture captions out as body text (example below). I think we could delete all text before and including "(photo by [...] )" and that would clear things up. I don't trust my regex that much, though.

"Members of the Islamist Syrian rebel group Jabhat al-Nusra ride on a vehicle mounted with an anti-aircraft weapon, along a damaged street in Deir el-Zour, eastern Syria, Feb. 5, 2014. (photo by REUTERS/Khalil Ashawi)\n\nThe fighting has escalated between Jabhat al-Nusra, the Islamic Front and their allies on the one hand, and the Islamic State of Iraq and al-Sham (ISIS) on the other, as it reached an unprecedented climax in the past few days, both in terms of expansion and violence.