whitethrash/GabyBots — pull request #6 by jeffmay (2012-11-16 05:56:06+00:00)
Title: #5 - created source spider

@Nearhan @anxiousmodernman

Created the SourceSpider. This code is working last I tested. Unfortunately it is hard to test since the feeds may change and add a source that isn't in the database.

I added a fixture called "moarscrapers.json" for the basic setup plus some possibly tested/untested scrapers. I realized that this was probably the time to add some test data and figure out how to add static sample tests.

Adding a suite of tests with fixtures will be the second phase of this branch, but we are no longer blocked on adding scrapers to get data from arbitrary sources (until this code breaks, of course).