publicscience/argos — pull request #135 by ferristseng (2014-05-06 05:06:28+00:00)
Title: progress on #90 -- have get_articles use a function rather than return a list

changed the test functions to emulate how `get_articles` was working previously.

test output:

```
............................................................................2014-05-06 00:57:44,891 - argos.core.digester.knowledge - INFO - Digesting the datasets to tests/data/knowledge/knodb...
2014-05-06 00:57:44,892 - argos.core.digester.knowledge - INFO - Using the datasets: tests/data/knowledge/images_en.ttl tests/data/knowledge/labels_en.ttl tests/data/knowledge/redirects_en.ttl
2014-05-06 00:57:44,892 - argos.core.digester.knowledge - INFO - Digestion complete.
.....................EE...............2014-05-06 00:58:07,266 - argos.core.membrane.collector - ERROR - Error extracting data for url http://some_url.com
Traceback (most recent call last):
File "/Users/ferristseng/Documents/prog/argos/argos/core/membrane/collector.py", line 90, in get_articles
entry_data, html = extractor.extract_entry_data(url)
File "", line 3, in extract_entry_data
File "/Users/ferristseng/.pythonz/pythons/CPython-3.3.3/lib/python3.3/unittest/mock.py", line 846, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/Users/ferristseng/.pythonz/pythons/CPython-3.3.3/lib/python3.3/unittest/mock.py", line 901, in _mock_call
raise effect
urllib.error.HTTPError: HTTP Error 404: None
..2014-05-06 00:58:07,878 - argos.core.membrane.collector - ERROR - Error extracting data for url http://some_url.com
Traceback (most recent call last):
File "/Users/ferristseng/Documents/prog/argos/argos/core/membrane/collector.py", line 90, in get_articles
entry_data, html = extractor.extract_entry_data(url)
File "", line 3, in extract_entry_data
File "/Users/ferristseng/.pythonz/pythons/CPython-3.3.3/lib/python3.3/unittest/mock.py", line 846, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/Users/ferristseng/.pythonz/pythons/CPython-3.3.3/lib/python3.3/unittest/mock.py", line 901, in _mock_call
raise effect
urllib.error.URLError:
.2014-05-06 00:58:08,401 - argos.core.membrane.collector - INFO - Fetching from foo...
.2014-05-06 00:58:09,300 - argos.core.membrane.collector - INFO - Fetching from foo...
2014-05-06 00:58:09,300 - argos.core.membrane.collector - INFO - Error fetching from foo.
.......................................2014-05-06 00:58:50,220 - argos.util.gullet - INFO - Total file size: 192.0 bytes
.2014-05-06 00:58:50,225 - argos.util.gullet - INFO - File already downloaded.
.2014-05-06 00:58:50,230 - argos.util.gullet - INFO - Total file size: 192.0 bytes
.2014-05-06 00:58:50,235 - argos.util.gullet - INFO - Total file size: 192.0 bytes
2014-05-06 00:58:50,235 - argos.util.gullet - INFO - Server does not allow resuming of downloads.
2014-05-06 00:58:50,235 - argos.util.gullet - INFO - Starting from the beginning! :D
.....
======================================================================
ERROR: test_company_profile (tests.core.knowledge_test.KnowledgeProfilesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/ferristseng/Documents/prog/argos/tests/core/knowledge_test.py", line 145, in test_company_profile
self.assertEqual(profile['type'], 'company')
KeyError: 'type'

======================================================================
ERROR: test_place_profile (tests.core.knowledge_test.KnowledgeProfilesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/ferristseng/Documents/prog/argos/tests/core/knowledge_test.py", line 153, in test_place_profile
self.assertEqual(profile['type'], 'place')
KeyError: 'type'

----------------------------------------------------------------------
Ran 165 tests in 191.900s

FAILED (errors=2)
```

not sure why the other 2 tests failed, but they were failing before I made the change