NiGhTTraX/hackernews-scraper — pull request #1 by claudiu-coman (2014-03-19 16:23:47+00:00)
Title: Implement comment scraping

## Need

I want to be able to fetch comments created since a timestamp.

## Deliverables

```gherkin
Scenario: Scraping comments
Given I have a timestamp
When I call the scraper
Then it will return all the comments created since that timestamp
```

```gherkin
Scenario: Scraping too many comments
Given I have a timestamp
And there are more than 50 * 20 comments since that timestamp
When I call the scraper
Then it will return all the comments created since that timestamp
And throw an exception informing me that there are more comments left that couldn't be fetched by the API
```

```gherkin
Scenario: No comments since timestamp
Given I have a timestamp
And there aren't any new comments since that timestamp
When I call the scraper
Then it will return an empty iterator
```

A returned comment could look like this
```json
{
"created_at": "2014-03-13T10:00:44.000Z",
"title": "DeepFace: Closing the Gap to Human-Level Performance in Face Verification",
"url": "https://www.facebook.com/publications/546316888800776/",
"comment_text": "blah blah blah",
"story_id": 7389728,
"story_title": "Mineral hints at bright blue rocks deep in the Earth",
"story_url": "http://www.bbc.com/news/science-environment-26553115",
"author": "mlla",
"points": 1,
"timestamp": 1394704844,
"comment_id": "7390857",
"parent_id": 7389984,
}
```
## Solution

#### Prerequisites
- glance over [the hacker news comments section](https://news.ycombinator.com/newcomments)
- [Algolia hacker news API](http://hn.algolia.com/api)
- [Requests library](http://docs.python-requests.org/en/latest/)

#### TODO
- [ ] Create a skeleton class
- [ ] Implement comment scraping

### Testing
Use mocking for response handling
- [ ] all fields are returned
- [ ] going back more than the supported number of pages raises an Exception
- [ ] when the API doesn't return anything, nothing is yielded and it doesn't break the code