sursh/blaggregator — pull request #68 by nnja (2014-12-15 00:02:10+00:00)
Title: throttle by number of new posts per crawl, instead of date

In regards to issue #56

I don't think this comment, made [here](https://github.com/sursh/blaggregator/blob/c8017f50f28a0d2b4bae088b0b0e1327870664c3/home/management/commands/crawlposts.py#L68-L72) is correct:

```
# Only post to zulip if the post was created recently
# so that new accounts don't spam zulip with their entire post list
```
Because based on [this](https://github.com/sursh/blaggregator/blob/c8017f50f28a0d2b4bae088b0b0e1327870664c3/home/views.py#L99-L106) method that's run in the view when a new blog is added, it appears that all crawled posts are added to the database, then [skipped over](https://github.com/sursh/blaggregator/blob/c8017f50f28a0d2b4bae088b0b0e1327870664c3/home/management/commands/crawlposts.py#L66) because they already exist. Because of this, none of the existing blog posts at time of creation get announced in Zulip.

Based on this, my proposed fix to the published date/crawled date problem is this:

Since after blog addition by a user, all existing blog posts are saved and ignored, we can assume that any blog post without a database equivalent is a new blog post, regardless of the date.

So, instead of throttling posts by date (< 2 days) we can throttle by amount of posts per author per crawl (in my code it's 2 or less).

Let me know what you think.

- Nina