ArchiveTeam/twitchtv-discovery-grab — pull request #2 by ianweller (2014-08-08 07:15:55+00:00)
Title: Add discover.py to discover users and videos

This creates a new item type, "user", which will fetch a list of discovered users and videos (both highlights and past broadcasts with more than 100 views).

We could import this list of users to start us off: https://github.com/ArchiveTeam/twitchtv-items/blob/master/seed/channel.txt

Tested with `python discover.py puncayshun test.txt.gz`:

$ gunzip -c test.txt.gz | grep ^user: | wc -l
27883
$ gunzip -c test.txt.gz | grep ^flv: | wc -l
107
# a random sample...
$ gunzip -c test.txt.gz | shuf | head
user:sleepyseal
user:coilerr
user:touchtheclouds
user:ironcross26
user:cutmyteeth
user:soapgoat
flv:c4840664
user:swaggdocta
user:jackthestripperdd7
user:godjesusitshim
$ gunzip -c test.txt.gz | tail
flv:c4173427
flv:a550640830
flv:c2422934
flv:c3590620
flv:c3966433
flv:c4802722
flv:c4212438
flv:a523859418
flv:c2683349
flv:c3431580

(Administrative note: I'll be at work for the bulk of tomorrow, so don't require my permission to make changes.)