Module Import alias alias TryExcept body ImportFrom alias handlers ExceptHandler type NameLoad body ImportFrom alias ImportFrom alias alias ClassDef bases NameLoad body Expr Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Str Expr Str FunctionDef arguments args NameParam defaults body Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd Str AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If UnaryOpNot NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad Return Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num Assign NameStore Num For NameStore SubscriptLoad NameLoad Index Str body If CompareEq SubscriptLoad NameLoad Index Str Num body Continue If Call AttributeLoad Call AttributeLoad SubscriptLoad NameLoad Index Str attr attr Str body Continue Assign NameStore SubscriptLoad NameLoad Index Str If Call AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str AttributeLoad NameLoad attr body AugAssignAdd NameStore Num Assign AttributeStore NameLoad attr BinOpAdd BinOpAdd AttributeLoad NameLoad attr Str Call NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad decorator_list decorator_list
N/A N/A re json N/A N/A urllib.parse urljoin N/A N/A N/A ImportError N/A urlparse urljoin N/A Crawler CrawlerError Pr0gramm N/A Crawler N/A N/A  pr0gramm.com image provider N/A __uri N/A __next N/A __site N/A __image_base_url http://img.pr0gramm.com/ N/A __api_base_url N/A __filter N/A N/A re compile ^/static/[\d]+ N/A __filterNextPage N/A 
        JSON From API
        {
            atEnd: <bool>
            atStart: <bool>
            error: ?
            items: [
                        ...
                        {
                            id: <int>
                            promoted: <int>
                            up: <int>
                            down: <int>
                            created: <int>
                            image: <string> z.B.: 2016/02/18/047b2e356d059074.jpg
                            thumb: <string>
                            fullsize: <string>
                            source: <string>
                            flags: <int>
                            user: <string>
                            mark: <int>
                        }
                        ...
                    ]
            ts: <int>
            cache: <string>
            rt: <int>
            qc: <int>
        }
     __build_uri N/A N/A uri N/A N/A N/A uri N/A staticmethod _restart_at_front N/A N/A self N/A N/A N/A N/A self __next N/A self __uri N/A __init__ N/A N/A self uri site N/A N/A N/A N/A self __site site N/A N/A self __uri N/A N/A N/A self __class__ __build_uri uri N/A N/A self __api_base_url N/A self __uri N/A N/A self __filterNextPage N/A N/A re compile N/A N/A ^/static/ N/A self __site /[\d]+ N/A N/A N/A self _restart_at_front N/A _crawl N/A N/A self N/A N/A N/A uri N/A self __next N/A N/A N/A N/A self __class__ _log debug N/A %s crawls url: %s N/A N/A N/A self __class__ __name__ uri N/A N/A remote uri N/A N/A N/A self __class__ _fetch_remote uri N/A N/A remote N/A N/A N/A N/A N/A self __class__ _log debug N/A %s crawled EMPTY url: %s N/A N/A N/A self __class__ __name__ uri N/A N/A data N/A N/A json loads remote N/A last_id 0 N/A images_added 0 N/A item N/A data N/A items N/A N/A N/A N/A item N/A promoted 0 N/A N/A N/A N/A N/A N/A N/A N/A item N/A image lower endswith .webm N/A N/A N/A last_id N/A item N/A id N/A N/A N/A self _add_image N/A urljoin N/A self __image_base_url N/A item N/A image N/A self __site N/A N/A images_added 1 N/A N/A self __next N/A N/A N/A self __api_base_url ?older= N/A str last_id N/A N/A N/A N/A self __class__ _log debug N/A %s added %s images on url: %s N/A N/A N/A self __class__ __name__ images_added uri N/A N/A