Module Import alias Import alias TryExcept body Import alias handlers ExceptHandler type NameLoad body ImportFrom alias ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Dict Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Dict Assign SubscriptStore NameLoad Index Str AttributeLoad AttributeLoad NameLoad attr attr Assign SubscriptStore NameLoad Index Str AttributeLoad AttributeLoad NameLoad attr attr If Call NameLoad AttributeLoad AttributeLoad NameLoad attr attr Str body Assign SubscriptStore NameLoad Index Str AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr If Call NameLoad AttributeLoad AttributeLoad NameLoad attr attr Str body Assign SubscriptStore NameLoad Index Str AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Assign SubscriptStore NameLoad Index Str AttributeLoad AttributeLoad NameLoad attr attr Assign SubscriptStore NameLoad Index Str NameLoad Return TupleLoad NameLoad NameLoad decorator_list decorator_list FunctionDef arguments args defaults body Assign NameStore ListLoad Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Assign NameStore Lambda arguments args NameParam defaults ListComp Call AttributeLoad Str attr NameLoad comprehension NameStore NameLoad Return Call AttributeLoad Str attr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad NameLoad Return NameLoad decorator_list
N/A N/A os N/A logging N/A N/A N/A jsonlogger N/A N/A N/A ImportError N/A pythonjsonlogger jsonlogger TaskAdapter N/A N/A logging LoggerAdapter N/A N/A  Enhance any log messages with extra information about the
    current context of the scraper.  __init__ N/A N/A self logger scraper N/A N/A N/A N/A N/A N/A super TaskAdapter self __init__ logger N/A N/A N/A self scraper scraper N/A process N/A N/A self msg kwargs N/A N/A N/A extra N/A N/A kwargs get extra N/A N/A N/A extra N/A scraperName N/A N/A self scraper name N/A N/A extra N/A scraperId N/A N/A self scraper id N/A N/A hasattr N/A N/A self scraper task_ctx name N/A N/A N/A extra N/A taskName N/A N/A N/A self scraper task_ctx name N/A N/A hasattr N/A N/A self scraper task_ctx id N/A N/A N/A extra N/A taskId N/A N/A N/A self scraper task_ctx id N/A N/A extra N/A scraperStartTime N/A N/A self scraper start_time N/A N/A kwargs N/A extra extra N/A N/A msg kwargs N/A N/A make_json_format N/A N/A N/A N/A N/A supported_keys N/A asctime created filename funcName levelname levelno lineno module msecs message name pathname process processName relativeCreated thread threadName N/A log_format N/A N/A N/A x N/A N/A N/A N/A %({0:s}) format i N/A i x N/A N/A N/A join N/A log_format supported_keys N/A log_path N/A N/A scraper N/A N/A N/A  Determine the file name for the JSON log.  N/A N/A N/A N/A os path join N/A N/A scraper config data_path N/A %s.jsonlog N/A scraper name N/A make_logger N/A N/A scraper N/A N/A N/A  Create two log handlers, one to output info-level ouput to the
    console, the other to store all logging in a JSON file which will
    later be used to generate reports.  N/A logger N/A N/A logging getLogger N/A N/A N/A logger setLevel N/A logging DEBUG N/A requests_log N/A N/A logging getLogger requests N/A N/A N/A requests_log setLevel N/A logging WARNING N/A json_handler N/A N/A logging FileHandler N/A log_path scraper N/A N/A N/A json_handler setLevel N/A logging DEBUG N/A json_formatter N/A N/A jsonlogger JsonFormatter N/A make_json_format N/A N/A N/A json_handler setFormatter json_formatter N/A N/A N/A logger addHandler json_handler N/A console_handler N/A N/A logging StreamHandler N/A N/A N/A console_handler setLevel N/A logging INFO N/A fmt %(name)s [%(levelname)-8s]: %(message)s N/A formatter N/A N/A logging Formatter fmt N/A N/A N/A console_handler setFormatter formatter N/A N/A N/A logger addHandler console_handler N/A logger N/A N/A logging getLogger N/A scraper name N/A logger N/A TaskAdapter logger scraper N/A logger N/A