Module ImportFrom alias ImportFrom alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ClassDef bases NameLoad body Assign NameStore Str Assign NameStore ListLoad Str Str Assign NameStore Str FunctionDef arguments args NameParam defaults vararg kwarg body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad If CompareEq Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Num body Assign NameStore Str Expr Call AttributeLoad NameLoad attr NameLoad Raise Call NameLoad NameLoad For NameStore Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body If CompareEq AttributeLoad NameLoad attr Str body If UnaryOpNot AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr keyword Call NameLoad NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Raise Call NameLoad NameLoad If CompareEq AttributeLoad NameLoad attr Str body If BoolOpOr UnaryOpNot AttributeLoad NameLoad attr UnaryOpNot AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr keyword Call NameLoad NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Raise Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults kwarg body Assign NameStore Str Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Pass decorator_list FunctionDef arguments args NameParam defaults body For NameStore Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr attr If CompareNotIn Str NameLoad body Assign SubscriptStore NameLoad Index Str Dict Assign SubscriptStore SubscriptLoad NameLoad Index Str Index Str NameLoad Assign SubscriptStore SubscriptLoad NameLoad Index Str Index Str NameLoad Expr Call AttributeLoad NameLoad attr If NameLoad body If CompareEq AttributeLoad NameLoad attr Str body Expr Yield Call NameLoad NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword NameLoad NameLoad orelse Expr Yield Call NameLoad NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword SubscriptLoad AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr keyword NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults kwarg body Assign NameStore SubscriptLoad AttributeLoad AttributeLoad SubscriptLoad NameLoad Index Str attr attr Index Str Assign NameStore SubscriptLoad AttributeLoad AttributeLoad SubscriptLoad NameLoad Index Str attr attr Index Str If CompareEq AttributeLoad SubscriptLoad NameLoad Index Str attr Num body If CompareEq AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr If CompareEq AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr orelse If CompareEq AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Str Assign NameStore SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Str If CompareEq AttributeLoad NameLoad attr Str body Return TryExcept body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr handlers ExceptHandler type NameLoad body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr Return If CompareEq Call NameLoad NameLoad Num body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr orelse If CompareEq AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr orelse If CompareNotEq SubscriptLoad NameLoad Index Num AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr keyword Call NameLoad NameLoad AttributeLoad NameLoad attr decorator_list decorator_list
N/A __future__ unicode_literals builtins str N/A logging scrapy signals scrapy.exceptions CloseSpider scrapy.http Request scrapy.xlib.pydispatch dispatcher dynamic_scraper.spiders.django_base_spider DjangoBaseSpider dynamic_scraper.models Scraper CheckerTest N/A DjangoBaseSpider N/A N/A name checker_test N/A mandatory_vars N/A ref_object scraper N/A command scrapy crawl checker_test -a id=SCRAPER_ID __init__ N/A N/A self N/A args kwargs N/A N/A N/A N/A self _set_ref_object Scraper kwargs N/A N/A self scraper N/A self ref_object N/A N/A N/A self _set_config kwargs N/A N/A N/A N/A N/A N/A self scraper checker_set count 0 N/A N/A msg No checkers defined for scraper! N/A N/A N/A logging error msg N/A N/A CloseSpider msg N/A checker N/A N/A N/A N/A self scraper checker_set all N/A N/A N/A N/A checker checker_type 4 N/A N/A N/A N/A checker checker_ref_url N/A N/A msg N/A N/A Please provide a reference url for your checker ({c}) (Command: {cr}). format c N/A str checker cr N/A self command N/A N/A N/A logging error msg N/A N/A CloseSpider msg N/A N/A N/A checker checker_type X N/A N/A N/A N/A N/A checker checker_x_path N/A N/A checker checker_ref_url N/A N/A msg N/A N/A Please provide the necessary x_path fields for your checker ({c}) (Command: {cr}). format c N/A str checker cr N/A self command N/A N/A N/A logging error msg N/A N/A CloseSpider msg N/A N/A N/A self _set_request_kwargs N/A N/A N/A self _set_meta_splash_args N/A N/A N/A dispatcher connect N/A self response_received signal N/A signals response_received N/A _set_config N/A N/A self N/A kwargs N/A N/A log_msg N/A N/A N/A N/A super CheckerTest self _set_config log_msg kwargs N/A spider_closed N/A N/A self N/A N/A N/A N/A start_requests N/A N/A self N/A N/A N/A checker N/A N/A N/A N/A self scraper checker_set all N/A N/A url N/A checker checker_ref_url N/A rpt N/A N/A N/A self scraper get_rpt_for_scraped_obj_attr N/A checker scraped_obj_attr N/A kwargs N/A N/A N/A N/A self dp_request_kwargs N/A N/A rpt page_type copy N/A N/A meta kwargs N/A N/A N/A kwargs N/A meta N/A N/A N/A N/A kwargs N/A meta N/A checker checker N/A N/A N/A kwargs N/A meta N/A rpt rpt N/A N/A N/A self _set_meta_splash_args N/A url N/A N/A N/A N/A rpt request_type R N/A N/A N/A N/A Request url callback N/A self parse method N/A rpt method dont_filter True kwargs N/A N/A N/A N/A FormRequest url callback N/A self parse method N/A rpt method formdata N/A N/A self dp_form_data N/A N/A rpt page_type dont_filter True kwargs N/A response_received N/A N/A self N/A kwargs N/A N/A checker N/A N/A N/A N/A kwargs N/A response request meta N/A checker N/A rpt N/A N/A N/A N/A kwargs N/A response request meta N/A rpt N/A N/A N/A N/A kwargs N/A response status 404 N/A N/A N/A N/A checker checker_type 4 N/A N/A N/A N/A self log N/A N/A Checker configuration working (ref url request returning 404) ({c}). format c N/A str checker N/A logging INFO N/A N/A N/A checker checker_type X N/A N/A N/A N/A self log N/A N/A A request of your checker ref url is returning 404. Your x_path can not be applied ({c})! format c N/A str checker N/A logging WARNING N/A N/A N/A N/A checker checker_type 4 N/A N/A N/A N/A self log N/A N/A Checker ref url request not returning 404 ({c})! format c N/A str checker N/A logging WARNING N/A parse N/A N/A self response N/A N/A N/A checker N/A N/A N/A response request meta N/A checker N/A rpt N/A N/A N/A response request meta N/A rpt N/A N/A N/A checker checker_type 4 N/A N/A N/A N/A N/A test_select N/A N/A N/A N/A response xpath N/A checker checker_x_path extract N/A N/A N/A ValueError N/A N/A N/A N/A self log N/A N/A Invalid x_path ({c})! format c N/A str checker N/A logging ERROR N/A N/A N/A N/A len test_select 0 N/A N/A N/A N/A self log N/A N/A Checker configuration not working (no elements found for xpath on reference url page) ({c})! format c N/A str checker N/A logging ERROR N/A N/A N/A N/A checker checker_x_path_result N/A N/A N/A N/A self log N/A N/A Checker configuration working (elements for x_path found on reference url page (no x_path result defined)) ({c}). format c N/A str checker N/A logging INFO N/A N/A N/A N/A test_select N/A 0 N/A checker checker_x_path_result N/A N/A N/A N/A self log N/A N/A Checker configuration not working (expected x_path result not found on reference url page) ({c})! format c N/A str checker N/A logging ERROR N/A N/A N/A N/A self log N/A N/A Checker configuration working (expected x_path result found on reference url page) ({c}). format c N/A str checker N/A logging INFO N/A N/A