Module Import alias Import alias Import alias ImportFrom alias Import alias Import alias Import alias ImportFrom alias alias alias alias alias ImportFrom alias identifier ImportFrom alias Assign NameStore Str ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults Str NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr BoolOpOr NameLoad Dict decorator_list decorator_list FunctionDef arguments args NameParam NameParam defaults vararg kwarg body Return Call NameLoad NameLoad Num Str decorator_list Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad FunctionDef arguments args NameParam NameParam defaults vararg kwarg body Raise Call AttributeLoad AttributeLoad NameLoad attr attr Str decorator_list Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num keyword NameLoad keyword Num keyword Num Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareIs AttributeLoad NameLoad attr NameLoad Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr Index Str Index Str Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Dict Str Str Assert CompareEq AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr Assert CompareEq SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Str Assert CompareEq SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assert CompareEq AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr With Call AttributeLoad NameLoad attr Str NameLoad body With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareLtELtE Num SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index Num Index Num Num Assign AttributeStore NameLoad attr Num Expr Call AttributeLoad NameLoad attr With Call AttributeLoad NameLoad attr Str NameLoad body With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Index Str Assert CompareEq NameLoad NameLoad Assign AttributeStore NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Index Str Assert CompareEq NameLoad Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assign AttributeStore NameLoad attr Dict Str Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Index Str Assert CompareEq NameLoad Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad keyword Num Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert UnaryOpNot AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Str With Call NameLoad NameLoad NameStore body Assert CompareEq Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert CompareEq NameLoad NameLoad With Call NameLoad NameLoad NameStore body Assert CompareEq Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad With Call NameLoad NameLoad NameStore body Assert CompareEq Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword ListLoad Call NameLoad Str Num Str Call NameLoad Str Num Str With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr keyword Call NameLoad Str Num Str With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword ListLoad Call NameLoad Str Num Str Call NameLoad Str Num Str With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr keyword Call NameLoad Str Num Str With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Num With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num keyword Num keyword NameLoad With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body With Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad Assign AttributeStore NameLoad attr Num With Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad With Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str keyword Num decorator_list FunctionDef arguments args defaults body Assign NameStore ListLoad FunctionDef arguments args defaults vararg kwarg body If NameLoad body Return Call NameLoad Str Num Str Expr Call AttributeLoad NameLoad attr Num Raise Call AttributeLoad NameLoad attr Str decorator_list Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call NameLoad keyword Num keyword Num With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr Assign NameStore ListLoad Assign NameStore Call NameLoad keyword Num keyword Num With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr Assert CompareNotIn Str SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assert CompareNotIn Str SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr Assert CompareIn Str SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assert CompareIn Str SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Assign SubscriptStore AttributeLoad NameLoad attr Index Str Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr Assert CompareIn Str SubscriptLoad SubscriptLoad NameLoad Index Str Index Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Lambda arguments args NameParam defaults Dict Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert CompareEq SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr Index Str Index Str BinOpAdd NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str Assert CompareEq SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr Index Str Index Str BinOpAdd NameLoad Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad Assign AttributeStore NameLoad attr Dict Str Str Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str attr Assert CompareEq SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Str Assign NameStore Call NameLoad Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr BinOpAdd NameLoad Str keyword Dict Str Str attr Assert CompareEq SubscriptLoad SubscriptLoad NameLoad Index Str Index Str Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr keyword Call NameLoad Str With Call AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Assert CompareEq AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args defaults body Assign NameStore ListLoad FunctionDef arguments args defaults vararg kwarg body If NameLoad body Return Call NameLoad Str Expr Call AttributeLoad NameLoad attr Num Raise Call NameLoad Str decorator_list Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad With Call AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call NameLoad keyword Num keyword Num Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Str Assert CompareEq AttributeLoad NameLoad attr Num Assign NameStore ListLoad Expr Call AttributeLoad NameLoad attr With Call AttributeLoad NameLoad attr Str NameLoad body Assign NameStore Call NameLoad keyword Num keyword Num Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Str keyword NameLoad Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad keyword Num Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str Num Assert CompareGt SubscriptLoad AttributeLoad NameLoad attr Index Str Num Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str BinOpDiv SubscriptLoad AttributeLoad NameLoad attr Index Str Num Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Str With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str Num Assert CompareGt SubscriptLoad AttributeLoad NameLoad attr Index Str NameLoad Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str BinOpDiv SubscriptLoad AttributeLoad NameLoad attr Index Str Num decorator_list FunctionDef arguments args defaults body Assign NameStore Call NameLoad With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str Num Expr Call AttributeLoad NameLoad attr Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str Num With Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str NameLoad body Expr Call AttributeLoad NameLoad attr Str Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Str Num decorator_list
N/A N/A os N/A glob N/A tempfile io BytesIO N/A mock N/A pytest N/A requests N/A Scraper HTTPError HTTPMethodUnavailableError urllib_URLError FTPError N/A _user_agent default_user_agent cache MemoryCache N/A HTTPBIN http://httpbin.org/ FakeResponse N/A object N/A __init__ N/A N/A self url code content encoding headers N/A utf-8 None N/A N/A N/A self url url N/A N/A self status_code code N/A N/A self content content N/A N/A self text N/A str content N/A N/A self encoding encoding N/A N/A self headers N/A headers N/A N/A N/A request_200 N/A N/A method url N/A args kwargs N/A N/A N/A FakeResponse url 200 ok N/A N/A mock_200 N/A N/A mock Mock wraps request_200 request_sslerror N/A N/A method url N/A args kwargs N/A N/A N/A N/A N/A requests exceptions SSLError sslfail N/A N/A mock_sslerror N/A N/A mock Mock wraps request_sslerror test_fields N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 100 raise_errors False retry_attempts -1 retry_wait_seconds 100 N/A N/A N/A s requests_per_minute 100 N/A N/A N/A s raise_errors False N/A N/A N/A s retry_attempts 0 N/A N/A N/A s retry_wait_seconds 100 N/A test_get N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A resp N/A N/A s get N/A HTTPBIN get?woo=woo N/A N/A N/A resp status_code 200 N/A N/A N/A N/A N/A N/A resp json N/A args N/A woo woo N/A test_post N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A resp N/A N/A s post N/A HTTPBIN post N/A woo woo N/A N/A N/A resp status_code 200 N/A resp_json N/A N/A resp json N/A N/A N/A N/A resp_json N/A form N/A woo woo N/A N/A N/A N/A resp_json N/A headers N/A Content-Type application/x-www-form-urlencoded N/A test_request_throttling N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 30 N/A N/A N/A s requests_per_minute 30 N/A mock_sleep N/A N/A mock Mock N/A N/A N/A mock patch time.sleep mock_sleep N/A N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://dummy/ N/A N/A N/A s get http://dummy/ N/A N/A N/A s get http://dummy/ N/A N/A N/A mock_sleep call_count 2 N/A N/A 1.8 N/A N/A N/A mock_sleep call_args N/A 0 N/A 0 2.2 N/A N/A s requests_per_minute 0 N/A N/A N/A mock_sleep reset_mock N/A N/A N/A mock patch time.sleep mock_sleep N/A N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://dummy/ N/A N/A N/A s get http://dummy/ N/A N/A N/A s get http://dummy/ N/A N/A N/A mock_sleep call_count 0 N/A test_user_agent N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A resp N/A N/A s get N/A HTTPBIN user-agent N/A ua N/A N/A N/A resp json N/A user-agent N/A N/A ua default_user_agent N/A N/A s user_agent a different agent N/A resp N/A N/A s get N/A HTTPBIN user-agent N/A ua N/A N/A N/A resp json N/A user-agent N/A N/A ua a different agent N/A test_user_agent_from_headers N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A N/A s headers N/A User-Agent from headers N/A resp N/A N/A s get N/A HTTPBIN user-agent N/A ua N/A N/A N/A resp json N/A user-agent N/A N/A ua from headers N/A test_404 N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A N/A N/A pytest raises HTTPError N/A s get N/A HTTPBIN status/404 N/A N/A s raise_errors False N/A resp N/A N/A s get N/A HTTPBIN status/404 N/A N/A N/A resp status_code 404 N/A test_500 N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A N/A N/A pytest raises HTTPError N/A s get N/A HTTPBIN status/500 N/A N/A s raise_errors False N/A resp N/A N/A s get N/A HTTPBIN status/500 N/A N/A N/A resp status_code 500 N/A test_caching N/A N/A N/A N/A N/A cache_dir N/A N/A tempfile mkdtemp N/A s N/A Scraper requests_per_minute 0 N/A N/A s cache_storage N/A MemoryCache N/A N/A s cache_write_only False N/A resp N/A N/A s get N/A HTTPBIN status/200 N/A N/A N/A resp fromcache N/A resp N/A N/A s get N/A HTTPBIN status/200 N/A N/A resp fromcache N/A path N/A N/A glob iglob N/A N/A N/A os path join cache_dir * N/A N/A N/A N/A os remove path N/A N/A N/A os rmdir cache_dir N/A test_urlretrieve N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A fname resp N/A N/A s urlretrieve http://dummy/ N/A N/A open fname f N/A N/A N/A N/A N/A f read ok N/A N/A N/A resp code 200 N/A N/A N/A os remove fname N/A N/A fh set_fname N/A N/A tempfile mkstemp N/A N/A fname resp N/A N/A s urlretrieve http://dummy/ set_fname N/A N/A fname set_fname N/A N/A open set_fname f N/A N/A N/A N/A N/A f read ok N/A N/A N/A resp code 200 N/A N/A N/A os remove set_fname N/A dirname N/A N/A N/A os path dirname set_fname N/A N/A fname resp N/A N/A s urlretrieve http://dummy/ dir dirname N/A N/A N/A N/A N/A os path dirname fname dirname N/A N/A open fname f N/A N/A N/A N/A N/A f read ok N/A N/A N/A resp code 200 N/A N/A N/A os remove fname N/A test_retry N/A N/A N/A N/A N/A s N/A Scraper retry_attempts 3 retry_wait_seconds 0.001 raise_errors False N/A mock_request N/A N/A mock Mock side_effect N/A N/A FakeResponse http://dummy/ 500 failure! N/A FakeResponse http://dummy/ 200 success! N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ N/A N/A N/A mock_request call_count 2 N/A mock_request N/A N/A mock Mock return_value N/A FakeResponse http://dummy/ 500 failure! N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ N/A N/A N/A resp status_code 500 N/A N/A N/A mock_request call_count 4 N/A test_retry_404 N/A N/A N/A N/A N/A s N/A Scraper retry_attempts 3 retry_wait_seconds 0.001 raise_errors False N/A mock_request N/A N/A mock Mock side_effect N/A N/A FakeResponse http://dummy/ 404 failure! N/A FakeResponse http://dummy/ 200 success! N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ retry_on_404 True N/A N/A N/A mock_request call_count 2 N/A N/A N/A resp status_code 200 N/A mock_request N/A N/A mock Mock return_value N/A FakeResponse http://dummy/ 404 failure! N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ retry_on_404 True N/A N/A N/A resp status_code 404 N/A N/A N/A mock_request call_count 4 N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ retry_on_404 False N/A N/A N/A resp status_code 404 N/A N/A N/A mock_request call_count 5 N/A test_retry_ssl N/A N/A N/A N/A N/A s N/A Scraper retry_attempts 5 retry_wait_seconds 0.001 raise_errors False N/A N/A N/A N/A mock patch object N/A requests Session request mock_sslerror N/A N/A N/A N/A pytest raises N/A N/A requests exceptions SSLError N/A N/A resp N/A N/A s get http://dummy/ retry_on_404 True N/A N/A N/A mock_sslerror call_count 1 N/A test_timeout N/A N/A N/A N/A N/A s N/A Scraper N/A N/A s timeout 0.001 N/A N/A N/A pytest raises N/A requests Timeout N/A N/A N/A N/A s get N/A HTTPBIN delay/1 N/A test_timeout_arg N/A N/A N/A N/A N/A s N/A Scraper N/A N/A N/A pytest raises N/A requests Timeout N/A N/A N/A N/A s get N/A HTTPBIN delay/1 timeout 0.001 N/A test_timeout_retry N/A N/A N/A N/A N/A count N/A side_effect N/A N/A N/A args kwargs N/A N/A count N/A N/A N/A FakeResponse http://dummy/ 200 success! N/A N/A N/A count append 1 N/A N/A N/A requests Timeout timed out :( N/A N/A mock_request N/A N/A mock Mock side_effect side_effect N/A s N/A Scraper retry_attempts 0 retry_wait_seconds 0.001 N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A N/A N/A pytest raises N/A requests Timeout N/A s get http://dummy/ N/A N/A N/A mock_request call_count 1 N/A N/A N/A mock_request reset_mock N/A count N/A N/A s N/A Scraper retry_attempts 2 retry_wait_seconds 0.001 N/A N/A N/A N/A mock patch object N/A requests Session request mock_request N/A N/A resp N/A N/A s get http://dummy/ N/A N/A N/A resp content success! N/A N/A N/A mock_request call_count 2 N/A test_disable_compression N/A N/A N/A N/A N/A s N/A Scraper N/A N/A s disable_compression True N/A data N/A N/A s get N/A HTTPBIN headers N/A djson N/A N/A data json N/A N/A compress N/A N/A djson N/A headers N/A Accept-Encoding N/A N/A gzip N/A N/A djson N/A headers N/A Accept-Encoding N/A N/A s disable_compression False N/A data N/A N/A s get N/A HTTPBIN headers N/A djson N/A N/A data json N/A N/A compress N/A N/A djson N/A headers N/A Accept-Encoding N/A N/A gzip N/A N/A djson N/A headers N/A Accept-Encoding N/A N/A N/A s headers N/A Accept-Encoding xyz N/A data N/A N/A s get N/A HTTPBIN headers N/A djson N/A N/A data json N/A N/A xyz N/A N/A djson N/A headers N/A Accept-Encoding N/A test_callable_headers N/A N/A N/A N/A N/A s N/A Scraper header_func N/A N/A N/A url N/A N/A X-Url url N/A data N/A N/A s get N/A HTTPBIN headers N/A N/A N/A N/A N/A N/A data json N/A headers N/A X-Url N/A HTTPBIN headers N/A data N/A N/A s get N/A HTTPBIN headers?shh N/A N/A N/A N/A N/A N/A data json N/A headers N/A X-Url N/A HTTPBIN headers?shh N/A test_headers_weirdness N/A N/A N/A N/A N/A s N/A Scraper N/A N/A s headers N/A accept application/json N/A data N/A N/A N/A N/A s get N/A HTTPBIN headers json N/A N/A N/A N/A data N/A headers N/A Accept application/json N/A s N/A Scraper N/A data N/A N/A N/A N/A s get N/A HTTPBIN headers headers N/A accept application/xml json N/A N/A N/A N/A data N/A headers N/A Accept application/xml N/A test_ftp_uses_urllib2 N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A urlopen N/A N/A mock Mock return_value N/A BytesIO ftp success! N/A N/A N/A mock patch scrapelib.urllib_urlopen urlopen N/A N/A r N/A N/A s get ftp://dummy/ N/A N/A N/A r status_code 200 N/A N/A N/A r content ftp success! N/A test_ftp_retries N/A N/A N/A N/A N/A count N/A side_effect N/A N/A N/A args kwargs N/A N/A count N/A N/A N/A BytesIO ftp success! N/A N/A N/A count append 1 N/A N/A urllib_URLError ftp failure! N/A N/A mock_urlopen N/A N/A mock Mock side_effect side_effect N/A N/A N/A mock patch scrapelib.urllib_urlopen mock_urlopen N/A N/A s N/A Scraper retry_attempts 2 retry_wait_seconds 0.001 N/A r N/A N/A s get ftp://dummy/ retry_on_404 True N/A N/A N/A r content ftp success! N/A N/A N/A mock_urlopen call_count 2 N/A count N/A N/A N/A N/A mock_urlopen reset_mock N/A N/A N/A mock patch scrapelib.urllib_urlopen mock_urlopen N/A N/A s N/A Scraper retry_attempts 0 retry_wait_seconds 0.001 N/A N/A N/A pytest raises FTPError N/A s get ftp://dummy/ retry_on_404 True N/A N/A N/A mock_urlopen call_count 1 N/A test_ftp_method_restrictions N/A N/A N/A N/A N/A s N/A Scraper requests_per_minute 0 N/A N/A N/A pytest raises HTTPMethodUnavailableError N/A s post ftp://dummy/ N/A test_basic_stats N/A N/A N/A N/A N/A s N/A Scraper N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://example.com N/A N/A N/A s get http://example.com N/A N/A N/A s get http://example.com N/A N/A N/A N/A s stats N/A total_requests 3 N/A N/A N/A N/A s stats N/A total_time 0 N/A N/A N/A N/A s stats N/A average_time N/A N/A N/A s stats N/A total_time 3 N/A three_time N/A N/A s stats N/A total_time N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://example.com N/A N/A N/A N/A s stats N/A total_requests 4 N/A N/A N/A N/A s stats N/A total_time three_time N/A N/A N/A N/A s stats N/A average_time N/A N/A N/A s stats N/A total_time 4 N/A test_reset_stats N/A N/A N/A N/A N/A s N/A Scraper N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://example.com N/A N/A N/A N/A s stats N/A total_requests 1 N/A N/A N/A s reset_stats N/A N/A N/A N/A s stats N/A total_requests 0 N/A N/A N/A N/A mock patch object N/A requests Session request mock_200 N/A N/A N/A N/A s get http://example.com N/A N/A N/A N/A s stats N/A total_requests 1 N/A