Module Import alias Import alias ImportFrom alias ImportFrom alias alias ImportFrom alias alias Import alias ImportFrom alias ImportFrom alias Import alias Assign NameStore Call AttributeLoad NameLoad attr Str ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Return NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Dict Assign NameStore Dict Assign NameStore Dict Assign NameStore Dict Assign NameStore Dict Assign NameStore Dict FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Num Num Num body Expr Str If CompareEq NameLoad Num body Assign AttributeStore NameLoad attr Num orelse Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad If CompareEq NameLoad Num body Assign AttributeStore NameLoad attr Num orelse Assign NameStore Call NameLoad NameLoad Assign AttributeStore NameLoad attr BinOpDiv Num NameLoad If CompareEq NameLoad Num body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr orelse Assign AttributeStore NameLoad attr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call NameLoad ListComp Call NameLoad NameLoad comprehension NameStore Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore ListComp TupleLoad SubscriptLoad NameLoad Index Num Call NameLoad SubscriptLoad NameLoad Index Num comprehension NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Return Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad If CompareEq NameLoad Num body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad Num orelse Assign NameStore BinOpDiv Num Call NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIn NameLoad AttributeLoad NameLoad attr body Return SubscriptLoad AttributeLoad NameLoad attr Index NameLoad orelse Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str If CompareEq NameLoad Num body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad AttributeLoad NameLoad attr orelse Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIn NameLoad AttributeLoad NameLoad attr body Return SubscriptLoad AttributeLoad NameLoad attr Index NameLoad orelse Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad Str NameLoad NameLoad Str Num NameLoad NameLoad NameLoad body Expr Str If CompareIs NameLoad NameLoad body Assign NameStore Dict If CompareIsNot NameLoad NameLoad body If Call NameLoad NameLoad NameLoad body For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad orelse Expr Call NameLoad NameLoad If CompareEq Call AttributeLoad NameLoad attr Str body Assign SubscriptStore NameLoad Index Str Str If CompareIsNot NameLoad NameLoad body Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assign SubscriptStore NameLoad Index Str NameLoad If CompareIsNot NameLoad NameLoad body Assign SubscriptStore NameLoad Index Str NameLoad Assign NameStore Dict Str Str Str Str Str Str Str Str Str Call NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad Call NameLoad Assign NameStore SubscriptLoad Call NameLoad SubscriptLoad NameLoad Index Str Index Num If CompareNotIn NameLoad AttributeLoad NameLoad attr body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad ListLoad If NameLoad body Expr Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr Num NameLoad orelse Expr Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Return SubscriptLoad NameLoad Index Str decorator_list FunctionDef arguments args NameParam NameParam defaults body If CompareNotIn NameLoad AttributeLoad NameLoad attr body Return NameLoad If CompareIn NameLoad AttributeLoad NameLoad attr body If CompareIn NameLoad AttributeLoad NameLoad attr body If CompareLt BinOpSub Call AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad body Return NameLoad orelse If CompareLt BinOpSub Call AttributeLoad NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AttributeLoad NameLoad attr body Return NameLoad If CompareIn NameLoad AttributeLoad NameLoad attr body If CompareIn NameLoad AttributeLoad NameLoad attr body If CompareGt SubscriptLoad AttributeLoad NameLoad attr Index NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad body Return NameLoad orelse If CompareGt SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AttributeLoad NameLoad attr body Return NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body While BoolOpAnd CompareLt Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareGt Call AttributeLoad NameLoad attr Num body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore NameLoad For NameStore NameLoad body If CompareEq Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Num body Delete SubscriptDel AttributeLoad NameLoad attr Index NameLoad orelse If Call AttributeLoad NameLoad attr NameLoad body Assign NameStore NameLoad Assign NameStore Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str NameLoad Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad BinOpAdd Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Num Num If UnaryOpNot NameLoad body Break If CompareGt Call AttributeLoad NameLoad attr Num body Expr Call AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body AugAssignSub SubscriptStore AttributeLoad NameLoad attr Index NameLoad Num Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body AugAssignSub SubscriptStore AttributeLoad NameLoad attr Index NameLoad Num Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign TupleStore NameStore NameStore NameStore SubscriptLoad Call NameLoad SubscriptLoad NameLoad Index Str Slice Num Num Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str keyword SubscriptLoad NameLoad Index Str If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad Call NameLoad keyword SubscriptLoad NameLoad Index Str orelse Expr Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad keyword SubscriptLoad NameLoad Index Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Return Dict Str Str Str Str NameLoad AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body If BoolOpAnd Call NameLoad NameLoad Str CompareIsNot AttributeLoad NameLoad attr NameLoad body Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Return NameLoad decorator_list decorator_list
N/A N/A urllib N/A time twisted.internet.defer Deferred twisted.internet reactor ssl twisted.web.client HTTPClientFactory _parse N/A dateutil.parser unicodeconverter convertToUTF8 OpenSSL SSL N/A logging N/A LOGGER N/A N/A logging getLogger main AllCipherSSLClientContextFactory N/A N/A ssl ClientContextFactory N/A N/A A context factory for SSL clients that uses all ciphers. getContext N/A N/A self N/A N/A N/A context N/A N/A SSL Context N/A self method N/A N/A N/A context set_cipher_list ALL N/A context N/A N/A RequestQueuer N/A object N/A N/A 
    HTTP Request Queuer
     N/A pending_reqs N/A N/A last_req N/A N/A active_reqs N/A N/A min_req_interval_per_hosts N/A N/A max_reqs_per_hosts_per_sec N/A N/A max_simul_reqs_per_hosts N/A __init__ N/A N/A self max_simultaneous_requests max_requests_per_host_per_second max_simultaneous_requests_per_host N/A 50 1 5 N/A N/A 
        Set the maximum number of simultaneous requests for a particular host.
        
        **Keyword arguments:**
          * *max_simultaneous_requests* -- Maximum number of simultaneous
            requests RequestQueuer should make across all hosts. (Default 50)
          * *max_requests_per_host_per_second* -- Maximum number of requests 
            per host per second. If set to 1, RequestQueuer will not make more 
            than 1 request per second to a host. If set to 0, RequestQueuer 
            will not limit the request rate. Can be overridden for an 
            individual host using ``setHostMaxRequestsPerSecond()`` (Default 1)
          * *max_simultaneous_requests_per_host* -- Maximum number of 
            simultaneous requests per host. If set to 1, RequestQueuer will
            not make more than 1 simultaneous request per host. If set to 0,
            RequestQueuer will not limit the number of simultaneous requests.
            Can be overridden for an individual host using 
            ``setHostMaxSimultaneousRequests()`` (Default 5)
  
         N/A N/A max_simultaneous_requests 0 N/A N/A N/A self max_simul_reqs 100000 N/A N/A N/A self max_simul_reqs N/A int max_simultaneous_requests N/A N/A self max_reqs_per_host_per_sec max_requests_per_host_per_second N/A N/A max_requests_per_host_per_second 0 N/A N/A N/A self min_req_interval_per_host 0 N/A N/A max_req_per_host_per_sec N/A float max_requests_per_host_per_second N/A N/A self min_req_interval_per_host N/A 1.0 max_req_per_host_per_sec N/A N/A max_simultaneous_requests_per_host 0 N/A N/A N/A self max_simul_reqs_per_host N/A self max_simul_reqs N/A N/A N/A self max_simul_reqs_per_host N/A int max_simultaneous_requests_per_host N/A getPending N/A N/A self N/A N/A N/A 
        Return the number of pending requests.
         N/A N/A sum N/A N/A len x N/A x N/A N/A N/A self pending_reqs values N/A getActive N/A N/A self N/A N/A N/A 
        Return the number of active requests.
         N/A N/A sum N/A N/A N/A self active_reqs values N/A getActiveRequestsByHost N/A N/A self N/A N/A N/A 
        Return a dictionary of the number of active requests by host.
         N/A N/A self active_reqs N/A getPendingRequestsByHost N/A N/A self N/A N/A N/A 
        Return a dictionary of the number of pending requests by host.
         N/A reqs N/A N/A N/A x N/A 0 N/A len N/A x N/A 1 N/A x N/A N/A N/A self pending_reqs items N/A N/A dict reqs N/A setHostMaxRequestsPerSecond N/A N/A self host max_requests_per_second N/A N/A N/A 
        Set the maximum number of requests per second for a particular host.
        
        **Arguments:**
         * *host* -- Hostname. (Example, ``"google.com"``)
         * *max_requests_per_second* -- Maximum number of requests to the
           host per second. If set to 1, RequestQueuer will not make more 
           than 1 request per second to the host. If set to 0, RequestQueuer 
           will not limit the request rate to the host.
         N/A N/A N/A self max_reqs_per_hosts_per_sec N/A host max_requests_per_second N/A N/A max_requests_per_second 0 N/A N/A N/A N/A self min_req_interval_per_hosts N/A host 0 N/A N/A min_req_interval N/A 1.0 N/A float max_requests_per_second N/A N/A N/A self min_req_interval_per_hosts N/A host min_req_interval N/A getHostMaxRequestsPerSecond N/A N/A self host N/A N/A N/A 
        Get the maximum number of requests per second for a particular host.
        
        **Arguments:**
         * *host* -- Hostname. (Example, ``"google.com"``)
         N/A N/A host N/A self max_reqs_per_hosts_per_sec N/A N/A N/A N/A self max_reqs_per_hosts_per_sec N/A host N/A N/A N/A self max_reqs_per_host_per_sec N/A setHostMaxSimultaneousRequests N/A N/A self host max_simultaneous_requests N/A N/A N/A 
        Set the maximum number of simultaneous requests for a particular host.
        
        **Arguments:**
         * *host* -- Hostname. (Example, ``"google.com"``)
         * *max_simultaneous_requests* -- Maximum number of simultaneous
           requests to the host. If set to 1, RequestQueuer will not make
           more than 1 simultaneous request to the host. If set to 0,
           RequestQueuer will not limit the number of simultaneous requests.
         N/A N/A max_simultaneous_requests 0 N/A N/A N/A N/A self max_simul_reqs_per_hosts N/A host N/A self max_simul_reqs N/A N/A N/A N/A self max_simul_reqs_per_hosts N/A host N/A int max_simultaneous_requests N/A getHostMaxSimultaneousRequests N/A N/A self host N/A N/A N/A 
        Get the maximum number of simultaneous requests for a particular host.
        
        **Arguments:**
         * *host* -- Hostname. (Example, ``"google.com"``)
         N/A N/A host N/A self max_simul_reqs_per_hosts N/A N/A N/A N/A self max_simul_reqs_per_hosts N/A host N/A N/A N/A self max_simul_reqs_per_host N/A getPage N/A N/A self url last_modified etag method postdata headers agent timeout cookies follow_redirect prioritize N/A None None GET None None RequestQueuer 60 None True False N/A N/A 
        Make an HTTP Request.

        **Arguments:**
         * *url* -- URL for the request.
         
        **Keyword arguments:**
         * *last_modified* -- Last modified date string to send as a request 
           header. (Default ``None``)
         * *etag* -- Etag string to send as a request header. (Default 
           ``None``)
         * *method* -- HTTP request method. (Default ``'GET'``)
         * *postdata* -- Dictionary of strings to post with the request. 
           (Default ``None``)
         * *headers* -- Dictionary of strings to send as request headers. 
           (Default ``None``)
         * *agent* -- User agent to send with request. (Default 
           ``'RequestQueuer'``)
         * *timeout* -- Request timeout, in seconds. (Default ``60``)
         * *cookies* -- Dictionary of strings to send as request cookies. 
           (Default ``None``).
         * *follow_redirect* -- Boolean switch to follow HTTP redirects. 
           (Default ``True``)
         * *prioritize* -- Move this request to the front of the request 
           queue. (Default ``False``)         

         N/A N/A headers None N/A N/A headers N/A N/A N/A postdata None N/A N/A N/A isinstance postdata dict N/A N/A key postdata N/A N/A N/A postdata N/A key N/A convertToUTF8 N/A postdata N/A key N/A postdata N/A N/A urllib urlencode postdata N/A N/A N/A convertToUTF8 postdata N/A N/A N/A N/A method lower post N/A N/A N/A headers N/A content-type application/x-www-form-urlencoded N/A N/A last_modified None N/A N/A time_tuple N/A N/A N/A N/A N/A dateutil parser parse last_modified timetuple N/A time_string N/A N/A time strftime %a, %d %b %Y %T %z time_tuple N/A N/A headers N/A If-Modified-Since time_string N/A N/A etag None N/A N/A N/A headers N/A If-None-Match etag N/A req N/A url method postdata headers agent timeout cookies follow_redirect deferred N/A convertToUTF8 url method postdata headers agent timeout cookies follow_redirect N/A Deferred N/A host N/A N/A _parse N/A req N/A url N/A 1 N/A N/A host N/A self pending_reqs N/A N/A N/A N/A self pending_reqs N/A host N/A N/A prioritize N/A N/A N/A N/A N/A N/A self pending_reqs N/A host insert 0 req N/A N/A N/A N/A N/A N/A self pending_reqs N/A host append req N/A N/A N/A self _checkActive N/A N/A req N/A deferred N/A _hostRequestCheck N/A N/A self host N/A N/A N/A N/A host N/A self pending_reqs N/A N/A False N/A N/A host N/A self last_req N/A N/A N/A host N/A self min_req_interval_per_hosts N/A N/A N/A N/A N/A N/A time time N/A N/A self last_req N/A host N/A N/A self min_req_interval_per_hosts N/A host N/A N/A False N/A N/A N/A N/A N/A N/A time time N/A N/A self last_req N/A host N/A self min_req_interval_per_host N/A N/A False N/A N/A host N/A self active_reqs N/A N/A N/A host N/A self max_simul_reqs_per_hosts N/A N/A N/A N/A N/A self active_reqs N/A host N/A N/A self max_simul_reqs_per_hosts N/A host N/A N/A False N/A N/A N/A N/A N/A self active_reqs N/A host N/A self max_simul_reqs_per_host N/A N/A False N/A True N/A _checkActive N/A N/A self N/A N/A N/A N/A N/A N/A N/A self getActive N/A self max_simul_reqs N/A N/A N/A self getPending 0 N/A N/A hosts N/A N/A N/A self pending_reqs keys N/A dispatched_requests False N/A host hosts N/A N/A N/A N/A len N/A N/A self pending_reqs N/A host 0 N/A N/A N/A N/A self pending_reqs N/A host N/A N/A N/A N/A self _hostRequestCheck host N/A N/A dispatched_requests True N/A req N/A N/A N/A N/A self pending_reqs N/A host pop 0 N/A d N/A N/A self _getPage req N/A N/A N/A d addCallback N/A self _requestComplete N/A req N/A deferred host N/A N/A N/A d addErrback N/A self _requestError N/A req N/A deferred host N/A N/A N/A self last_req N/A host N/A N/A time time N/A N/A N/A self active_reqs N/A host N/A N/A N/A N/A self active_reqs get host 0 1 N/A N/A dispatched_requests N/A N/A N/A N/A N/A N/A self getPending 0 N/A N/A N/A N/A reactor callLater 0.1 N/A self _checkActive N/A _requestComplete N/A N/A self response deferred host N/A N/A N/A N/A N/A self active_reqs N/A host 1 N/A N/A N/A self _checkActive N/A N/A N/A deferred callback response N/A None N/A _requestError N/A N/A self error deferred host N/A N/A N/A N/A N/A self active_reqs N/A host 1 N/A N/A N/A self _checkActive N/A N/A N/A deferred errback error N/A None N/A _getPage N/A N/A self req N/A N/A N/A N/A scheme host port N/A N/A _parse N/A req N/A url N/A 0 3 N/A factory N/A HTTPClientFactory N/A req N/A url method N/A req N/A method postdata N/A req N/A postdata headers N/A req N/A headers agent N/A req N/A agent timeout N/A req N/A timeout cookies N/A req N/A cookies followRedirect N/A req N/A follow_redirect N/A N/A scheme https N/A N/A N/A N/A reactor connectSSL host port factory N/A AllCipherSSLClientContextFactory timeout N/A req N/A timeout N/A N/A N/A N/A reactor connectTCP host port factory timeout N/A req N/A timeout N/A N/A N/A N/A factory deferred addCallback N/A self _getPageComplete factory N/A N/A N/A N/A factory deferred addErrback N/A self _getPageError factory N/A N/A factory deferred N/A _getPageComplete N/A N/A self response factory N/A N/A N/A N/A response headers status message response N/A factory response_headers N/A int N/A factory status N/A factory message N/A _getPageError N/A N/A self error factory N/A N/A N/A N/A N/A hasattr factory response_headers N/A N/A factory response_headers None N/A N/A N/A N/A error value headers N/A factory response_headers N/A error N/A N/A