Module Import alias Import alias Import alias TryExcept body ImportFrom alias alias handlers ExceptHandler type NameLoad body ImportFrom alias alias Import alias TryExcept body ImportFrom alias handlers ExceptHandler type NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Expr Call AttributeLoad NameLoad attr Num ImportFrom alias alias alias alias alias Assign NameStore Str TryExcept body ImportFrom alias alias handlers ExceptHandler type NameLoad body ImportFrom alias alias Assign NameStore Dict Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Assign NameStore Dict Str Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore NameLoad Assign NameStore Call NameLoad NameLoad If CompareNotIn SubscriptLoad NameLoad Index Num ListLoad Str Str Str body Expr Call AttributeLoad NameLoad attr BinOpMod Str SubscriptLoad NameLoad Index Num Return If CompareEq SubscriptLoad NameLoad Index Num Str body Assign NameStore NameLoad If CompareGt Call NameLoad SubscriptLoad NameLoad Index Num Num body Assign NameStore SubscriptLoad NameLoad Index Num orelse Assign NameStore Str Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Num attr Str If CompareGt Call NameLoad NameLoad Num body Assign NameStore SubscriptLoad NameLoad Index Num orelse Assign NameStore NameLoad Assign NameStore SubscriptLoad NameLoad Index Num Assign NameStore SubscriptLoad NameLoad Index Num Return TupleLoad NameLoad NameLoad NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad For NameStore Call AttributeLoad NameLoad attr Str NameLoad body Assign NameStore SubscriptLoad NameLoad Slice Num Num Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad body Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad For NameStore Call AttributeLoad NameLoad attr Str NameLoad body Assign NameStore SubscriptLoad NameLoad Slice Num Num For NameStore NameLoad body If NameLoad body Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Assign NameStore Call AttributeLoad Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad NameLoad attr Str Str attr Str Str Expr Yield NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad NameLoad For NameStore Call NameLoad Call NameLoad NameLoad body For NameStore Call NameLoad Num body Assign NameStore SubscriptLoad NameLoad Slice Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad Expr Yield Call NameLoad Call AttributeLoad Str attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad For NameStore NameLoad body If Call AttributeLoad NameLoad attr NameLoad body AugAssignAdd NameStore SubscriptLoad NameLoad Index NameLoad orelse AugAssignAdd NameStore NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad For NameStore NameLoad body AugAssignAdd NameStore BinOpAdd NameLoad Str Return Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad Call NameLoad NameLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad NameLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad NameLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad NameLoad Str Str decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad NameLoad Str Str decorator_list ClassDef bases body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults Str Num NameLoad Num Str NameLoad Dict NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad If CompareIs NameLoad NameLoad body If NameLoad body Assign NameStore Num orelse Assign NameStore Num Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr NameLoad TryExcept body Assign AttributeStore NameLoad attr IfExp NameLoad Call AttributeLoad NameLoad attr NameLoad Call NameLoad handlers ExceptHandler type NameLoad name NameStore body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Assign AttributeStore NameLoad attr Call NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults Str NameLoad NameLoad NameLoad NameLoad NameLoad body Assign NameStore AttributeLoad NameLoad attr If NameLoad body AugAssignAdd AttributeStore NameLoad attr Num If CompareGtE AttributeLoad NameLoad attr Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore NameLoad orelse Assign AttributeStore NameLoad attr Num If CompareIs NameLoad NameLoad body Assign NameStore AttributeLoad NameLoad attr For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body If CompareIs NameLoad NameLoad body Assign NameStore Dict Assign SubscriptStore NameLoad Index NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad If CompareIsNot NameLoad NameLoad body Assign NameStore Call NameLoad Lambda arguments args NameParam defaults Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr orelse Assign NameStore Dict Assign NameStore Dict If UnaryOpNot CompareIn Str NameLoad body Assign SubscriptStore NameLoad Index Str Str If UnaryOpNot CompareIn Str NameLoad body Assign SubscriptStore NameLoad Index Str Str If UnaryOpNot CompareIn Str NameLoad body Assign SubscriptStore NameLoad Index Str Str Assign NameStore Call NameLoad ListLoad NameLoad NameLoad NameLoad If NameLoad body If CompareIn NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad NameLoad NameLoad Return SubscriptLoad AttributeLoad NameLoad attr Index NameLoad orelse Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad If NameLoad body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad If NameLoad body Assign TupleStore NameStore NameStore NameLoad If CompareIn AttributeLoad NameLoad attr ListLoad Num Num Num body If NameLoad body If Call AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Assign NameStore Call NameLoad NameLoad If CompareIsNot NameLoad NameLoad body Assign TupleStore NameStore NameStore NameStore NameStore NameStore NameLoad If UnaryOpNot NameLoad body Assign NameStore Num If CompareEq NameLoad Str body Assign NameStore AttributeLoad NameLoad attr If CompareIs NameLoad NameLoad body Assign NameStore AttributeLoad NameLoad attr If UnaryOpNot Call AttributeLoad NameLoad attr Str body Assign NameStore BinOpAdd Str NameLoad If CompareEq TupleLoad NameLoad NameLoad NameLoad TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad NameLoad keyword NameLoad orelse Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad orelse Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str Call AttributeLoad NameLoad attr Str Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Assign NameStore AttributeLoad NameLoad attr TryFinally body TryExcept body Assign TupleStore NameStore NameStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Assign NameStore Call NameLoad If CompareGt AttributeLoad NameLoad attr Num body Assign SubscriptStore NameLoad Index Str Num If BoolOpAnd CompareGtE AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr body Import alias identifier Assign SubscriptStore NameLoad Index Str Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad NameLoad NameLoad If CompareLtE AttributeLoad NameLoad attr Num body If CompareGt AttributeLoad NameLoad attr Num body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr TryExcept body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad keyword NameLoad handlers ExceptHandler type AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str AttributeLoad NameLoad attr Return AugAssignAdd AttributeStore NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Assign NameStore TupleLoad NameLoad NameLoad handlers ExceptHandler type TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore NameLoad finalbody Expr Call AttributeLoad AttributeLoad NameLoad attr attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad Num Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Assign NameStore Call NameLoad If CompareGt NameLoad NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Return Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad If CompareIs NameLoad NameLoad body Return Assign TupleStore NameStore NameStore NameLoad TryExcept body Assign NameStore Call NameLoad NameLoad handlers ExceptHandler body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Return Assign NameStore Call NameLoad Str For NameStore NameLoad body TryExcept body Assign NameStore SubscriptLoad NameLoad Index Str If CompareIsNot NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad If BoolOpAnd CompareNotEq SubscriptLoad NameLoad Index Num Str CompareNotEq AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Continue If CompareNotIn SubscriptLoad NameLoad Index Num ListLoad Str Str Str body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Continue Assign NameStore SubscriptLoad NameLoad Index Num If UnaryOpNot Call AttributeLoad NameLoad attr Str body Assign NameStore BinOpAdd Str NameLoad If CompareGt Call NameLoad SubscriptLoad NameLoad Index Num Num body Assign NameStore Call NameLoad TupleLoad Str Str NameLoad SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Return NameLoad If CompareNotIn NameLoad AttributeLoad NameLoad attr body Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpMod Str NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword BinOpAdd NameLoad Num keyword NameLoad If NameLoad body Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad If BoolOpOr UnaryOpNot AttributeLoad NameLoad attr UnaryOpNot AttributeLoad NameLoad attr body Raise Call NameLoad Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Str If CompareNotEq Call NameLoad NameLoad Num body Raise Call NameLoad Str TryExcept body If CompareEq AttributeLoad NameLoad attr Str body If CompareIs NameLoad NameLoad body Raise Call NameLoad Str Return Call NameLoad SubscriptLoad NameLoad Index Num Call NameLoad SubscriptLoad NameLoad Index Num orelse If CompareEq AttributeLoad NameLoad attr Str body Return Call NameLoad SubscriptLoad NameLoad Index Num Call NameLoad SubscriptLoad NameLoad Index Num orelse Raise Call NameLoad Str handlers ExceptHandler type NameLoad body Raise Call NameLoad Str decorator_list decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str If CompareNotEq Call NameLoad NameLoad Call NameLoad NameLoad body Return NameLoad If CompareEq NameLoad NameLoad body Return NameLoad For NameStore NameLoad body If CompareNotEq Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad body Return NameLoad Return NameLoad decorator_list
N/A N/A re N/A sys N/A socket N/A N/A urlparse urlparse urlunparse N/A N/A N/A ImportError N/A urllib.parse urlparse urlunparse N/A logging N/A N/A bs4 BeautifulSoup N/A N/A N/A ImportError N/A N/A N/A N/A N/A sys stderr write You need to get BeautifulSoup installed
 N/A N/A N/A N/A sys stderr write Do it now, as privileged user/root run: pip install beautifulsoup4
ow N/A N/A N/A sys exit 1 proxy NullProxy HttpProxy Socks5Proxy httplib socks N/A __license__ 
Copyright (c) 2016, {Sandro Gauci|Wendel G. Henrique}
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimer in the documentation
      and/or other materials provided with the distribution.
    * Neither the name of EnableSecurity or Trustwave nor the names of its contributors
      may be used to endorse or promote products derived from this software
      without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
 N/A N/A urllib quote unquote N/A N/A N/A ImportError N/A urllib.parse quote unquote N/A unicodemapping N/A / \ ' " > < # ! $ * @ . _ ( ) , % - ; : | & + = a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R s S t T u U v V w W x X y Y z Z 0 1 2 3 4 5 6 7 8 9 %u0020 %u2215 %u2215 %u02b9 %u0022 %u003e %u003c %uff03 %uff01 %uff04 %uff0a %u0040 %uff0e %uff3f %uff08 %uff09 %uff0c %u0025 %uff0d %uff1b %uff1a %uff5c %uff06 %uff0b %uff1d %uff41 %uff21 %uff42 %uff22 %uff43 %uff23 %uff44 %uff24 %uff45 %uff25 %uff46 %uff26 %uff47 %uff27 %uff48 %uff28 %uff49 %uff29 %uff4a %uff2a %uff4b %uff2b %uff4c %uff2c %uff4d %uff2d %uff4e %uff2e %uff4f %uff2f %uff50 %uff30 %uff51 %uff31 %uff52 %uff32 %uff53 %uff33 %uff54 %uff34 %uff55 %uff35 %uff56 %uff36 %uff57 %uff37 %uff58 %uff38 %uff59 %uff39 %uff5a %uff3a %uff10 %uff11 %uff12 %uff13 %uff14 %uff15 %uff16 %uff17 %uff18 %uff19 N/A homoglyphicmapping N/A ' %ca%bc oururlparse N/A N/A target N/A N/A N/A log N/A N/A logging getLogger urlparser N/A ssl False N/A o N/A urlparse target N/A N/A N/A o N/A 0 N/A http https N/A N/A N/A N/A log error N/A scheme %s not supported N/A o N/A 0 N/A N/A N/A N/A o N/A 0 https N/A N/A ssl True N/A N/A N/A len N/A o N/A 2 0 N/A N/A path N/A o N/A 2 N/A N/A path / N/A tmp N/A N/A N/A o N/A 1 split : N/A N/A N/A len tmp 1 N/A N/A port N/A tmp N/A 1 N/A N/A port None N/A hostname N/A tmp N/A 0 N/A query N/A o N/A 4 N/A N/A hostname port path query ssl N/A modifyurl N/A N/A path modfunc log N/A N/A N/A path path N/A N/A N/A log debug N/A path is currently %s path N/A m N/A N/A re findall (\[.*?\]) path N/A N/A ourstr N/A m N/A 1 -1 N/A newstr N/A modfunc ourstr N/A N/A N/A log debug N/A String was %s ourstr N/A N/A N/A log debug N/A String became %s newstr N/A path N/A N/A path replace m newstr N/A N/A N/A log debug N/A the path is now %s path N/A path N/A modifypath N/A N/A path newstrs log encode N/A True N/A N/A N/A N/A log debug N/A path is currently %s path N/A m N/A N/A re findall (\[.*?\]) path N/A N/A ourstr N/A m N/A 1 -1 N/A newstr newstrs N/A N/A encode N/A N/A newstr N/A quote newstr N/A N/A N/A log debug N/A String was %s ourstr N/A N/A N/A log debug N/A String became %s newstr N/A newpath N/A N/A N/A N/A N/A N/A path replace m newstr replace ] replace [ N/A N/A newpath N/A bruteforceascii N/A N/A ourstr N/A N/A N/A listourstr N/A list ourstr N/A pos N/A range N/A len ourstr N/A N/A i N/A range 256 N/A N/A newlistourstr N/A listourstr N/A N/A N/A newlistourstr N/A pos N/A chr i N/A N/A N/A quote N/A N/A join newlistourstr N/A unicodeurlencode N/A N/A ourstr N/A N/A N/A newstr N/A str N/A character ourstr N/A N/A N/A N/A unicodemapping has_key character N/A N/A newstr N/A unicodemapping N/A character N/A N/A newstr character N/A newstr N/A nullify N/A N/A ourstr N/A N/A N/A newstr N/A str N/A character ourstr N/A N/A newstr N/A character N/A N/A quote newstr N/A replacechars N/A N/A ourstr origchar newchar N/A N/A N/A newstr N/A N/A ourstr replace origchar newchar N/A newstr N/A nullifyspaces N/A N/A ourstr N/A N/A N/A N/A quote N/A replacechars ourstr N/A slashspaces N/A N/A ourstr N/A N/A N/A N/A replacechars ourstr / N/A tabifyspaces N/A N/A ourstr N/A N/A N/A N/A replacechars ourstr N/A crlfspaces N/A N/A ourstr N/A N/A N/A N/A replacechars ourstr N/A backslashquotes N/A N/A ourstr N/A N/A N/A N/A replacechars ourstr ' '' N/A waftoolsengine N/A N/A __init__ N/A N/A self target port ssl debuglevel path followredirect extraheaders proxy N/A www.microsoft.com 80 False 0 / True N/A False N/A N/A 
        target: the hostname or ip of the target server
        port: defaults to 80
        ssl: defaults to false
         N/A N/A self target target N/A N/A port None N/A N/A ssl N/A N/A port 443 N/A N/A port 80 N/A N/A self port N/A int port N/A N/A self ssl ssl N/A N/A self debuglevel debuglevel N/A N/A self cachedresponses N/A dict N/A N/A self requestnumber 0 N/A N/A self path path N/A N/A self redirectno 0 N/A N/A self followredirect followredirect N/A N/A self crawlpaths N/A list N/A N/A self extraheaders extraheaders N/A N/A N/A N/A self proxy N/A proxy N/A N/A self _parse_proxy proxy N/A NullProxy N/A N/A N/A Exception N/A e N/A N/A N/A N/A N/A self log critical N/A Proxy disabled: %s e N/A N/A self proxy N/A NullProxy N/A request N/A N/A self method path usecache cacheresponse headers comingfromredir N/A GET None True True None False N/A N/A followredirect N/A self followredirect N/A comingfromredir N/A N/A N/A self redirectno 1 N/A N/A N/A self redirectno 5 N/A N/A N/A N/A N/A self log error We received way too many redirects.. stopping that N/A followredirect False N/A N/A N/A self redirectno 0 N/A N/A path None N/A N/A path N/A self path N/A hdr N/A N/A N/A self extraheaders keys N/A N/A N/A headers None N/A N/A headers N/A N/A N/A headers N/A hdr N/A N/A self extraheaders N/A hdr N/A N/A headers None N/A N/A knownheaders N/A map N/A N/A N/A x N/A N/A N/A x lower N/A N/A headers keys N/A N/A knownheaders N/A N/A headers N/A N/A N/A N/A user-agent knownheaders N/A N/A N/A headers N/A User-Agent Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.0 N/A N/A N/A accept-charset knownheaders N/A N/A N/A headers N/A Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7 N/A N/A N/A accept knownheaders N/A N/A N/A headers N/A Accept */* N/A k N/A str N/A method path headers N/A usecache N/A N/A N/A k N/A N/A N/A self cachedresponses keys N/A N/A N/A N/A N/A self log debug N/A Using cached version of %s, %s N/A method path N/A N/A N/A self cachedresponses N/A k N/A N/A N/A N/A N/A self log debug N/A %s not found in %s N/A k N/A N/A N/A self cachedresponses keys N/A r N/A N/A self _request method path headers N/A cacheresponse N/A N/A N/A N/A self cachedresponses N/A k r N/A r N/A N/A N/A response responsebody r N/A N/A N/A response status N/A 301 302 307 N/A N/A followredirect N/A N/A N/A N/A response getheader location N/A N/A newloc N/A N/A response getheader location N/A N/A N/A N/A self log info N/A Redirected to %s newloc N/A pret N/A oururlparse newloc N/A N/A pret None N/A N/A N/A target port path query ssl pret N/A N/A port N/A N/A port 80 N/A N/A target N/A N/A target N/A self target N/A N/A port None N/A N/A port N/A self port N/A N/A N/A N/A path startswith / N/A N/A path N/A / path N/A N/A N/A target port ssl N/A N/A self target N/A self port ssl N/A N/A r N/A N/A self request method path usecache cacheresponse headers comingfromredir True N/A N/A N/A N/A N/A self log warn N/A Tried to redirect to a different server %s newloc N/A N/A N/A N/A N/A self log warn N/A %s is not a well formatted url N/A N/A response getheader location N/A r N/A _request N/A N/A self method path headers N/A N/A N/A original_socket N/A socket socket N/A N/A N/A N/A N/A N/A conn_factory connect_host connect_port query_path N/A N/A N/A self proxy prepare N/A self target N/A self port path N/A self ssl N/A params N/A dict N/A N/A N/A sys hexversion 33947648 N/A N/A N/A params N/A timeout 4 N/A N/A N/A N/A sys hexversion 34015488 N/A self ssl N/A N/A ssl ssllib N/A N/A params N/A context N/A N/A ssllib _create_unverified_context N/A h N/A conn_factory connect_host connect_port params N/A N/A N/A self debuglevel 10 N/A N/A N/A N/A self debuglevel 1 N/A N/A N/A N/A h set_debuglevel N/A self debuglevel N/A N/A N/A N/A N/A N/A self log info N/A Sending %s %s N/A method path N/A N/A N/A h request method query_path headers headers N/A N/A N/A N/A socket error N/A N/A N/A N/A N/A self log warn N/A Could not initialize connection to %s N/A self target N/A N/A N/A self requestnumber 1 N/A response N/A N/A h getresponse N/A responsebody N/A N/A response read N/A N/A N/A h close N/A r N/A response responsebody N/A N/A N/A N/A N/A socket error N/A socket timeout N/A httplib BadStatusLine N/A N/A N/A N/A N/A self log warn Hey.. they closed our connection! N/A r None N/A N/A N/A N/A N/A self proxy terminate N/A r N/A querycrawler N/A N/A self path curdepth maxdepth N/A None 0 1 N/A N/A N/A N/A N/A self log debug N/A Crawler is visiting %s path N/A localcrawlpaths N/A list N/A N/A curdepth maxdepth N/A N/A N/A N/A N/A self log info N/A maximum depth %s reached maxdepth N/A N/A r N/A N/A self request path path N/A N/A r None N/A N/A N/A N/A response responsebody r N/A N/A N/A soup N/A BeautifulSoup responsebody N/A N/A N/A N/A N/A N/A N/A self log warn could not parse the response body N/A N/A tags N/A soup a N/A tag tags N/A N/A N/A N/A href N/A tag N/A href N/A N/A href None N/A N/A tmpu N/A urlparse href N/A N/A N/A N/A tmpu N/A 1 N/A N/A self target N/A tmpu N/A 1 N/A N/A N/A N/A N/A self log debug N/A Ignoring link because it is not on the same site %s href N/A N/A N/A N/A tmpu N/A 0 N/A http https N/A N/A N/A N/A N/A self log debug N/A Ignoring link because it is not an http uri %s href N/A N/A path N/A tmpu N/A 2 N/A N/A N/A N/A path startswith / N/A N/A path N/A / path N/A N/A N/A len N/A tmpu N/A 4 0 N/A N/A location N/A urlunparse N/A path N/A tmpu N/A 3 N/A tmpu N/A 4 N/A N/A N/A N/A self log info N/A Found query %s location N/A href N/A N/A path N/A self crawlpaths N/A N/A href N/A unquote path N/A N/A N/A N/A self log debug N/A adding %s for crawling href N/A N/A N/A N/A self crawlpaths append href N/A N/A N/A localcrawlpaths append href N/A N/A N/A KeyError N/A N/A N/A nextpath localcrawlpaths N/A N/A r N/A N/A self querycrawler path nextpath curdepth N/A curdepth 1 maxdepth maxdepth N/A r N/A N/A r N/A _parse_proxy N/A N/A self proxy N/A N/A N/A parts N/A urlparse proxy N/A N/A N/A N/A parts scheme N/A N/A parts netloc N/A N/A N/A Exception Invalid proxy specified, scheme required N/A netloc N/A N/A N/A parts netloc split : N/A N/A N/A len netloc 2 N/A N/A N/A Exception Proxy port unspecified N/A N/A N/A N/A N/A parts scheme socks5 N/A N/A N/A socks None N/A N/A N/A Exception socks5 proxy requires PySocks N/A N/A Socks5Proxy N/A netloc N/A 0 N/A int N/A netloc N/A 1 N/A N/A N/A N/A parts scheme http N/A N/A N/A HttpProxy N/A netloc N/A 0 N/A int N/A netloc N/A 1 N/A N/A N/A Exception Unsupported proxy scheme N/A N/A N/A ValueError N/A N/A N/A Exception Invalid port number N/A N/A scrambledheader N/A N/A header N/A N/A N/A c connection N/A N/A N/A len header N/A len c N/A N/A False N/A N/A header c N/A N/A False N/A character c N/A N/A N/A N/A N/A c count character N/A N/A header count character N/A N/A False N/A True N/A