Module ImportFrom alias ImportFrom alias ImportFrom alias alias TryExcept body ImportFrom alias handlers ExceptHandler body Assign NameStore NameLoad ClassDef bases NameLoad body Expr Str Assign NameStore TupleLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str TupleLoad AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str TupleLoad AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str NameLoad NameLoad TupleLoad Num AttributeLoad NameLoad attr Str TupleLoad AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr NameLoad NameLoad FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body If BoolOpAnd CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareIsNot AttributeLoad NameLoad attr NameLoad CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Return Expr Call AttributeLoad NameLoad attr While NameLoad body Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr If CompareEq NameLoad AttributeLoad NameLoad attr body Break If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Dict Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Dict Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Num body If CompareEq NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr Dict Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr NameLoad orelse Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If BoolOpAnd CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareIsNot AttributeLoad NameLoad attr NameLoad CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr NameLoad TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Return Expr Call AttributeLoad NameLoad attr Str If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr If CompareNotEq AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults body Return decorator_list decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore ListComp BinOpMod Str TupleLoad NameLoad NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Return BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad Str attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Return BoolOpAnd Call NameLoad NameLoad AttributeLoad NameLoad attr CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Return UnaryOpNot CompareEq NameLoad NameLoad decorator_list decorator_list
N/A thrift.Thrift * thrift.transport TTransport thrift.protocol TBinaryProtocol TProtocol N/A N/A thrift.protocol fastbinary N/A N/A N/A N/A fastbinary None CrawlUri N/A object N/A N/A 
  The main strcut for CrawlUris.

  This contains some metadata and if possible the saved web page.

  Attributes:
   - url
   - effective_url
   - current_priority
   - begin_processing
   - end_processing
   - req_header
   - rep_header
   - content_body
   - status_code
   - req_time
   - queue_time
   - optional_vars
   N/A thrift_spec N/A None N/A 1 N/A TType STRING url None None N/A 2 N/A TType STRING effective_url None None N/A 3 N/A TType I16 current_priority None None N/A 4 N/A TType I64 begin_processing None None N/A 5 N/A TType I64 end_processing None None N/A 6 N/A TType MAP req_header N/A N/A TType STRING None N/A TType STRING None None N/A 7 N/A TType MAP rep_header N/A N/A TType STRING None N/A TType STRING None None N/A 8 N/A TType STRING content_body None None N/A 9 N/A TType I16 status_code None None N/A 10 N/A TType DOUBLE req_time None None N/A 11 N/A TType DOUBLE queue_time None None N/A 12 N/A TType MAP optional_vars N/A N/A TType STRING None N/A TType STRING None None __init__ N/A N/A self url effective_url current_priority begin_processing end_processing req_header rep_header content_body status_code req_time queue_time optional_vars N/A None None None None None None None None None None None None N/A N/A N/A self url url N/A N/A self effective_url effective_url N/A N/A self current_priority current_priority N/A N/A self begin_processing begin_processing N/A N/A self end_processing end_processing N/A N/A self req_header req_header N/A N/A self rep_header rep_header N/A N/A self content_body content_body N/A N/A self status_code status_code N/A N/A self req_time req_time N/A N/A self queue_time queue_time N/A N/A self optional_vars optional_vars N/A read N/A N/A self iprot N/A N/A N/A N/A N/A N/A iprot __class__ N/A TBinaryProtocol TBinaryProtocolAccelerated N/A isinstance N/A iprot trans N/A TTransport CReadableTransport N/A N/A self thrift_spec None N/A fastbinary None N/A N/A N/A N/A fastbinary decode_binary self N/A iprot trans N/A N/A self __class__ N/A self thrift_spec N/A N/A N/A N/A iprot readStructBegin N/A True N/A N/A N/A fname ftype fid N/A N/A iprot readFieldBegin N/A N/A ftype N/A TType STOP N/A N/A N/A N/A fid 1 N/A N/A N/A ftype N/A TType STRING N/A N/A N/A self url N/A N/A iprot readString N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 2 N/A N/A N/A ftype N/A TType STRING N/A N/A N/A self effective_url N/A N/A iprot readString N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 3 N/A N/A N/A ftype N/A TType I16 N/A N/A N/A self current_priority N/A N/A iprot readI16 N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 4 N/A N/A N/A ftype N/A TType I64 N/A N/A N/A self begin_processing N/A N/A iprot readI64 N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 5 N/A N/A N/A ftype N/A TType I64 N/A N/A N/A self end_processing N/A N/A iprot readI64 N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 6 N/A N/A N/A ftype N/A TType MAP N/A N/A N/A self req_header N/A N/A N/A _ktype1 _vtype2 _size0 N/A N/A iprot readMapBegin N/A _i4 N/A xrange _size0 N/A N/A _key5 N/A N/A iprot readString N/A _val6 N/A N/A iprot readString N/A N/A N/A self req_header N/A _key5 _val6 N/A N/A N/A iprot readMapEnd N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 7 N/A N/A N/A ftype N/A TType MAP N/A N/A N/A self rep_header N/A N/A N/A _ktype8 _vtype9 _size7 N/A N/A iprot readMapBegin N/A _i11 N/A xrange _size7 N/A N/A _key12 N/A N/A iprot readString N/A _val13 N/A N/A iprot readString N/A N/A N/A self rep_header N/A _key12 _val13 N/A N/A N/A iprot readMapEnd N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 8 N/A N/A N/A ftype N/A TType STRING N/A N/A N/A self content_body N/A N/A iprot readString N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 9 N/A N/A N/A ftype N/A TType I16 N/A N/A N/A self status_code N/A N/A iprot readI16 N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 10 N/A N/A N/A ftype N/A TType DOUBLE N/A N/A N/A self req_time N/A N/A iprot readDouble N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 11 N/A N/A N/A ftype N/A TType DOUBLE N/A N/A N/A self queue_time N/A N/A iprot readDouble N/A N/A N/A N/A iprot skip ftype N/A N/A N/A fid 12 N/A N/A N/A ftype N/A TType MAP N/A N/A N/A self optional_vars N/A N/A N/A _ktype15 _vtype16 _size14 N/A N/A iprot readMapBegin N/A _i18 N/A xrange _size14 N/A N/A _key19 N/A N/A iprot readString N/A _val20 N/A N/A iprot readString N/A N/A N/A self optional_vars N/A _key19 _val20 N/A N/A N/A iprot readMapEnd N/A N/A N/A N/A iprot skip ftype N/A N/A N/A N/A iprot skip ftype N/A N/A N/A iprot readFieldEnd N/A N/A N/A iprot readStructEnd N/A write N/A N/A self oprot N/A N/A N/A N/A N/A N/A oprot __class__ N/A TBinaryProtocol TBinaryProtocolAccelerated N/A N/A self thrift_spec None N/A fastbinary None N/A N/A N/A N/A N/A oprot trans write N/A N/A fastbinary encode_binary self N/A N/A self __class__ N/A self thrift_spec N/A N/A N/A N/A oprot writeStructBegin CrawlUri N/A N/A N/A self url None N/A N/A N/A N/A oprot writeFieldBegin url N/A TType STRING 1 N/A N/A N/A oprot writeString N/A self url N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self effective_url None N/A N/A N/A N/A oprot writeFieldBegin effective_url N/A TType STRING 2 N/A N/A N/A oprot writeString N/A self effective_url N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self current_priority None N/A N/A N/A N/A oprot writeFieldBegin current_priority N/A TType I16 3 N/A N/A N/A oprot writeI16 N/A self current_priority N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self begin_processing None N/A N/A N/A N/A oprot writeFieldBegin begin_processing N/A TType I64 4 N/A N/A N/A oprot writeI64 N/A self begin_processing N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self end_processing None N/A N/A N/A N/A oprot writeFieldBegin end_processing N/A TType I64 5 N/A N/A N/A oprot writeI64 N/A self end_processing N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self req_header None N/A N/A N/A N/A oprot writeFieldBegin req_header N/A TType MAP 6 N/A N/A N/A oprot writeMapBegin N/A TType STRING N/A TType STRING N/A len N/A self req_header N/A N/A kiter21 viter22 N/A N/A N/A self req_header items N/A N/A N/A N/A oprot writeString kiter21 N/A N/A N/A oprot writeString viter22 N/A N/A N/A oprot writeMapEnd N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self rep_header None N/A N/A N/A N/A oprot writeFieldBegin rep_header N/A TType MAP 7 N/A N/A N/A oprot writeMapBegin N/A TType STRING N/A TType STRING N/A len N/A self rep_header N/A N/A kiter23 viter24 N/A N/A N/A self rep_header items N/A N/A N/A N/A oprot writeString kiter23 N/A N/A N/A oprot writeString viter24 N/A N/A N/A oprot writeMapEnd N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self content_body None N/A N/A N/A N/A oprot writeFieldBegin content_body N/A TType STRING 8 N/A N/A N/A oprot writeString N/A self content_body N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self status_code None N/A N/A N/A N/A oprot writeFieldBegin status_code N/A TType I16 9 N/A N/A N/A oprot writeI16 N/A self status_code N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self req_time None N/A N/A N/A N/A oprot writeFieldBegin req_time N/A TType DOUBLE 10 N/A N/A N/A oprot writeDouble N/A self req_time N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self queue_time None N/A N/A N/A N/A oprot writeFieldBegin queue_time N/A TType DOUBLE 11 N/A N/A N/A oprot writeDouble N/A self queue_time N/A N/A N/A oprot writeFieldEnd N/A N/A N/A self optional_vars None N/A N/A N/A N/A oprot writeFieldBegin optional_vars N/A TType MAP 12 N/A N/A N/A oprot writeMapBegin N/A TType STRING N/A TType STRING N/A len N/A self optional_vars N/A N/A kiter25 viter26 N/A N/A N/A self optional_vars items N/A N/A N/A N/A oprot writeString kiter25 N/A N/A N/A oprot writeString viter26 N/A N/A N/A oprot writeMapEnd N/A N/A N/A oprot writeFieldEnd N/A N/A N/A oprot writeFieldStop N/A N/A N/A oprot writeStructEnd validate N/A N/A self N/A N/A N/A N/A N/A __repr__ N/A N/A self N/A N/A N/A L N/A N/A %s=%r N/A key value N/A N/A key value N/A N/A N/A self __dict__ iteritems N/A N/A %s(%s) N/A N/A N/A self __class__ __name__ N/A N/A ,  join L N/A __eq__ N/A N/A self other N/A N/A N/A N/A N/A isinstance other N/A self __class__ N/A N/A self __dict__ N/A other __dict__ N/A __ne__ N/A N/A self other N/A N/A N/A N/A N/A self other N/A N/A