Module FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Str Assign NameStore Str If BoolOpAnd AttributeLoad NameLoad attr CompareIn Str AttributeLoad NameLoad attr body Assign TupleStore NameStore NameStore Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Str If BoolOpAnd CompareEq NameLoad Str AttributeLoad NameLoad attr CompareGtE Call NameLoad AttributeLoad NameLoad attr Num body Assign NameStore Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Slice Num attr Assign NameStore Call AttributeLoad NameLoad attr Str If CompareNotEq NameLoad Num body Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str BinOpAdd NameLoad Num Return Call NameLoad SubscriptLoad NameLoad Slice BinOpAdd NameLoad Num NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Str Assign NameStore Str For NameStore Call AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr If Call AttributeLoad NameLoad attr Str body Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Str Index Num Assign NameStore Call AttributeLoad NameLoad attr Str Str orelse Assign NameStore NameLoad Return TupleLoad NameLoad NameLoad decorator_list
N/A get_content_type_encoding N/A N/A curi N/A N/A N/A 
    Determine the content encoding based on the `Content-Type` Header.

    `curi` is the :class:`CrawlUri`.
     N/A content_type text/plain N/A charset N/A N/A N/A curi rep_header N/A Content-Type N/A curi rep_header N/A N/A N/A content_type charset N/A extract_content_type_encoding N/A N/A curi rep_header N/A Content-Type N/A N/A N/A charset N/A curi content_body N/A N/A len N/A curi content_body 512 N/A N/A first_bytes N/A N/A N/A N/A curi content_body N/A 512 lower N/A ctypestart N/A N/A first_bytes find content-type N/A N/A ctypestart -1 N/A N/A ctypestart N/A N/A first_bytes find content=" ctypestart N/A ctypeend N/A N/A first_bytes find " N/A ctypestart 9 N/A N/A extract_content_type_encoding N/A first_bytes N/A N/A ctypestart 9 ctypeend N/A N/A content_type charset N/A extract_content_type_encoding N/A N/A content_type_string N/A N/A N/A 
    Extract the content type and encoding information.
     N/A charset N/A content_type N/A part N/A N/A content_type_string split ; N/A N/A part N/A N/A N/A N/A part strip lower N/A N/A N/A part startswith charset N/A N/A charset N/A N/A N/A part split = N/A 1 N/A charset N/A N/A charset replace - _ N/A N/A content_type part N/A N/A content_type charset N/A