Module Expr Str ImportFrom alias Import alias Import alias Import alias Import alias ImportFrom alias Import alias identifier FunctionDef arguments args NameParam defaults body Expr Str Return SubscriptLoad AttributeLoad NameLoad attr Index Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body If CompareEq NameLoad NameLoad body Return NameLoad decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareEq Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Num body Raise NameLoad Str If AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr NameLoad If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr Str Assign AttributeStore AttributeLoad NameLoad attr attr Str orelse Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr Str Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr If CompareEq AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad orelse Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str TryExcept body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Num handlers ExceptHandler body Raise NameLoad Str Expr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num For NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice Num attr AttributeLoad NameLoad attr AugAssignAdd SubscriptStore NameLoad Index Call NameLoad NameLoad Num Assign NameStore Num For NameStore NameLoad body If CompareGtE Call NameLoad NameLoad Num body If BoolOpAnd CompareEq SubscriptLoad NameLoad Index Num Str CompareEq SubscriptLoad NameLoad Index Num Str body AugAssignAdd NameStore Num AugAssignAdd SubscriptStore NameLoad Index NameLoad Num If CompareGt NameLoad Num body Break Expr Call AttributeLoad NameLoad attr If BoolOpAnd CompareEq Call NameLoad NameLoad Num CompareEq Call NameLoad NameLoad Num body If CompareEq SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num body Return AttributeLoad NameLoad attr orelse If CompareEq SubscriptLoad NameLoad Index Num Num body Return AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad If BoolOpAnd CompareGt NameLoad Num CompareGt NameLoad Num body If CompareEq NameLoad NameLoad body Return AttributeLoad NameLoad attr orelse Return AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIs NameLoad NameLoad body Assign NameStore Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr Str attr Num TryExcept body Return Call AttributeLoad Call AttributeLoad NameLoad attr attr NameLoad handlers ExceptHandler body Raise NameLoad Str orelse Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If BoolOpOr UnaryOpNot AttributeLoad NameLoad attr CompareEq Call NameLoad AttributeLoad NameLoad attr Num body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str For NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr body Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Break orelse Print Str For NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Slice Num attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad Break Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num For NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body AugAssignAdd SubscriptStore NameLoad Index Call NameLoad NameLoad Num AugAssignAdd NameStore Num If CompareGt NameLoad Num body Break Expr Call AttributeLoad NameLoad attr If CompareEq Call NameLoad NameLoad Num body Return Str orelse Return Str decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Num For NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body AugAssignAdd NameStore Num Expr Call AttributeLoad NameLoad attr Return NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str Pass decorator_list
N/A N/A  Used to identify characteristics of a file
    contains:
      - FileTyper class
    todo:
      - explore details around quoting flags - they seem very inaccurate

    See the file "LICENSE" for the full license governing this code. 
    Copyright 2011 Ken Farmer
 __future__ division N/A fileinput N/A collections N/A csv N/A os pprint pprint N/A common comm get_quote_number N/A N/A quote_name N/A N/A N/A  used to help applications look up quote names typically provided by
        users.
     N/A N/A N/A csv __dict__ N/A N/A N/A quote_name upper N/A get_quote_name N/A N/A quote_number N/A N/A N/A  used to help applications look up quote numbers typically provided by
        users.
     N/A N/A key value N/A N/A N/A csv __dict__ items N/A N/A N/A value quote_number N/A N/A key N/A FileTyper N/A object N/A N/A  Determines type of file - mostly using csv.Sniffer()
        Populates public variables:
          - format_type
          - csv_quoting
          - dialect
          - record_cnt
     __init__ N/A N/A self fqfn delimiter rec_delimiter has_header quoting N/A None None None None N/A N/A   fqfn = fully qualified file name
         N/A N/A self _delimiter delimiter N/A N/A self _has_header has_header N/A N/A self _quoting quoting N/A N/A self fqfn fqfn N/A N/A self format_type None N/A N/A self fixed_length None N/A N/A self field_cnt None N/A N/A self record_cnt None N/A N/A self csv_quoting None N/A N/A self dialect None N/A analyze_file N/A N/A self N/A N/A N/A  analyzes a file to determine the structure of the file in terms
            of whether or it it is delimited, what the delimiter is, etc.
         N/A N/A N/A N/A N/A os path getsize N/A self fqfn 0 N/A N/A IOErrorEmptyFile Empty File N/A N/A self _delimiter N/A N/A N/A self dialect N/A csv Dialect N/A N/A N/A self dialect delimiter N/A self _delimiter N/A N/A N/A self dialect skipinitialspace False N/A N/A N/A self _quoting None N/A N/A N/A self _quoting N/A csv QUOTE_MINIMAL N/A N/A N/A self dialect quoting N/A self _quoting N/A N/A N/A self dialect quotechar " N/A N/A N/A self dialect lineterminator N/A N/A N/A self dialect N/A N/A self _get_dialect N/A N/A N/A self dialect lineterminator N/A N/A self _quoting N/A N/A self dialect quoting N/A N/A self _delimiter N/A N/A self dialect delimiter N/A N/A N/A N/A self dialect quoting N/A csv QUOTE_NONE N/A N/A N/A self csv_quoting False N/A N/A N/A self csv_quoting True N/A N/A self format_type N/A N/A self _get_format_type N/A N/A N/A self dialect has_header N/A N/A self _get_has_header N/A self _has_header N/A N/A self _has_header N/A N/A self dialect has_header N/A N/A self field_cnt N/A N/A self _get_field_cnt N/A N/A self record_cnt N/A N/A self _count_records N/A N/A self dialect N/A _get_dialect N/A N/A self N/A N/A N/A  gets the dialect for a file
            Uses the csv.Sniffer class 
            Then performs additional processing to try to improve accuracy of
            quoting.
         N/A csvfile N/A open N/A self fqfn rb N/A N/A N/A dialect N/A N/A N/A N/A csv Sniffer sniff N/A N/A csvfile read 50000 N/A N/A N/A N/A IOError could not analyse file - you may want to provide explicit csv delimiter, quoting, etc N/A N/A N/A csvfile close N/A N/A dialect quoting N/A N/A self _get_dialect_quoting dialect N/A dialect N/A _get_dialect_quoting N/A N/A self dialect N/A N/A N/A  Since Sniffer tends to default to QUOTE_MINIMAL we're going to try to
            get a more accurate guess.  In the event that there's an extremely
            consistent set of data that is either all quoted or not quoted at
            all we will return that appropriate value.
         N/A total_field_cnt N/A N/A collections defaultdict int N/A quoted_field_cnt N/A N/A collections defaultdict int N/A rec_cnt 0 N/A rec N/A N/A fileinput input N/A self fqfn N/A N/A fields N/A N/A N/A rec N/A -1 split N/A dialect delimiter N/A N/A total_field_cnt N/A N/A len fields 1 N/A quoted_cnt 0 N/A field fields N/A N/A N/A N/A len field 2 N/A N/A N/A N/A N/A field N/A 0 " N/A N/A field N/A -1 " N/A N/A quoted_cnt 1 N/A N/A quoted_field_cnt N/A quoted_cnt 1 N/A N/A rec_cnt 1000 N/A N/A N/A N/A N/A fileinput close N/A N/A N/A N/A len total_field_cnt 1 N/A N/A len quoted_field_cnt 1 N/A N/A N/A N/A total_field_cnt N/A 0 N/A quoted_field_cnt N/A 0 N/A N/A N/A csv QUOTE_ALL N/A N/A N/A N/A quoted_field_cnt N/A 0 0 N/A N/A N/A csv QUOTE_NONE N/A N/A common_field_key common_field_pct N/A N/A comm get_common_key total_field_cnt N/A N/A common_quoted_field_key common_quoted_field_pct N/A N/A comm get_common_key quoted_field_cnt N/A N/A N/A common_field_pct 0.95 N/A common_quoted_field_pct 0.95 N/A N/A N/A common_field_key common_quoted_field_key N/A N/A N/A csv QUOTE_ALL N/A N/A N/A csv QUOTE_NONE N/A N/A dialect quoting N/A _get_has_header N/A N/A self has_header N/A N/A N/A  if the has_header flag was already provided, then just return it
            back.
            Otherwise, figure out whether or not there's a header based on
            the first 50,000 bytes
         N/A N/A has_header None N/A N/A sample N/A N/A N/A open N/A self fqfn r read 50000 N/A N/A N/A N/A N/A N/A N/A csv Sniffer has_header sample N/A N/A N/A N/A IOError Could not complete header analysis.  It may help to provide explicit header info N/A N/A has_header N/A _get_field_cnt N/A N/A self N/A N/A N/A  determines the number of fields in the file.

            To do:  make it less naive - it currently assumes that all #recs
            will have the same number of fields.  It should be improved t 
            deal with bad data.
         N/A N/A N/A N/A self _delimiter N/A N/A len N/A self _delimiter 1 N/A N/A csvfile N/A open N/A self fqfn r N/A row N/A N/A csv reader csvfile N/A self dialect N/A N/A row_len N/A len row N/A N/A N/A csvfile close N/A N/A N/A file_type._get_field_cnt - DEPRECATED FUNCTIONALITY N/A rec N/A N/A fileinput input N/A self fqfn N/A N/A fields N/A N/A N/A rec N/A -1 split N/A self _delimiter N/A N/A N/A fileinput close N/A row_len N/A len fields N/A N/A row_len N/A _get_format_type N/A N/A self N/A N/A N/A  Determines format type based on whether or not all records
            are of the same length.

            Returns either 'csv' or 'fixed'
         N/A rec_length N/A N/A collections defaultdict int N/A rec_cnt 0 N/A rec N/A N/A fileinput input N/A self fqfn N/A N/A N/A rec_length N/A N/A len rec 1 N/A rec_cnt 1 N/A N/A rec_cnt 1000 N/A N/A N/A N/A N/A fileinput close N/A N/A N/A len rec_length 1 N/A N/A fixed N/A N/A csv N/A _count_records N/A N/A self N/A N/A N/A  Returns the number of records in the file
         N/A rec_cnt 0 N/A dummy N/A N/A fileinput input N/A self fqfn N/A N/A rec_cnt 1 N/A N/A N/A fileinput close N/A rec_cnt N/A N/A IOErrorEmptyFile N/A IOError N/A N/A Error due to empty file
     N/A N/A