Package: get-reader
get-readerThis module provides aget_reader()function that returns reader
objects similar to those returned bycsv.reader(). This package:reduces common boilerplate code for handling files and reading
recordsreads data from CSV, pandas, SQL connections, MS Excel, DBF, and squintprovides a single interface across Python versions (including
seamless Unicode-aware CSV support for Python 2)is easy to incorporate into your own projects:has no hard dependenciesruns on Python 2.6, 2.7, 3.2 through 3.8, PyPy, PyPy3, and Jythonis freely available under the Apache License, version 2can be easily vendored directly into your codebase if you don't
want to include it as a dependencyOpen a UTF-8 encoded CSV:fromget_readerimportget_readerreader=get_reader('myfile.csv')forrowinreader:print(', '.join(row))In the above example, file handling is managed automatically by the
reader object. The file is automatically closed when the iterator is
exhausted or when the object is deleted. It also handles Unicode in
Python 2 without changes.Open a Latin-1 (ISO-8859-1) encoded CSV file:reader=get_reader('myfile.csv',encoding='latin-1')forrowinreader:print(', '.join(row))Use the reader as a context manager:withget_reader('myfile.csv')asreader:forrowinreader:print(', '.join(row))In this example,readerautomatically closes its internal file object
when exiting thewithblock even if the for-loop doesn't finish
exhausting thereader.Access other data sources:# From a pandas DataFrame, Series, Index, or MultiIndex.df=pd.DataFrame([...])reader=get_reader(df)# requires pandas# From a database connection.connection=...reader=get_reader(connection,'SELECT col1, col2 FROM mytable;')# From an Excel file--must install with 'excel' option.reader=get_reader('myfile.xlsx')# From a DBF file--must install with 'dbf' option.reader=get_reader('myfile.dbf')# From a squint Select, Query, or Result.select=...reader=get_reader(select({'col1':'col2'}).sum())Call constructors directly to override auto-detect behavior:# Specify tab-delimited data from a text file.reader=get_reader.from_csv('myfile.dat',delimiter='\t')InstallTheget_readermodule has no hard dependencies; is tested on
Python 2.6, 2.7, 3.2 through 3.8, PyPy, PyPy3, and Jython; and
is freely available under the Apache License, version 2.You can installget_readerusingpip:pipinstallget_readerTo install optional support for MS Excel and DBF files (dBase,
Foxpro, etc.), use the following:pipinstallget_reader[excel,dbf]Python 2 Support StatementWhile official support for Python 2 ends on January 1, 2020, this
project will continue to support older versions as long as the
existing ecosystem provides the ability to run automated tests
on those older versions.Referenceget_reader(obj, *args, **kwds)Return aReaderobject which will iterate over records in
the givenobj—like acsv.reader(). The givenobjmay
be one of the following:CSV file (string path or file object)iterable of dictionary rowsdatabase connection (should be DBAPI2 compatible)pandas DataFrame, Series, Index, or MultiIndexsquint Select, Query, or ResultIf optional extras are installed,objmay also be:MS Excel file pathDBF file pathWhenobjis a file path, theReadercontains a file object
that is handled internally. When given a file-likeobj(rather
than a path), users are responsible for properly closing this
file themselves.The givenobjis checked against supported types and
automatically passed to the appropriate constructor if a match is
found. Ifobjis a string, it is treated as a file path whose
extension determines its content type. Any *argsand **kwdsare passed along to the matching constructor:fromget_readerimportget_reader# CSV file.reader=get_reader('myfile.csv')# Database connection.connection=...reader=get_reader(connection,'SELECT col1, col2 FROM mytable;')# Pandas DataFrame.df=pd.DataFrame([...])reader=get_reader(df)# Excel file.reader=get_reader('myfile.xlsx',worksheet='Sheet2')If theobjtype cannot be determined automatically, users can
call the constructor methods directly.Constructor Methodsget_reader.from_csv(csvfile,encoding='utf-8',dialect='excel', **kwds)Return a reader object which will iterate over lines in the
givencsvfile. Thecsvfilecan be a string (treated as a
file path) or any object which supports the iterator protocol
and returns a string each time its__next__()method is
called—file objects and list objects are both suitable. Ifcsvfileis a file object, it should be opened withnewline=''.fromget_readerimportget_readerreader=get_reader.from_csv('myfile.tab',delimiter='\t')Using explicit file handling:fromget_readerimportget_readerwithopen('myfile.csv')ascsvfile:reader=get_reader.from_csv(fh)get_reader.from_dicts(records,fieldnames=None)Return a reader object which will iterate over the given
dictionaryrecords. This can be thought of as converting acsv.DictReader()into a plain, non-dictionarycsv.reader().fromget_readerimportget_readerdictrows=[{'A':1,'B':'x'},{'A':2,'B':'y'},]reader=get_reader.from_dicts(dictrows)This method assumes that record contents are consistent. If the first
record is a dictionary, it is assumed that all following records will
be dictionaries with matching keys.get_reader.from_sql(connection,table_or_query)Return a reader object which will iterate over the records
from a given database table or over the records returned from
a SQL query. Theconnectionshould be a DBAPI2 compatible
database connection andtable_or_querymust be a string
with a table name or a SQL query.Read records from a specified table:fromget_readerimportget_readerconnection=...reader=get_reader.from_sql(connection,'mytable')Read records from the results of a SQL query:reader=get_reader.from_sql(connection,'SELECT col1, col2 FROM mytable;')get_reader.from_excel(path,worksheet=0)Return a reader object which will iterate over lines in the given
Excel worksheet. Thepathmust specify an XLSX or XLS file andworksheetmust specify the index or name of the worksheet to
load (defaults to the first worksheet).Load first worksheet:fromget_readerimportget_readerreader=get_reader.from_excel('mydata.xlsx')Specific worksheets can be loaded by name (a string) or index
(an integer):reader=get_reader.from_excel('mydata.xlsx','Sheet 2')get_reader.from_pandas(obj,index=True)Return a reader object which will iterate over records in
a pandasDataFrame,Series,IndexorMultiIndex.importpandasaspdfromget_readerimportget_readerdf=pd.DataFrame(...)reader=get_reader.from_pandas(df)get_reader.from_dbf(filename,encoding=None, **kwds)Return a reader object which will iterate over lines in the given
DBF file (from dBase, FoxPro, etc.).fromget_readerimportget_readerreader=get_reader.from_dbf('myfile.dbf')get_reader.from_squint(obj,fieldnames=None)Return a reader object which will iterate over the records returned
from a squintSelect,Query, orResult. If thefieldnamesargument is not provided, this function tries to construct names
using the values from the underlying object.importsquintfromget_readerimportget_readerselect=squint.Select(...)reader=get_reader.from_squint(select)classReader(iterable,closefunc=<no value>)An iterator which will produce rows from the giveniterable. The
giveniterableshould produce non-string sequences. An optionalclosefuncmay be provided to close associated resources (files,
database cursors, etc.) once the reader is no longer needed—it will
be automatically called when:the iterable is exhaustedexiting awithstatement (if used as a context manager)the Reader is garbage collectedReader.close()Closes any associated resources (callsclosefuncearly):fromget_readerimportReaderreader=Reader(...,closefunc=...)reader.close()# <- Explicitly close resources.If the resources have already been closed, this method passes
without error.classReaderLike()An abstract class that can be used for type checking. Objects
will test asReaderLikeif they are one of the following:instance of theReaderclassobject returned bycsv.reader()non-exhaustible iterable that produces non-string sequencesSee the following examples:>>>isinstance(get_reader(csvfile),ReaderLike)True>>>isinstance(csv.reader(csvfile),ReaderLike)True>>>list_of_lists=[['col1','col2'],['a','b']]>>>isinstance(list_of_lists,ReaderLike)True>>>list_of_strings=['col1,col2','a,b']>>>isinstance(list_of_strings,ReaderLike)False>>>list_of_sets=[{'col1','col2'},{'a','b'}]>>>isinstance(list_of_sets,ReaderLike)FalseFreely licensed under the Apache License, Version 2.0(C) Copyright 2018 – 2019 Shawn Brown.