Package: wq.io
wq.iois a Pythonic library for consuming
(input), iterating over, and generating (output) external data resources
in various formats. wq.io facilitates interoperability between thewq
frameworkand other systems and formats.wq.io isdesigned to be customized,
with abase classand modular mixin
classes that handleloading,parsing, andmappingexternal data to a convenient
API.Somewhat coincidentally,https://wq.iois also the URL for the
website describing the wq framework as a whole. The documentation
for wq.io (the library) is available on wq.io (the website) athttps://wq.io/wq.io.Getting Started# Recommended: create virtual environment
# python3 -m venv venv
# . venv/bin/activate# Install entire wq suite (recommended)pipinstallwq# Install only wq.iopipinstallwq.io# To enable wq.io's GIS supportpipinstallgeopandas# includes Shapely & Fiona# To enable wq.io's Excel write supportpipinstallxlwt# xls supportpipinstallxlsxwriter# xlsx support
# (xls/xlsx read support is enabled by default)Seethe wq documentationfor more information.Featureswq.io provides a general purpose API for loading, iterating over, and
writing tabular datasets. The basic idea is to avoid needing to remember
the unique usage of e.g.csv,xlrd, orxml.etreeevery time one needs to work with external data. Instead, wq.io
abstracts these libraries into a consistent interface that works as aniterableofnamedtuples.
Whenever possible, the field names for a dataset are automatically
determined from the source file, e.g. the column headers in an Excel
spreadsheet.fromwq.ioimportExcelFileIOdata=ExcelFileIO(filename='example.xls')forrowindata:print(row.name,row.date)wq.io provides a number of built-in classes like the above, including aCsvFileIO,XmlFileIO, andJsonFileIO. There is also a
convenience function,load_file(), that attempts to automatically
determine which class to use for a given file.fromwq.ioimportload_filedata=load_file('example.csv')forrowindata:print(row.name,row.date)All of the included*FileIOclasses support both reading and writing
to external files, though write support for Excel files requires
additional libraries (xlwtandxlsxwriter) that aren’t listed
as dependencies.Network Clientwq.io also provides network-capable equivalents of each of the above
classes, to facilitate loading data from third party webservices.fromwq.ioimportJsonNetIOclassWebServiceIO(JsonNetIO):url="http://example.com/api"data=WebServiceIO(params={'type':'all'})forrowindata:print(row.timestamp,row.value)The powerfulrequestslibrary is used
internally to load data over HTTP.Pandas AnalysisWhenPandasis installed, theas_dataframe()method on wq.io classes can be used to create aDataFrame,
enabling more extensive analysis possibilities.instance=WebServiceIO(params={'type':'all'})df=instance.as_dataframe()print(df.value.mean())GIS SupportWhenFionaandShapelyare installed, wq.io
can also open and create shapefiles and other OGR-compatible geographic
data formats.fromwq.ioimportShapeIOdata=ShapeIO(filename='sites.shp')forid,siteindata.items():print(id,site.geometry.wkt)Extending wq.ioEachIOclass is composed of mixin classes
(loaders,parsers, andmappers) that handle the various steps
of the process. By extending these mixin or the pre-mixed classes above,
it is straightforward toextend wq.ioto support arbitrary formats. Theclimata
libraryprovides a number of
examples of customIOclasses for loading climate and hydrology
data.