Package: pd2hts
===========================================================================pd2hts - Read and write Pandas objects from/to Hydrognomon timeseries files===========================================================================.. image:: https://travis-ci.org/aptiko/pd2hts.svg?branch=master:alt: Build button:target: https://travis-ci.org/aptiko/pd2htsRequirements============Python 3, PandasReference=========``pd2hts`` contains the following functions:* ``read(f, start_date=None, end_date=None)`` reads filelike object``f`` that contains a time series in text format, and returns a pandasDataFrame with date index, value, and flags. There must be no newlinetranslation in ``f`` (open it with ``open(..., newline='\n')``. If``start_date`` and/or ``end_date`` are specified, it skips rowsoutside the range.* ``read_file(f)`` reads filelike object ``f`` that contains a timeseries in file format, and returns a pandas DataFrame which also hasattributes ``unit``, ``title``, ``comment``, ``time_zone``,``time_step``, ``timestamp_rounding``, ``timestamp_offset``,``interval_type``, ``variable``, ``precision`` and ``location``. Forthe meaning of these attributes, see section "File format" below. Allthese attributes are informational; they aren't otherwise taken intoaccount in the pandas object.In particular, ``time_step`` and the other time-step-relatedattributes don't necessarily mean that the pandas object will have arelated frequency. In fact, raw time series may be irregular butactually have a time step. For example, a ten-minute time series mightend in :10, :20, etc., but at some point there might be anirregularity and it could continue with :31, :41, etc. Strictlyspeaking, such a time series has an irregular step. However, whenstored in a database, specifying that its time step is ten minutes(because that's what it is, ten minutes with irregularities) can helppeople who browse or search the database contents.The ``location`` attribute is a dictionary that has items``abscissa``, ``ordinate``, ``srid``, ``altitude``, and ``asrid``.* ``write(pd, f)`` writes pandas DataFrame ``pd``, which should be atime series with value and flags, to filelike object ``f``, in textformat. In accordance with the :ref:`text format specification<textformat>`, time series are written using the CR-LF sequence toterminate lines.  In order to produce fully compliant files, careshould be taken that ``f``, or any subsequent operations on ``f``, donot perform text translation; otherwise, it may result in lines beingterminated with CR-CR-LF. If ``f`` is a file, it should have beenopened in binary mode.* ``write_file(pd, f, version=4)`` is like ``write()`` but uses fileformat. If ``pd`` has any of the extra attributes mentioned in``read_file()``, these are written to the file.Formats=======There are two formats: the *text format* is generic text format, withoutmetadata; the *file format* is like the text format, but additionallycontains headers with metadata... _textformat:Text format-----------The text format for a time series is us-ascii, one line per record,like this:2006-12-23 18:34,18.2,RANGEThe three fields are comma-separated and must always exist.  In the datefield, the time may be missing. The character that separates the datefrom the time may be either a space or a lower case ``t``, or a capital``T`` (this module produces text format using a space as date separator,but can read text format that uses ``t`` or ``T``). The second fieldalways uses a dot as the decimal separator and may be empty.  The thirdfield is usually empty but may contain a list of space-separated flags.The line separator should be the CR-LF sequence used in MS-DOS andWindows systems. Code that produces text format should always use CR-LFto end lines, but code that reads text format should be able to alsoread lines that end in LF only, as well as CR-CR-LF (for reasonsexplained in the ``write()`` function above).In order to improve performance in file writes, the maximum length ofeach time series record line is limited to 255 characters.Flags should be encoded in ASCII; there must be no characters withcode greater than 127... _fileformat:File format-----------The file format is like this::Version=2Title=My timeseriesUnit=°C2006-12-23 18:34,18.2,RANGE2006-12-23 18:44,18.3,In other words, the file format consists of a header that specifiesparameters in the form ``Parameter=Value``, followed by a blank line,followed by the timeseries in text format. The same conventions for lineterminators apply here as for the text format. The encoding of theheader section is UTF-8.Client and server software should recognize UTF-8 files with or withoutUTF-8 BOM (Byte Order Mark) in the begining of file.  Writes may or maynot include the BOM, according OS. (Usually Windows software attachesthe BOM at the beginning of the file).Parameter names are case insensitive.  There may be white space oneither side of the equal sign, which is ignored. Trailing white space onthe line is also ignored. A second equal sign is considered to be partof the value. The value cannot contain a newline, but there is a way tohave multi-lined parameters explained in the Comment parameter below.All parameters except Version are optional: either the value can beblank or the entire ``Parameter=Value`` can be missing; the onlyexception is the Comment parameter.The parameters available are:**Version**There are four versions:* Version 1 files are long obsolete. They did not have a headersection.* Version 2 files must have ``Version=2`` as the first line of thefile. All other parameters are optional. The file may not containunrecognized parameters; software reading files with unrecognizedparameters may raise an error.* Version 3 files do not have the *Version* parameter. At least one ofthe other parameters must be present. Unrecognized parameters areignored when reading. The deprecated parameter names*Nominal_offset* and *Actual_offset* are used instead of the newerones *Timestamp_rounding* and *Timestamp_offset*.* Version 4 files are the same as Version 3, except for the names ofthe parameters *Timestamp_rounding* and *Timestamp_offset*.**Unit**A symbol for the measurement unit, like ``°C`` or ``mm``.**Count**The number of records in the time series. If present, it need not beexact; it can be an estimate. Its primary purpose is to enableprogress indicators in software that takes time to read large timeseries files. In order to determine the actual number of records,the records need to be counted.**Title**A title for the time series.**Comment**A multiline comment for the time series. Multiline comments arestored by specifying multiple adjacent Comment parameters, likethis::Comment=This timeseries is extremely importantComment=because the comment that describes itComment=spans five lines.Comment=Comment=These five lines form two paragraphs.The Comment parameter is the only parameter where a blank value issignificant and indicates an empty line, as can be seen in theexample above.**Timezone**The time zone of the timestamps, in the format :samp:`{XXX}(UTC{+HHmm})`, where *XXX* is a time zone name and *+HHmm* is theoffset from UTC. Examples are ``EET (UTC+0200)`` and ``VST(UTC-0430)``.**Time_step**A comma-separated pair of integers; the number of minutes and monthsin the time step (one of the two mut be zero). If missing, the timeseries is without time step.**Timestamp_rounding**A comma-separated pair of integers indicating the number of minutesand months that must be added to a round timestamp to get to thenominal timestamp.  For example, if an hourly time series hastimestamps that end in :13, such as 01:13, 02:13, etc., then itsrounding is 13 minutes, 0 months, i.e., ``(13, 0)``. Monthly timeseries normally have a nominal timestamp of ``(0, 0)``, thetimestamps usually being of the form 2008-02-01 00:00, meaning"February 2008" and usually rendered by application software as "Feb2008" or "2008-02". Annual timestamps have a nominal timestamp whichnormally has 0 minutes, but may have nonzero months; for example, acommon rounding in Greece is 9 months (0=January), which means thatan annual timestamp is of the form 2008-10-01 00:00, normallyrendered by application software as 2008-2009, and denoting thehydrological year 2008-2009.``timestamp_rounding`` may be None, meaning that the timestamps canbe irregular.*Timestamp_rounding* is named differently in older versions. See the*Version* parameter above for more information.**Timestamp_offset**A comma-separated pair of integers indicating the number of minutesand months that must be added to the nominal timestamp to get to theactual timestamp. The timestamp offset for small time steps, such asup to daily, is usually zero, except if the nominal timestamp is thebeginning of an interval, in which case the timestamp offset isequal to the length of the time step, so that the actual timestampis the end of the interval. For monthly and annual time steps, thetimestamp offset is usually 1 and 12 months respectively.  For amonthly time series, a timestamp offset of (-475, 1) means that2003-11-01 00:00 (often rendered as 2003-11) denotes the interval2003-10-31 18:05 to 2003-11-30 18:05.*Timestamp_offset* is named differently in older versions. See the*Version* parameter above for more information.**Interval_type**Has one of the values ``sum``, ``average``, ``maximum``,``minimum``, and ``vector_average``. If absent it means that thetime series values are instantaneous, they do not refer tointervals.**Variable**A textual description of the variable, such as ``Temperature`` or``Precipitation``.**Precision**The precision of the time series values, in number of decimal digitsafter the decimal separator. It can be negative; for example, aprecision of -2 indicates values accurate to the hundred, such as100, 200, 300 etc.**Location**, **Altitude**(Versions 3 and later.) *Location* is three numbers,space-separated: abscissa, ordinate, and EPSG SRID. *Altitude* isone or two space-separated numbers: the altitude and the EPSG SRIDfor altitude. The altitude SRID may be omitted.License=======Written by Antonis Christofides.| Copyright (C) 2015-2011 National Technical University of Athens| Copyright (C) 2013-2014 TEI of Epirus| Copyright (C) 2016 Antonis ChristofidesThis program is free software: you can redistribute it and/or modifyit under the terms of the GNU General Public License as published bythe Free Software Foundation, either version 3 of the License, or(at your option) any later version.This program is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty ofMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See theGNU General Public License for more details.You should have received a copy of the GNU General Public Licensealong with this program.  If not, see <http://www.gnu.org/licenses/>.