Package: tidal-parser
TiDaLTiDaL is atime-seriesdatalogging utility for C++ applications. It consists of two parts:A header-only C++ library for logging numerical data (scalars orEigenvectors/matrices), with an associated timestamp, to a fileA Python module for parsing log-file data into numpy arrays for further processing and/or plottingTable of ContentsLog formatC++ loggingUsageScalar StreamVector StreamMatrix StreamIncluding in your projectPython parsingLog formatEach log file can contain multiple named "streams," where each stream is an independent time series. The time stamps need not be synchronized between streams. Each stream can be one of the following types:Scalar: More properly, a tuple of any combination of the supported scalar types (signed and unsigned integers, single- or double-precision floating point numbers, or boolean). Labels for each of the fields in the tuple can also be written to the log file.Vector: A one-dimensional vector of any of the supported scalar typesMatrix: A two-dimensional matrix of any of the supported scalar typesData is stored in a binary format in the log files, with minimal metadata overhead. As a result, logging and parsing are fast and the files are compact.C++ loggingNote: Requires C++17 support.UsageExample usage can also be seen in the fileexamples/example.cpp.Each log file is managed by aLogobject:#include<tidal/tidal.h>tidal::Loglog("filename.bin");Streams are added by calling the appropriateadd_<type>_stream()method on theLogobject, which then returns astd::shared_ptr<>to theStreamobject that provides alog()function for actually writing data to that stream in the log file. Streams can be added at any time, even if data has already been logged to other streams.Specific instructions for each stream type are given below.Scalar StreamA scalar stream can contain any number of elements that are of the supported scalar types.
The supported scalar types areuint8_t,int8_t,uint16_t,int16_t,uint32_t,int32_t,uint64_t,int64_t,float,double, andbool. Equivalent types such asintinstead ofint32_tare also supported.For example, a scalar stream named "My Scalar Stream" and consisting of adouble, anint, and aboolwould be added and then used asautoscalar_stream=log.add_scalar_stream<double,int,bool>("My Scalar Stream");uint64_ttimestamp=1234567;scalar_stream->log(timestamp,23.5,42,true);The SFINAE implementation of the library ensures at compile time that only the correct number and types of values are passed to thelog()function. The type ofscalar_streamin this example isstd::shared_ptr<tidal::Log::ScalarStream<double,int,bool>>.Additionally, scalar streams support adding labels for each of the fields, such asscalar_stream->set_labels("x","y","z");where the correct number of labels is enforced at compile time.Vector StreamA vector stream is used to log fixed-size Eigen column vectors, such asEigen::Vector3dorEigen::Matrix<double, 6, 1>. Vector streams are added by calling theadd_vector_stream<>()method, whose template arguments are the datatype and length of the Eigen vector.For example, a vector stream named "My Vector Stream" for vectors of typeEigen::Vector3dwould be added and then used as#include<eigen3/Eigen/Core>autovector_stream=log.add_vector_stream<double,3>("My Vector Stream");uint64_ttimestamp=1234567;Eigen::Vector3dx;x<<1.0,2.0,3.0;vector_stream->log(timestamp,x);The type ofvector_streamin this example isstd::shared_ptr<tidal::Log::VectorStream<double,3>>.Matrix StreamA matrix stream is used to log fixed-size Eigen matrices, such asEigen::Matrix3dorEigen::Matrix<double, 6, 4>. Matrix streams are added by calling theadd_matrix_stream<>()method, whose template arguments are the datatype, rows, and columns of the Eigen matrix.For example, a matrix stream named "My Matrix Stream" for matrices of typeEigen::Matrix3dwould be added and then used as#include<eigen3/Eigen/Core>automatrix_stream=log.add_matrix_stream<double,3,3>("My Matrix Stream");uint64_ttimestamp=1234567;Eigen::Matrix3dX=Eigen::Matrix3d::Identity();matrix_stream->log(timestamp,X);The type ofmatrix_streamin this example isstd::shared_ptr<tidal::Log::MatrixStream<double,3,3>>.Including in your projectThe entire logging library is contained in the header filetidal.h. The easiest way to include this library in your project is probably to add it as a submodule.For example, with a CMake-based project using Git, I might add this library in thelib/tidaldirectory withgitsubmoduleaddhttps://github.com/dpkoch/tidal.gitlib/tidalThen in myCMakeLists.txtI may have something likeinclude(# other include directorieslib/tidal/include)After which I can include the header file with#include<tidal/tidal.h>Python parsingThe python parsing package is available fromPython Package Indexas thetidal-parserpackage. This can be installed with pip:python3-mpipinstall--usertidal-parserA log file is parsed by creating aParserobject with the location of the log file as its argument:fromtidal_parserimportParserlog=Parser('/path/to/my/log/file.bin')The data is then accessed as numpy arrays through thetimeanddatamembers of theParserobject. These members are Python dictionaries whose keys are the stream names specified by the C++add_<type>_stream()methods described above.The following examples show how the streams added in the C++ snippets above might be accessed:fromtidal_parserimportParserlog=Parser("filename.bin")# scalar streamscalar_time=log.time['My Scalar Stream']x=log.data['My Scalar Stream']['x']y=log.data['My Scalar Stream']['y']z=log.data['My Scalar Stream']['z']# vector streamvector_time=log.time['My Vector Stream']vector_data=log.data['My Vector Stream']# returns nx3 array, where n is the number of timestepsv_0=log.data['My Vector Stream'][:,0]# get the 1st element of the vector across all timesteps# matrix streammatrix_time=log.time['My Matrix Stream']matrix_data=log.data['My Matrix Stream']# returns nx3x3 array, where n is the number of timestepsm_1_2=log.data['My Matrix Stream'][:,1,2]# get the (1,2) element of the matrix across all timesteps