Package: gwdama
This package aims at providing a unified and easy to use interface to access Gravitational Wave (GW) data and output somewell organiseddatasets, ready to be used for Machine Learning projects or Data Analysis purposes (source properties, noise studies, etc.).Data Preparation WorkflowThe typical use case of this package is data acquisition and preparation, which can be seen, in apipeline, as the preliminary stage for Data Analysis. Although it is primarily meant for GW data, it is built to be sufficiently generic to handle any data type. The basic workflow that can be implemented with this package is the following:Data acquisition: Data from GW detectors can be fetched from different sources, such as thelocalstorage or, fromremote, from theGravitational Wave Open Science Center(GWOSC). Local data can be natively read inGW frame file format(gwf) or inhdf5 format. Preprocessed datasets byGWdamaare saved in the latter format. When combined with other Python packages, for examplePandas, other kind of data can be read and manipulated withGWdama. In practice,evrything that can be mapped to a NumPy typeis a valid data format;Organisation into groups:Rawandprocesseddata, from variousacquisition channelscan be organised in a hierarchical way ingroups and subgroups, each containing their own metadata and methods;Data visualization and pre-processing: This package includes some functions commonly used to visualise andpre-process data, such as filtering and spectral analysis methods. These operations can be performed in the “data preparation” stage of a pipeline, before storing the final dataset and/or forwarding it to the subsequent Data Analysis. This processed data is conveniently organised into new groups, and the “raw” ones can be removed to save memory;Reading and Writing: Once the dataset specific for a task has been created, this can be saved to disk intohdf5 format, preserving all the hierarchical group and sub-group structure and the metadata. This can be readily read back byGWdamafor further data manipulation and preparation.GW data manager package overviewGWdamacurrently comprises the main classGwDataManager, which behaves as a multi-purpose and multi-format container for data. This is based on theh5py.File classwith the addition of the methods and the attributes to import and manipulate GW data. Differently from the common application ofh5py.Fileobjects, aGwDataManagerinstance is, by default, set to occupy only a temporary file or some space in the RAM, which is authomatically deleted by python once the program is closed. Refer to thefull documentationfor further details.InsideGwDataManagerobjects, data is stored intoDatasetobjects, organised into a hierarchical structure ofh5py.Groupsand sub-groups. These Datasets are created within an instance ofGwDataManagerwith the usual methods ofh5py:create_dataset(name, shape, dtype). They contain data, typically of numeric type, and some attributes (or metadata). For example, for GW data, and in general all time series, it is important the information of when they have been recorded, and at which sampling frequency. A name and a unit are also useful. These can be conveniently added and customised. Also, aGwDataManagerobject contains attributes for itself.InstallationGWdamacan be installed viapip:$pipinstallgwdamaand requires Python 3.6.0 or higher. The previous command automatically fulfils all the required dependencies (like onnumpy,matplotlib), so you are ready to start generating datasets and making plots.Further details can be found in thedocumentation.Alternatively, it can be installed via Conda, specifyingfdirenzoas channel (soon it will available viaconda-forge):$condainstall-cfdirenzogwdamaQuick startA dataset of, say, random numbers can be readily created with the aid ofnumpy.randomroutines:>>> from gwdama.io import GwDataManager
>>> import numpy as np

>>> dama = GwDataManager("my_dama")
>>> dama.create_dataset('random', data=np.random.normal(0, 1, (10,)))Thestring representationof theGwDataManagerclass provides a quick look at its structure and its attributes:>>> print(dama)
my_dama:
  └── random

  Attributes:
     dama_name : my_dama
    time_stamp : 20-07-28_19h36m47sOther attributes can be added to both theGwDataManagerobject and theDatasetstherein:>>> dama.attrs['owner'] = 'Francesco'  # The new attribute "owner" is added with value "Francesco"
>>> dama.show_attrs
my_dama:
  └── random

  Attributes:
     dama_name : my_dama
         owner : Francesco
    time_stamp : 20-07-28_19h36m47sDatasetscan be accessed from theirkeys, as reported in the structure shown above, with a syntax similar to that for Python dictionaries:>>> dset = dama['random']       # 'random' is the dataset key
>>> dset.attrs['t0'] = 0        # It is conveninet to use gps times
>>> dset.attrs['fsample'] = 10  # measured in Hz

>>> dset.show_attrs
fsample : 10
     t0 : 0To get the data contained in this dataset, call its attributedata:>>> dset.data
array([-0.73796689, -1.34206706, -0.97898291, -0.19846702,
       -0.85056961,  0.20206334,  0.84720009,  0.19527366,
       -0.9246727 , -0.04808732])Writing and reading datasetsSo far, data is stored on temporary or volatile memory. To secure it to disk, we can call the write method of ourGwdataManagerobject:>>> out_f = 'out_dataset.h5'
>>> write_gwdama(out_f)Then remember toclose your previous filebefore leaving the session:>>> dama.close()
>>> del dama       # Redundant...NoteThis operation is automatically performed every time the session is closed. However, it is good practice to do this manually every time there is no more need of a certain variable.Toread backthe data:>>> new_dama = GwDataManager(out_f)  # Same namse as the line above
Reading dama
>>> print(new_dama)
my_dama:
  └── random

  Attributes:
     dama_name : my_dama
         owner : Francesco
    time_stamp : 20-07-30_12h19m32sRead open dataOpen data can be accessed from both online and local virtual disks provided byCVMFS.From online GWOSCGW strain data can be read by means of the.read_gwdata()method. This basically takes as input an interval of time, which can be provided as afloatin gps units or in UTC, in a human readible format (see next example), besides the label of the detector (H1,L1orV1):>>> event_gps = 1186746618                                      # GW170814
>>> dama = GwDataManager()                                      # Default name 'mydama' assigned
>>> dama.read_gwdata(event_gps - 50, event_gps +10, ifo='L1',   # Required params
                     data_source="gwosc-online",                # data source (optional, already implicit)
                     dts_key='online')                          # group key (optional, but useful)From local CVMFSCernVM-FS must be installed and configured on your computer. Refer to itsdescription on the GWOSC websiteor tothis Quick start guide.Assuming your data are stored at the following path (you can always modify it by passing it as a parameter toread_gwdata()):cvmfs_path = '/data2/cvmfs/gwosc.osgstorage.org/gwdata/'data can be read with:>>> start='2017-06-08 01:00:00'  # starting UTC time as a string
>>> end='2017-06-08 02:00:00'    # ending time as a string
>>> ifo='H1'                     # interfereometer tag

>>> rate='4k'                    # sample rate: 4k or 16k
>>> frmt='hdf5'                  # format of the data: gwf or hdf5

>>> dama.read_gwdata(start, end, data_source="gwosc-cvmfs", ifo=ifo, data_format=frmt)Changelog0.5.2Added the optionalreturn_outputparameter to.read_gwdata(...)to allow (if True) aving a Dataset or a Group as the output of this method. The corrisponding data is added in any case to theGwDataManagerobject.0.5.1.plot()method forDatasetclass. Mainly aimed at time sereis data, witht0andsample_rateattributes;0.5.0If one passesffl_specorffl_pathorgwf_pathparameter toread_gwdata, thendata_sourceis automatically set tolocal;Some parameter names have been slightly simplified. E.g.:m_data_source->data_source;histmethod ofDataset <https://gwnoisehunt.gitlab.io/gwdama/dataset.html>`_s now has a ``ax`parameter to specify an existing matplotlib axes.0.4.5Added interface with GWpy;Multi-Taper Method.0.4.1Methods:hist,duration;Attributes:groups;Preprocessing functions:PSD,whiten,taper.0.4.0Implemented support for data on Virgo Farm.