Package: marlin-data
marlin_data| the data adapter© Rahul Tandon, RS Aqua 2024Aboutmarlin_dataallows access to RSA's acoustic signature database. The module requests the RSA acoustic dataset required in order to run a machine learning (ML). Current version ofmarlin_dataprovides a predefined dataset for proof of concept and tutorial purposes. Future versions will allow for more specific datasets as well as definig training and validation datasets separately.NOTE| Numerical data inmarlin_datais defined using Python's numpy and pandas library.Dependenciesimport numpy as np import pandas as pd import requests, json import logging import dotenv import os, sys from dataclasses import dataclass import randomInstallationfrom marlin_data import *Quick StartAccessible DataDatafeed InstanceEach iteration over the Marlin data feed will provide once instance of the datafeed and simulation data / snapshots. Frequency time series in numpy and a pandas dataframe is available along with descrictive metadata. Feed data is an iterable class which can be looped over allowing for an incremental datafeed.Feed Instance->frequency_ts_npThe frequency time series for this data snapshot ( NumPy Array )for feed_instance in data_feed: frequency_np_array = feed_instance['frequency_ts_np']Feed Instance->frequency_ts_pdThe frequency time series for this data snapshot (pandasDataFrame)for feed_instance in data_feed: frequency_data_frame = feed_instance['frequency_ts_pd']Feed Instance->meta_dataMetadata for the associated snapshot. Metadata is a dictionary with the following defined fields:snapshot_id: unique id for the snapshotdata_frame_start: timestamp for the start of the snapshot data framedata_frame_end: timestamp for the end of the snapshot data framelistener_location: geo location of the listening device for snapshotlocation_name: human friendly name of listening locationframe_delta_t: delta t for the snapshot (s)sample_rate: frequency recorder sample rateSignature DataSignatureDatadataclass is accessible fromMarlinData->signature_data. Key values are stored inMarlinData->signature_index.@dataclass
    class SignatureData:
            frequency_ts_np : np.array
            frequency_ts_pd : pd.DataFrame
            meta_data : {}Metadata for the associated snapshot. Metadata is a dictionary with the following defined fields:snapshot_id: unique id for the snapshotdata_frame_start: timestamp for the start of the snapshot data framedata_frame_end: timestamp for the end of the snapshot data framelistener_location: geo location of the listening device for snapshotlocation_name: human friendly name of listening locationframe_delta_t: delta t for the snapshot (s)sample_rate: frequency recorder sample rateSimulation DataSimulationDatadataclass is accessible fromMarlinData->simulation_data. Key values are stored inMarlinData->simulation_index.@dataclass
    class SimulationData:
        frequency_ts_np : np.array
        frequency_ts_pd : pd.DataFrame
        meta_data : {}
        snapshot : bool = TrueMetadata for the associated snapshot. Metadata is a dictionary with the following defined fields:snapshot_id: unique id for the snapshotdata_frame_start: timestamp for the start of the snapshot data framedata_frame_end: timestamp for the end of the snapshot data framelistener_location: geo location of the listening device for snapshotlocation_name: human friendly name of listening locationframe_delta_t: delta t for the snapshot (s)sample_rate: frequency recorder sample rateDownloading dataInstantiate aMarlinDataclass.` marlin_data = MarlinData(load_args={})`Download signature data from RSA signature database.`marlin_data.download_signatures(load_args={})`Download simulation / ML run data required for datafeed.`marlin_data.download_simulation_snapshots(load_args={})`load_args:limit: maximum number of downloads (Require in init())signature_ids: vector of signature idsss_ids: vector of snapshot ids◊location: vector of locationsConnecting to datafeedCreate the Marlin datafeed. This data feed can be iterated over in order to simulate a data feed into a model.`data_feed = MalrinDataStreamer()`Connect the downloaded data to the datafeed instance.`data_feed.init_data(marlin_data.simulation_data, marlin_data.simulation_index) `Iterate over the datafeed.for data_inst in data_feed:
            print (data_inst)`