DOI: 10.5061/dryad.v41ns1s91
title: Data and code from: The effects of carbonate precipitation on the physical properties of basalt

description: This dataset contains the data and software acquired and produced in the
 study "The Effects of Carbonate Precipitation on the Physical
 Properties of Basalt". Included in the data are the raw ultrasonic
 waveforms, recorded flow rates, pressures, temperatures, permeabilities,
 and relevant X-Ray computed tomography (CT) data for each of the five
 experiments (B31, B37, B35, B32, and B41). The most important software
 used for processing the raw data is also included, written in Python.
description: # Data and code from: The effects of carbonate precipitation on the
 physical properties of basalt Dataset DOI:
 [10.5061/dryad.v41ns1s91](https://doi.org/10.5061/dryad.v41ns1s91) ##
 Description of the data and file structure This dataset contains the raw
 data acquired in the study "The Effects of Carbonate Precipitation on
 the Physical Properties of Basalt". The raw data come in three main
 formats: X-Ray computed tomography (CT) images, binary waveform files for
 ultrasonic data, and data recorded by the suite of auxiliary sensors used
 in the experiment (e.g., fluid pressure, pump flow rate, outlet sensor
 flow rate, etc.). Each data type and how it can be used is explained
 below. ## Time-lapse CT Data The timelapse CT data comes in the format of
 a stack of .tiff images. These images are the product of 3D reconstruction
 and processing from Nikon and VisualStudio Max software. Each 3D rock scan
 is composed of approximately 2000 individual horizontal slices, which,
 when stacked together, form a 3D volume. The pixel/voxel spacing for all
 scans is 11.5 micron (although exact dimensions can be found in the json
 file associated with each scan). Processing of the CT data was performed
 mainly using the open source software package 3D Slicer (see
 [here](https://www.slicer.org/)) running on a Linux Ubuntu OS. The ANTs
 registration module was used within 3D Slicer for volume
 registration/alignment, and can be downloaded as a plugin from the module
 downloader wizard within Slicer. The processing workflow was largely
 automated using the Python interface that can be run within Slicer. The
 workflow script which handled most of this automatic workflow for
 processing timelapse CT data is included in the scripts directory. ##
 Waveform Data Ultrasonic waveform data is included for the relevant
 experiments. Ultrasonic waveforms are saved in NumPy .npy binary format.
 The time of acquisition is included in the file name (as Unix time), along
 with the trigger delay in percent of the record (td) and the sampling
 interval in ns (si). Each NumPy file contains either 2 or 4 channels in a
 structured array format, which is accessible using the dtype names as
 fields (dtype names can be determined by accessing the array.dtype.names
 field). The channel voltage level is contained in each heading for each
 channel so that the integer data can be converted into voltages. ## Sensor
 Data The third type of data included here are the remaining data acquired
 by the sensors in the experiment. These include the outlet flow rate
 sensor, upstream pressure, pump flow rate, and ambient temperature. Each
 of these datasets is saved in .npy format in a structured array, with the
 time of acquisition recorded in the first columns. In addition, a .json
 file has information that explains the acquisition state for each
 experiment and has comments on the experimental setup. Note that some
 experiments have multiple folders of sensor data as the recording software
 had to be restarted multiple times during the course of the experiment. 
 ## Files and variables ### Directory Structure ``` . ├── B31 │   ├──
 experiment_data │   │   └── sensorview_data │   │   └──
 b31_calcite_precip_sodium-carbonate │   │   ├── ArduinoSLF3x │   │   │  
 └──
 b31_calcite_precip_sodium-carbonate_experiment-data_2024-05-23_17-00-36_ArduinoSLF3x.npy │   │   ├── b31_calcite_precip_sodium-carbonate_experiment-config_2024-05-23_17-00-36.json │   │   ├── Permeability │   │   │   └── b31_calcite_precip_sodium-carbonate_experiment-data_2024-05-23_17-00-36_Permeability.npy │   │   └── SyringePumpPID │   │   └── b31_calcite_precip_sodium-carbonate_experiment-data_2024-05-23_17-00-36_SyringePumpPID.npy │   └── waveforms │   ├── b31_calcite_precip_sodium-carbonate_pulsing_ae_classifications.csv │   ├── b31_calcite_precip_sodium-carbonate_waveform*.mat │   ├── b31_calcite_precip_sodium-carbonate_waveform_si-0v8ns_td-50v*.npy ├── B32 │   ├── CT │   │   ├── B32_bubbles_volumes_sensor-locations.csv │   │   ├── B32_calcite_volumes_full_master.csv │   │   └── ImageStacks_B32_adjusted │   │   ├── scan[0-38]_lvl6700_win7500 │   │   │   ├── B32_sodium-carbonate_precip_*.tif │   │   ├── scan_Dry_PreExp_lvl6700_win7500 │   │   │   ├── B32_sodium-carbonate_precip_scan_Dry_PreExp_*.tif │   │   └── scan_VacuumOvenDry_PostExp_HighRes_lvl6700_win7500 │   │   ├── B32_sodium-carbonate_precip_scan_VacuumOvenDry_PostExp_HighRes_*.tif │   ├── experiment_data │   │   └── sensorview_data │   │   ├── B32_sodium-carbonate_CT-1[-1-57] │   │   │   ├── ArduinoSLF3x │   │   │   │   └── B32_sodium-carbonate_CT-1*_experiment-data_*_ArduinoSLF3x.npy │   │   │   ├── B32_sodium-carbonate_CT-1*_experiment-config_*.json │   │   │   ├── Permeability │   │   │   │   └── B32_sodium-carbonate_CT-1*_experiment-data_*_Permeability.npy │   │   │   └── SyringePumpPID │   │   │   └── B32_sodium-carbonate_CT-1*_experiment-data_*_SyringePumpPID.npy │   └── waveforms │   ├── 20250115-0001_si-2v0ns_td-50v003997us_npy_*.npy │   └── b32_calcite_precip_sodium-carbonate_pulsing_ae_classifications.csv ├── B35 │   └── experiment_data │   └── sensorview_data │   ├── B35_sodium-carbonate_precip_CT1[-1-13] │   │   ├── ArduinoSLF3x │   │   │   └── B35_sodium-carbonate_precip_CT1*_experiment-data_*_ArduinoSLF3x.npy │   │   ├── B35_sodium-carbonate_precip_CT1*_experiment-config_*.json │   │   ├── Permeability │   │   │   └── B35_sodium-carbonate_precip_CT1*_experiment-data_*_Permeability.npy │   │   └── SyringePumpPID │   │   └── B35_sodium-carbonate_precip_CT1*_experiment-data_*_SyringePumpPID.npy ├── B37 │   ├── CT │   │   ├── IcelandicBasalt_JS_B37_post_01_tifStack │   │   │   ├── IcelandicBasalt_JS_B37_post_01_*.tif │   │   └── IcelandicBasalt_JS_B37_pre_01_tifStack │   │   ├── IcelandicBasalt_JS_B37_pre_01_*.tif │   ├── experiment_data │   │   └── sensorview_data │   │   ├── b37_calcite_precip_sodium-carbonate-[8-10]* │   │   │   ├── AirPressureAndTemperature │   │   │   │   └── b37_calcite_precip_sodium-carbonate_experiment-data_*_AirPressureAndTemperature.npy │   │   │   ├── ArduinoSLF3x │   │   │   │   └── b37_calcite_precip_sodium-carbonate_experiment-data_*_ArduinoSLF3x.npy │   │   │   ├── b37_calcite_precip_sodium-carbonate_experiment-config_*.json │   │   │   ├── Permeability │   │   │   │   └── b37_calcite_precip_sodium-carbonate_experiment-data_*_Permeability.npy │   │   │   └── SyringePumpPID │   │   │   └── b37_calcite_precip_sodium-carbonate_experiment-data_*_SyringePumpPID.npy │   └── waveforms │   ├── b37_calcite_precip_sodium-carbonate_pulsing_ae_classifications.csv │   ├── b37_calcite_precip_sodium-carbonate_waveform*.mat │   ├── b37_calcite_precip_sodium-carbonate_waveform_si-0v8ns_td-*_npy_*.npy ├── B41 │   ├── CT │   │   ├── B41_calcite_volumes_full_thr25.csv │   │   ├── B41_calcite_volumes_pore1549.csv │   │   ├── B41_calcite_volumes_pore3224.csv │   │   ├── B41_calcite_volumes_pore6036.csv │   │   ├── B41_calcite_volumes_pore8864.csv │   │   ├── B41_calcite_volumes_sensors.csv │   │   └── ImageStacks_B41_adjusted │   │   ├── scan[1-59]_lvl8000_win7168 │   │   │   ├── B41_sodium-carbonate_precip_scan[1-59]_*.tif │   │   ├── scan_Dry_PostExp_lvl8000_win7168 │   │   │   ├── B41_sodium-carbonate_precip_scan_Dry_PostExp_*.tif │   │   └── scan_Dry_PreExp_lvl8000_win7168 │   │   ├── B41_sodium-carbonate_precip_scan_Dry_PreExp_*.tif │   ├── experiment_data │   │   └── sensorview_data │   │   ├── B41_after24_P* │   │   │   ├── ArduinoSLF3x │   │   │   │   └── B41_*_experiment-data_*_ArduinoSLF3x.npy │   │   │   ├── B41_*_experiment-config_*.json │   │   │   ├── Permeability │   │   │   │   └── B41_*_experiment-data_*_Permeability.npy │   │   │   └── SyringePumpPID │   │   │   └── B41_*_experiment-data_*_SyringePumpPID.npy │   │   ├── B41_sodium-carbonate_CT │   │   │   ├── ArduinoSLF3x │   │   │   │   └── B41_sodium-carbonate_CT_experiment-data_2025-03-27_15-56-06_ArduinoSLF3x.npy │   │   │   ├── B41_sodium-carbonate_CT_experiment-config_2025-03-27_15-56-06.json │   │   │   ├── Permeability │   │   │   │   └── B41_sodium-carbonate_CT_experiment-data_2025-03-27_15-56-06_Permeability.npy │   │   │   └── SyringePumpPID │   │   │   └── B41_sodium-carbonate_CT_experiment-data_2025-03-27_15-56-06_SyringePumpPID.npy │   │   └── B41_sodium-carbonate_CT_post-24hr │   │   ├── ArduinoSLF3x │   │   │   └── B41_sodium-carbonate_CT_post-24hr_experiment-data_2025-03-28_16-56-51_ArduinoSLF3x.npy │   │   ├── B41_sodium-carbonate_CT_post-24hr_experiment-config_2025-03-28_16-56-51.json │   │   ├── Permeability │   │   │   └── B41_sodium-carbonate_CT_post-24hr_experiment-data_2025-03-28_16-56-51_Permeability.npy │   │   └── SyringePumpPID │   │   └── B41_sodium-carbonate_CT_post-24hr_experiment-data_2025-03-28_16-56-51_SyringePumpPID.npy │   └── waveforms │   ├── 20250327-0001_si-*ns_td-*us_npy_*.npy │   └── b41_calcite_precip_sodium-carbonate_pulsing_ae_classifications.csv ├── manuals │   └── worksflow_for_3d-slicer_ct_processing.odt └── scripts ├── ct_scripts │   ├── adjust_levels_and_equalize.py │   ├── calculate_porosity.py │   ├── calculate_precipitate_volume_in_zones.py │   ├── calculate_precipitate_volume.py │   ├── count_nonzero_pixels.ijm │   ├── output_animation.txt │   └── slicer_process_timelapse_scans.py └── waveform_scripts ├── b37_calcite_precip_ultrasonics_proc.py ├── helper_functions.py └── __pycache__ ``` 
 
 ## Folders, Files, and Variables The experimental data is stored in the
 folder corresponding to each sample number (B31, B32, B35, B37, and B41).
 Within each sample folder, there are one or more folders titled
 experiment_data, waveforms, and/or CT. ##### experiment_data The
 experiment data folders contain the "sensorview_data" files
 which record the data from the sensors used in each experiment (e.g.
 permeability, flow rate, experiment configuration, etc.), as described in
 the "Sensor Data" section above. There are often multiple
 folders in the "sensorview_data" folder, as the acquisition
 program sometimes had to be restarted multiple times over the course of a
 single experiment. The order that these individual acquisitions were
 acquired is given in the folder/file names, and these data can be stitched
 together to obtain the full sensor data over the course of the experiment.
 ##### waveforms These folders contain the waveform data, as described
 above. In addition, each waveform folder contains a csv file which is
 named* ...pulsing_*ae_classifications.csv. This is a tabulated list of all
 the waveform files recorded, with the headers: file_index: The waveform
 file number in order of recording file_name: The full name of the waveform
 file, with the UNIX time stamp of recording waveform_type: This is either
 "pulse" or "ae", referring to active pulsing of the
 ultrasonic sensors or a passive acoustic emission. The vast majority of
 waveforms are pulses. ##### CT These folders contain the semi-processed
 micro X-Ray CT images in tiff format. Each sub-folder corresponds to an
 individual scan. Pre- and Post- experiment scans are identified in their
 folder names. These are longer-duration (~2 hr) scans performed before and
 after the experiments. Folders that have "scan*" in their name
 (where star is a number from 0-59) correspond to short-duration (6-12 min)
 scans performed during the flow-through experiments. In most cases, these
 data have been pre-processed to equalize the grey values/histograms across
 all scans in an experiment. The adjusted midpoint and window of the
 histograms for each image are given by the lvl and win numbers in the
 folder names. In addition to the folders containing the tif files, there
 are several .csv files containing tabulated data from the processing steps
 of the CT data in some of the CT folders. These all contain the volumes of
 precipitates for each scan either for the full sample or individual pores
 (as identified in the file name). The variables in each of these files are
 as follows: scan_name: The name of the scan unix_time: The time the scan
 was started in seconds since the epoch relative_time: Number of elapsed
 seconds to the start of the scan since the first scan readable_time: A
 readable timestamp for the start time of each scan calcite_volume_voxels:
 The number of voxels within the scan that were above the threshold
 identified for precipitates. The threshold for each experiment was
 constant, and is identified in the file name of each csv file.
 calcite_volume_cm3: The total volume of precipitates in cubic centimeters.
 ## Code/software Example scripts used in the processing of these data are
 included in the scripts directory. The description of the function of each
 script is included in the headers of each file. 3D Slicer, FIJI/ImageJ
 (using the 3D Script plugin), and Python were used extensively to process
 and analyze this data. A processing workflow written in Python is included
 (slicer_process_timelapse_scans.py) that runs within 3D Slicer to automate
 many of the tasks of processing the CT data. ## Manuals A detailed manual
 describing the processing steps in 3D slicers is included in the manuals
 directory. This processing workflow is implemented in the processing
 workflow script (slicer_process_timelapse_scans.py).