Package: e2epyppg
End-to-End PPG Processing Pipeline (e2epyppg)Welcome to the PPG Signal Processing Pipeline, a comprehensive package designed for extracting accurate Heart Rate (HR) and Heart Rate Variability (HRV) data from Photoplethysmogram (PPG) signals.IntroductionThis project provides a robust pipeline for processing PPG signals, extracting reliable HR and HRV parameters. The pipeline encompasses various stages, including filtering, signal quality assessment (SQA), signal reconstruction, peak detection and interbeat interval (IBI) extraction, and HR and HRV computation.PreprocessingThe input raw PPG signal undergoes filtering to remove undesired frequencies. A second-order Butterworth bandpass filter is employed, allowing frequencies within a specific range to pass while rejecting frequencies outside.Signal Quality Assessment (SQA)SQA involves identifying clean and noisy parts within PPG signals. Our SQA approach requires PPG signals in a fixed length, which necessitates segmenting the input signals. To this end, we apply a moving window segmentation technique, where the PPG signals are divided into overlapping segments, each spanning 30 seconds, by sliding a window over the signal. The SQA process includes PPG feature extraction and classification, employing a one-class support vector machine model to distinguish between "Reliable" and "Unreliable" segments.Noise ReconstructionNoisy parts within PPG signals, shorter than a specified threshold, are reconstructed using a deep convolutional generative adversarial network (GAN). The GAN model includes a generator trained to produce synthetic clean PPG segments and a discriminator to differentiate real from synthetic signals. The reconstruction process is applied iteratively to ensure denoising.Peak Detection and IBI ExtractionSystolic peaks in PPG signals are identified using a deep-learning-based method with dilated Convolutional Neural Networks (CNN) architecture. PPG peak detection enables the extraction of IBI values that serve as the basis for obtaining HR and HRV. IBI represents the time duration between two consecutive heartbeats and is computed by measuring the time interval between systolic peaks within the PPG signals.HR and HRV ExtractionHR and HRV parameters are computed from the extracted IBIs. A variety of metrics are calculated, including:HR: Heart rateMeanNN: The mean of the RR intervalsSDNN: The standard deviation of the RR intervalsSDANN: The standard deviation of average RR intervalsSDNNI: The mean of the standard deviations of RR intervalsRMSSD: The square root of the mean of the squared successive differences between adjacent RR intervalsSDSD: The standard deviation of the successive differences between RR intervalsCVNN: The standard deviation of the RR intervals (SDNN) divided by the mean of the RR intervals (MeanNN)CVSD: The root mean square of successive differences (RMSSD) divided by the mean of the RR intervals (MeanNN)MedianNN: The median of the RR intervalsMadNN: The median absolute deviation of the RR intervalMCVNN: The median absolute deviation of the RR intervals (MadNN) divided by the median of the RR intervals (MedianNN)IQRNN: The interquartile range (IQR) of the RR intervalsPrc20NN: The 20th percentile of the RR intervalsPrc80NN: The 80th percentile of the RR intervalspNN50: The proportion of RR intervals greater than 50ms, out of the total number of RR intervalspNN20: The proportion of RR intervals greater than 20ms, out of the total number of RR intervalsMinNN: The minimum of the RR intervalsMaxNN: The maximum of the RR intervalsTINN: A geometrical parameter of the HRV, or more specifically, the baseline width of the RR intervals distribution obtained by triangular interpolation, where the error of least squares determines the triangle. It is an approximation of the RR interval distributionHTI: The HRV triangular index, measuring the total number of RR intervals divided by the height of the RR intervals histogramULF: The spectral power of ultra low frequencies (by default, .0 to .0033 Hz)VLF: The spectral power of very low frequencies (by default, .0033 to .04 Hz)LF: The spectral power of low frequencies (by default, .04 to .15 Hz)HF: The spectral power of high frequencies (by default, .15 to .4 Hz)VHF: The spectral power of very high frequencies (by default, .4 to .5 Hz)LFHF: The ratio obtained by dividing the low frequency power by the high frequency powerLFn: The normalized low frequency, obtained by dividing the low frequency power by the total powerHFn: The normalized high frequency, obtained by dividing the low frequency power by the total powerLnHF: The log transformed HFSD1: Standard deviation perpendicular to the line of identitySD2: Standard deviation along the identity lineSD1/SD2: ratio of SD1 to SD2. Describes the ratio of short term to long term variations in HRVS: Area of ellipse described by SD1 and SD2 (pi * SD1 * SD2)InstallationInstall thee2epyppgpackage using the following pip command:pipinstalle2epyppgUsage1. End-to-End HR and HRV Extraction Functione2e_hrv_extraction()End-to-end HR and HRV extraction from an input raw PPG signal.Args:input_sig(np.ndarray): The input PPG signal.sampling_rate(int): The sampling rate of the input signal.window_length_sec(int, optional): The desired window length for HR and HRV extraction in seconds (default: 60).peak_detection_method(str, optional): The method used for peak detection. Valid inputs: 'kazemi', 'nk' (Neurokit), and 'heartpy' (default: 'kazemi').Returns:hrv_data(pd.DataFrame): A DataFrame containing HRV parameters ('None' if no clean segments detected in the signal).Reference:Feli, M., Kazemi, K., Azimi, I., Wang, Y., Rahmani, A., & Liljeberg, P. (2023). End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE.Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.e2e_ppg_pipelineimporte2e_hrv_extraction# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20# Set the desired window length for HR and HRV features extraction (in seconds)window_length_sec=60# Set the peak detection method (optional)peak_detection_method='kazemi'# Call the end-to-end functionhrv_data=e2e_hrv_extraction(input_sig,sampling_rate,window_length_sec,peak_detection_method)2. PPG Signal Quality Assessment (SQA) Functionsqa()Perform PPG Signal Quality Assessment (SQA).This function assesses the quality of a PPG signal by classifying its segments as reliable (clean) or unreliable (noisy) using a pre-trained model. The clean indices represent parts of the PPG signal that are deemed reliable, while the noisy indices indicate parts that may be affected by noise or artifacts.Args:sig(np.ndarray): PPG signal.sampling_rate(int): Sampling rate of the PPG signal.filter_signal(bool, optional): True if the signal has not been filtered using a bandpass filter.Returns:clean_indices(list): A list of clean indices.noisy_indices(list): A list of noisy indices.Reference:Feli, M., Azimi, I., Anzanpour, A., Rahmani, A. M., & Liljeberg, P. (2023). An energy-efficient semi-supervised approach for on-device photoplethysmogram signal quality assessment. Smart Health, 28, 100390.Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.ppg_sqaimportsqa# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20# Set this parameter True if the signal has not been filtered:filter_signal=True# Call the PPG signal quality assessment functionclean_indices,noisy_indices=sqa(input_sig,sampling_rate,filter_signal)3. PPG Signal Reconstruction Functionreconstruction()Reconstruct noisy PPG signals using Generative Adversarial Network (GAN).Args:sig(np.ndarray): Original PPG signal.clean_indices(list): List of indices representing clean parts (obtained from sqa()).noisy_indices(list): List of indices representing noisy parts (obtained from sqa()).sampling_rate(int): Sampling rate of the signal.filter_signal(bool, optional): True if the signal has not been filtered using a bandpass filter.Returns:ppg_signal(np.ndarray): Reconstructed PPG signal (if reconstruction is applied; otherwise, returns the original signal).clean_indices(list): Updated indices of clean parts (if reconstruction is applied; otherwise, returns the original indices of clean parts).noisy_indices(list): Updated indices of noisy parts (if reconstruction is applied; otherwise, returns the original indices of noisy parts).Reference:Wang, Y., Azimi, I., Kazemi, K., Rahmani, A. M., & Liljeberg, P. (2022, July). Ppg signal reconstruction using deep convolutional generative adversarial network. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 3387-3391). IEEE.Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.ppg_sqaimportsqafrome2epyppg.ppg_reconstructionimportreconstruction# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20# Set this parameter True if the signal has not been filtered:filter_signal=True# Call the PPG signal quality assessment functionclean_indices,noisy_indices=sqa(input_sig,sampling_rate,filter_signal)# Call the PPG reconstruction functionppg_reconstructed,updated_clean_indices,updated_noisy_indices=reconstruction(input_sig,clean_indices,noisy_indices,sampling_rate,filter_signal)4. Clean Segment Extraction Functionclean_seg_extraction()Scan the clean parts of the signal and extract clean segments based on the input window length.Args:sig(np.ndarray): Input PPG signal.noisy_indices(list): List of noisy segment indices (obtained from sqa() or reconstruction()).window_length(int): Desired window length for clean segment extraction in terms of samples (not seconds).Returns:clean_segments(list): List of clean PPG segments with the specified window length and their starting index.Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.ppg_sqaimportsqafrome2epyppg.ppg_reconstructionimportreconstructionfrome2epyppg.ppg_clean_extractionimportclean_seg_extraction# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20window_length_sec=60# Set this parameter True if the signal has not been filtered:filter_signal=True# Call the PPG signal quality assessment functionclean_indices,noisy_indices=sqa(input_sig,sampling_rate,filter_signal)# Call the PPG reconstruction functionppg_reconstructed,updated_clean_indices,updated_noisy_indices=reconstruction(input_sig,clean_indices,noisy_indices,sampling_rate,filter_signal)# Set the window length for HR and HRV extraction in terms of sampleswindow_length=window_length_sec*sampling_rate# Call the clean segment extraction functionclean_segments=clean_seg_extraction(ppg_reconstructed,updated_noisy_indices,window_length)5. Peak Detection Functionpeak_detection()Detect peaks in clean PPG segments using the specified peak detection method.Args:clean_segments(list): List of clean PPG segments with the specified window length and their starting index (obtained from clean_seg_extraction()).sampling_rate(int): Sampling rate of the PPG signal.method(str, optional): Peak detection method. Valid inputs: 'kazemi', 'nk' (Neurokit), and 'heartpy'. (default: 'kazemi').Returns:total_peaks(list): List of lists, each containing the detected peaks for a corresponding clean segment.References:Kazemi method: Kazemi, K., Laitala, J., Azimi, I., Liljeberg, P., & Rahmani, A. M. (2022). Robust ppg peak detection using dilated convolutional neural networks. Sensors, 22(16), 6054.Neurokit method: Makowski, D., Pham, T., Lau, Z. J., Brammer, J. C., Lespinasse, F., Pham, H., ... & Chen, S. A. (2021). NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior research methods, 1-8.HeartPY method: Van Gent, P., Farah, H., Nes, N., & van Arem, B. (2018, June). Heart rate analysis for human factors: Development and validation of an open source toolkit for noisy naturalistic heart rate data. In Proceedings of the 6th HUMANIST Conference (pp. 173-178).Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.ppg_sqaimportsqafrome2epyppg.ppg_reconstructionimportreconstructionfrome2epyppg.ppg_clean_extractionimportclean_seg_extractionfrome2epyppg.ppg_peak_detectionimportpeak_detection# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20window_length_sec=60# Set this parameter True if the signal has not been filtered:filter_signal=True# Call the PPG signal quality assessment functionclean_indices,noisy_indices=sqa(input_sig,sampling_rate,filter_signal)# Call the PPG reconstruction functionppg_reconstructed,updated_clean_indices,updated_noisy_indices=reconstruction(input_sig,clean_indices,noisy_indices,sampling_rate,filter_signal)# Set the window length for HR and HRV extraction in terms of sampleswindow_length=window_length_sec*sampling_rate# Call the clean segment extraction functionclean_segments=clean_seg_extraction(ppg_reconstructed,updated_noisy_indices,window_length)# Set the peak detection method (optional)peak_detection_method='kazemi'# Call the peak detection functionpeaks=peak_detection(clean_segments,sampling_rate,peak_detection_method)6. HR and HRV Extraction Functionhrv_extraction()Calculate HR and HRV parameters from clean segments peak indices.Args:clean_segments(list): List of clean PPG segments and their starting index (obtained from clean_seg_extraction()).peaks(list): List of lists, each containing the detected peaks for a corresponding clean segment (obtained from peak_detection()).sampling_rate(int): Sampling rate of the PPG signal.window_length(int): Desired window length for HR and HRV extraction in terms of samples (not seconds).Returns:hrv_data(pd.DataFrame): DataFrame containing HRV parameters for all clean segments.Usage Example:frome2epyppg.utilsimportget_datafrome2epyppg.ppg_sqaimportsqafrome2epyppg.ppg_reconstructionimportreconstructionfrome2epyppg.ppg_clean_extractionimportclean_seg_extractionfrome2epyppg.ppg_peak_detectionimportpeak_detectionfrome2epyppg.ppg_hrv_extractionimporthrv_extraction# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)file_name="201902020222_Data.csv"input_sig=get_data(file_name=file_name)sampling_rate=20window_length_sec=60# Set this parameter True if the signal has not been filtered:filter_signal=True# Call the PPG signal quality assessment functionclean_indices,noisy_indices=sqa(input_sig,sampling_rate,filter_signal)# Call the PPG reconstruction functionppg_reconstructed,updated_clean_indices,updated_noisy_indices=reconstruction(input_sig,clean_indices,noisy_indices,sampling_rate,filter_signal)# Set the window length for HR and HRV extraction in terms of sampleswindow_length=window_length_sec*sampling_rate# Call the clean segment extraction functionclean_segments=clean_seg_extraction(ppg_reconstructed,updated_noisy_indices,window_length)# Set the peak detection method (optional)peak_detection_method='kazemi'# Call the peak detection functionpeaks=peak_detection(clean_segments,sampling_rate,peak_detection_method)# Call the HR and HRV feature extraction functionhrv_data=hrv_extraction(clean_segments,peaks,sampling_rate,window_length)CitationIf you use our work in your research, please consider citing the following papers:End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction(PPG Pipeline Paper)Conference:2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)Authors:Mohammad Feli, Kianoosh Kazemi, Iman Azimi, Yuning Wang, Amir Rahmani, Pasi Liljeberg@inproceedings{feli2023end,
  title={End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction},
  author={Feli, Mohammad and Kazemi, Kianoosh and Azimi, Iman and Wang, Yuning and Rahmani, Amir and Liljeberg, Pasi},
  booktitle={2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)},
  year={2023},
  organization={IEEE}
}An Energy-Efficient Semi-Supervised Approach for On-Device Photoplethysmogram Signal Quality Assessment(PPG Signal Quality Assessment Paper)Journal:Smart HealthAuthors:Mohammad Feli, Iman Azimi, Arman Anzanpour, Amir M Rahmani, Pasi Liljeberg@article{feli2023energy,
  title={An Energy-Efficient Semi-Supervised Approach for On-Device Photoplethysmogram Signal Quality Assessment},
  author={Feli, Mohammad and Azimi, Iman and Anzanpour, Arman and Rahmani, Amir M and Liljeberg, Pasi},
  journal={Smart Health},
  volume={28},
  pages={100390},
  year={2023},
  publisher={Elsevier}
}PPG Signal Reconstruction Using Deep Convolutional Generative Adversarial Network(PPG Reconstruction Paper)Conference:2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)Authors:Yuning Wang, Iman Azimi, Kianoosh Kazemi, Amir M Rahmani, Pasi Liljeberg@inproceedings{wang2022ppg,
  title={PPG Signal Reconstruction Using Deep Convolutional Generative Adversarial Network},
  author={Wang, Yuning and Azimi, Iman and Kazemi, Kianoosh and Rahmani, Amir M and Liljeberg, Pasi},
  booktitle={2022 44th Annual International Conference of the IEEE Engineering in Medicine \& Biology Society (EMBC)},
  pages={3387--3391},
  year={2022},
  organization={IEEE}
}Robust PPG Peak Detection Using Dilated Convolutional Neural Networks(PPG Peak Detection Paper)Journal:SensorsAuthors:Kianoosh Kazemi, Juho Laitala, Iman Azimi, Pasi Liljeberg, Amir M Rahmani@article{kazemi2022robust,
  title={Robust PPG Peak Detection Using Dilated Convolutional Neural Networks},
  author={Kazemi, Kianoosh and Laitala, Juho and Azimi, Iman and Liljeberg, Pasi and Rahmani, Amir M},
  journal={Sensors},
  volume={22},
  number={16},
  pages={6054},
  year={2022},
  publisher={MDPI}
}AcknowledgmentsWe would like to express our sincere gratitude toProf Pasi LiljebergandProf Amir M. Rahmanifor their guidance and support throughout this project.ContributingWe welcome contributions to enhance and extend the capabilities of the PPG Signal Processing Pipeline.LicenseThis project is licensed under the terms of theMIT License.