Task 4-2: Apply Filters (30 points)
You are required to implement the functions in task_4_2.py.
In this task, you are now given multiple noisy signals. Please use the filters that have been included in
our tutorials to smooth these noisy data.
1. task_4_2_1.pickle: You are supposed to filter it in apply_filter_1(). The sampling
rate is 10Hz.
2. task_4_2_2.pickle: You are supposed to filter it in apply_filter_2(). The sampling
rate is 20Hz.
Note: You can pick the filters according to the shape of the signals. Smoothing is a quite subjective task,
but as a programming task, we have some quantitative metrics. The data for Task 4-2 are synthetic,
meaning we possess the original datasets without any added noise. This allows for a direct comparison
between the filtered output and the true, noise-free signal to objectively evaluate the effectiveness of
the filtering process. Therefore, we can evaluate your results using the following metrics:
• SNR (Signal-to-Noise Ratio): This measures the level of the desired signal relative to the back-
ground noise. A higher SNR means the signal is clearer compared to the noise.
• RMSE (Root Mean Square Error): RMSE quantifies the difference between values predicted by a
model or filter and the values actually observed from the environment that is being modeled or
about which predictions are being made. Lower RMSE values indicate a closer fit to the original
data.
COMP3516@HKU 1
Programming Task for Lab 4 Due: 2025-03-20
• Variations of the first derivative: This metric assesses the smoothness of a signal by measuring
the changes in its rate of change. Smoother signals have less variation in their first derivative.

from scipy.signal import butter, sosfiltfilt
from scipy.signal import savgol_filter
from hampel import hampel
import numpy as np
import os.path as osp
import pickle
import matplotlib.pyplot as plt


class task_4_2:
def __init__(self, data_root="./data/"):
"""
Initializes the task_4_1 class by loading signal data from a specified path.

Parameters:
data_root (str): The root directory where the signal data file is located.
The default value is "./data/".

Attributes:
data (np.ndarray): Loaded signal data.
fs (int): Sampling rate in Hz, initialized from the loaded data.
"""
file_1_n = "task_4_2_1.pickle"
file_2_n = "task_4_2_2.pickle"
with open(osp.join(data_root, file_1_n), "rb") as f:
data1 = pickle.load(f)
with open(osp.join(data_root, file_2_n), "rb") as f:
data2 = pickle.load(f)
self.data1, self.fs1 = data1["values"], data1["fs"]
self.data2, self.fs2 = data2["values"], data2["fs"]
self.clean1 = data1["clean"]
self.clean2 = data2["clean"]

def apply_filter_1(self):
"""
Applies a specified filtering technique to smooth the noisy signal stored in self.data1.

This method is designed to process signals sampled at self.fs1, specifically for the data
provided in 'task_4_2_1.pickle' (self.data1). The goal is to filter out noise and extract meaningful
information from the signal using an appropriate digital filter.

The filter and its parameters should be chosen based on the characteristics of the noise
and the signal of interest.

Returns:
np.ndarray: The filtered signal.

>>> test = task_4_2(data_root="./data/")
>>> filtered = test.apply_filter_1()
>>> np.all(filtered != None)
True
>>> len(filtered) == len(test.clean1)
True
>>> rmse = test._compute_rmse(test.clean1, filtered)
>>> rmse < 0.2
True
>>> snr = test._compute_snr(test.clean1, filtered)
>>> snr > 20
True
>>> dv = test._compute_derivative_variation(filtered)
>>> dv < 0.1
True
"""
filtered = None
# >>>>>>>>>>>>>>> YOUR CODE HERE <<<<<<<<<<<<<<<
# TODO:
# >>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<
filtered = np.array(filtered, dtype=np.float64)
return filtered

def apply_filter_2(self):
"""
Applies a different specified filtering technique to smooth the noisy signal stored in self.data2.

Similar to apply_filter_1, this method targets signals sampled at self.fs2, but it is tailored
for the data provided in 'task_4_2_2.pickle' (self.data2). It involves using a potentially different
digital filter or parameters to address the unique characteristics of this signal and its
associated noise.

The chosen filter should effectively reduce noise while preserving the signal's integrity.

Returns:
np.ndarray: The filtered signal.

>>> test = task_4_2(data_root="./data/")
>>> filtered = test.apply_filter_2()
>>> np.all(filtered != None)
True
>>> len(filtered) == len(test.clean2)
True
>>> rmse = test._compute_rmse(test.clean2, filtered)
>>> rmse < 0.2
True
>>> snr = test._compute_snr(test.clean2, filtered)
>>> snr > 35
True
>>> dv = test._compute_derivative_variation(filtered)
>>> dv < 0.1
True
"""
filtered = None
# >>>>>>>>>>>>>>> YOUR CODE HERE <<<<<<<<<<<<<<<
# TODO:
# >>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<
filtered = np.array(filtered, dtype=np.float64)
return filtered

def _compute_snr(self, clean, filtered):
"""Helper function to compute SNR in dB."""
signal_power = np.mean(clean ** 2)
noise = filtered - clean
noise_power = np.mean(noise ** 2)
if noise_power == 0:
return float('inf')
return 10 * np.log10(signal_power / noise_power)

def _compute_rmse(self, clean, filtered):
"""Helper function to compute RMSE."""
return np.sqrt(np.mean((filtered - clean) ** 2))

def _compute_derivative_variation(self, signal):
"""Helper function to compute standard deviation of first derivative."""
derivative = np.diff(signal)
return np.std(derivative)

consider this hint:
Should we only have to pick one filter for each of the apply_filter functions and adjust its parameters, or are there multiple steps within each function? Thanks!
PermalinkReply
 It is recommended to use multiple filters

Task 4-1: apply_bandpass: the last doctest should be
>>> round(filtered[10], 2), round(filtered[-1], 2)
(0.44, 0.06)

Task 4-2: Remove the duplicated `True` for apply_filter_2()