## 3 Proposed Approaches
### Sinusoidal Activation Function  
The \(GELU\) activation function used in the UniTS model is a nonlinear activation function based on the Gaussian error function, proposed by [PERSON] and [PERSON] in 2016 [20]. Its definition is presented by (1):  
\[GELU(x)=0.5x(1+tanh(\sqrt{\frac{2}{\pi}}(x+0.044715x^{3}))) \tag{1}\]The _GELU_ activation function smoothly approaches zero for negative values and gradually increases for positive values. This characteristic enables the _GELU_ to outperform traditional rectified linear unit (_ReLU_) activation functions in many deep learning models, as it effectively handles the negative input data, thereby enhancing the model's performance. The formula of the _ReLU_ activation function is presented in (2). [PERSON] and [PERSON] proposed the _ReLU_ activation function in 2010 [21]. The _ReLU_ activation function maintains linearity for positive values and directly outputs zero for negative values. This feature makes the _ReLU_ activation function computationally efficient due to its simple calculation method, which reduces model complexity during training. \[ReLU(x)=max(0,x) \tag{2}\]  
However, the _GELU_ and _ReLU_ activation functions have limitations in capturing periodic data features, thereby hindering the extraction of repeating patterns in such data. Specifically, periodic data often exhibit conspicuous cyclic fluctuations and repetitive structures, which may be attenuated when processed through the _GELU_ and _ReLU_ activation functions. This could result in a diminished capacity of the model to discern these crucial features. Hence, for datasets containing significant periodic characteristics, it might be necessary to integrate alternative methods or activation functions to more effectively capture these features. [PERSON] and [PERSON] explored the potential of the _SAF_ in deep learning [22]. Despite being less commonly utilized compared to other activation functions, the _SAF_ has demonstrated superiority in capturing periodic features in specific scenarios. In a study by [PERSON], [PERSON], and their colleagues [23], researchers investigated the application of sinusoidal activation functions in deep neural networks for time-series modeling problems. They initialized the network with fast Fourier transform (_FFT_) and implemented dynamic parameter optimization, concluding that the _SAF_ effectively captures periodic patterns in data and outperforms traditional activation functions in certain tasks. Additionally, [PERSON] et al. discovered that neural networks with periodic activation functions (such as the _SAF_) are particularly adept at representing complex natural signals and their derivatives [24]. These networks excel at fitting and representing details of time-series data, making them especially suitable for processing periodic signals such as ENFs. The mathematical representation of the _SAF_ is depicted in (3). This function directly applies a sinusoidal transform to the input value, \(x\), converting the linear input into a periodic output, enabling the model to effectively capture the periodic characteristics of the input signal. Figure 2 illustrates the schematic diagrams of the _ReLU_, _GELU_, and _SAF_. \[SAF(x)=sin(x) \tag{3}\]  
To compare the impact of different activation functions on processing the original periodic signal, we conducted simulations using periodic signals and applied them successively to the _SAF_, _ReLU_, and _GELU_, as depicted in Figure 3. The unique sinusoidal activation mechanism of the _SAF_ enables it to more effectively capture and replicate the periodic variations in the signal, thereby preserving a greater proportion of its original periodic characteristics during processing. Figure 2: Illustration of activation functions. As previously mentioned, the ENF signal exhibits a robust periodicity. In order to bolster the model's capacity for handling ENF data, we incorporated the sinusoidal function as the activation function.