# Physics-Informed Neural Networks for Cantilever Dynamics and Fluid-Induced Excitation
## 2 Proposed Approach
### Variants in Activation  
The last layer of a PINN is typically constructed by combining hidden variables linearly, which can be expressed as follows:  
\[\sum_{i}W_{i}\sigma_{i}(f(x,t))+b \tag{4}\]  
where \(f\) represents the non-linear operator of hidden layers except the last layer, and \(W_{i}\) and \(b\) correspond to the weights and biases in the last layer, respectively. The activation function, \(\sigma\), plays a significant role in describing the mathematical expression of the PINN prediction. In this regard, the activation could express the expected mathematical representation of the solution curve. Although the \(\tanh\) activation widely used in PINNs has a number of merits [29], there exist limitations that can deteriorate the expressivity of the neural networks in some problems [30; 31].  
First, the \(\tanh\) activation function exhibits poor performance in reproducing periodic behavior [30]. This could be a serious limitation of the PINN since periodic functions are fundamental base functions that construct solution curves for many physical problems. Second, the \(\tanh\) activation hardly conveys a tendency described from the second-order expansion of the given input vector \(x_{j}\). The first non-linear term in the Taylor series expansion of \(\tanh\), \(\tanh\bigl{(}x_{j}\bigr{)}=x_{j}-\frac{x_{j}^{3}}{3}+\frac{2x_{j}^{5}}{15}+O(x _{j}^{7})\), appears at the third order, which implies that the second-order tendency could not be directly transferred to the next layer. This limitation can be resolved by adding a number of layers, as the second-order variants are reconstructed from the increased higher-order terms. However, this leads to the addition of unnecessary layers as well.  
To address the above-described problems, this study employed an alternative activation function, \(x_{j}+\sin^{2}\bigl{(}x_{j}\bigr{)}\), proposed by [PERSON] et al. (2020). The Taylor series expansion of \(\sin\bigl{(}x_{j}\bigr{)}\) is given by \(\sin\bigl{(}x_{j}\bigr{)}=x_{j}-\frac{x_{j}^{3}}{3!}+\frac{x_{j}^{5}}{3!}+O(x _{j}^{7})\), leading to \(x_{j}+\sin^{2}\bigl{(}x_{j}\bigr{)}=x_{j}+x_{j}^{2}-\frac{2x_{j}t}{3!}+O(x_{j} ^{6})\). This function ensures the inclusion of second-order tendencies in non-linear trends, as shown in the Taylor series expansion, capturing both linear and higher-order tendencies. However, this activation introduces a correlation between the phase and scale in the output, as it directly bypasses the input, \(x_{j}\), and incorporates a periodic function, \(\sin^{2}\bigl{(}x_{j}\bigr{)}\), with \(x_{j}\) as its phase. To overcome this limitation, a linear scaling layer, \(L_{S}\bigl{(}x_{j}\bigr{)}=wx_{j}\), can be added after the activation, resulting in the output \(L_{S}(x_{j}+\sin^{2}(x_{j}))\). The scaling constant \(w\) is determined during training. Although the linear scaling layer may seem unnecessary, it allows the network to explore the outcome of the activation without being affected by the correlation between phase and scale. This approach improves the convergence and robustness of the neural network in experiments. Similarly, adding a linear scaling layer before the activation can also improve performance, which is known as adaptive activation [32].