# A Multi-Scale Unsupervised Feature Extraction Network with Structured Layer-Wise Decomposition
## 2 Materials and Methods
### Loss Functions
This expression corresponds to the squared norm of the 2D gradient vector, which serves to penalize large changes in the feature map between adjacent positions. In practice, neural networks operate on discrete grid-based data, so we approximate the continuous partial derivatives using forward finite differences:  
\[\frac{\partial f^{(1)}_{i,j}}{\partial x}\approx f^{(1)}_{i+1,j}-f^{(1)}_{i,j},\quad\frac{\partial f^{(1)}_{i,j}}{\partial y}\approx f^{(1)}_{i,j+1}-f^{(1)} _{i,j} \tag{4}\]  
These approximations in Equation (4) estimate the local change in activation values by computing the difference between a pixel and its immediate neighbor in the corresponding direction. Forward difference usage is computationally efficient and is widely used in discrete image processing. Substituting these approximations into Equation (2), we obtain the discretized version of the smooth layer loss in Equation (5). \[\mathcal{L}_{\text{smooth}}\approx\lambda_{1}\sum_{i,j}\biggl{[}\left(f^{(1) }_{i+1,j}-f^{(1)}_{i,j}\right)^{2}+\left(f^{(1)}_{i,j+1}-f^{(1)}_{i,j}\right)^ {2}\biggr{]} \tag{5}\]  
This formulation penalizes high-frequency components in the spatial structure of the feature map by minimizing the squared differences between adjacent pixels. As a result, it promotes locally smooth outputs and reduces visual artifacts such as noise. The hyperparameter \(\lambda_{1}\) controls the relative strength of this regularization term in the total loss function. The entire loss term is differentiable and compatible with automatic differentiation frameworks. In the implementation, it can be computed efficiently using convolutional filters or element-wise operations on tensor slices. In addition to encouraging global smoothness, it is often desirable to preserve important local details such as edges and texture boundaries. To this end, we introduce a detail layer loss, which penalizes the absolute magnitude of the spatial gradients. This formulation aligns with the concept of total variation (TV) regularization, a well-known technique in signal processing and computer vision. In Equation (3), the loss of detail layer computes the total variation by summing the absolute values of the first-order derivatives across both spatial directions. It encourages the solution to be smooth in part while allowing for sudden changes when necessary. As in the previous section, we approximate the partial derivatives on a discrete grid using forward finite differences in Equation (6). \[\frac{\partial f_{i,j}^{(2)}}{\partial x}\approx f_{i+1,j}^{(2)}-f_{i,j}^{(2)},\quad\frac{\partial f_{i,j}^{(2)}}{\partial y}\approx f_{i,j+1}^{(2)}-f_{i,j}^ {(2)} \tag{6}\]  
Substituting these into Equation (3) yields the discretized form of the loss of detail layer in Equation (7). \[Loss(\hat{f})=\sum_{i,j}(\hat{f}_{i,j}-f_{i,j})^{2}+\lambda_{1}\sum_{i,j} \Bigl{(}\
abla f_{i,j}^{(1)}\Bigr{)}^{2}+\lambda_{2}\sum_{i,j}\Bigl{|}\
abla f _{i,j}^{(2)}\Bigr{|}+\lambda_{3}\sum_{i,j}\log(1+f_{i,j}^{(3)}) \tag{2}\]0  
This discretized formulation corresponds to the first-order total variation in two dimensions. It is widely used to preserve structure while reducing minor variations and noise. The hyperparameter \(\lambda_{2}\) controls the strength of the regularization. Higher values increase edge preservation and denoising at the potential cost of over-smoothing fine textures. To further regulate the scale and dynamics of the output values, we introduce a residual layer loss based on a logarithmic penalty. This function discourages large magnitudes in the output activations while maintaining numerical stability, particularly for small values. The residual loss is defined as:  
\[Loss(\hat{f})=\sum_{i,j}(\hat{f}_{i,j}-f_{i,j})^{2}+\lambda_{1}\sum_{i,j} \Bigl{(}\
abla f_{i,j}^{(1)}\Bigr{)}^{2}+\lambda_{2}\sum_{i,j}\Bigl{|}\
abla f _{i,j}^{(2)}\Bigr{|}+\lambda_{3}\sum_{i,j}\log(1+f_{i,j}^{(3)}) \tag{2}\]1  
In Equation (8), \(x_{i,j}\) denotes the activation value at position \((i,j)\), and \(\lambda_{3}\) is a regularization coefficient.