# Deep-Learning Integration of CNN-Transformer and U-Net for Bi-Temporal SAR Flash-Flood Detection
## 3 Methodology
### Benchmark Models
#### 3.6.5 Hybrid CNN [66]  
Model architecture: Our model uses a hybrid CNN (both 3D and 2D convolutions) following the Keras functional API. Input tensors of dimensions [p \(\times\) p \(\times\) channels] are reshaped to [p \(\times\) p \(\times\) channels \(\times\) 1]. It is composed of two 3D convolutional layers, the first with 64 filters (kernel size 3 \(\times\) 3 \(\times\) 2) and the second with 32 filters (kernel size 3 \(\times\) 3 \(\times\) 1), all using ReLU activation. After these layers, feature maps are reshaped into a single dimension by combining the dimensions of filter and feature-map channels along which we will perform 2D convolutions. A 2D convolutional layer using 16 filters (3 \(\times\) 3 kernel size) and the rectified linear activation is then applied. This architecture is designed such that the network can learn spatiotemporal patterns through 3D convolutions and treat fused features using 2D convolutions. Finally, the feature maps are flattened to a one-dimensional vector and go through a fully connected layer with 16 hidden units and ReLU activation. A dropout layer with a 0.5 rate for dropout is used to avoid overfitting before the output layer, which has [output_shape] units with Softmax activation, and returns the probability values of classes. 3.6.6. U-Net [67]  
The model architecture uses a modified U-Net CNN with an encoder-decoder structure and skip connections via the Keras functional API. It has three components: the encoder path, a bottleneck, and a decoder path, each using specialized convolutional blocks. However, these blocks perform a 3 \(\times\) 3 convolution followed by batch normalization, ReLU activation, and a 1 \(\times\) 1 convolution with the same normalization and activation. This dual-convolution technique supports spatial feature extraction and channel refinement. The encoder path comprises four blocks, combining a convolutional block with 1 \(\times\) 1 max pooling, with filter counts increasing (8, 16, 32, 64) for hierarchical feature extraction. Each encoder block retains features before pooling through skip connections for the decoder. The bottleneck contains a convolutional block with 128 filters, linking the encoder and decoder while managing abstract features. The decoder mirrors the encoder with four blocks, starting with a 1 \(\times\) 1 transposed convolution (upsampling) and concatenating corresponding skip features, merging low-level spatial data with high-level semantics. Filter counts decrease (64, 32, 16, 8), reconstructing spatial resolution and reducing feature complexity. After the decoder, global average pooling condenses dimensions to 1 \(\times\) 1, summarizing information across the feature map. The network ends with a dense output layer with [output_shape] units and sigmoid activation for binary classification. 3.6.7. Vision Transformer (ViT) [43]  
The ViT model is a patch-based technique that processes input tensors, utilizing a transformer architecture for feature extraction and classification. The patches layer segments the input image (3 \(\times\) 3) into non-overlapping 1 \(\times\) 1 patches, which are projected into a higher-dimensional space (projection_dim = 64) via the PatchEncoder layer with learnable position embeddings. The transformer backbone includes four layers with: a multihead self-attention mechanism (4 heads), layer normalization (epsilon = 1 \(\times\) 10\({}^{-6}\)) with residual connections, and a multilayer perceptron (MLP) with GELU activation, consisting of two dense layers sized [64, 128] and a dropout rate of 0.1. Following the transformer layers, the architecture normalizes the final encoded representation, flattens spatial dimensions, applies dropout regularization at 0.5, and includes a final MLP head with two dense layers sized [32, 64] and a dropout rate of 0.5, finishing with an output layer using Softmax activation for classification. The model uses the Adam optimizer with a learning rate of 0.001 and weight decay of 0.0001.