## 4 Streed-Net
### Encoder  
The encoder structure depicted in Figure 3 is the first block of the STREED-Net architecture. It is composed of an initial convolutional layer, a series of _residual units_, and a final convolution layer. Unlike similar approaches (e.g., STAR [6]), the proposed encoder structure introduces three novel aspects: (i) each layer is time-distributed, meaning that the model learns from a sequence of frames (for time coherence) instead of focusing on each frame singularly. (ii) it applies further convolutions after the residual unit, so that to reduce the frame size and (iii) it applies a BN after each convolution to avoid gradient disappear/explode problems and achieve faster and more efficient reported optimization [42; 43]. Unlike other works from the literature, where the distant temporal information is also used (from the previous day and previous week), the encoder takes as input a four-dimensional tensor \(F\in\mathbb{R}^{H\times N\times M\times 2}\). This tensor is a sequence of consecutive three-dimensional frames conveying flow information of nearby periods (with regard to the prediction time \(t^{\prime}\)). Such a tensor (also referred to as _closeness_ in the literature [7]) is obtained by selecting \(p\) points preceding the prediction time \(t^{\prime}\), i.e., the sequence \([F_{t^{\prime}-p},F_{t^{\prime}-(p+1)},...,F_{t^{\prime}-1}]\). In this way, STREED-Net can focus on the most recent dynamics only. Each frame in \(F\) is processed by the convolution layer to extrapolate spatial information. It is worth noting that in Figure 2, the encoder is represented by a collection of identical blocks in parallel execution on the input frames instead of (as in reality) a single convolution applied sequentially. Such a representation is used to highlight that a time-distributed layer is trained by taking into account all input frames simultaneously. The use  
Figure 3: Encoder. Figure 2: STREED-Net Architecture. of this approach leads the model to identify temporal (that is, inter-frame) dynamics, rather than looking only to spatial dependencies within each frame. Each convolutional layer is followed by a ReLU activation function and a \(BN\) layer. Formally, we have:  
\[E_{t}^{(0)}=BN(ReLU(W_{e}^{(0)}*F_{t}+b_{e}^{(0)})) \tag{3}\]  
where \(E_{t}^{(0)}\) corresponds to the output of the first convolutional layer, \(F_{t}\) is one of \(p\) frames in input to the model and \(*\) the convolution operator. \(W_{e}^{(0)}\) and \(b_{e}^{(0)}\) are the weights and biases of the respective convolutional operation. Next, \(\mathbf{L}\) encoder blocks (see Figure 3) are placed. Each of these blocks is composed of a residual unit followed by a downsampling layer:  
\[E_{t}^{(l)}=Downsampling(ResUntil(E_{t}^{(l-1)})) \tag{4}\]  
where \(E_{t}^{(l-1)}\) and \(E_{t}^{(l)}\) correspond respectively to the input and output of the encoder block; \(l\) takes values in \(\{1,...,\mathbf{L}\}\). The residual units have been implemented as a sequence of two convolutional layers whose output is eventually summed to the block input. Mathematically, in STREED-Net the residual unit are defined as follows:  
\[c_{1}=BN(ReLU(W_{1}^{(l)}*E_{t}^{(l-1)}+b_{1}^{(l)})) \tag{5}\] \[c_{2}=BN(ReLU(W_{2}^{(l)}*c_{1}+b_{2}^{(l)}))\] (6) \[ERU_{t}^{(l)}=E_{t}^{(l-1)}+c_{2} \tag{7}\]  
where \(E_{t}^{(l-1)}\) is the residual unit input and \(ERU_{t}^{(l)}\) (_Encoder Residual Unit_) is used to indicate the result of \(ResUntil(E_{t}^{(l-1)})\). \(*\) is the convolution operator, \(W_{1}^{(l)}\), \(W_{2}^{(l)}\) and \(b_{1}^{(l)}\), \(b_{2}^{(l)}\) are the weights and biases of the respective convolutional operations. For what concerns the _Downsampling_, it has been implemented as:  
\[E_{t}^{(l)}=BN(ReLU(W_{ds}^{(l)}*ERU_{t}^{(l)}+b_{ds}^{(l)})) \tag{8}\]  
where \(W_{ds}^{(l)}\), \(b_{ds}^{(l)}\) and \(*\) indicate a convolutional layer with _kernel size_ and _stride_ parameters set to halve the height and width of the input frame.