# Fast Flow Reconstruction via Robust Invertible \(n\times n\) Convolution
## 2 Related Work  
The generative models can be divided into two groups, that is, _Generative Adversarial Networks_ and _Flow-based Generative Models_. In the first group, _Generative Adversarial Networks_[33] provide an appropriate solution to model the data generation. The discriminative model learns to distinguish the real data from the fake samples produced using a generative model. Two models are trained as they are playing a mini-max game. Meanwhile, in the second group, the _Flow-based Generative Models_[29; 31; 32] are constructed using a  
Figure 1: Reconstruction results using our proposed approach. sequence of _invertible_ and _tractable_ transformations. Unlike GAN, the model explicitly learns the data distribution \(p(\mathbf{x})\) and therefore the loss function is efficiently employed with the log-likelihood. In this section, we discuss several types of flow-based layers that are commonly used in flow-based generative models. An overview of several invertible functions is provided in the Table 1. In particular, all functions easily obtain the reverse function and tractability of a Jacobian determinant. The symbols \(\odot\), / denote element-wise multiplication and division. \(h,w\) denotes the height and width of the input/output. \(c,i,j\) are the depth channel index and spatial indices, respectively. **Coupling Layers:** NICE [31] and RealNVP [32] presented coupling layers with a normalizing flow by stacking a sequence of invertible bijective transformation functions. The bijective function is designed as an affine coupling layer, which is a tractable form of Jacobian determinant. RealNVP can work in a multi-scale architecture to build a more efficient model for large inputs. To further improve the propagation step, the authors applied batch normalization and weight normalization during training. Later, [PERSON] et. al. [43] presented a continuous mixture cumulative distribution function to improve the density modeling of coupling layers. In addition to improving the expressiveness of transformations of coupling layers [43], utilized multi-head self-attention layers [44] in the transformations. **Inverse Autoregressive Convolution:** [PERSON] et al. [45] introduced autoregressive autoencoders by constructing an extension of a non-variational autoencoder that can estimate distributions and is straightforward in computing their Jacobian determinant. Masked autoregressive flow [36] is a type of normalizing flow, where the transformation layer is built as an autoregressive neural network. Inverse autoregressive flow [30] formulates the conditional probability of the target variable as an autoregressive model. **Invertible \(1\times 1\) Convolution:** [PERSON] et al. [29] proposed simplifying the architecture via invertible \(1\times 1\) convolutions. Learning a permutation matrix is a discrete optimization that is not amenable to gradient ascent. However, the permutation operation is simply a special case of a linear transformation with a square matrix. We can pursue this work with convolutional neural networks, as permuting the channels is equivalent to a \(1\times 1\) convolution operation with an equal number of input and output channels. Therefore, the authors replace the fixed permutation with learned \(1\times 1\) convolution operations. **Activation Normalization:**[29] performs an affine transformation using scale and bias parameters per channel. This layer simply shifts and scales the activations with data-dependent initialization that normalizes the activations given an initial minibatch of data. This allows the scaling down of the minibatch size to \(1\) (for large images) and the scaling up of the size of the model. **Invertible \(n\times n\) Convolution:** Since the invertible \(1\times 1\) convolution is not flexible, [PERSON] et al. [35] proposed an invertible \(n\times n\) convolution generalized from the \(1\times 1\) convolutions. The authors presented two methods to produce the invertible convolutions: (1) _Emerging Convolution_ and (2) _Invertible Periodic Convolutions_.