# Fast Flow Reconstruction via Robust Invertible \(n\times n\) Convolution
## 5 Experiments
### Quantitative Experiments  
**Datasets and Metric:** We evaluate our invertible \(n\times n\) convolution on CIFAR-10 (Figure 3(a)) and ImageNet (Figure 3(b)) with \(32\times 32\) and \(64\times 64\) image sizes. We use bits per dimension as the criteria with which to evaluate models. We compare our method against RealNVP [32], Glow [29] and Emerging Convolution [35]. We adopt the network structures of Glow and replace all invertible \(1\times 1\) convolutions of Glow with our invertible \(n\times n\) convolutions. For the data preprocessing, we follow the same process as in RealNVP [32].  
**Network Configurations:** In the CIFAR experiment, the depth of flow \(K\) and the number of levels \(L\) are set to 32 and 3, respectively. Meanwhile, the depth of flow in ImageNet experiments is set to 48, the numbers of levels of ImageNet \(32\times 32\) and ImageNet \(64\times 64\) experiments are set to 3 and 4, respectively. We use the Adam optimizer [47] to optimize the networks in which batch size and learning rate are set to 64 (per GPU) and 0.001, respectively. We choose Normal Distribution as the prior distribution \(p\,z(z)\sim\mathcal{N}(\mathbf{z};0,\mathbf{I})\) in all experiments.  
The shift function \(\mathcal{S}\) will be not inverse if the \(\alpha_{c}=0\) (\(\exists\ c\in[1...C]\)). Hence, in the training process, we will first initialize \(\alpha_{c}=1\) and \(\beta_{c}=0\) (\(1\leq c\leq C\)). During the learning processing, we keep \(\alpha_{c}\) (\(1\leq c\leq C\)) as a different 0 to guarantee that the shift function \(\mathcal{S}\) is inverse and to guarantee the tractability of the Jacobian determinant. Training models on high-dimensional data requires large memory. To be able to train with a large batch size, we simultaneously and distributively trained the models on four GPUs via Horovod ([[https://github.com/horovod/horovod](https://github.com/horovod/horovod)]([https://github.com/horovod/horovod](https://github.com/horovod/horovod)), accessed on 8 July 2021) and TensorFlow ([[https://tensorflow.org](https://tensorflow.org)]([https://tensorflow.org](https://tensorflow.org)), accessed on 8 July 2021) frameworks.  
**Results:** Table 2 shows our experimental results. In particular, our proposal helps to improve the generative models on ImageNet \(32\times 32\) and ImageNet \(64\times 64\) datasets, which are more challenging than CIFAR-10. In particular, our proposed method achieves a state-of-the-art performance in which the bit per dimension results in ImageNet \(32\times 32\), and ImageNet \(64\times 64\) is \(\mathbf{3.96}\) and \(\mathbf{3.74}\), respectively. In comparison, the Emerging Convolution [35] and Glow achieve similar results in both ImageNet \(32\times 32\) and ImageNet \(64\times 64\) benchmarks, which are 4.09 and 3.81, respectively. Meanwhile, the corresponding results of RealNVP on these benchmarks are 4.28 and 3.98, respectively. As shown by the results, our proposed invertible \(n\times n\) convolution provides a better generative capability than the stand-alone invertible \(1\times 1\) convolution. Since Emerging Convolution uses invertible auto-regressive convolution, our proposal is, therefore, less complicated and has faster inference than Emerging Convolution. In the CIFAR-10 benchmark, although our model does not perform as well as Glow [29] and Emerging Convolution [35], we find it interesting that our method gains competitive results with a small number of modifications. The gap in performance is partially caused by the small amount of CIFAR-10 data that is inefficient for training the well-generalized convolution.  
Figure 4: The examples from the CIFAR dataset (**a**), ImageNet dataset (**b**) and Celeb-HQ dataset (**c**).