# RCA-PixelCNN: Residual Causal Attention PixelCNN for Pulsar Candidate Image Lossless Compression
## 4 Experiments
### Results and Analysis  
#### 4.4.1 Experiment Setting and Results  
The model in this paper was implemented using the PyTorch framework on an NVIDIA GeForce GTX 1080 GPU. The implementation details of the model structure are as follows: At the front end of the network, a large-sized convolutional layer with a \(7\times 7\) kernel of type A is used, with 3 input channels and 128 output channels, followed by a BatchNorm layer and a ReLU activation function layer. Immediately following is a small-sized 3 \(\times\) 3 convolutional layer with a kernel of type B, also having 128 input and output channels, and including a BatchNorm layer and a ReLU activation function layer. There are seven Residual Causal layers, where the residual branch structure is Conv B\(\_1\times 1\rightarrow\)BatchNorm\(\rightarrow\)ReLU \(\rightarrow\)Conv B\(\_3\times 3\rightarrow\)BatchNorm\(\rightarrow\)ReLU \(\rightarrow\)Conv B\(\_1\times 1\). The first convolutional layer has 128 input channels and 64 output channels, the second convolutional layer has 64 input and output channels, and the last convolutional layer has 64 input channels and 128 output channels. The main path is connected to the residual branch by adding their information. The residual-causal-attention block was injected as a plugin into the main network, and its implementation details were introduced in Section 3.2. Finally, there is a \(1\times 1\) Causal Convolution layer with a \(1\times 1\) kernel of type B, having 128 input channels and 256 output channels. As can be seen, we do not stack Conv B layers to expand the receptive field; instead, we use Residual Causal Attention and Residual Casual Layers. The optimizer used is Adam, with an initial learning rate of 0.01 and beta parameters of 0.9 and 0.99. In order to make a fair comparison of the density modeling abilities between RCA-PixelCNN and the GMM (Gaussian mixture model) as well as STM (Student's t mixture model), the evaluation metric used is the average negative log likelihood. Both training and testing data are sourced from the HTRU1 dataset. Our RCA-PixelCNN model directly models 2D image matrices and 3D data volumes, whereas the GMM and STM can only model 1D data sequences. Since the HTRU1 images are of size 64 \(\times\) 64, they are flattened to 1D, resulting in a dimensionality of 4096. Due to the high dimensionality, convergence of models like the GMM and STM becomes challenging. To address this issue, for the training set, we randomly extract 800,000 8 \(\times\) 8 image patches from the training data. Through model selection experiments, we determine the optimal number of mixture components for the GMM and STM to be 8. To avoid overfitting or underfitting, the models are trained on the training set, and their performance in terms of average negative log likelihood is evaluated using the test set. As shown in Table 1, we compare the experimental results of RCA-PixelCNN with other models on the HTRU1 test dataset. The average negative log-likelihood score for the mixtures of the Gaussian model is 3.51 bits per pixel (bpp), which performs well in fitting natural images. However, the mixtures of the Student T model, which is typically better suited for long-tailed distributions, perform worse than the GMM, suggesting that the candidate pulse images do not follow a long-tailed distribution. The PixelCNN model achieves an average negative log-likelihood score of 3.11 bpp, indicating the best performance among the compared models. This highlights the superior modeling capability of neural-network-based methods compared to traditional approaches. Within the PixelCNN framework, the RCA-PixelCNN model with the added Res-Causal-Attention module exhibits the best performance, outperforming the standard PixelCNN by 0.33 bpp. This demonstrates that deep-learning-based modeling methods outperform traditional data distribution modeling approaches, and the inclusion of the Res-Causal-Attention module further enhances the performance of PixelCNN.