SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(b)
Figure 9: (a): FER performance of the triple decoder with the theoretical limit for the WiMAX LDPC code (b): FER performance over the Rayleigh channel for the WiMAX LDPC code.
Extending the proposed method to include more than two decoders is feasible and can be applied in a similar manner. For instance, in the case of the WiMax LDPC code (shown in Fig. 6(a)), uncorrected words from the error-floor region of the base + post decoder (i.e.,  $\mathrm{E_b / N_0}$  4.5 dB) are collected to train a third decoder. Fig. 9(a) shows that the aid of this third decoder (labeled as "Base+Post+Third) reduces the error-floor further. It is worth mentioning that introducing this third decoder might lead to increased decoding latency. Moreover, the process of collecting uncorrected words would be time-consuming, especially in the very low FER region. Addressing these challenges could be a promising direction for subsequent research. In addition, we add a graph showing the finite length performance limit [44].
We want to note that the concept of multi-stage decoding has been employed in the previous works [45, 46, 47], particularly in the field of coding theory. Our study has similarities with theses previous approaches, as we also carries out a two-stage decoding. However, unlike physically divided decoders in [45, 46, 47], our method utilizes a single LDPC decoder. While the proposed decoder is conceptually divided into two stages (base/post) based on iterations, in practice, we employ only a single decoder with distinct weight parameter sets. Moreover, our approach has a novelty in that the post decoder is 'trained' dependent on the results of the base decoder.
7.6 Application to a different channel
The concept of training using uncorrected words can be consistently applied regardless of the channel type. In Fig. 9(b), we include the results for the Rayleigh channel with the scale parameter  $\alpha = 1$ , showing the effectiveness of our proposed method holds across different channel types.
7.7 Application to long length codes
Figure 10: FER performance for the long 5G LDPC code of length 1248.
We show the results for the (1248, 1056) 5G LDPC code in Fig. 10. This result demonstrates the proposed method is also effective for long codes. There is little improvement in waterfall performance with the NMS technique at such long lengths. However, the performance improvement in the error-floor region using our proposed method is clearly evident.
7.8 Comparison with the augmented neural decoders
Table 5: Comparison with [5], [34], [6] in terms of the waterfall performance of short codes. The results are measured by the negative natural logarithm of BER. Higher is better.
Architecture	Vanilla NBP	NBP+HyperNet.	Transformer
Method	BP	Orig. NBP [5]	Proposed	Hyper [34]	ECCT [6]
Eb/N0	4	5	6	4	5	6	4	5	6	4	5	6	4	5	6
Polar (64,48)	4.74	5.94	7.42	4.70	5.93	7.55	4.93	6.64	8.77	4.92	6.44	8.39	6.36	8.46	11.09
BCH (63,51)	4.58	5.82	7.42	4.64	6.21	8.21	4.72	6.42	8.96	4.80	6.44	8.58	5.66	7.89	11.01
MacKay (96,48)	8.15	11.29	14.29	8.66	11.52	14.32	8.26	11.83	15.85	8.90	11.97	14.94	8.39	12.24	16.41
In Table 5, we compare the waterfall performance of various neural decoders for short length codes. To ensure a fair comparison with other works, we employ the NBP decoder and utilize the soft-BER loss function for BER performance optimization while making use of full diversity weights. Out of a total of 50 iterations, 20 iterations are allocated for base decoding and 30 iterations for post decoding.
The result reveals a notable performance improvement over the original NBP [5], which employs the same vanilla NBP architecture. The improvement is particularly pronounced at high SNR. We achieve even better performance at high SNR than the work [34], which adds hyper-networks to the vanilla NBP architecture. In comparison to the ECCT [6] using the transformer architecture, our method shows inferior performance for high density codes (Polar, BCH), yet it achieves fairly comparable performance for LDPC codes. Owing to its intricate transformer structure, the ECCT requires higher training and decoding complexity than NMS decoders [6], implying its current limited practicality. Nevertheless, it is worth emphasizing that our proposed training method can be adaptable across any architecture. In other words, there's promising potential in future research that integrates the proposed training methods with the architecture with hyper-networks or the transformer architecture.
7.9 Ablation study
Table 6: Ablation analysis.
O: Boosting X: Conv.	O: Block-wise X: One-shot	O: Proposed sharing X: Full diversity	FER (at Eb/N0 5.0dB)
X	X	X	3.14 × 10-6
O	X	X	2.77 × 10-7
O	O	X	1.84 × 10-7
O	O	O	1.85 × 10-7
In Table 6, we provide the ablation study for the WiMAX LDPC code. The result shows that the boosting learning is the core technique to reduce the FER performance and the block-wise training also contributes the FER reduction, while the proposed sharing technique does not involve performance degradation.