Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
Authors: Hee-Youl Kwak
Dae-Young Yun
Yongjune Kim
Sang-Hyo Kim
Jong-Seon No
Venue: NeurIPS 2023 poster (2023)
Keywords: Error-floor
Low-density parity-check codes
Boosting learning
Training shcedule
weight sharing
Neural decoders
Min-sum
TL;DR: We show that the machine learning techniques can solve the error-floor issues of low-density parity-check codes, which can extend the applications of LDPC codes to 6G communication, crypto systems, and storage systems.

Abstract: Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor.

Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
Youl Kwak University of Ulsan ghy1 mhy11220gmail.. com
Dae-Young Yun Seoul National University
Yongjune Kim POSTECH yongjune@postech..ee ac.
bigbow12040snu. ac
Sang-Hyo Kim
Jong-Seon No Scoul National University j janoesnu. ac kr
Sungkyunkwan University iasshkimeskku edu
Abstract
Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the етгоr rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error and deploying LDPC codes in scenarios de- manding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First. by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block Mock-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively low- ers the error- error-floor with minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance com- pared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at hhttpps: :// :/:g:://ttii githuu............... com gtmm//hyy222/LPCCC__ Errrrr_rrrrr.. _Floor
Introduction
The field of learning-based decoding for error-correctnng codes began with research training neural networks to produce the information vector when given a distorted codeword [| These works assume arbitrary neural network with prior knowledge of decoding algorithms, and accordingly, face the challenge of learning a decoding algorithm. In contrast, model-based neural decoders designed by mapping well-known graph-base iterative decoding algorithm, such as belief propagation (BP) and min-sum (MS) decoding algorithms, to a neural network and then training algoorithng
its weights [5). Compared to the arbitrary network approaches the error correction transformer 6). model-based neural decoders offer the advantages of guaranteeing the performance of existing iterative algorithms and using hardware architectures 7 that are already well optimized for iterative decoding algorithms.
"Corresponding authors.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

LDPC codes have been incorporated into WiMAX and 5G communication systems 8 91 owing to their strong crror-correcting capabilitiess and low decoding complexity [T, However, more advanced LDPC coding technology needs to be developed for diverse communication environments lying in the scope of future 6G systems. In particular, for environments that require extremely low frame error rate (FER) such as the next generation ultra-reliable and low-latency communications (XURLLC) 12], it is crucial to mitigate the error-floor in the decoding of LDPC codes. The error-floor phenomenon refers to an abnormal phenomenon where the FER does not decrease as rapidly as in the waterfall region M [3] The error-floor phenomenon also should be addressed for systems demanding very high reliability, such as solid-state drive (SSD) storage [14, DNA storage [15], and cryptosystems [16]. However, enhancing other features of LDPC codes often inadvertently reinforces the error-floor phenomenon optimizing LDPC codes for superior waterfall performance or decoding by low complexity decoders such as quantized MS decoders [17]. Therefore, research focused on alleviating the error-floor, especially when decoding LDPC codes optimized for performance with low decoding complexity, has become significant. Such advancements will broaden the applications of LDPC codes.
side effect. For instance, the error-floor tends to be intensified when
1.1 Main contributions
With this need in mind, we focus on how to train a low-complexity neural MS (NMS) decoder to prevent the error-floor in well designed LDPC codes. The main contributions of the paper are threefold as follows.
Boosting learning using uncorrected words: We first leverage the boosting learning technique [8] 19 that employs a sequential training approach for multiple classifiers, wherein subsequent classifiers concentrate on the data samples that preceding classifiers incorrectly classify. Inspired by this method, we divide the neural decoder into two cascaded neural decoders and train the first decoder to be focused on the waterfall performance, while training the second decoder to be specialized in handling the uncorrected words that are not corrected by the first decoder due to the error-floor phenomenon. Uncorrected words in the error-floor region mostly contain small-error patterns related to trapping sets or absorbing sets [. which can be effectively corrected by weighting decoding messages. As a result, a significant performance improvement in the error-floor region is achieved by boosting learning.
Block-wise training schedule with retraining: To mitigate the error-floor, iterative decoding typically requires large number of decoding iterations, often exceeding 50 17 20 21 22]. However, NMS decoders encompassing many iterations can undergo the vanishing gradient problem in training (23. To address this problem, we propose a new training schedule inspired by block- wise training The proposed block wise training schedule divides the entire decoding iterations into sub-blocks and trains these sub-blocks in sequential manner. Additionally, rather than fixing the weights trained from previous blocks, we retrain them to escape from local minima. As a result, the proposed schedule enables to train numerous weights for all 50 iterations successfully while outperforming both the multi-loss method (5) and the iter-by-iter schedule (26].
methods (24 25
The weight sharing technique is way to reduce the number of trainable weights by grouping specific weights to share a common value. The waterfall performance does not severely degrade even if we bundle all weights for each iteration [7|26] 26). However, our observations indicate that this does not hold true in the error-floor region, implying that higher degree of weight diversity is necessary to correct error patterns causing the flror. To obtain sufficient diversity with a minimal number of weights, we dynamically assign different weights to unsatisfied check nodes (UCNs) and satisfied check nodes (SCNs) in the decoding process. By utilizing only two weight values for SCNs and UCNs each itcration, we achieve the performance of the NMS decoder using different weights for every edge. This method reduces the number of weights to be trained to only 2.6% of the original number of weights.
Weight sharing technique with dynamic weight allocation:
We employ these training methods on a range of representative LDPC codes adopted for standards such as WiMAX R IEEE 802.1 In (28. and 5G new radio 9. The FER point the onset of the error-floor diminishes by over two orders of magnitude for all codes compared to conventional weighted MS (WMS) decoding 229. Compared to existing NMS decoding approaches [27 26), our
proposed scheme exhibits notably enhanced capability to suppress the error-floor. This scheme also
achieves a similar performance as the otate-of-the-art post- protssiogsssing method in 22 with only a third of the iterations.

---

Table 1: Comparison between model-basee neural decoders
Reference
Codes
Target region
Decoders
Training sample
Training schedule
Weight sharing
Waterfall,
Block-wise
Spatial with
This work
Standard LDPC
MS
Uncorrected words
Ervor-flor
with retraining
UCN weights
BCH
Waterfall
BP. MS
Received words
One-shot
Temporal
Standard I. IDC.
Waterfall
BE, MS
Received words
lter-hy-lee -ter
Spatial
30
Short LDPC
Waterfall
BP
Absorbing set
One-shot
Temporal
Waterfall,
Regular LDPC
FAID
Received words
One-shot
Tempural
Brvor-florr
Short LDPC
Waterfall,
FAID
Trapping
One-shot
Temporal
Envor-fooo
Standard LDPC
Waterfall
Layered
Received words
Iter-by-lter
UCN weights
1.2
Related works
We compare the proposed training scheme with the existing schemes for NMS decoders Table First, all works except [1 32 aim to improve the waterfall performance. Although the scope of the works [3]] includes the error-floo performance, they assumed specifie conditions of binary symmetric channels (BSC) and FAID, while we deal with the more general situation of additive white Gaussian noise (AWGN) channels and MS decoding. Regarding the training sample selection, the
training samples can be received words randomly taken from the AWGN channel 5 27
31 33] or
codewords with erroneous trapping sets (or absorbing sets) 30 32
However, to use the method in
30 32. trapping sets should be enumerated, which is only applicable short LDPC codes and not
feasible for medium to large length standard LDPC codes. In contrast, the proposed boosting method, which generates training samples through decoding with linear complexity, be applied even to codes of several thousand lengths.
For scheduling of training. it is common to train all weights at once (One-shot training) 5 and some works sequentially train the weights corresponding to a single iteration locally [27 33), while we train a block of iterations with retraining. In terms of the weight sharing techniques, confirm that the proposed sharing technique using UCN weights is superior to the spatial or temporal sharing technique used in [5] 27 30 31 32) Meanwhile, method of assigning different weights to UCNs has been introduced in [33, but they applied the same UCN weight to all CNs belonging to a proto CN when at least one CN is unsatisfied, whereas we pinpoint specific UCNs and apply weights individually.
There have been studies adding hypernetworks to the Vanilla NMS decoder 34 35 36
or using a transformer architecture (6 to improve the waterfall performance at the expense of increased training and decoding costs. While the proposed training techniques are broadly applicable to these augmented neural decoders, this work primarily aims to improve the error-floor performance of the Vanilla NMS decoder under practical conditions.
2 Preliminaries
2.1 LDPC codes
In this paper, we consider quasi-cyclic (QC) LDPC codes, which have been adopted various applications due to their implementation advantages [3] 37]. The Tanner graph of QC-LDPC code, consisting of n = Nz VNs and m = Mr CNs, can be obtained by lifting the protograph, composed of M proto CNs and N proto VNs, with lifting factor 38) Let E be the total number of edges in the protograph. As a running example, we use the WiMAX QC-LDPC code of length n 576
and code-rate 3/4 with N = 24, M 6, E 88, and 24 g
2.2 Neural min-sum decoding
For iteration e, let ma represent the message from CN to VN and let m represent the message from VN e to CN c. The neighboring nodes of are represented by N(x). The initial (0)
(0) conditions are m m me- 0 for the channel LLR mah of VN v. For é 1,..., Z, the
3

---

Input nodes (++ my?
output nodes (-+ mag)
bilden
hidden
layer
laye:
layer
layer
(b)
Figure 1: (a) The Tanner graph of an LDPC code and (b) the neural network corresponding NMS decoding with maximum iteration of = 2.
NMS decoding algorithm
S
updates the messages as follows
m p-
(E)
m
+
Σ
m
(1)
d'E(())
m m
(f)
II d'EN(c))
sgn
(1) (m
min v'Ei(c)
|m!0
mm
where We and we-o are called the VN weight and CN weight, respectively. At the last iteration output LLRs m. are computed as m.; = m +ZEEN())) me-vv"
By quantizing me-te, m mo and m the quantized NMS decoding algorithm is obtained. The quantized decoders are widely used in practical applications due its low complexity and commonly employed in existing error-floor researches [17 20 21 22 Therefore, we use it to ensure fair
comparison. Specifically, we use 5-bit uniform quantization with a maximum magnitude of 7.5 and
step size of 0.5 for the quantized NMS decoder in [20 21 22
2.3 Training weights for the NMS decoder
If all the weights in ( and are set to 1, NMS decoding is equivalent to MS decoding or VN weights ( are 1 and CN weights (6) have the same value, the decoder operates as the WMS decoder [40]. The NMS decoder gains performance improvement over the WMS or MS decoder by greatly increasing the diversity of weights. However, the full diversity weights increase the training complexity and require a large amount memory to store the weights. Therefore, previous studies used weight sharing techniques by assigning the same value to weights with the same attributes. First, since this paper deals with QC-LDPC codes, we use the protograph weight sharing technique (26] by default, assigning the
weight value the VNs (or CNs) belonging
proto VN (or CN). Then,
the weights to be trained are represented by (why. We, wo } for a proto VN Up and proto CN cp- The total number of weights is then (N + E'}. If we employ spatial weight sharing in (27]. only one VN weight and one CN weight remain for each iteration, and the weights are (fo(f) u. ( with total number of 24. On the other hand, by using temporal weight sharing (55 eliminate differences between iterations, the weights are [w., Wop-v J. and the number is (N + E).
(E)
The neural network in Fig. bbb corresponds to NMS decoding of the Tanner graph in Fig. [ with 2=2 2. The input to this neural network is the channel LLR vector (m ..... mnsh) and the output is the output LLR vector hidden layer has number of nodes equal the number of edges in the Tanner graph. In the odd hidden layers, the VN to CN message operation in is performed, while in the even hidden layers, the CN to VN message operation in (2) is performed. The input layer is also connected the odd hidden layers, which corresponds to the addition of the channel LLR in T The messages from the
). For each iteration, two hidden layers are arranged, and each
26-th hidden layer to the (2e + 1)-th hidden layer are weighted by w v. and the messages from the

---

input nodes to the (28 + 1-tth hidden layer are weighted by w As our goal is to reduce the FER in the error-floor region, we use the FER loss, & 1 - sgn mini
32
Proposed training method
Algorithm 1 Training method
Input: 8 d= + Eg. R2
I: Base decoder training: Train the weights ws (u19), w/ with the received words sampled at Ri 2: Collecting the uncorrected words: Run base decoder with the received words sampled at R2 and collect the received words when the decoder
Base decoder Training
Collecting the uncorrected words
Post decoder Traininng
Received words sampled
Received words sampled R2
Uncorrected wards
Uncorrected words
3: Post decoder training:
weights we (w(9), wlf, ai/11)7 Hett,1 with the uncorrected words block- training schedule.
Decoding with We
Collect uncormated wurds
Output: Trained weights Wo. Wp
Train Wa
Train Wp
(a)
(b)
Figure 2: The proposed training method represented by (a) an algorithm and (b) a block diagram.
In this section, we introduce the proposed training methods through three subsections. The organized training algorithm is shown in Fig. 2
3.1 Boosting learning using uncorrected words
For the boosting learning approach, we first divide the entire decoding process into two stages: the base decoding stage {1.... {1} and the post decoding stage (61 + + 1,.....4. 4 + lg = Z}. Training of base decoder follows the conventional training method: the received words sampled from E re- gion R, are used as training samples. Specifically, we set & = 20 and R, {2.0, 2.5, 3.0, 3.5, 4.0). which corresponds the waterfall region of WMS decoding. We use the spatial sharing technique (i.e..
WB (w(), (()) since this achieves comparable performance to the full diversity weights in the waterfall region.
In Fig. a). the base NMS decoder is compared with the MS decoder and the WMS decoder with single weight of 0.75 for e = 20. The WMS decoding performance has severe error-floor even though its waterfall performance better than the MS decoding performance. Compared to the MS and WMS decoders, the NMS decoder for 20 performs better over the training range R2, On the other hand, the NMS decoder performs worse than the MS decoder in the error-floor region (e.g.. 4.5 dB), which is outside R, To improve the performance the error-floor region, straightforward approach is extending the training range R1 to include the error-floor region. However, the FER of the base decoder for received words from the error-floor region is very low, resulting an almost negligible FER loss. Consequently, integrating the error-floor region into the training range does not impact the weight update process.
Before training the post decoder, we first collect uncorrected words that the trained base decoder fails to correct among the received words sampled from region R2. Then, the uncorrected words serve as training samples for the post decoder, which is distinct from the conventional training methods. The post decoder trains the weights (w!f) wep wep- ) with the aim of correcting the uncorrected words. After completing the training. the trained weights are used for the NMS decoding algorithm in ( (2). From the perspective of the NMS decoder. performs continuous decoding up to iteration using the trained weights, but for the sake of discussion, we assume as if there are two cascaded decoders (base and post) in the perspective of training. Note that we employ the full diversity weights for the post decoder confirm the best performance but we will introduce the shared weights wp (used in Fig- 2 in the next subsection. We also set l2 10, & 30 for this experiment, and subsequently extend the maximum number of iterations in the following subsection.
To analyze the effectiveness of the proposed boosting learning, compare the following three cases.
5

---

10
-Q-Cax lose decoding
10
C-
---Cre 2 ahea lous decoding
Crue desuting
Ta Case 8 abet poot deodling
10
Distrleation
500
10
600
10
300
10
200
10
100
107
Bigh% (c13)
Number
arror
(a)
(h)
Figure 3: (a) Decoding performances of the MS, WMS, NMS decoders and (b) Error distributions after base and post decoding for Case and Case 2.
Case Uncorrected words sampled at 4.5 dB in the error-floor region (i.e., R2 4.5).
Case 2 Uncorrected words sampled at 3.5 dB in the waterfall region (i.e., R2 3.5).
Case 3 Received words sampled at 4.5 dB without filtering.
Regarding Case and Case 2, we collect total of 60,000 uncorrected words, allocating 50,000 for training, 5,000 for validation, and remaining 5,000 for test. Training is conducted for 100 epochs. Fig. Fig. shows the distribution of the number of eπrоrs after base decoding and post decoding for the test samples used in Case 1 and Case 2. For Case 1, the uncorrected words collected in the error-floor region mainly have a small number of errors since most of decoding errors are trapped in small trapping sets, so the distribution is concentrated on small numbers (see Case after base decoding). For ease of use, we refer to codewords with fewer than 11 remaining errors as small-error words. The post decoder, which is mainly trained on these small-error words, corrects significant number of small-error words (see Case after post decoding). Out the total 5,000 test samples, 68.5% of samples are corrected by the post decoder, resulting that the test FER for the test samples is 0.315. This means that, when decoding for received words the AWGN channel, the resulting FER at E,/N, = 4.5 dB after post decoding is 0.315 times of the FER after base decoding shown in Fig 3(a). In other words, the post decoder is successfully trained to correct small-crror words inducing the error-floor.
On the other hand, Case 2, where the samples are collected from the waterfall region, has distribution that is widespread across all areas (see Case 2 after base decoding). In addition, the distribution remains almost the same after post decoding (see Case after post decoding). which means that the post decoder fails to reduce the FER sufficiently. For the test samples, the test FER at En/ /Noo 3.5 dB after post decoding 0.77 times of the FER after base decoding whose difference is not noticeable shown in Fig 3 a). Comparing the results of the two cases, conclude that composing mainly with small-errop words facilitates the post decoder to learn to correct small-crrop words more effectively. As result, Fig. a] shows that Case I mitigates the error-floor more than Case 2. Meanwhile, for Case 3, where all received words are used as training samples without filtering, almost all of them are corrected during base decoding. Since the post stage training is mainly performed on codewords without errors, the loss function becomes almost Then, the weights the post decoder unchanged from the initial value and accordingly, the performance approaches the MS decoding performance, as shown in Fig. Bla).
3.2 Block-wise training schedule
In the previous subsection, the number of iterations for the post decoder is set to Eg = 10. To lower the error-floor further, large number of iterations is required, so we set & = 30, = 50. However, deep neural decoders with a large iteration number are prone to suffer from the vanishing gradient problem. In order to tackle this issue, we propose a block-wiss training schedule which is shown in Fig. a). The proposed training schedule locally trains the weights corresponding to block of A iterations at each training stage. In the first stage, the weights belonging to the first block trained,

---

Channdaert [9]
One abet [1] 2
A. 5,5g 10.7
30utn box 1)
Ae 5. 10.7
Training with alf one inital valuce
trining with grvws iniial vniiul
Htr-by--ie [2]
weights
Stage
8338
0.6
0.4
Stage 2
Stage 3
20
30
lleation
Iteration
(b)
Figure 4: (a): Block- training schedule and (b): Evolution of the test FER across iterations.
Edges weighted by 1y(1
Edges weighted @ 99
Test FER
Number of weights
Correct
Full diversity
0.112
(N + E 3360
Spatial sharing
0.168
282 60
Eroor
Temporal sharing
0,186
(N + E) 112
Unsatisfied
Proposed sharing
0.111
362 90
Satisfied CN
Figure 5: Illustration of the proposed weight sharing technique and comparison with other sharing techniques.
and in the next stage, the weights of the subsequent A iterations are trained. At this point, the weight values of previous A2 iterations, which are already trained in the first stage, are further trained by taking the result of the first stage training as its initial state. This retraining, which is not used in the iter-by-iter training schedule (26), assists in preventing the learning process from falling into local minimum. Note that the method of one-shot training [5) corresponds to the case of A, = f2, A and the iter-by-iter training schedule [26] is equivalent to the case of A A2 0.
We employ greedy approach to determine the optimal values for A₁ and A2, resulting that A 5, Ag = 10 offers superior performance in terms of the test FER. Fig. 4bb shows the evolution of test FER as function of the iteration number. In the case of one-shot training [5). the vanishing gradient problem hinders training the weights of earlier iterations, so the test FER stays roughly the same until iteration 40 and starts to fall thereafter. The same behavior is observed for / = 40. Thus, the test FERs of L = 40 and L 50 are almost the same. Next, since the iter-by-iter training schedule (26 finds the optimal weight for each individual iteration, the test FER falls even at the 21) without encountering the vanishing gradient problem. However, this local optimization leads to a degraded local minimum, and consequently, the test FER gradually and slowly decreases with each iteration. Likewise, the multi-loss method (5) shows a similar result. In contrast, the block-wise training schedule with A 5, A = O shows superior test FER at iteration 25 compared to the other training schedules because it results in a better solution by training the weights of multiple iterations simultaneously. Moreover, the schedule with retraining (i.e., A 5, A2 = 10) outperforms the schedule without retraining (i.e., A 5, A = 0) at iteration 30 though it shows worse result at iteration 25. This implies that through retraining, the weights intermediate iterations have been adjusted to preprocess error patterns thereby leading to stronger correction capabilities in the final iteration. As result, the maximum of 50 iterations, the proposed training schedule with A 5, Ag = 10 provides the better test FER value compared to the other training schedules shown in Fig, b): 0.11 for the block-wise, 0.16 for the multi-loss, 0.18 for the one-shot, 0.37 for the iter-by-iter,
first iteration of the post decoder (i.e.,
3.3 Weight sharing technique using UCN weights
Assuming the techniques proposed thus far (in detail, uncorrected words at Fr/No = 4.5 dB and training schedule with A 5, A2 10) are used, we compare the weight sharing techniques
7

---

in Fig. [ Compared to the full diversity weights, the spatial and temporal sharing techniques sigmificantly reduce the number of distinct weights, but cause performance degradation. In contrast, the proposed sharing technique that introduces a new weight type called UCN weight shows almost identical performance while using only about 2.6% of the weights compared to the full diversity weights. The proposed sharing technique assigns different weights to UCNs and SCNs as shown in Fig. [ This is feasible because the decoder knows whether a CN satisfies the check equation or not. Using the spatial sharing technique and distinguishing between SCN weight w(() and UCN weight w()) the proposed sharing technique can be represented as (w w(f), w)()) Je for iteration é and the total number of distinct weights becomes 362.
Techniques using different weights for UCNs and SCNs have also been proposed in (3 However, the work [41 uses only one suppression factor represent the UCN weight (i.e., a(()) (1
))) making the UCN weight dependent on the CN weight. As result, due to the limited degree of freedom for the UCN weight, is difficult to obtain the decoding diversity for effectively removing various types of error patterns. Moreover, in [33), if at least one of the CNs belonging to single proto CN is unsatisfied, all CNs from the proto CN are weighted by the UCN weight. This approach, which applies the same weight to large number of CNs tied together the proto level, is not suitable for correcting words with a small number of UCNs, because it does not separately handle individual CNs like the proposed method.
4 Performance evaluation
10
RS
Det
On WSte Idt 2-
10
po1 190
10
Propeced MoR
FER.
10
10
10
10
3
2.5
3.5
4.5
5
5.5
6.5
2
3
8u%% (4B)
8r/% (4BB)
8y/% (4E3)
(a)
(b)
(c)
Figure 6: FER performances of (a): WiMAX LDPC (length 576, rate 3/4), (b): IEEE802. 1 In LDPC (length 648, rate 5/6 6). (c): 5G LDPC (length 552, rate 1/2) codes.
In this section, we compare the proposed and other conventional decoding schemes in terms of the decoding performance. All simulations are performed by NVIDIA GeForce RTX 3090 GPU and AMD Ryzen 9 5950X 16-Core Processor CPU. For training, the weights are trained by the Adam optimizer
42 with learning rate of 0.001. We evaluate the decoding performance using Monte Carlo methods with at least 500 uncorrected words for each FER point. Fig. O a) shows the FER performances of the proposed scheme for the WiMAX LDPC code. The proposed scheme incorporates the i) boosting learning with uncorrected words, ii) block-wise training schedule with retraining, and iii) spatial weight sharing with UCN weights. The performance is compared with MS decoding, WMS decoding, and existing NMS decoding schemes in 26] Among the neural decoder studies listed in Table we exclude comparison with the studies that use FAID and layered decoding, that require enumerating trapping sets and absorbing sets. In addition, we choose not to compare with augmented neural networks 6 34] since our approach does not increase model complexity to deal with the low-error-rate region of long codes. A comparative analysis for short codes in the waterfall region can be found in the appendix.
For the NMS decoding schemes in 26) the base decoder is used for iterations from to 20 like the proposed scheme, and the training methods introduced in [5] 26 are employed for the post stage. The full diversity weights are used for the schemes in 5 26 to observe their best performance. For the scheme in (5), received words in the waterfall region (F,/ /No 2-4 dB) are used as training samples, and the weights for the post stage are trained all at once without training schedule. For the scheme
8

---

Table Complexity comparison for the WIMAX LDPC of (N, M, E, 2, = (24, 6, 88, 24, 15.6).
where a is the comparison count for each CN (i.e., a d, + logd, - for the CN degree d, [3]
Complexily per iteration
Total complexity
Memory for weights
Addition
Comparison C
Multiplication M
(A+2C* M)
MS. =
E: 2112
542400
NMS [S] 26 2 50
2E 4224
aM: 2256
(2E 1 N)=
3760
676800
Proposed NMS, F 50
4800
130
Post processing 8 150
22
E= 2112
1627200
in [26], received words from the Eb/No points where the MS decoder achieves bit error rate of 10 are used training samples, and the iter- iter-by-iter training schedule is employed. The remaining hyper-pane parameters are set in the same way as in the proposed scheme. As shown in Fig. aa), the conventional NMS decoders in 26] show good performance in the waterfall region (E,/Ng 2-4 dB), but the error-floor occurs from 4 dB. This is because the training samples are composed of received words without filtering. In contrast, the proposed scheme shows excellent performance in both the waterfall and error-floor regions, and the error-floor phenomenon is barely noticeable down to FER of 10 In particular, comparing the results of ? = 20 and i = 50, it is confirmed that the post decoder successfully removes the error-floor.
In addition, we compare the proposed scheme with the state-of-the-are post processing scheme in [22] We directly reference the simulation results from 222] As shown in Fig. (a), the scheme in [2 shows a similar or worse performance to the proposed scheme, but has disadvantages of having very high decoding complexity and latency since it consumes large number of iterations = 150. Table compares the schemes in terms of the decoding complexity. The NMS decoder has more multiplications than the MS decoder by (E + N) due the weighting operation. The number of other operations is the same as in the MS decoder. Total complexity is evaluated with assumption that the comparison
as twice as complex than the addition A and multiplication M 43 The additional memory for storing the weights of the proposed scheme is 37 which is much lower than those of [5] 266] which exploit full weight diversity. Since the scheme in [22] does not use weighting, the complexity per iteration is lower than the proposed NMS scheme, but the total complexity is more than twice as high as the proposed NMS scheme due to the higher number of iterations. Moreover, additional complexity is required for the error path detector (22). Fig. ((b), (c), similar results are observed for the IEEE802.11 1 In LDPC and 5G LDPC codes, where the proposed scheme outperforms the other schemes and achieves an FER of 10-7 without severe error-floor.
5 Conclusions
This paper proposed training methods for the NMS decoder of LDPC codes to enhance the error-floor performance. Using uncorrected words from the base decoder, we trained the post decoder to be specialized for error patterns causing the error-floor, promoting decoding diversity in the cascaded base and post decoders. We also proposed training schedule to circumvent the vanishing gradient and local minimum problems, and a weight sharing technique that significantly reduces the number of distinct weights without sacrificing performance. The proposed NMS decoder using the trained weights showed the excellent waterfall and error-floor performances for several standard I.DPC codes. Along with the performance improvement, the proposed training scheme has the advantage of being flexibly applicable regardless of the types of channel, code, and decoding algorithm. This scheme can also be implemented directly on hardware architectures without additional costs, and can be directly utilized with no prior analysis of the target code and decoding algorithm.
6 Acknowledgments
This work was supported by Samsung Electronics Co., Ltd (I023041 1005859-01, by Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government [2021- 0-00746, Development of Tbps wireless communication technology], by the National Research Foundation of Korea (NRF) grant funded by the Korea govermment (MSIT) (No. RS- RS-2023-00212200),

---

and by the National Research Foundation of Korea (NRF) grant funded by the Korea government
(MSIT) (No. RS- RS-2023-00247197).
References
[1] Jehoshua Bruck and Mario Blaum. Neural networks, error-correcting codes, and polynomials over the binary n-cube. IEEE Transactions on Information Theory, 35(5))976-99) 987, 1989.
[2] William Caid and Robert W Means. Neural network error correcting decoders for block and convolutional codes. In [Proceedings] GLOBECOM 99: IEEE Global Telecommunications Conference and Exhibition, pages 1028-1031. IEEE, 1990.
[3] LG Tallini and P Cull. Neural nets for decoding error-corecting codes. In IEEE Technical Applications Conference and Workshops. Northcon/95. Conference record, page 89. IEEE, 1995.
[4] Tobias Gruber, Sebastian Cammerer, Jakob Hoydis, and Stephan ten Brink, On deep learning- based channel decoding. In 2017 51st Annual Conference on Information Sciences and Systems (CISS), pages 1-6. IEEE, 2017.
[5] Eliya Nachmani, Elad Marciano, Loren Lugosch, Warren J Gross, David Burshtein, and Yair Be' brye.y. Deep learning methods for improved decoding of linear codes. IEEE Journal of Selected Topics in Signal Processing, 12(1):119-11) 2018.
[6] Yoni Choukroun and Lior Wolf. Error correction code transformer. In Advances in Neural Information Processing Systems.
[7] Shuai Shao, Peter Hailes, Tsang- Yi Wang, Jwo-Yul Wu, Robert G Maunder, Bashir M Al- Hashimi, and Lajos Hanzo. Survey of turbo, LDPC, and polar decoder ASIC implementations. IEEE Communications Surveys & Tutorials, 21(3):2309-235 2233,223333000000 2019.
[8] IEEE standard for local and metropolitan area networks-part 16: Air interface for fixed and mobile broadband wireless access systems amendment 2: Physical and medium access con- trol layers for combined fixed and mobile operation in licensed bands and corrigendum 1. IEEEStandard 802.16e, 2020.
[9] Technical specification group radio access network; NR; multiplexing and channel coding (release 17), document ts 38.212 v17.5.0. 3rd Generation Partnership Project, 2023.
[10] Tom Richardson and Shrinivas Kudekar. Design of low-density parity check codes for 5G new radio. IEEE Communications Magazine, 56(3):28-34, 2018.
[11] Tom Richardson and Ruediger Urbanke. Modern Coding Theory, Cambridge University Press, 2008.
[12] Een-Kee Hong. Inkyu Lee, Byonghyo Shim, Young- Ko, Sang- Kim, Sangheon Pack, Kyunghan Lee, Sunwoo Kim, Jae-Hyun Kim, Yoan Shin, et al. 6G R&D vision: Requirements and candidate technologies. Journal of Communications and Networks, 24(2):232 225,, 2022.
[13] William Ryan and Shu Lin. Channel Codes: Classical and Modern. Cambridge University Press, 2009.
[14] Guiqiang Dong, Ningde Xie, and Tong Zhang. On the use soft-decision error-correction codes in NAND flash memory. IEEE Transactions on Circuits and Systems I: Regular Papers, 58(2):429 439, 2010.
[15] Shubham Chandak, Kedar Tatwawadi, Billy Lau, Jay Mardia, Matthew Kubit, Joachim Neu, Pe- ter Griffin, Mary Wootters, Tsachy Weissman, and Hanlee Ji. Improved read/write cost tradeoff in DNA-based data storage using LDPC codes. 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 147-156. IEEE, 2019.
10

---

[16] Marco Baldi, Marco Bodrato, and Franco Chiaraluce. A new analysis of the McEliece cryptosys- tem based on QC-LDPC codes. In Security and Cryptography for Networks: 6th International Conference, SCN 2008, Amalfi, Italy, September 10-12, 2008. Proceedings 6, pages 246-261 Springer, 2008.
[17] Xiaojie Zhang and Paul H Siegel. Quantized iterative message passing decoders with low enror floor for I.PCC codes. IEEE Transactions on Communications, 12(1)) 2013.
[18] Yoav Freund and Robert E Schapire. A theioric- generalization of on-line learning and an application boosting. Journal of Computer and System Sciences, 55(1): 199--199, 1997.
[19] Yoav Freund, Robert Schapire, and Naoki Abe. A short introduction to boosting. Journal- Jupanese Society For Artificial Intelligence, 14(771 (47771-880:1611 1999.
[20] Jingyu Kang, Qin Huang, Shu Lin, and Khaled Abdel-Ghaffar. An iterative decoding al- gorithm with backtracking to lower the error-floors of LDPC codes. IEEE Transactions on Communications, 59(1):64-733 2010.
[21] Soonyoung Kang. Jackyun Moon. Jeongseok Ha, and Jinwoo Shin. Breaking the trapping sets in LDPC codes: Check node removal and collaborative decoding. IEEE Transactions on Communications, 64(1):15-22 2015.
[22] Seokju Han, Jieun Oh, Kyungmok Oh, and Jeongseok Ha. Deep-learning for breaking the trapping sets in low-density parity-check codes. IEEE Transactions on Communications, 70(5):2909-299) 2923, 2022.
[23] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 256-256. JMLR Workshop and Conference Proceedings, 2010.
[24] Arild Nøkland and Lars Hiller Eidnes. Training neural networks with local error signals. In International Conference on Machine Learning, pages 4850.. PMLR, 2019.
[25] Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Greedy layerwise learning can scale to imagenet. In International Conference on Machine Learning, pages 583-593. PMLR, 2019.
[26] Jincheng Dai, Kailin Tan, Zhongwei Si, Kai Niu, Mingzhe Chen, H Vincent Poor, and Shuguang Cui. Learning to decode protograph LDPC codes. IEEE Journal on Selected Areas in Commu- nications, 19(7):198311998 2021.
[27] Mengke Lian, Fabrizio Carpi, Christian Häger, and Henry Pfister. Learned belief proppaattongnno decoding with simple scaling and SNR adaptation. In 2019 IEEE International Symposium on Information Theory (ISIT), pages 161-165. IEEE, 2019.
[28] IEEE standard for information technology telecommunications and information exchange between systems local and metropolitan area networks specific requirements -part 11: Wire- less lan medium access control (MAC) and physical layer (PHY) specificatio. IEEE Standard 802. 11, 2016.
[29] Jinghu Chen, Ajay Dholakia, Evangelos Eleftheriou, Marc PC Fossorier, and Xiao-Yu Yu Hu. Reduced--complextty decoding of LDPC codes. IEEE Transactions on Communications, 5388) 2005.
[30] Joachim Rosseel, Valérian Mannoni, Inbar Fijalkow, and Valentin Savin. Decoding short LDPC codes via BP-RNN diversity and reliability-baset post-processing. IEEE Transactions Communications, 70(12):7830-7842 2022.
[31] Xin Xiao, Bane Vasié, Ravi Tandon, and Shu Lin. Designing finite alphabet iterative decoders of LDPC codes via recurrent quantized neural networks. IEEE Transactions on Communications, 68(7):393333974 2020.
11

---

[32] Xin Xiao, Nithin Raveendran, Bane Vasié, Shu Lin, and Ravi Tandon. FAID diversity via neural networks. In 2021 11th International Symposium on Topics Coding (ISTC), pages 1-5. IEEE, 2021.
[33] Nemin Shah and Yash Vasavada. Neural layered decoding of 5G LDPC codes. IEEE Communi- cations Letters, 25(11):3599) 35330333-399 2021.
[34] Eliya Nachmani and Lior Wolf. Hyper-graph- decoders for block codes. Advances in Neural Information Processing Systems, 32, 2019.
[35] Sebastian Cammerer, Jakob Hoydis, Fayçal Ait Aoudia, and Alexander Keller. Graph neural networks for channel decoding. In 2022 IEEE Globecom Workshops (GC Wkshps), pages 486-491. IEEE, 2022.
[36] Eliya Nachmani and Lior Wolf. Autoregressive belief propagation for decoding block codes. arXiv preprint arXiv:2 /1780, 2021.
[37] Mare PC Fossorier. Quasicyclic low-density parity-check codes from circulant permutation matrices. IEEE transactions on information theory, 1788) 2004.
[38] Jeremy Thorpe. Low-density parity-check (LDPC) codes constructed from protographs. IPN progress report, 42(154):42-154, 2003.
[39] Mare PC Fossorier, Miodrag Mihaljevic, and Hideki Imai. Reduced complexity iterative decoding of low-density parity check codes based on belief propagation. IEEE Transactions on Communications, 47(5):673-66 680, 1999.
[40] Jinghu Chen, Ajay Dholakia, Evangelos Fleftheriou, Marc PC Fossorier, and Xiao Yu Hu. Reduced-complexity decoding of LDPC codes. IEEE Transactions on Communications, 1388):1288-1299 2005.
[41] Xiaofu Wu, Yue Song, Ming Jiang, and Chumming Zhao. Adaptive-normalizcdlfffscc min-sum algorithm. IEEE Communications Letters, 14(7):667-669, 2010.
[42] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.699 2014.
[43] Heshani Gamage, Nandana Rajatheva, and Matti Latva-Aho. Channel coding for enhanced mobile broadband communication 5G systems. In 2017 European Conference on Networks and Communications (EuCNC), pages 1-6. IEEE, 2017.
[44] Yury Polyanskiy, H Vincent Poor, and Sergio Verdú. Channel coding rate in the finite blocklength regime. IEEE Transactions on Information Theory, 56(5):2307-2359, 2010.
[45] Shaohua Yang, Yang Han, Xuebin Wu, Roger Wood, and Rick Galbraith. A soft decodable concatenated LDPC code. IEEE Transactions Magnetics, 51(11):1-4, 2015.
[46] Jieun Oh, Jeongseok Ha, Hyegyeong Park, and Jaekyun Moon. RS-LDPC concatenated coding for the modern tape storage channel. IEEE Transactions on Communications, 64(1):59-69, 2015.
[47] Hyegyeong Park and Jackyun Moon. Improving SSD read latency via coding. IEEE Transactions on Computers, 69(12):1808 1118889992111111111 2020.
12

---

7 Appendix
7.1 A better trade-off between the waterfall and error-floor performances
10
10'
cxtvorta (Poppsod)
WIMAX,
Single large
-0-WSAAX, WMS Flast
10
--8 WIMAX WMS Quaatied
10
-2 W.MAX, Pripveed NS58, Quaneove
FIR
10
30
10
10
RL,% (4B)
Ba/D6 (dI3)
(a)
(b)
Figure 7: (a): Decoding performance with ceammizzednonnootiiiize codes and floating/quanzzized decoders (b): Comparison between the two networks and single large network.
Typically, LDPC code design assumes the availability of a floating-point decoder and optimizes the code maximize waterfall performance. In Fig. a), it's evident that the optimized WiMAX LDPC code outperforms the non-optimized regular LDPC code in terms of the waterfall performance when the floating point MS decoder employed. Additionally, the WMS decoder enhances the performance by assigning weights. As shown in Fig. a WMS decoding of the WiMAX LDPC code exhibits the best waterfall performance when floating-point operations are available. However, for practical hardware implementation, quantization of decoding messages is essential, but it leads to performance degradation. Notably, WMS decoding suffers substantial performance degradation with early onset of the error-floor. A side-by-side comparison of the quantized WMS decoding performances between the non-optimized and WiMAX LDPC codes reveals that the performance loss due to quantization is more pronounced in optimized LDPC codes. Consequently, there's 's need for a method that ensures high performance in both the waterfall and error-floor regions with low complexity. This work accomplishes this goal by leveraging machine learning techniques. In Fig. (7)). the proposed quantized NMS decoder achieves the floating WMS decoding performance for the WiMAX LDPC code in both waterfall and error-floor regions.
7.2 Boosting learning of two networks v.s. conventional learning of single large network
In Fig. (b),. we compare the proposed method with the method that uses a single decoder, which is trained by large single network at once. For the proposed method, the boosting learning is employed with two networks. Both have the same model size. Fig. 7 shows that simply increasing the model size doesn't I significantly mitigate the error-floor. In contrast, the two-stage network using boosting learning achieves greater decoding diversity and demonstrates superior efficiency in alleviating the error-floor.
7.3 Optimization of parameters of the block-wise training schedule
Table 3: Test FER values of the biock-wise training with various A and Az.
A Ag
0
5
10
15
30
0.376
0.144
0.123
0.122
0.150
0.193
0.134
0.112
0.127
0.137
10
0.142
0.142
0.13
0.128
0.13
30
0.179
To fine-tune the parameters A and A2 used in the block-wise training schedule, we compare the test FER values across a range of A₁ and A2. The comparison is done for the post decoder training
13

---

with l2 = 30 and is summarized in Table For the case where A2 = 0 (i.e., without retraining). the performance improves as A increases from 10. This can be attributed the training of more weights simultaneously, aiding in escaping from local minima. However, if A becomes too large, the weights in front are not adequately trained due to the vanishing gradient problem, causing performance degradation. In addition, the retraining iteration number A₂ grows for given A₁ value, a similar trend is noted: performance enhances up to a certain point and then plateaus. According to Table the best performance is achieved with A 5 and 2 10.
7.4 Discussion on the trained weights
Table 4: Trained weights
Weights
1~ 10
0.74
0.98
0.96
1.10
1.20
1.15
1.22
1.19
1.17
1.09
11 ~ 20
1.15
1.09
1.07
1.04
1.05
1.07
1.00
0.98
0.90
0.81
VW (
21 ~ 30
1.14
0.97
0.93
0.69
0.66
0.58
0.57
0.51
0.49
0.45
0.48
0.41
0.37
0.38
0.38
0.39
0.32
0.33
0.32
0.34
~ 50
0.39
0.37
0.31
0.31
0.34
0.45
0.39
0.37
0.42
0.42
1~ 10
0.74
0.71
0.66
0.69
0.67
0.70
0.75
0.71
0.73
0.76
11 20
0.73
0.70
0.77
0.79
0.84
0.84
0.86
0.82
0.81
0.97
CW (
0.19
0.30
0.54
0.58
0.62
0.59
0.73
0.73
0.74
0.78
31 ~ 40
0.72
0.78
0.79
0.81
0.88
0.82
0.88
0.85
0.97
0.79
41 ~ 50
0.81
0.82
0.85
0.91
1.19
0.86
0.96
1.10
1.18
1.62
I 10
0.74
0.71
0.66
0.69
0.67
0.70
0.75
0.71
0.73
0.76
0.73
0.77
0.79
0.84
0.84
0.86
0.82
0.81
0.97
UCW (0))
21 ~ 30
0.58
0.80
0.83
0.73
0.81
0.86
0.72
0.78
0.73
0.73
31 ~ 40
0.59
0.71
0.64
0.67
0.67
0.72
0.65
0.65
0.66
0.70
41 50
0.77
0.70
0.72
0.81
0.92
0.58
0.71
0.76
0.87
1.57
4- VW --- CW with
Pcase]
Plase2
1.6
1.4
UCW a/9
Number of samples
3000
1.2
2500
Wiigos
2000
0.8
0.6
5
-
0.2
500
0
S withoot Phuse
Phasol
Phooo?
d withoot Phase
0
Ileration
Iteration
(a)
(b)
Figure 8: Evolution of the weights and the number of samples as function of iteration.
Table shows the resulting weights trained using the proposed training methods. Owing to the weight sharing technique, only three types weights, VW w((), CW (9()) and UCW w(()) exist for each iteration. Thus, weight variations across iterations can be depicted in two dimensions, shown in Fig. 8 a), enabling analysis of the trained results. Without the weight sharing technique, there would be (N + E') weights per iteration, making 2D graphical representation unfeasible. Fig. 8(()) shows that the weights for the base decoding stage, where the CW and VW are roughly 0.75 and are similar the weights of conventional WMS decoding. As result, the waterfall performance of WMS decoding and NMS decoding are nearly indistinguishable. The NMS decoding technique offers minimal advantage in the waterfall region.
On the other hand, for the post decoding stage, the weights undergo significant changes. iteration 21, the VW increases while the CW decreases substantially. This means that the channel LLR values are given more weight when performing the sum operation at each VN, while the messages coming from CNs are attenuated. As result, the decoding process reverts to the initial decoding state (with

---

high error rate). This leads to an increase in the number of samples having more than 6 errors (denoted by e > 6), shown in Fig. Bb), and a decrease in the number of samples with S This intentional regression aims to break free from trapped error patterns. From iteration 22 onward, the CW gradually increases, the VW decreases, and the count of e S 6 samples starts to increase again. After iteration 27, the CW and VW show less variations. We term the period from iteration 21 to 26 as Phase and the period following iterations as Phase2. At iteration 20, the numbers of samples with E I 6 and > 6 are roughly equal, but through Phase 1, samples with > 6 transform into samples with e I 6. Consequently, by the end of Phase I, there are more samples with e S 6 than samples with 6. The small-sized error samples diminished steadily in Phase 2.
The result without Phase 1 is also shown in Fig. B(b). Without Phase 1, the process of transforming into small-sized eτror samples is omitted. Consequently. the decoding proceeds in a situation where the number of samples with e S and e > similar. Although Phase effectively correct most of the e < 6 samples, the > 6 samples remain in significant numbers as they are initially abundant. This indicates the need for the pre-processing of Phase 1.
7.5 Multiple stage decoders
10
11000000000
WMS, 50
7-03
Propoemed NAS,
Payoool XDS Thoe Puo) 2-
Papooed 806 (uue Pos Tanily $)
10
10
10
10
10
2
5
0
10
12
By/% (dD)
5,0% (4B)
(a)
(b)
Figure 9: (a): FER performance of the triple decoder with the theoretical limit for the WiMAX LDPC
code (b): FER performance over the Rayleigh channel for the WiMAX LDPC code.
Extending the proposed method to include more than two decoders is feasible and can be applied in similar manner. For instance, in the case of the WiMax LDPC code (shown in Fig. ((a)), uncorrected words from the error- floor region of the base + post decoder (i.c., E, 4.5 dB) are collected to train a third decoder. Fig. 9 (a) shows that the aid of this third decoder (labeled as "Base+Post+TThe hird) reduces the error-floor further. It is worth mentioning that introducing this third decoder might lead to increased decoding latency. Moreover, the process of collecting uncorrected words would be time-consuming, especially in the very low FER region. Addressing these challenges could be promising direction for subsequent research. In addition, we add a graph showing the finite length
performance limit 24
We want note that the concept of multi-stage decoding has been employed in the previous works [4]46]]] 46 (4), particularly in the field of coding theory. Our study has similarities with theses previous approaches, as we also carries out two-stage decoding. However, unlike physically divided decoders in 45 46 47). our method utilizes a single LDPC decoder. While the proposed decoder is conceptually divided into two stages (base/post) based on iterations, in practice, we employ only a single decoder with distinct weight parameter sets. Moreover, our approach has novelty in that the post decoder is 'trained' dependent on the results of the base decoder.
7.6 Application to different channel
The concept of training using uncorrected words can be consistently applied regardless of the channel type. In Fig. Ib), we include the results for the Rayleigh channel with the scale parameter a = 1, showing the effectiveness of our proposed method holds across different channel types.
15

---

7.7 Application to long length codes
MAI,,
KOIR, 7
10
FEN
10
8.%%, ((B)
Figure 10: FER performance for the long 5G LDPC code of length 1248.
We show the results for the (1248, 1056) 5G LDPC code in Fig. T0 This result demonstrates the proposed method is also effective for long codes. There is little improvement in waterfall performance with the NMS technique at such long lengths. However, the performance improvement the error-floor region using our proposed method clearly evident.
7.8 Comparison with the augmented neural decoders
Table Comparison with 5
in terms of the waterfall performance of short codes. The results are measured by the negative natural logarithm of BER. Higher is better.
Architecture
Vanilla NBP
NiP-HHyperNet.
Transformer BCCT [
Method
BP
Orig. NBP 5
Proposed
Hyper (34)
/NoNo
Polar (64,48)
4.74
5.94
7.42
4.70
5.93
7.55
4.93
6.64
8.77
4.92
6.44
8.39
6.36
8.46
11.09
BCH (63,51)
4.58
5.82
7.42
4.64
6.21
8.21
4.72
6.42
8.96
4.80
6.44
8.58
5.66
7.89
11.01
MacKay (96.48)
16.41
In Table 5 we compare the waterfall performance of various neural decoders for short length codes. To ensure fair comparison with other works, we employ the NBP decoder and utilize the soft-BER loss function for BER performance optimization while making use of full diversity weights. Out of total of 50 iterations, 20 iterations are allocated for base decoding and 30 iterations for post decoding.
The result reveals a notable performance improvement over the original NBP (5), which employs the same vanilla NBP architecture. The improvement particularly pronounced at high SNR. We achieve even better performance at high SNR than the work (34), which adds
to the vanilla NBP architecture. comparison the ECCT 6 using the transformer architecture, our method shows inferior performance for high density codes (Polar, BCH), yet it achieves fairly comparable performance for LDPC codes. Owing its intricate transformer structure, the ECCT requires higher training and decoding complexity than NMS decoders [6) implying its current limited practicality. Nevertheless, it is worth emphasizing that our proposed training method can be adaptable across any architecture. In other words, there's promising potential in future research that integrates the proposed training methods with the architecture with neper-networks or the transformer architecture.
7.9
Ablation study
Table 6: Ablation analysis.
O: Boosting
O: Block-wise
O: Proposed sharing
X: Conv.
X: One-shot
X: Full diversity
FER (at Eg/No 5.0dB)
X
X
3.14 x 10-6
X
2.77 10-7
0
O
1.84 X 10
0
1.85 x 10
16

---

In Table we provide the ablation study for the WIMAX LDPC code. The result shows that the boosting learning is the core technique reduce the FER performance and the blsek-wise training also contributes the FER reduction, while the proposed sharing technique does not involve performance degradation.
17


Reviews:
[{"review_id": "oaZGR820v3", "reviewer": "Reviewer_uaWt", "summary": "\nThis paper presents novel training techniques for the NMS decoder of LDPC codes aimed at enhancing performance in the error floor region. The proposed decoding methods comprise two stages: a base decoder and a post decoder designed specifically for uncorrected codewords from the base decoder. In order to tackle the issue of vanishing gradient during training, a block-wise training schedule is introduced in this paper. By assigning distinct weights to unsatisfied check nodes, the error floor can be lowered with a small number of weights to be trained.", "questions": "In order to harness the potential of boosting learning, this study introduces a division of the decoding network into two components, with the post decoding stage aiding the base decoding process. A natural extension of this approach is to explore the possibility of incorporating more than two decoders. It would be intriguing to learn from the authors whether they have considered this scenario and if they can provide insights into the expected outcomes. Including discussions and potential answers to this question would enhance the manuscript's appeal and stimulate further interest in the research.", "limitations": "N/A - This work does not appear to have any discernible negative societal impact.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The impact of the proposed decoding technique is substantial, as evidenced by the figures which clearly demonstrate a significant improvement in code performance, particularly in the error floor region.\n2. The analysis and interpretation of the results in Figure 3 were particularly valuable, providing a clear understanding of the distinctions between the methods employed for selecting training samples.", "weaknesses": "1. The proposed decoding method in this paper builds upon the concept of two-stage decoding and incorporates a post decoder that complements the base decoder. It is important to acknowledge that this idea is not entirely new within the field of coding theory for LDPC codes, as there has been prior research exploring similar approaches. The followings are some of the examples sharing a similar philosophy and it would be valuable to mention them as relevant previous work in this direction.\nS. Yang, Y. Han, X. Wu, R. Wood and R. Galbraith, \"A soft decodable concatenated LDPC code,\" 2015 IEEE International Magnetics Conference (INTERMAG), Beijing, China, 2015\nJ. Oh, J. Ha, H. Park and J. Moon, \"RS-LDPC Concatenated Coding for the Modern Tape Storage Channel,\" in IEEE Transactions on Communications, vol. 64, no. 1, pp. 59-69, Jan. 2016\nH. Park and J. Moon, \"Improving SSD Read Latency via Coding,\" in IEEE Transactions on Computers, vol. 69, no. 12, pp. 1809-1822, 1 Dec. 2020\n\n2. The clarity of the presentation style could be enhanced. For example, the specific problem addressed by the paper, such as the type of channel considered, is not explicitly stated. The reader must infer this information primarily from the related work section. Providing a clear and explicit description of the problem, including the type of channel under consideration, would greatly improve the overall clarity of the paper.\n\n3. Nitpicks\n- It would greatly enhance the presentation if the theoretical performance limit based on the code rate could be included in the plots, such as in Figures 3 and 6.\n- Typo: On line 121, page 3, \"protogrph\" should be corrected to \"protograph\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable comments and the clarification.\n\n## Related works sharing a similar philosophy\n\nThank you for informing us about the related research. In those studies, they employ a two-stage decoding where the outer and inner decoders perform decoding subsequently or iteratively. Our proposed study bears similarities, as our study also carries out a two-stage decoding. However, unlike these concatenated codes, our method utilizes a single LDPC decoder. While the proposed decoder is conceptually divided into two stages (base/post) based on iterations, in practice, we employ only a single decoder with distinct weight parameter sets. Consequently, our approach avoids the need for the additional (or separate) parity bits that are typically associated with concatenated codes. Moreover, our approach has a novelty in that the post decoder is 'trained' dependent on the results of the base decoder. Nonetheless, we agree with the comment that our and previous works share the philosophy of complementary multiple decoders. We believe they are worth mentioning as relevant previous works.\n\n\n## Clarity of the presentation\nAs you mentioned, there was no explicit introduction of the channel model in the original manuscript. In the revised version, we will specify that the underlying channel is the AWGN channel. Additionally, we will add information about the sampling points of the post decoder and the size of the neural network.\n\n## Performance limit\nAs you suggested, we have added a graph showing the finite length capacity in the attached file, Figure R.1(a). The finite length performance limit was referenced from the paper [a].\n\n[a]: Y. Polyanskiy, ”Channel coding rate in the finite block-length regime,” IEEE Trans. on IT, vol. 56, no. 5, 2010\n\n## Multiple decoders\nIt's possible to extend to the case of more than two decoders in the same manner. For instance, in the WiMax LDPC code result (Fig. 6(a)), uncorrected words from the error floor region of the base + post decoder (i.e., Eb/N0 5dB) can be collected to train a third decoder. In Figure R.1(a) of the attached file, we present numerical results that demonstrate the error floor's further reduction with the aid of this third decoder (labeled as “Base+Post+Third”). It's worth mentioning that introducing this third decoder might lead to increased decoding latency. Moreover, the process of collecting uncorrected words would be time-consuming, especially in the very low FER region. Addressing these challenges could be a promising direction for subsequent research.\n"}, {"review_id": "Dhc2YN8fyE", "reviewer": "Reviewer_zXhE", "summary": "The paper proposes training methods to optimize neural min-sum (NMS) decoders that are robust to the error-floor phenomenon of LDPC codes. The proposed methods include: (1) dividing the decoding network into two neural networks and training the post network to be specialized for uncorrected codewords that failed in the first network; (2) introducing a block-wise training schedule that locally trains a block of weights while retraining the preceding block; and (3) assigning different weights to unsatisfied check nodes. The proposed methods are applied to standard LDPC codes, and the results show that they achieve the best decoding performance in the error-floor region compared to other decoding methods in the literature. The proposed NMS decoder is designed only by modifying the training methodology, without adding any additional modules. Therefore, it can be seamlessly incorporated into the well-established architectures of LDPC decoders and can be immediately utilized in practical domains that demand exceptionally low error rates.\n\n", "questions": "1. Does the proposed method can be also beneficial to improve the results of neural decoder for other types of codes? such as BCH and Polar?\n2. How does the model behave with other type of noise (other than Gaussian)? \n3. Can you tie the weights of the different network to get improvement in terms of model size?\n4. What happen if you take more then two NN? does the performance improve?", "limitations": ".", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper introduce novel ideas:\n1. Boosting learning using uncorrected codewords - The authors propose a boosting learning technique for LDPC code decoding. The technique divides the decoding network into two cascaded networks, where the first network focuses on the waterfall performance and the second network focuses on the error-floor region. The second network is able to correct uncorrected codewords that are not corrected by the first network due to the error-floor phenomenon. This results in a significant performance improvement in the error-floor region.\n2. Block-wise training schedule with retraining - The authors propose a new training schedule for NMS decoders that mitigates the vanishing gradient problem. The proposed schedule divides the entire decoding iterations into sub-blocks and trains the blocks in a sequential manner. Additionally, the weights trained from previous blocks are retrained to escape from local minima. This results in a significant performance improvement over the one-shot training method and the iter-by-iter schedule.\n3. Weight sharing technique with dynamic weight allocation - The authors propose a new weight sharing technique for NMS decoders that improves the performance in the error-floor region. The technique dynamically assigns different weights to unsatisfied check nodes (UCNs) and satisfied check nodes (SCNs) in the decoding process. This results in a significant performance improvement with a 2.6% reduction in the number of weights to be trained.\n", "weaknesses": ".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and constructive comments.\n\n## Other types of codes\nThe proposed boosting learning method is more effective in the error floor region than in the waterfall region. Therefore, its impact is more prominent with LDPC codes than with Polar or BCH codes, where the error floor doesn't manifest as prominently. However, boosting learning can be applied regardless of the code type and is also applicable targeting the waterfall region. To demonstrate this, we added simulation results of our method when applied to Polar and BCH in the waterfall region to the attached file's Table R.1. When compared to the SOTA model-based approach [15] in terms of the *waterfall performance*, a comparable performance is achieved. However, while [15] requires additional training/decoding complexities, our proposed method does not. In other words, our approach achieves “the *error floor SOTA* result” and exhibits comparable performance to “the *waterfall SOTA* result” [15] for other types of codes.\n\n## Other types of noise\nThe concept of training using uncorrected words can be consistently applied regardless of the channel type. In the attached file, Figure R.2, we included the results for the Rayleigh channel, showing the effectiveness of our proposed method holds across different channel types.\n\n## Tie the weights\nThe key mechanism for performance improvement is that the base and post decoders obtain decoding diversity by using different weights. Therefore, sharing weights between them would likely be ineffective. Exploring new weight sharing (or tying) techniques to reduce the model size can be an attractive avenue for future research.\n\n## Multiple decoders\nIt's possible to extend to the case of more than two decoders in the same manner. For instance, in the WiMax LDPC code result (Fig. 6(a)), uncorrected words from the error floor region of the base + post decoder (i.e., Eb/N0 5dB) can be collected to train a third decoder. In Figure R.1(a) of the attached file, we present numerical results that demonstrate the error floor's further reduction with the aid of this third decoder (labeled as “Base+Post+Third”). It's worth mentioning that introducing this third decoder might lead to increased decoding latency. Moreover, the process of collecting uncorrected words would be time-consuming, especially in the very low FER region. Addressing these challenges could be a promising direction for subsequent research.\n"}, {"review_id": "sJI6BUZIk7", "reviewer": "Reviewer_MFD6", "summary": "The author studies LDPC code's neural min-sum decoder's error floor problem, by solely change the training method:\n\n(1) boosting learning method, which use first network to do majority of decoding to achieve waterfall region, and second network deal with small error residuals. \n(2) relief deep decoding iteration's vanish gradient problem by block-wise training. \n(3) bundle weight in dynamic manner, to reduce the number of weights to be trained. \n\nWith above techniques, the experiment runs on varies LDPC settings, shows that proposed NMS has better performance on error-floor regions, with same computational resource as NMS. Since NMS and MS have similar complexity, it is believed that proposed NMS could be possibly adapted to future LDPC decoder implementation. ", "questions": "N/A", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "(1) The paper works directly on training of NMS algorithm, which requires minimal change on existing low complexity NMS solutions. All three training methods are reasonable and are supported by experiments. The residual learning (termed as boosting learning) empirically shows advantage on error-floor regions. The block-wise training schedule does improve performance with large number of iterations. Finally the weight sharing technique adds a structural sparse limitation to neural decoder, which only uses a small portion of weights. \n\n(2) The evaluation is extensive, on multiple LDPC settings at Figure 6, we see that proposed NMS method shows better error-floor performance, compared to existing NMS and canonical decoders.\n", "weaknesses": "Here are my concerns:\n\n(1) The major contribution on Machine Learning sense, is limited. The proposed boosting learning method is common, as residual learning in ML sense; the proposed training schedule and weight sharing are also widely-used ML methods. The core contribution is on applying ML techniques in the context of channel coding field. In some sense, the targeted audience shall be channel coding researchers, rather than general ML researchers.\n\n(2) The experiments are not conducted in long block length (length>1000, which is common for QC-LDPC), which is widely used in 5G/WIFI systems. It might be interesting to check the performance on long block length, where block length coding gain is significant, and NMS shows less advantage against canonical methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comment.\n\n## Contribution of this work\nWe agree with your opinion that the contribution to machine learning techniques can be seen as limited. However, we believe this manuscript aligns well with the scope of NeurIPS under the category of “application of machine learning”. The primary emphasis of our work is on the novel application of ML techniques to the domain of coding theory. In a similar vein, previous works [a]-[d], presented at ML conferences, have primarily aimed at broadening the spectrum of ML applications. A unique aspect of our research is the adoption of boosting learning, a method yet to be explored in coding theory. Our boosting approach for LDPC codes achieves the SOTA performance in the error-floor region compared to other LDPC decoding strategies.\n\n[a] 2018 NeurIPS, “Deepcode Feedback Codes via Deep Learning”\n\n[b] 2019 NeurIPS, “Hyper-Graph-Network Decoders for Block Codes”\n\n[c] 2020 NeurIPS, “Learning to Decode Reinforcement Learning for Decoding of Sparse Graph-Based Channel Codes” \n\n[d] 2021 ICML, “Cyclically Equivariant Neural Decoders for Cyclic Codes”\n\n[e] 2022 NeurIPS, “Error Correction Code Transformer”\n\n## Long length code\nAs you suggested, we’ve added the results for the (1248, 1056) 5G LDPC code to the attached file's Figure R.1(c). This result demonstrates the proposed method is also effective for long codes. As you mentioned, there is little improvement in waterfall performance with the NMS technique at such long lengths. However, the performance improvement in the error floor region using our proposed method is clearly evident.\n"}, {"review_id": "zUblmWMSaJ", "reviewer": "Reviewer_jBEt", "summary": "The paper proposes a training framework for the NMS decoder of LDPC codes in order to enhance the error-floor performance of these codes. \\\nThe NMS decoder iterations are split into two cascaded parts, the first so-called based decoder is trained for decoding waterfall parts, and the second (post decoder) is for decoding uncorrected codewords caused by the error-floor.\\\nThe paper suggests the use of a training schedule reusing learned weights to tackle the vanishing gradient of many iterations NMS.\\\nThe proposed decoder using the trained weights showed improved error-floor performances for several standard LDPC codes and can be efficiently integrated into current NMS solutions.", "questions": "1) Please address the NMS subsection from the Weaknesses section.\\\nIt would be beneficial to have an ablation study of the final accuracy for NMS augmented with the different components of the proposed methods. Also, different training strategies of the NMS must be considered.\n\n2) Please address the clarity and Misc from the Weaknesses section.\n\nI believe adding clarity, addressing the ablations, and adding more experiments would increase the quality, impact, and rating of the paper.", "limitations": "No limitations are discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper addresses an important issue present in BP/MS-based decoders on these codes.\n\nThe solution is simple and does not require any architectural modification.\n\nThe performance seems important and is obtained without adding any complexity compared to the original NMS decoder.", "weaknesses": "#### NMS\n I am not sure the comparison with NMS is fair for multiple reasons.\n\n1) The training of NMS can be performed in several ways that should not allow zero loss. For example, the observations in lines 224 and 225 can be easily solved by using large batches and/or using batches spanning many Eb/N0 values.\n\n\n2) The vanishing gradient phenomenon has been solved in [5] using a multi-loss objective, which clearly prevents the gradients to collapse at each layer.\nAlso, in Figure 4b) NMS and the method should be compared at equal number of iterations.\nNMS's test FERs with l=40 and l=50 are not that far from the  l=50 proposed method FER.\n\n3) NMS remains non-negligibly better in the waterfall region.\n\n4) Many NMS algorithms have emerged since [5] and I am wondering if the comparison with the original NMS [5](2018) is fair and conclusive in order to judge the paper.\n\n#### Clarity:\nFigure 2 is not clear. For example, the differentiation between the NMS and the neural network is not clear; the base and post NN should be linked by an arrow.\n\nThe paper contains a very large amount of text which can become cumbersome. I believe adding clear Algorithm(s) would simplify the comprehension a lot.\n\n\n#### Misc:\nTypos: \n\nline 88: we a new \\\nline 121: protogrph", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing insightful comments.\n\n## Schemes for avoiding zero loss\nThank you for your detailed feedback. Lines 224, 225 are about the Case 3. In Case 3, we use received words sampled at 4.5dB (the error floor region of the base decoder) as training samples for the post decoder without any filtering. At 4.5 dB, the FER of the base decoder is at the level of $2\\times 10^{-5}$, so even if the batch size is increased to 100, on average, only 0.02 samples produce a non-zero loss. Furthermore, if we span the sampling Eb/N0 to include lower values (in the Waterfall region), it doesn't foster the post decoder training that's robust to the error floor. In other words, the core idea of our proposed method is to sample from the error floor region and use only uncorrected words by filtering, ensuring effective training for the post decoder. We will refine this section to eliminate any ambiguity and preclude potential misinterpretations.\n\n## Multi-loss objective function\nAs you suggested, we included a comparison with the multi-loss method (see the attached file, Figure R.1(d)). Like the iter-by-iter method, the multi-loss method also rapidly reduces the FER value in the initial iterations. However, by iteration 50, block-wise training outperforms the multi-loss method. This can be attributed to the fact that error patterns occurred in the error floor region can be better corrected when all iterations within a block collaborate closely, rather than optimizing each individual iteration. This is discussed in Section 4 of the supplementary material. While the graph in Figure R.1(d) seems not to show a significant difference in the final test FER, the precise values are 0.179 for at once, 0.156 for multi-loss, and 0.112 for block-wise, which means that the block-wise training method achieves reductions of 40% and 28% respectively.\n\n## NMS remains non-negligibly better in the waterfall region\nThe proposed post decoder is trained for the error floor region, so even with the addition of the post decoder, the waterfall performance remains largely unchanged. (compare the proposed method with iteration 20 and 50 in Figure 6(a)) Consequently, as you rightly pointed out, our proposed method in Figure 6(a) exhibits a slightly degraded waterfall performance compared to other NMS methods. Compared to [5], our methodology incurs a minor reduction of 0.05dB in the waterfall region (@FER $10^{-2}$). However, it can achieve an appreciable SNR gain of 1dB in the error floor region (@FER $10^{-7}$). Such a trade-off will be beneficial, especially for the target applications of our work, e.g., extremely ultra-reliable communications for 6G and storage systems.\n\n## Comparison with other works\nAs you've pointed out, several follow-up studies have been conducted after [5]. Unlike our work focuses on the error floor performance, the majority of the works have concentrated on improving waterfall performance. The state-of-the-art (SOTA) model-based approaches in waterfall performance is the hyper graph network [15]. A comparison with [15] in the waterfall region is added to the attached file. Our proposed boosting learning is also applicable in the waterfall region. As seen in Table R.1, our method achieves comparable performance with [15] solely through the enhancement of the training technique without any additional modules. \n\nWhat we would like to emphasize is that our proposed method has achieved the SOTA result in terms of the error floor performance under a low complexity environment. Specifically, our work assumes scenarios suitable for application areas like storage systems and XURLLC for 6G, encompassing:\n\ni) Error floor region (FER <=1e-7, BER<=1e-9)\n\nii) Target metric: FER performance\n\niii) MS algorithm\n\niv) Without additional training/decoding cost\n\nv) Hardware-Feasible\n\nvi) Moderate and long code length (>=500)\n\nIn contrast, [15] targets short-length BER performance in the waterfall region (BER~1e-7), incurs additional training/decoding costs, and is constrained to the BP algorithm. This makes it difficult to make a fair comparison. Technically, due to memory limitations in TensorFlow, it was impossible for us to train the [15] technique on the WiMax LDPC code of length 576. Moreover, the evaluation of the technique [15] in the waterfall region takes several days and it is infeasible to evaluate the error-floor performance.\n\nOur proposed method is the first in research on learning-based decoders to directly target the error floor performance. For reference, the SOTA work sharing the same environment as this paper is [33], and our proposed NMS showed better performance with about a third of the latency and without additional modules. In the revised paper, we will emphasize that the SOTA in the waterfall is the existing works [15], and will highlight that the goal of our study is the error floor. Thank you for clarifying the contribution of the research.\n\n## Clarity and ablation study\nThank you for the valuable suggestion. For the clarity of the paper, we've extensively revised the schematics as shown in Figure R.5 of the attached file and added an algorithm. Additionally, we included an ablation study in the attached file. The result shows that the boosting learning is the core technique to reduce the FER performance and the block-wise training also contributes the FER reduction, while the proposed sharing does not involve the performance degradation.\n"}, {"review_id": "PfAMAKcYg0", "reviewer": "Reviewer_UbgL", "summary": "\n1.\tIn the proposed work authors proposed \n         a.\tNeural Min-Sum (NMS) decoders \n        b.\tNMS decoder with block-wise training schedule that locally trains a block of weights while retraining the preceding block.\n        c.\tThe different weights are assigned to the unsatisfied check nodes and training is carried out.\n2.\tThe contribution of the proposed work are.\n     a.\tBoosting learning using uncorrected codewords\n     b.\tBlock-wise training schedule with retraining\n     c.\tWeight sharing technique with dynamic weight allocation\n3.\tSufficient results are presented and discussed.\n", "questions": "1.\tHow the weights of the NMS decoders are initialized.\n2.\tIs it possible to design ASIC hardware of NMS decoder \n3.\tWhat about the latency of the decoder.\n4.\tThe neural decoders with recent state of art techniques with their advantages should be discussed in detail.\n5.\tThe experimental set up should be clearly mentioned with parameters used for the training.\n6.\tHow exactly the weights are updated in the training process of LDPC CODES mentioned in the figure 1 and what is the impact of weights in the training process should be clearly mentioned.\n", "limitations": "Since the authors mentioned that proposed work can be taken into hardware architecture.\nBut clear architectute of the proposed method is not highlighted.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Novelty of the work is good.\nSufficient results are presented and discussed.", "weaknesses": "The experimetnal set up should be elaboprated more with parameters selected for training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and constructive comments.\n\n## Experimental setup\nAs suggested, we’ve provided a more detailed account of the experimental setup in the revised version, including the channel type, the training Eb/N0 for the post decoder for each code, the number of hidden layers, and the number of nodes in the neural network.\n\n## Initial values\nInitial values are set to 1. We’ve tried various values, but there wasn't a significant difference. Research on effective initialization is an interesting topic, and we plan to pursue it in follow-up studies.\n\n## Implementation by ASIC\nThe proposed method requires no additional modules; hence, once trained, the NMS decoder can be implemented in the same manner as the WMS decoder. There have been numerous studies [a]-[c] on the ASIC implementation of the WMS decoder. As a result, the proposed NMS decoder can also be fabricated using ASIC hardware.\n\n[a] W. Zhang, S. Chen, X. Bai, and D. Zhou, “A full layer parallel QCLDPC decoder for WiMAX and Wi-Fi,” in Proc. IEEE 11th Int. Conf. ASIC (ASICON), Nov. 2015, pp. 1–4.\n\n[b] T. H. Tran, Y. Nagao, H. Ochi, and M. Kurosaki, “ASIC design of 7.7 Gbps multi-mode LDPC decoder for IEEE 802.11ac,” in Proc. IEEE 14th Int. Symp. Commun. Inf. Technol. (ISCIT), 2014, pp. 259–263.\n\n[c] S. Shao et al, “Survey of Turbo, LDPC, and Polar Decoder ASIC Implementations”,  IEEE Communications surveys & Tutorials, Vol. 21, no. 3, pp. 2309-2333, 2019\n\n## Comparison with recent works\nConventional neural decoder works have primarily targeted improvements in waterfall performance, whereas our study achieved improvements in error floor performance. Nevertheless, we include a comparison with the latest neural decoder study [15] in the attached file’s Table R.1. Compared to [15], our method achieves similar performance with lower training/decoding complexity.\n\nA comparison in the error floor region is not able to conduct with [15], as verifying their error floor performance is infeasible, and there are numerous differences in the experimental environments. The state-of-the-art (SOTA) result with the same experimental setting with our work is from [33], so we performed the comparison with that. The distinctions between our research and these prior works will be highlighted in the revised version.\n\n\n## How to update the weights\nFigure 1(b) corresponds to the neural network for the LDPC code depicted in Figure 1(a). The connections between the nodes in the network vary depending on the code, and each edge has an assigned weight. Here, these weights do not have any special functions; they are just typical weights of neural networks. Therefore, the training of these weights is done using standard backpropagation, and in this paper, we employed the Adam optimizer.\n\n## HW architecture\nIt seems that some ambiguous expressions may have led to misunderstandings. What we meant was that the proposed method can be directly applied to the HW architecture used in WMS, not that we are proposing a new architecture. We will make corrections to the revised version to ensure clear communication of this point.\n"}]

Author rebuttal:
### Dear Reviewers and Area Chair,


We sincerely appreciate the time and effort you've taken to review our paper. 

Your insightful feedback has undoubtedly enhanced our work. 

We've carefully addressed each of your remarks and inquiries, providing detailed responses for each one.

We hope that our responses have addressed any concerns.

If you have further questions or need clarifications, please don't hesitate to ask.



Meta-review: This paper studies how to optimise the training of neural-based min-sum decoders for LDPC codes. The objective is to reduce the error floor behaviour of the code (which is known to be an issue in applications targeting a very low block error rate).

The problem being tackled (i.e., reducing the error floor in LDPC decoding) is important, the impact of the proposed technique is substantial, and the numerical results are convincing. The rebuttal has helped to clarify some issues raised by the reviewers, and the final reviews are uniformly positive (although with different levels of enthusiasm). After my own reading of the reviews, rebuttal and paper, I agree with this positive evaluation and recommend acceptance. I encourage the authors to incorporate the comments and the additional simulation results in the camera ready.
Decision: Accept (poster)