Title: Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
Venue: NeurIPS 2023
Accepted: yes
Keywords: Error-floor

Low-density parity-check codes

Boosting learning

Training shcedule

weight sharing

Neural decoders

Min-sum
Abstract: Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor.

Boosting Learning for LDPC Codes to Improve the Error-Floor Performance

Abstract

Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .

1 Introduction

The field of learning-based decoding for error-correcting codes began with research on training neural networks to produce the information vector when given a distorted codeword [1, 2, 3, 4]. These works assume an arbitrary neural network with no prior knowledge of decoding algorithms, and accordingly, face the challenge of learning a decoding algorithm. In contrast, model-based neural decoders are designed by mapping a well-known graph-based iterative decoding algorithm, such as belief propagation (BP) and min-sum (MS) decoding algorithms, to a neural network and then training its weights [5]. Compared to the arbitrary network approaches or the error correction transformer [6], model-based neural decoders offer the advantages of guaranteeing the performance of existing iterative algorithms and using hardware architectures [7] that are already well optimized for iterative decoding algorithms.

LDPC codes have been incorporated into WiMAX and 5G communication systems [8, 9], owing to their strong error-correcting capabilities and low decoding complexity [10, 11]. However, more advanced LDPC coding technology needs to be developed for diverse communication environments lying in the scope of future 6G systems. In particular, for environments that require extremely low frame error rate (FER) such as the next generation ultra-reliable and low-latency communications (xURLLC) [12], it is crucial to mitigate the error-floor in the decoding of LDPC codes. The error-floor phenomenon refers to an abnormal phenomenon where the FER does not decrease as rapidly as in the waterfall region [11, 13]. The error-floor phenomenon also should be addressed for systems demanding very high reliability, such as solid-state drive (SSD) storage [14], DNA storage [15], and cryptosystems [16]. However, enhancing other features of LDPC codes often inadvertently reinforces the error-floor phenomenon as a side effect. For instance, the error-floor tends to be intensified when optimizing LDPC codes for superior waterfall performance or decoding by low complexity decoders such as quantized MS decoders [17]. Therefore, research focused on alleviating the error-floor, especially when decoding LDPC codes optimized for performance with low decoding complexity, has become significant. Such advancements will broaden the applications of LDPC codes.

1.1 Main contributions

With this need in mind, we focus on how to train a low-complexity neural MS (NMS) decoder to prevent the error-floor in well designed LDPC codes. The main contributions of the paper are threefold as follows.

Boosting learning using uncorrected words: We first leverage the boosting learning technique [18, 19] that employs a sequential training approach for multiple classifiers, wherein subsequent classifiers concentrate on the data samples that preceding classifiers incorrectly classify. Inspired by this method, we divide the neural decoder into two cascaded neural decoders and train the first decoder to be focused on the waterfall performance, while training the second decoder to be specialized in handling the uncorrected words that are not corrected by the first decoder due to the error-floor phenomenon. Uncorrected words in the error-floor region mostly contain small-error patterns related to trapping sets or absorbing sets [11], which can be effectively corrected by weighting decoding messages. As a result, a significant performance improvement in the error-floor region is achieved by boosting learning.

Block-wise training schedule with retraining: To mitigate the error-floor, iterative decoding typically requires a large number of decoding iterations, often exceeding 50 [17, 20, 21, 22]. However, NMS decoders encompassing many iterations can undergo the vanishing gradient problem in training [23]. To address this problem, we propose a new training schedule inspired by block-wise training methods [24, 25]. The proposed block-wise training schedule divides the entire decoding iterations into sub-blocks and trains these sub-blocks in a sequential manner. Additionally, rather than fixing the weights trained from previous blocks, we retrain them to escape from local minima. As a result, the proposed schedule enables to train numerous weights for all 50 iterations successfully while outperforming both the multi-loss method [5] and the iter-by-iter schedule [26].

Weight sharing technique with dynamic weight allocation: The weight sharing technique is a way to reduce the number of trainable weights by grouping specific weights to share a common value. The waterfall performance does not severely degrade even if we bundle all weights for each iteration [27, 26]. However, our observations indicate that this does not hold true in the error-floor region, implying that a higher degree of weight diversity is necessary to correct error patterns causing the error-floor. To obtain sufficient diversity with a minimal number of weights, we dynamically assign different weights to unsatisfied check nodes (UCNs) and satisfied check nodes (SCNs) in the decoding process. By utilizing only two weight values for SCNs and UCNs each iteration, we achieve the performance of the NMS decoder using different weights for every edge. This method reduces the number of weights to be trained to only 2.6% of the original number of weights.

We employ these training methods on a range of representative LDPC codes adopted for standards such as WiMAX [8], IEEE 802.11n [28], and 5G new radio [9]. The FER point at the onset of the error-floor diminishes by over two orders of magnitude for all codes compared to conventional weighted MS (WMS) decoding [29]. Compared to existing NMS decoding approaches [27, 26], our proposed scheme exhibits a notably enhanced capability to suppress the error-floor. This scheme also achieves a similar performance as the state-of-the-art post-processing method in [22], with only a third of the iterations.

1.2 Related works

We compare the proposed training scheme with the existing schemes for NMS decoders in Table 1. First, all works except [31, 32] aim to improve the waterfall performance. Although the scope of the works in [31, 32] includes the error-floor performance, they assumed specific conditions of binary symmetric channels (BSC) and FAID, while we deal with the more general situation of additive white Gaussian noise (AWGN) channels and MS decoding. Regarding the training sample selection, the training samples can be received words randomly taken from the AWGN channel [5, 27, 31, 33], or codewords with erroneous trapping sets (or absorbing sets) [30, 32]. However, to use the method in [30, 32], trapping sets should be enumerated, which is only applicable to short LDPC codes and not feasible for medium to large length standard LDPC codes. In contrast, the proposed boosting method, which generates training samples through decoding with linear complexity, can be applied even to codes of several thousand lengths.

For scheduling of training, it is common to train all weights at once (One-shot training) [5] and some works sequentially train the weights corresponding to a single iteration locally [27, 33], while we train a block of iterations with retraining. In terms of the weight sharing techniques, we confirm that the proposed sharing technique using UCN weights is superior to the spatial or temporal sharing technique used in [5, 27, 30, 31, 32]. Meanwhile, a method of assigning different weights to UCNs has been introduced in [33], but they applied the same UCN weight to all CNs belonging to a proto CN when at least one CN is unsatisfied, whereas we pinpoint specific UCNs and apply weights individually.

There have been studies adding hypernetworks to the Vanilla NMS decoder [34, 35, 36] or using a transformer architecture [6] to improve the waterfall performance at the expense of increased training and decoding costs. While the proposed training techniques are broadly applicable to these augmented neural decoders, this work primarily aims to improve the error-floor performance of the Vanilla NMS decoder under practical conditions.

2 Preliminaries

2.1 LDPC codes

In this paper, we consider quasi-cyclic (QC) LDPC codes, which have been adopted in various applications due to their implementation advantages [13, 37]. The Tanner graph of a QC-LDPC code, consisting of n = Nz VNs and m = Mz CNs, can be obtained by lifting the protograph, composed of M proto CNs and N proto VNs, with a lifting factor z [37, 38]. Let E be the total number of edges in the protograph. As a running example, we use the WiMAX QC-LDPC code of length n = 576 and code-rate 3 / 4 with N = 24 , M = 6 , E = 88 , and z = 24 [8] .

2.2 Neural min-sum decoding

For iteration ℓ , let m ( ℓ ) c → v represent the message from CN c to VN v and let m ( ℓ ) v → c represent the message from VN v to CN c . The neighboring nodes of x are represented by N ( x ) . The initial conditions are m (0) v → c = m ch v , m (0) c → v = 0 for the channel LLR m ch v of VN v . For ℓ = 1 , . . . , ℓ , the

𝑣1

𝑣2

𝑣3

𝑐 1

𝑐2

𝑣1

𝑣2

𝑣3

𝑣1

𝑒2

𝑣2

𝑣3

𝑒1

𝑒2

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

ഥ

𝑤 1

(1)

𝑒1

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

𝑣1

𝑣2

𝑣3

𝑣1

𝑣2

𝑣3

ഥ

𝑤 1

(1)

ഥ

𝑤 2

(1)

ഥ

𝑤 3

(1)

𝑤 1→1

(1)

𝑤 2→1

(1)

ഥ

𝑤 1

(2)

ഥ

𝑤 1

(2)

ഥ

𝑤 2

(2)

ഥ

𝑤 3

(2)

𝑤 1→1

(2)

𝑤 2→1

(2)

𝑤 1→2

(2)

𝑣𝑖

: Input nodes (

ch

)

𝑣𝑖

: output nodes (

→ 𝑚𝑣𝑖

o

)

1 st

hidden

layer

𝑤 2→3

(2)

2 nd

hidden

layer

3 rd

hidden

layer

4 th

hidden

layer

(a)

𝑐 1

𝑐2

𝑣1

𝑒2

𝑣2

𝑣3

𝑒1

𝑒2

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

𝑣1

𝑣2

𝑣3

ഥ

𝑤 1

(1)

𝑒1

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

𝑒1

𝑒2

𝑒3

𝑒4

𝑣1

𝑣2

𝑣3

𝑣1

𝑣2

𝑣3

ഥ

𝑤 1

(1)

ഥ

𝑤 2

(1)

ഥ

𝑤 3

(1)

𝑤 1→1

(1)

𝑤 2→1

(1)

ഥ

𝑤 1

(2)

ഥ

𝑤 1

(2)

ഥ

𝑤 2

(2)

ഥ

𝑤 3

(2)

𝑤 1→1

(2)

𝑤 2→1

(2)

𝑤 1→2

(2)

𝑣𝑖

: Input nodes (

ch

)

𝑣𝑖

: output nodes (

→ 𝑚𝑣𝑖

o

)

1 st

hidden

layer

𝑤 2→3

(2)

2 nd

hidden

layer

3 rd

hidden

layer

4 th

hidden

layer

(b)

NMS decoding algorithm [5] updates the messages as follows

13

13

where w ( ℓ ) v and w ( ℓ ) c → v are called the VN weight and CN weight, respectively. At the last iteration ℓ , output LLRs m o v are computed as m o v = m ch v + ∑ c ′ ∈N ( v ) m ( ℓ ) c ′ → v .

By quantizing m ( ℓ ) v → c , m ( ℓ ) c → v , and m ch v , the quantized NMS decoding algorithm is obtained. The quantized decoders are widely used in practical applications due to its low complexity and commonly employed in existing error-floor researches [17, 20, 21, 22]. Therefore, we use it to ensure a fair comparison. Specifically, we use 5-bit uniform quantization with a maximum magnitude of 7 . 5 and a step size of 0 . 5 for the quantized NMS decoder as in [20, 21, 22].

2.3 Training weights for the NMS decoder

If all the weights in (1) and (2) are set to 1 , NMS decoding is equivalent to MS decoding [39], or if VN weights w ( ℓ ) v are 1 and CN weights w ( ℓ ) c → v have the same value, the decoder operates as the WMS decoder [40]. The NMS decoder gains performance improvement over the WMS or MS decoder by greatly increasing the diversity of weights. However, the full diversity weights increase the training complexity and require a large amount of memory to store the weights. Therefore, previous studies used weight sharing techniques by assigning the same value to weights with the same attributes. First, since this paper deals with QC-LDPC codes, we use the protograph weight sharing technique [26] by default, assigning the same weight value to the VNs (or CNs) belonging to a proto VN (or CN). Then, the weights to be trained are represented by { w ( ℓ ) v p , w ( ℓ ) c p → v p } for a proto VN v p and a proto CN c p . The total number of weights is then ( N + E ) ℓ . If we employ spatial weight sharing in [27], only one VN weight and one CN weight remain for each iteration, and the weights are { w ( ℓ ) , w ( ℓ ) } ℓ ℓ =1 , with a total number of 2 ℓ . On the other hand, by using temporal weight sharing [5] to eliminate differences between iterations, the weights are w , w → , and the number is ( N + E ) .

The neural network in Fig. 1(b) corresponds to NMS decoding of the Tanner graph in Fig. 1(a) with ℓ = 2 . The input to this neural network is the channel LLR vector ( m ch 1 , . . . , m ch n ) , and the output is the output LLR vector ( m o 1 , . . . , m o n ) . For each iteration, two hidden layers are arranged, and each hidden layer has a number of nodes equal to the number of edges in the Tanner graph. In the odd hidden layers, the VN to CN message operation in (1) is performed, while in the even hidden layers, the CN to VN message operation in (2) is performed. The input layer is also connected to the odd hidden layers, which corresponds to the addition of the channel LLR in (1). The messages from the 2 ℓ -th hidden layer to the (2 ℓ +1) -th hidden layer are weighted by w ( ℓ ) c → v , and the messages from the

{ v p c p v p }

𝑣1

𝑣2

𝑣3

input nodes to the (2 ℓ +1) -th hidden layer are weighted by w ( ℓ ) v . As our goal is to reduce the FER in the error-floor region, we use the FER loss, 1 2 [ 1 -sgn ( min 1 ≤ v ≤ N m o v ) ] [32].

3 Proposed training method

Algorithm 1 Training method

Input:

/lscript = /lscript 1 + /lscript 2 , R 1 , R 2 1: Base decoder training : Train the weights w B = { w ( /lscript ) , w ( /lscript ) } /lscript 1 /lscript =1 with the received words sampled at R 1

2: Collecting the uncorrected words : Run the base decoder with the received words sampled at R 2 and collect the received words when the decoder fails

3: Post decoder training : Train the weights w P = { w ( /lscript ) , w ( /lscript ) , ˆ w ( /lscript ) } /lscript /lscript = /lscript 1 +1 with the uncorrected words and the block-wise training schedule.

Output:

Trained weights

w

B

,

w

P

(a)

Received words

sampled at

ℛ1

Train

𝐰B

Decoding

with

𝐰B

Received words

sampled at

ℛ2

Collect

uncorrected

words

Uncorrected

words

Train

𝐰𝐏

Uncorrected

words

Base decoder

Training

Post decoder

Training

Collecting the

uncorrected words

Fail

(b)

In this section, we introduce the proposed training methods through three subsections. The organized training algorithm is shown in Fig. 2.

3.1 Boosting learning using uncorrected words

5 For the boosting learning approach, we first divide the entire decoding process into two stages: the base decoding stage { 1 , . . . , ℓ 1 } and the post decoding stage { ℓ 1 +1 , . . . , ℓ 1 + ℓ 2 = ℓ } . Training of base decoder follows the conventional training method: the received words sampled from E b / N 0 region R 1 are used as training samples. Specifically, we set ℓ 1 = 20 and R 1 = { 2 . 0 , 2 . 5 , 3 . 0 , 3 . 5 , 4 . 0 } , which corresponds the waterfall region of WMS decoding. We use the spatial sharing technique (i.e., w B = { w ( ℓ ) , w ( ℓ ) } ℓ 1 ℓ =1 ) since this achieves comparable performance to the full diversity weights in the waterfall region.

In Fig. 3(a), the base NMS decoder is compared with the MS decoder and the WMS decoder with a single weight of 0 . 75 for ℓ = 20 . The WMS decoding performance has a severe error-floor even though its waterfall performance is better than the MS decoding performance. Compared to the MS and WMS decoders, the NMS decoder for ℓ = 20 performs better over the training range R 1 . On the other hand, the NMS decoder performs worse than the MS decoder in the error-floor region (e.g., 4 . 5 dB), which is outside R 1 . To improve the performance in the error-floor region, a straightforward approach is extending the training range R 1 to include the error-floor region. However, the FER of the base decoder for received words from the error-floor region is very low, resulting in an almost negligible FER loss. Consequently, integrating the error-floor region into the training range does not impact the weight update process.

Before training the post decoder, we first collect uncorrected words that the trained base decoder fails to correct among the received words sampled from region R 2 . Then, the uncorrected words serve as training samples for the post decoder, which is distinct from the conventional training methods. The post decoder trains the weights { w ( ℓ ) v p , w ( ℓ ) c p → v p } ℓ ℓ = ℓ 1 +1 with the aim of correcting the uncorrected words. After completing the training, the trained weights are used for the NMS decoding algorithm in (1)-(2). From the perspective of the NMS decoder, it performs continuous decoding up to iteration ℓ using the trained weights, but for the sake of discussion, we assume as if there are two cascaded decoders (base and post) in the perspective of training. Note that we employ the full diversity weights for the post decoder to confirm the best performance but we will introduce the shared weights w P (used in Fig. 2) in the next subsection. We also set l 2 = 10 , ℓ = 30 for this experiment, and subsequently extend the maximum number of iterations in the following subsection.

To analyze the effectiveness of the proposed boosting learning, we compare the following three cases.

1

2

2.5

3

3.5

4

4.5

5

5.5

6

10

-7

10

-6

10

-5

10

-4

10

-3

10

-2

10

-1

10

0

(a)

5

10

15

20

25

30

35

40

45

50

0

100

200

300

400

500

600

700

800

(b)

Case 1 Uncorrected words sampled at 4 . 5 dB in the error-floor region (i.e., R 2 = 4 . 5 ).

Case 3 Received words sampled at 4 . 5 dB without filtering.

Case 2 Uncorrected words sampled at 3 . 5 dB in the waterfall region (i.e., R 2 = 3 . 5 ).

Regarding Case 1 and Case 2, we collect a total of 60 , 000 uncorrected words, allocating 50 , 000 for training, 5 , 000 for validation, and remaining 5 , 000 for test. Training is conducted for 100 epochs. Fig. 3(b) shows the distribution of the number of errors after base decoding and post decoding for the test samples used in Case 1 and Case 2. For Case 1, the uncorrected words collected in the error-floor region mainly have a small number of errors since most of decoding errors are trapped in small trapping sets, so the distribution is concentrated on small numbers (see Case 1 after base decoding). For ease of use, we refer to codewords with fewer than 11 remaining errors as small-error words. The post decoder, which is mainly trained on these small-error words, corrects a significant number of small-error words (see Case 1 after post decoding). Out of the total 5 , 000 test samples, 68 . 5% of samples are corrected by the post decoder, resulting that the test FER for the test samples is 0 . 315 . This means that, when decoding for received words of the AWGN channel, the resulting FER at E b / N 0 = 4 . 5 dB after post decoding is 0 . 315 times of the FER after base decoding as shown in Fig 3(a). In other words, the post decoder is successfully trained to correct small-error words inducing the error-floor.

On the other hand, Case 2, where the samples are collected from the waterfall region, has a distribution that is widespread across all areas (see Case 2 after base decoding). In addition, the distribution remains almost the same after post decoding (see Case 2 after post decoding), which means that the post decoder fails to reduce the FER sufficiently. For the test samples, the test FER at E b / N 0 = 3 . 5 dB after post decoding is 0 . 77 times of the FER after base decoding whose difference is not noticeable as shown in Fig 3(a). Comparing the results of the two cases, we conclude that composing mainly with small-error words facilitates the post decoder to learn to correct small-error words more effectively. As a result, Fig. 3(a) shows that Case 1 mitigates the error-floor more than Case 2. Meanwhile, for Case 3, where all received words are used as training samples without filtering, almost all of them are corrected during base decoding. Since the post stage training is mainly performed on codewords without errors, the loss function becomes almost 0 . Then, the weights of the post decoder are unchanged from the initial value 1 , and accordingly, the performance approaches the MS decoding performance, as shown in Fig. 3(a).

3.2 Block-wise training schedule

In the previous subsection, the number of iterations for the post decoder is set to ℓ 2 = 10 . To lower the error-floor further, a large number of iterations is required, so we set ℓ 2 = 30 , ℓ = 50 . However, deep neural decoders with a large iteration number are prone to suffer from the vanishing gradient problem. In order to tackle this issue, we propose a block-wise training schedule which is shown in Fig. 4(a). The proposed training schedule locally trains the weights corresponding to a block of ∆ 1 iterations at each training stage. In the first stage, the weights belonging to the first block are trained,

Δ1

Fixed weights

Training with

all one

initial values

Re-training with

given

initial values

Δ2

Stage

1

Iteration

Stage

2

Stage

3

Δ1

Δ2

Δ1

(a)

20

25

30

35

40

45

50

0

0.2

0.4

0.6

0.8

1

(b)

17

Test FER

Number of weights

Full diversity

0

.

112

(

N

+

E

)

ℓ

2

= 3360

Spatial sharing

0

.

168

2

ℓ

2

= 60

Temporal sharing

0

.

186

(

N

+

E

) = 112

Proposed sharing

0

.

111

3

ℓ

2

= 90

: Error VN

: Unsatisfied CN

: Satisfied CN

: Correct VN

: Edges weighted by

𝑤 (ℓ)

: Edges weighted by

ෝ

𝑤 (ℓ)

Test FER

Number of weights

Full diversity

0.112

Spatial sharing

0.168

2ℓ2 = 60

Temporal sharing

0.186

= 112

Proposed sharing

0.111

3ℓ2 = 90

18 and in the next stage, the weights of the subsequent ∆ 1 iterations are trained. At this point, the weight values of previous ∆ 2 iterations, which are already trained in the first stage, are further trained by taking the result of the first stage training as its initial state. This retraining, which is not used in the iter-by-iter training schedule [26], assists in preventing the learning process from falling into a local minimum. Note that the method of one-shot training [5] corresponds to the case of ∆ 1 = ℓ 2 , ∆ 2 = 0 , and the iter-by-iter training schedule [26] is equivalent to the case of ∆ 1 = 1 , ∆ 2 = 0 .

We employ a greedy approach to determine the optimal values for ∆ 1 and ∆ 2 , resulting that ∆ 1 = 5 , ∆ 2 = 10 offers superior performance in terms of the test FER. Fig. 4(b) shows the evolution of test FER as a function of the iteration number. In the case of one-shot training [5], the vanishing gradient problem hinders training the weights of earlier iterations, so the test FER stays roughly the same until iteration 40 and starts to fall thereafter. The same behavior is observed for ℓ = 40 . Thus, the test FERs of ℓ = 40 and ℓ = 50 are almost the same. Next, since the iter-by-iter training schedule [26] finds the optimal weight for each individual iteration, the test FER falls even at the first iteration of the post decoder (i.e., ℓ = 21 ) without encountering the vanishing gradient problem. However, this local optimization leads to a degraded local minimum, and consequently, the test FER gradually and slowly decreases with each iteration. Likewise, the multi-loss method [5] shows a similar result. In contrast, the block-wise training schedule with ∆ 1 = 5 , ∆ 2 = 0 shows a superior test FER at iteration 25 compared to the other training schedules because it results in a better solution by training the weights of multiple iterations simultaneously. Moreover, the schedule with retraining (i.e., ∆ 1 = 5 , ∆ 2 = 10 ) outperforms the schedule without retraining (i.e., ∆ 1 = 5 , ∆ 2 = 0 ) at iteration 30 though it shows a worse result at iteration 25 . This implies that through retraining, the weights of intermediate iterations have been adjusted to preprocess error patterns thereby leading to stronger correction capabilities in the final iteration. As a result, at the maximum of 50 iterations, the proposed training schedule with ∆ 1 = 5 , ∆ 2 = 10 provides the better test FER value compared to the other training schedules as shown in Fig. 4(b): 0 . 11 for the block-wise, 0 . 16 for the multi-loss, 0 . 18 for the one-shot, 0 . 37 for the iter-by-iter, .

3.3 Weight sharing technique using UCN weights

Assuming the techniques proposed thus far (in detail, uncorrected words at E b / N 0 = 4 . 5 dB and training schedule with ∆ 1 = 5 , ∆ 2 = 10 ) are used, we compare the weight sharing techniques

in Fig. 5. Compared to the full diversity weights, the spatial and temporal sharing techniques significantly reduce the number of distinct weights, but cause performance degradation. In contrast, the proposed sharing technique that introduces a new weight type called UCN weight shows almost identical performance while using only about 2.6% of the weights compared to the full diversity weights. The proposed sharing technique assigns different weights to UCNs and SCNs as shown in Fig. 5. This is feasible because the decoder knows whether a CN satisfies the check equation or not. Using the spatial sharing technique and distinguishing between SCN weight w ( ℓ ) and UCN weight ˆ w ( ℓ ) , the proposed sharing technique can be represented as { w ( ℓ ) , w ( ℓ ) , ˆ w ( ℓ ) } ℓ for iteration ℓ and the total number of distinct weights becomes 3 ℓ 2 .

Techniques using different weights for UCNs and SCNs have also been proposed in [33, 41]. However, the work [41] uses only one suppression factor ρ to represent the UCN weight (i.e., ˆ w ( ℓ ) = (1 + ρ ) w ( ℓ ) ), making the UCN weight dependent on the CN weight. As a result, due to the limited degree of freedom for the UCN weight, it is difficult to obtain the decoding diversity for effectively removing various types of error patterns. Moreover, in [33], if at least one of the CNs belonging to a single proto CN is unsatisfied, all z CNs from the proto CN are weighted by the UCN weight. This approach, which applies the same weight to a large number of CNs tied together at the proto level, is not suitable for correcting words with a small number of UCNs, because it does not separately handle individual CNs like the proposed method.

4 Performance evaluation

2

3

4

5

6

7

10

-6

10

-4

10

-2

10

0

(a)

2.5

3

3.5

4

4.5

5

5.5

6

6.5

10

-6

10

-4

10

-2

10

0

(b)

1

2

3

4

5

10

-6

10

-4

10

-2

10

0

(c)

In this section, we compare the proposed and other conventional decoding schemes in terms of the decoding performance. All simulations are performed by NVIDIA GeForce RTX 3090 GPU and AMD Ryzen 9 5950X 16-Core Processor CPU. For training, the weights are trained by the Adam optimizer [42] with a learning rate of 0 . 001 . We evaluate the decoding performance using Monte Carlo methods with at least 500 uncorrected words for each FER point. Fig. 6(a) shows the FER performances of the proposed scheme for the WiMAX LDPC code. The proposed scheme incorporates the i) boosting learning with uncorrected words, ii) block-wise training schedule with retraining, and iii) spatial weight sharing with UCN weights. The performance is compared with MS decoding, WMS decoding, and existing NMS decoding schemes in [5, 26]. Among the neural decoder studies listed in Table 1, we exclude comparison with the studies that use FAID and layered decoding, or that require enumerating trapping sets and absorbing sets. In addition, we choose not to compare with augmented neural networks [6, 34] since our approach does not increase model complexity to deal with the low-error-rate region of long codes. A comparative analysis for short codes in the waterfall region can be found in the appendix.

For the NMS decoding schemes in [5, 26], the base decoder is used for iterations from 1 to 20 like the proposed scheme, and the training methods introduced in [5, 26] are employed for the post stage. The full diversity weights are used for the schemes in [5, 26] to observe their best performance. For the scheme in [5], received words in the waterfall region ( E b / N 0 2 -4 dB) are used as training samples, and the weights for the post stage are trained all at once without a training schedule. For the scheme

in [26], received words from the E b / N 0 points where the MS decoder achieves bit error rate of 10 -3 are used as training samples, and the iter-by-iter training schedule is employed. The remaining hyper-parameters are set in the same way as in the proposed scheme. As shown in Fig. 6(a), the conventional NMS decoders in [5, 26] show good performance in the waterfall region ( E b / N 0 2 -4 dB), but the error-floor occurs from 4 dB. This is because the training samples are composed of received words without filtering. In contrast, the proposed scheme shows excellent performance in both the waterfall and error-floor regions, and the error-floor phenomenon is barely noticeable down to FER of 10 -7 . In particular, comparing the results of ℓ = 20 and ℓ = 50 , it is confirmed that the post decoder successfully removes the error-floor.

In addition, we compare the proposed scheme with the state-of-the-art post processing scheme in [22]. We directly reference the simulation results from [22]. As shown in Fig. 6(a), the scheme in [22] shows a similar or worse performance to the proposed scheme, but it has disadvantages of having very high decoding complexity and latency since it consumes a large number of iterations ℓ = 150 . Table 2 compares the schemes in terms of the decoding complexity. The NMS decoder has more multiplications than the MS decoder by ( E + N ) z due to the weighting operation. The number of other operations is the same as in the MS decoder. Total complexity is evaluated with assumption that the comparison C is as twice as complex than the addition A and multiplication M [43]. The additional memory for storing the weights of the proposed scheme is 3 ℓ which is much lower than those of [5, 26] which exploit full weight diversity. Since the scheme in [22] does not use weighting, the complexity per iteration is lower than the proposed NMS scheme, but the total complexity is more than twice as high as the proposed NMS scheme due to the higher number of iterations. Moreover, additional complexity is required for the error path detector [22]. In Fig. 6(b), (c), similar results are observed for the IEEE802.11n LDPC and 5G LDPC codes, where the proposed scheme outperforms the other schemes and achieves an FER of 10 -7 without a severe error-floor.

5 Conclusions

This paper proposed training methods for the NMS decoder of LDPC codes to enhance the error-floor performance. Using uncorrected words from the base decoder, we trained the post decoder to be specialized for error patterns causing the error-floor, promoting decoding diversity in the cascaded base and post decoders. We also proposed a training schedule to circumvent the vanishing gradient and local minimum problems, and a weight sharing technique that significantly reduces the number of distinct weights without sacrificing performance. The proposed NMS decoder using the trained weights showed the excellent waterfall and error-floor performances for several standard LDPC codes. Along with the performance improvement, the proposed training scheme has the advantage of being flexibly applicable regardless of the types of channel, code, and decoding algorithm. This scheme can also be implemented directly on hardware architectures without additional costs, and can be directly utilized with no prior analysis of the target code and decoding algorithm.

6 Acknowledgments

This work was supported by Samsung Electronics Co., Ltd (IO230411-05859-01), by Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government [20210-00746, Development of Tbps wireless communication technology], by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00212103),

and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00247197).

References

Jehoshua Bruck and Mario Blaum. Neural networks, error-correcting codes, and polynomials over the binary n-cube. IEEE Transactions on Information Theory , 35(5):976-987, 1989.

William R Caid and Robert W Means. Neural network error correcting decoders for block and convolutional codes. In [Proceedings] GLOBECOM'90: IEEE Global Telecommunications Conference and Exhibition , pages 1028-1031. IEEE, 1990.

LG Tallini and P Cull. Neural nets for decoding error-correcting codes. In IEEE Technical Applications Conference and Workshops. Northcon/95. Conference record , page 89. IEEE, 1995.

Tobias Gruber, Sebastian Cammerer, Jakob Hoydis, and Stephan ten Brink. On deep learningbased channel decoding. In 2017 51st Annual Conference on Information Sciences and Systems (CISS) , pages 1-6. IEEE, 2017.

Eliya Nachmani, Elad Marciano, Loren Lugosch, Warren J Gross, David Burshtein, and Yair Be'ery. Deep learning methods for improved decoding of linear codes. IEEE Journal of Selected Topics in Signal Processing , 12(1):119-131, 2018.

Yoni Choukroun and Lior Wolf. Error correction code transformer. In Advances in Neural Information Processing Systems .

Shuai Shao, Peter Hailes, Tsang-Yi Wang, Jwo-Yuh Wu, Robert G Maunder, Bashir M AlHashimi, and Lajos Hanzo. Survey of turbo, LDPC, and polar decoder ASIC implementations. IEEE Communications Surveys & Tutorials , 21(3):2309-2333, 2019.

IEEE standard for local and metropolitan area networks-part 16: Air interface for fixed and mobile broadband wireless access systems amendment 2: Physical and medium access control layers for combined fixed and mobile operation in licensed bands and corrigendum 1. IEEEStandard 802.16e , 2020.

Technical specification group radio access network; NR; multiplexing and channel coding (release 17), document ts 38.212 v17.5.0. 3rd Generation Partnership Project , 2023.

Tom Richardson and Shrinivas Kudekar. Design of low-density parity check codes for 5G new radio. IEEE Communications Magazine , 56(3):28-34, 2018.

Tom Richardson and Ruediger Urbanke. Modern Coding Theory . Cambridge University Press, 2008.

Een-Kee Hong, Inkyu Lee, Byonghyo Shim, Young-Chai Ko, Sang-Hyo Kim, Sangheon Pack, Kyunghan Lee, Sunwoo Kim, Jae-Hyun Kim, Yoan Shin, et al. 6G R&D vision: Requirements and candidate technologies. Journal of Communications and Networks , 24(2):232-245, 2022.

William Ryan and Shu Lin. Channel Codes: Classical and Modern . Cambridge University Press, 2009.

Guiqiang Dong, Ningde Xie, and Tong Zhang. On the use of soft-decision error-correction codes in NAND flash memory. IEEE Transactions on Circuits and Systems I: Regular Papers , 58(2):429-439, 2010.

Shubham Chandak, Kedar Tatwawadi, Billy Lau, Jay Mardia, Matthew Kubit, Joachim Neu, Peter Griffin, Mary Wootters, Tsachy Weissman, and Hanlee Ji. Improved read/write cost tradeoff in DNA-based data storage using LDPC codes. In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 147-156. IEEE, 2019.

Marco Baldi, Marco Bodrato, and Franco Chiaraluce. A new analysis of the McEliece cryptosystem based on QC-LDPC codes. In Security and Cryptography for Networks: 6th International Conference, SCN 2008, Amalfi, Italy, September 10-12, 2008. Proceedings 6 , pages 246-262. Springer, 2008.

Xiaojie Zhang and Paul H Siegel. Quantized iterative message passing decoders with low error floor for LDPC codes. IEEE Transactions on Communications , 62(1):1-14, 2013.

Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences , 55(1):119-139, 1997.

Yoav Freund, Robert Schapire, and Naoki Abe. A short introduction to boosting. JournalJapanese Society For Artificial Intelligence , 14(771-780):1612, 1999.

Jingyu Kang, Qin Huang, Shu Lin, and Khaled Abdel-Ghaffar. An iterative decoding algorithm with backtracking to lower the error-floors of LDPC codes. IEEE Transactions on Communications , 59(1):64-73, 2010.

Soonyoung Kang, Jaekyun Moon, Jeongseok Ha, and Jinwoo Shin. Breaking the trapping sets in LDPC codes: Check node removal and collaborative decoding. IEEE Transactions on Communications , 64(1):15-26, 2015.

Seokju Han, Jieun Oh, Kyungmok Oh, and Jeongseok Ha. Deep-learning for breaking the trapping sets in low-density parity-check codes. IEEE Transactions on Communications , 70(5):2909-2923, 2022.

Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 249-256. JMLR Workshop and Conference Proceedings, 2010.

Arild Nøkland and Lars Hiller Eidnes. Training neural networks with local error signals. In International Conference on Machine Learning , pages 4839-4850. PMLR, 2019.

Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Greedy layerwise learning can scale to imagenet. In International Conference on Machine Learning , pages 583-593. PMLR, 2019.

Jincheng Dai, Kailin Tan, Zhongwei Si, Kai Niu, Mingzhe Chen, H Vincent Poor, and Shuguang Cui. Learning to decode protograph LDPC codes. IEEE Journal on Selected Areas in Communications , 39(7):1983-1999, 2021.

Mengke Lian, Fabrizio Carpi, Christian Häger, and Henry D Pfister. Learned belief-propagation decoding with simple scaling and SNR adaptation. In 2019 IEEE International Symposium on Information Theory (ISIT) , pages 161-165. IEEE, 2019.

IEEE standard for information technology-telecommunications and information exchange between systems local and metropolitan area networks-specific requirements-part 11: Wireless lan medium access control (MAC) and physical layer (PHY) specificatio. IEEE Standard 802.11 , 2016.

Jinghu Chen, Ajay Dholakia, Evangelos Eleftheriou, Marc PC Fossorier, and Xiao-Yu Hu. Reduced-complexity decoding of LDPC codes. IEEE Transactions on Communications , 53(8):1288-1299, 2005.

Joachim Rosseel, Valérian Mannoni, Inbar Fijalkow, and Valentin Savin. Decoding short LDPC codes via BP-RNN diversity and reliability-based post-processing. IEEE Transactions on Communications , 70(12):7830-7842, 2022.

Xin Xiao, Bane Vasi´ c, Ravi Tandon, and Shu Lin. Designing finite alphabet iterative decoders of LDPC codes via recurrent quantized neural networks. IEEE Transactions on Communications , 68(7):3963-3974, 2020.

Xin Xiao, Nithin Raveendran, Bane Vasi´ c, Shu Lin, and Ravi Tandon. FAID diversity via neural networks. In 2021 11th International Symposium on Topics in Coding (ISTC) , pages 1-5. IEEE, 2021.

Nemin Shah and Yash Vasavada. Neural layered decoding of 5G LDPC codes. IEEE Communications Letters , 25(11):3590-3593, 2021.

Eliya Nachmani and Lior Wolf. Hyper-graph-network decoders for block codes. Advances in Neural Information Processing Systems , 32, 2019.

Sebastian Cammerer, Jakob Hoydis, Fayçal Aït Aoudia, and Alexander Keller. Graph neural networks for channel decoding. In 2022 IEEE Globecom Workshops (GC Wkshps) , pages 486-491. IEEE, 2022.

Eliya Nachmani and Lior Wolf. Autoregressive belief propagation for decoding block codes. arXiv preprint arXiv:2103.11780 , 2021.

Marc PC Fossorier. Quasicyclic low-density parity-check codes from circulant permutation matrices. IEEE transactions on information theory , 50(8):1788-1793, 2004.

Jeremy Thorpe. Low-density parity-check (LDPC) codes constructed from protographs. IPN progress report , 42(154):42-154, 2003.

Marc PC Fossorier, Miodrag Mihaljevic, and Hideki Imai. Reduced complexity iterative decoding of low-density parity check codes based on belief propagation. IEEE Transactions on Communications , 47(5):673-680, 1999.

Jinghu Chen, Ajay Dholakia, Evangelos Eleftheriou, Marc PC Fossorier, and Xiao-Yu Hu. Reduced-complexity decoding of LDPC codes. IEEE Transactions on Communications , 53(8):1288-1299, 2005.

Xiaofu Wu, Yue Song, Ming Jiang, and Chunming Zhao. Adaptive-normalized/offset min-sum algorithm. IEEE Communications Letters , 14(7):667-669, 2010.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Heshani Gamage, Nandana Rajatheva, and Matti Latva-Aho. Channel coding for enhanced mobile broadband communication in 5G systems. In 2017 European Conference on Networks and Communications (EuCNC) , pages 1-6. IEEE, 2017.

Yury Polyanskiy, H Vincent Poor, and Sergio Verdú. Channel coding rate in the finite blocklength regime. IEEE Transactions on Information Theory , 56(5):2307-2359, 2010.

Shaohua Yang, Yang Han, Xuebin Wu, Roger Wood, and Rick Galbraith. A soft decodable concatenated LDPC code. IEEE Transactions on Magnetics , 51(11):1-4, 2015.

Jieun Oh, Jeongseok Ha, Hyegyeong Park, and Jaekyun Moon. RS-LDPC concatenated coding for the modern tape storage channel. IEEE Transactions on Communications , 64(1):59-69, 2015.

Hyegyeong Park and Jaekyun Moon. Improving SSD read latency via coding. IEEE Transactions on Computers , 69(12):1809-1822, 2020.

7 Appendix

7.1 A better trade-off between the waterfall and error-floor performances

2

3

4

5

6

7

10

-6

10

-4

10

-2

10

0

(a)

2

2.5

3

3.5

4

4.5

5

5.5

6

10

-6

10

-4

10

-2

10

0

(b)

Typically, LDPC code design assumes the availability of a floating-point decoder and optimizes the code to maximize waterfall performance. In Fig. 7(a), it's evident that the optimized WiMAX LDPC code outperforms the non-optimized regular LDPC code in terms of the waterfall performance when the floating-point MS decoder is employed. Additionally, the WMS decoder enhances the performance by assigning weights. As shown in Fig. 7(a), WMS decoding of the WiMAX LDPC code exhibits the best waterfall performance when floating-point operations are available. However, for practical hardware implementation, quantization of decoding messages is essential, but it leads to performance degradation. Notably, WMS decoding suffers substantial performance degradation with an early onset of the error-floor. A side-by-side comparison of the quantized WMS decoding performances between the non-optimized and WiMAX LDPC codes reveals that the performance loss due to quantization is more pronounced in optimized LDPC codes. Consequently, there's a need for a method that ensures high performance in both the waterfall and error-floor regions with low complexity. This work accomplishes this goal by leveraging machine learning techniques. In Fig. 7(a), the proposed quantized NMS decoder achieves the floating WMS decoding performance for the WiMAX LDPC code in both waterfall and error-floor regions.

7.2 Boosting learning of two networks v.s. conventional learning of a single large network

In Fig. 7(b), we compare the proposed method with the method that uses a single decoder, which is trained by a large single network at once. For the proposed method, the boosting learning is employed with two networks. Both have the same model size. Fig. 7(b) shows that simply increasing the model size doesn't significantly mitigate the error-floor. In contrast, the two-stage network using boosting learning achieves greater decoding diversity and demonstrates superior efficiency in alleviating the error-floor.

7.3 Optimization of parameters of the block-wise training schedule

To fine-tune the parameters ∆ 1 and ∆ 2 used in the block-wise training schedule, we compare the test FER values across a range of ∆ 1 and ∆ 2 . The comparison is done for the post decoder training

with ℓ 2 = 30 and is summarized in Table 3. For the case where ∆ 2 = 0 (i.e., without retraining), the performance improves as ∆ 1 increases from 1 to 10 . This can be attributed to the training of more weights simultaneously, aiding in escaping from local minima. However, if ∆ 1 becomes too large, the weights in front are not adequately trained due to the vanishing gradient problem, causing performance degradation. In addition, as the retraining iteration number ∆ 2 grows for a given ∆ 1 value, a similar trend is noted: performance enhances up to a certain point and then plateaus. According to Table 3, the best performance is achieved with ∆ 1 = 5 and ∆ 2 = 10 .

7.4 Discussion on the trained weights

5

10

15

20

25

30

35

40

45

50

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Phase1

Phase2

(a)

5

10

15

20

25

30

35

40

45

50

0

500

1000

1500

2000

2500

3000

3500

4000

Phase1

Phase2

(b)

Table 4 shows the resulting weights trained using the proposed training methods. Owing to the weight sharing technique, only three types of weights, VW w ( ℓ ) , CW w ( ℓ ) , and UCW ˆ w ( ℓ ) , exist for each iteration. Thus, weight variations across iterations can be depicted in two dimensions, as shown in Fig. 8(a), enabling analysis of the trained results. Without the weight sharing technique, there would be ( N + E ) weights per iteration, making a 2D graphical representation unfeasible. Fig. 8(a) shows that the weights for the base decoding stage, where the CW and VW are roughly 0 . 75 and 1 , are similar to the weights of conventional WMS decoding. As a result, the waterfall performance of WMS decoding and NMS decoding are nearly indistinguishable. The NMS decoding technique offers minimal advantage in the waterfall region.

On the other hand, for the post decoding stage, the weights undergo significant changes. At iteration 21 , the VW increases while the CW decreases substantially. This means that the channel LLR values are given more weight when performing the sum operation at each VN, while the messages coming from CNs are attenuated. As a result, the decoding process reverts to the initial decoding state (with

a high error rate). This leads to an increase in the number of samples having more than 6 errors (denoted by e > 6 ), as shown in Fig. 8(b), and a decrease in the number of samples with e ≤ 6 . This intentional regression aims to break free from trapped error patterns. From iteration 22 onward, the CWgradually increases, the VW decreases, and the count of e ≤ 6 samples starts to increase again. After iteration 27 , the CW and VW show less variations. We term the period from iteration 21 to 26 as Phase 1 and the period following iterations as Phase2. At iteration 20 , the numbers of samples with e ≤ 6 and e > 6 are roughly equal, but through Phase 1, samples with e > 6 transform into samples with e ≤ 6 . Consequently, by the end of Phase 1, there are more samples with e ≤ 6 than samples with e > 6 . The small-sized error samples diminished steadily in Phase 2.

The result without Phase 1 is also shown in Fig. 8(b). Without Phase 1, the process of transforming into small-sized error samples is omitted. Consequently, the decoding proceeds in a situation where the number of samples with e ≤ 6 and e > 6 is similar. Although Phase 2 effectively correct most of the e ≤ 6 samples, the e > 6 samples remain in significant numbers as they are initially abundant. This indicates the need for the pre-processing of Phase 1.

7.5 Multiple stage decoders

2

3

4

5

6

7

10

-6

10

-4

10

-2

10

0

(a)

4

5

6

7

8

9

10

11

12

10

-6

10

-4

10

-2

10

0

(b)

Extending the proposed method to include more than two decoders is feasible and can be applied in a similar manner. For instance, in the case of the WiMax LDPC code (shown in Fig. 6(a)), uncorrected words from the error-floor region of the base + post decoder (i.e., E b / N 0 4.5 dB) are collected to train a third decoder. Fig. 9(a) shows that the aid of this third decoder (labeled as "Base+Post+Third) reduces the error-floor further. It is worth mentioning that introducing this third decoder might lead to increased decoding latency. Moreover, the process of collecting uncorrected words would be time-consuming, especially in the very low FER region. Addressing these challenges could be a promising direction for subsequent research. In addition, we add a graph showing the finite length performance limit [44].

We want to note that the concept of multi-stage decoding has been employed in the previous works [45, 46, 47], particularly in the field of coding theory. Our study has similarities with theses previous approaches, as we also carries out a two-stage decoding. However, unlike physically divided decoders in [45, 46, 47], our method utilizes a single LDPC decoder. While the proposed decoder is conceptually divided into two stages (base/post) based on iterations, in practice, we employ only a single decoder with distinct weight parameter sets. Moreover, our approach has a novelty in that the post decoder is 'trained' dependent on the results of the base decoder.

7.6 Application to a different channel

The concept of training using uncorrected words can be consistently applied regardless of the channel type. In Fig. 9(b), we include the results for the Rayleigh channel with the scale parameter α = 1 , showing the effectiveness of our proposed method holds across different channel types.

7.7 Application to long length codes

3

4

5

6

7

8

10

-6

10

-4

10

-2

10

0

We show the results for the (1248, 1056) 5G LDPC code in Fig. 10. This result demonstrates the proposed method is also effective for long codes. There is little improvement in waterfall performance with the NMS technique at such long lengths. However, the performance improvement in the error-floor region using our proposed method is clearly evident.

7.8 Comparison with the augmented neural decoders

In Table 5, we compare the waterfall performance of various neural decoders for short length codes. To ensure a fair comparison with other works, we employ the NBP decoder and utilize the soft-BER loss function for BER performance optimization while making use of full diversity weights. Out of a total of 50 iterations, 20 iterations are allocated for base decoding and 30 iterations for post decoding.

The result reveals a notable performance improvement over the original NBP [5], which employs the same vanilla NBP architecture. The improvement is particularly pronounced at high SNR. We achieve even better performance at high SNR than the work [34], which adds hyper-networks to the vanilla NBP architecture. In comparison to the ECCT [6] using the transformer architecture, our method shows inferior performance for high density codes (Polar, BCH), yet it achieves fairly comparable performance for LDPC codes. Owing to its intricate transformer structure, the ECCT requires higher training and decoding complexity than NMS decoders [6], implying its current limited practicality. Nevertheless, it is worth emphasizing that our proposed training method can be adaptable across any architecture. In other words, there's promising potential in future research that integrates the proposed training methods with the architecture with hyper-networks or the transformer architecture.

7.9 Ablation study

In Table 6, we provide the ablation study for the WiMAX LDPC code. The result shows that the boosting learning is the core technique to reduce the FER performance and the block-wise training also contributes the FER reduction, while the proposed sharing technique does not involve performance degradation.

Peer reviews:
comments: SUMMARY:
1.	In the proposed work authors proposed 
         a.	Neural Min-Sum (NMS) decoders 
        b.	NMS decoder with block-wise training schedule that locally trains a block of weights while retraining the preceding block.
        c.	The different weights are assigned to the unsatisfied check nodes and training is carried out.
2.	The contribution of the proposed work are.
     a.	Boosting learning using uncorrected codewords
     b.	Block-wise training schedule with retraining
     c.	Weight sharing technique with dynamic weight allocation
3.	Sufficient results are presented and discussed.

STRENGTHS:
Novelty of the work is good.
Sufficient results are presented and discussed.

WEAKNESSES:
The experimetnal set up should be elaboprated more with parameters selected for training.

QUESTIONS:
1.	How the weights of the NMS decoders are initialized.
2.	Is it possible to design ASIC hardware of NMS decoder 
3.	What about the latency of the decoder.
4.	The neural decoders with recent state of art techniques with their advantages should be discussed in detail.
5.	The experimental set up should be clearly mentioned with parameters used for the training.
6.	How exactly the weights are updated in the training process of LDPC CODES mentioned in the figure 1 and what is the impact of weights in the training process should be clearly mentioned.

LIMITATIONS:
Since the authors mentioned that proposed work can be taken into hardware architecture.
But clear architectute of the proposed method is not highlighted.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-17

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a training framework for the NMS decoder of LDPC codes in order to enhance the error-floor performance of these codes. \\\nThe NMS decoder iterations are split into two cascaded parts, the first so-called based decoder is trained for decoding waterfall parts, and the second (post decoder) is for decoding uncorrected codewords caused by the error-floor.\\\nThe paper suggests the use of a training schedule reusing learned weights to tackle the vanishing gradient of many iterations NMS.\\\nThe proposed decoder using the trained weights showed improved error-floor performances for several standard LDPC codes and can be efficiently integrated into current NMS solutions.\n\nSTRENGTHS:\nThe paper addresses an important issue present in BP/MS-based decoders on these codes.\n\nThe solution is simple and does not require any architectural modification.\n\nThe performance seems important and is obtained without adding any complexity compared to the original NMS decoder.\n\nWEAKNESSES:\n#### NMS\n I am not sure the comparison with NMS is fair for multiple reasons.\n\n1) The training of NMS can be performed in several ways that should not allow zero loss. For example, the observations in lines 224 and 225 can be easily solved by using large batches and/or using batches spanning many Eb/N0 values.\n\n\n2) The vanishing gradient phenomenon has been solved in [5] using a multi-loss objective, which clearly prevents the gradients to collapse at each layer.\nAlso, in Figure 4b) NMS and the method should be compared at equal number of iterations.\nNMS's test FERs with l=40 and l=50 are not that far from the  l=50 proposed method FER.\n\n3) NMS remains non-negligibly better in the waterfall region.\n\n4) Many NMS algorithms have emerged since [5] and I am wondering if the comparison with the original NMS [5](2018) is fair and conclusive in order to judge the paper.\n\n#### Clarity:\nFigure 2 is not clear. For example, the differentiation between the NMS and the neural network is not clear; the base and post NN should be linked by an arrow.\n\nThe paper contains a very large amount of text which can become cumbersome. I believe adding clear Algorithm(s) would simplify the comprehension a lot.\n\n\n#### Misc:\nTypos: \n\nline 88: we a new \\\nline 121: protogrph\n\nQUESTIONS:\n1) Please address the NMS subsection from the Weaknesses section.\\\nIt would be beneficial to have an ablation study of the final accuracy for NMS augmented with the different components of the proposed methods. Also, different training strategies of the NMS must be considered.\n\n2) Please address the clarity and Misc from the Weaknesses section.\n\nI believe adding clarity, addressing the ablations, and adding more experiments would increase the quality, impact, and rating of the paper.\n\nLIMITATIONS:\nNo limitations are discussed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author studies LDPC code's neural min-sum decoder's error floor problem, by solely change the training method:\n\n(1) boosting learning method, which use first network to do majority of decoding to achieve waterfall region, and second network deal with small error residuals. \n(2) relief deep decoding iteration's vanish gradient problem by block-wise training. \n(3) bundle weight in dynamic manner, to reduce the number of weights to be trained. \n\nWith above techniques, the experiment runs on varies LDPC settings, shows that proposed NMS has better performance on error-floor regions, with same computational resource as NMS. Since NMS and MS have similar complexity, it is believed that proposed NMS could be possibly adapted to future LDPC decoder implementation.\n\nSTRENGTHS:\n(1) The paper works directly on training of NMS algorithm, which requires minimal change on existing low complexity NMS solutions. All three training methods are reasonable and are supported by experiments. The residual learning (termed as boosting learning) empirically shows advantage on error-floor regions. The block-wise training schedule does improve performance with large number of iterations. Finally the weight sharing technique adds a structural sparse limitation to neural decoder, which only uses a small portion of weights. \n\n(2) The evaluation is extensive, on multiple LDPC settings at Figure 6, we see that proposed NMS method shows better error-floor performance, compared to existing NMS and canonical decoders.\n\nWEAKNESSES:\nHere are my concerns:\n\n(1) The major contribution on Machine Learning sense, is limited. The proposed boosting learning method is common, as residual learning in ML sense; the proposed training schedule and weight sharing are also widely-used ML methods. The core contribution is on applying ML techniques in the context of channel coding field. In some sense, the targeted audience shall be channel coding researchers, rather than general ML researchers.\n\n(2) The experiments are not conducted in long block length (length>1000, which is common for QC-LDPC), which is widely used in 5G/WIFI systems. It might be interesting to check the performance on long block length, where block length coding gain is significant, and NMS shows less advantage against canonical methods.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes training methods to optimize neural min-sum (NMS) decoders that are robust to the error-floor phenomenon of LDPC codes. The proposed methods include: (1) dividing the decoding network into two neural networks and training the post network to be specialized for uncorrected codewords that failed in the first network; (2) introducing a block-wise training schedule that locally trains a block of weights while retraining the preceding block; and (3) assigning different weights to unsatisfied check nodes. The proposed methods are applied to standard LDPC codes, and the results show that they achieve the best decoding performance in the error-floor region compared to other decoding methods in the literature. The proposed NMS decoder is designed only by modifying the training methodology, without adding any additional modules. Therefore, it can be seamlessly incorporated into the well-established architectures of LDPC decoders and can be immediately utilized in practical domains that demand exceptionally low error rates.\n\nSTRENGTHS:\nThe paper introduce novel ideas:\n1. Boosting learning using uncorrected codewords - The authors propose a boosting learning technique for LDPC code decoding. The technique divides the decoding network into two cascaded networks, where the first network focuses on the waterfall performance and the second network focuses on the error-floor region. The second network is able to correct uncorrected codewords that are not corrected by the first network due to the error-floor phenomenon. This results in a significant performance improvement in the error-floor region.\n2. Block-wise training schedule with retraining - The authors propose a new training schedule for NMS decoders that mitigates the vanishing gradient problem. The proposed schedule divides the entire decoding iterations into sub-blocks and trains the blocks in a sequential manner. Additionally, the weights trained from previous blocks are retrained to escape from local minima. This results in a significant performance improvement over the one-shot training method and the iter-by-iter schedule.\n3. Weight sharing technique with dynamic weight allocation - The authors propose a new weight sharing technique for NMS decoders that improves the performance in the error-floor region. The technique dynamically assigns different weights to unsatisfied check nodes (UCNs) and satisfied check nodes (SCNs) in the decoding process. This results in a significant performance improvement with a 2.6% reduction in the number of weights to be trained.\n\nWEAKNESSES:\n.\n\nQUESTIONS:\n1. Does the proposed method can be also beneficial to improve the results of neural decoder for other types of codes? such as BCH and Polar?\n2. How does the model behave with other type of noise (other than Gaussian)? \n3. Can you tie the weights of the different network to get improvement in terms of model size?\n4. What happen if you take more then two NN? does the performance improve?\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents novel training techniques for the NMS decoder of LDPC codes aimed at enhancing performance in the error floor region. The proposed decoding methods comprise two stages: a base decoder and a post decoder designed specifically for uncorrected codewords from the base decoder. In order to tackle the issue of vanishing gradient during training, a block-wise training schedule is introduced in this paper. By assigning distinct weights to unsatisfied check nodes, the error floor can be lowered with a small number of weights to be trained.\n\nSTRENGTHS:\n1. The impact of the proposed decoding technique is substantial, as evidenced by the figures which clearly demonstrate a significant improvement in code performance, particularly in the error floor region.\n2. The analysis and interpretation of the results in Figure 3 were particularly valuable, providing a clear understanding of the distinctions between the methods employed for selecting training samples.\n\nWEAKNESSES:\n1. The proposed decoding method in this paper builds upon the concept of two-stage decoding and incorporates a post decoder that complements the base decoder. It is important to acknowledge that this idea is not entirely new within the field of coding theory for LDPC codes, as there has been prior research exploring similar approaches. The followings are some of the examples sharing a similar philosophy and it would be valuable to mention them as relevant previous work in this direction.\nS. Yang, Y. Han, X. Wu, R. Wood and R. Galbraith, \"A soft decodable concatenated LDPC code,\" 2015 IEEE International Magnetics Conference (INTERMAG), Beijing, China, 2015\nJ. Oh, J. Ha, H. Park and J. Moon, \"RS-LDPC Concatenated Coding for the Modern Tape Storage Channel,\" in IEEE Transactions on Communications, vol. 64, no. 1, pp. 59-69, Jan. 2016\nH. Park and J. Moon, \"Improving SSD Read Latency via Coding,\" in IEEE Transactions on Computers, vol. 69, no. 12, pp. 1809-1822, 1 Dec. 2020\n\n2. The clarity of the presentation style could be enhanced. For example, the specific problem addressed by the paper, such as the type of channel considered, is not explicitly stated. The reader must infer this information primarily from the related work section. Providing a clear and explicit description of the problem, including the type of channel under consideration, would greatly improve the overall clarity of the paper.\n\n3. Nitpicks\n- It would greatly enhance the presentation if the theoretical performance limit based on the code rate could be included in the plots, such as in Figures 3 and 6.\n- Typo: On line 121, page 3, \"protogrph\" should be corrected to \"protograph\".\n\nQUESTIONS:\nIn order to harness the potential of boosting learning, this study introduces a division of the decoding network into two components, with the post decoding stage aiding the base decoding process. A natural extension of this approach is to explore the possibility of incorporating more than two decoders. It would be intriguing to learn from the authors whether they have considered this scenario and if they can provide insights into the expected outcomes. Including discussions and potential answers to this question would enhance the manuscript's appeal and stimulate further interest in the research.\n\nLIMITATIONS:\nN/A - This work does not appear to have any discernible negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-11"}