# Ternary Syndrome Decoding with Large Weight

Rémi Bricout, André Chailloux, Thomas Debris-Alazard, Matthieu Lequesne

► To cite this version:

Rémi Bricout, André Chailloux, Thomas Debris-Alazard, Matthieu Lequesne. Ternary Syndrome Decoding with Large Weight. SAC 2019 - 26th International Conference Selected Areas in Cryptography, Aug 2019, Waterloo, Canada. pp.437-466, 10.1007/978-3-030-38471-5_18 . hal-02420997

HAL Id: hal-02420997

https://inria.hal.science/hal-02420997

Submitted on 20 Dec 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Ternary Syndrome Decoding with Large Weight

Rémi Bricout¹,², André Chailloux², Thomas Debris-Alazard¹,², and Matthieu Lequesne¹,²

¹ Sorbonne Universités, UPMC Univ Paris 06

² Inria, Paris

`{remi.bricout, andre.chailloux, thomas.debris, matthieu.lequesne}@inria.fr`

**Abstract.** The Syndrome Decoding problem is at the core of many code-based cryptosystems. In this paper, we study ternary Syndrome Decoding in large weight. This problem has been introduced in the Wave signature scheme but has never been thoroughly studied. We perform an algorithmic study of this problem which results in an update of the Wave parameters. On a more fundamental level, we show that ternary Syndrome Decoding with large weight is a really harder problem than the binary Syndrome Decoding problem, which could have several applications for the design of code-based cryptosystems.

*Keywords.* Post-quantum cryptography, Syndrome Decoding problem, Subset Sum algorithms.

## 1 Introduction

Syndrome decoding is one of the oldest problems used in coding theory and cryptography [McE78]. It is known to be NP-complete [BMvT78] and its average case variant is still believed to be hard forty years after it was proposed, even against quantum computers. This makes code-based cryptography a credible candidate for post-quantum cryptography. There has been numerous proposals of post-quantum cryptosystems based on the hardness of the Syndrome Decoding (SD) problem, some of which were proposed for the NIST standardization process for quantum-resistant cryptographic schemes. Most of them are qualified for the second round of the competition [ABB+17, ACP+17, AMAB+17, BBC+19, BCL+17]. It is therefore a significant task to understand the computational hardness of the Syndrome Decoding problem.

Informally, the Syndrome Decoding problem is stated as follows. Given a matrix $\mathbf{H} \in \mathbb{F}_q^{(n-k) \times n}$, a vector $\mathbf{s} \in \mathbb{F}_q^{n-k}$ and a weight $w \in [0, n]$, the goal is to find a vector $\mathbf{e} \in \mathbb{F}_q^n$ such that $\mathbf{H}\mathbf{e}^{\top} = \mathbf{s}^{\top}$ and $|\mathbf{e}| = w$, where $|\mathbf{e}|$ denotes the Hamming weight, namely $|\mathbf{e}| = |\{i : \mathbf{e}_i \neq 0\}|$. The binary case, i.e. when $q = 2$ has been extensively studied. Even before this problem was used in cryptography, Prange [Pra62] constructed a clever algorithm for solving the binary problem using a method now referred to as Information Set Decoding (ISD).

### 1.1 Binary vs. Ternary Case

The binary case of the SD problem has been thoroughly studied. Its complexity is always studied for relative weight $W := \frac{w}{n} \in [0, 0.5]$ because the case $W > 0.5$ is equivalent (see Remark 2 in Section 2). However, this argument is no longer valid in the general case ($q \ge 3$). Indeed, the large weight case does not behave similarly to the small weight case, as we can see on Figure 1.

The general case $q \ge 3$ has received much less attention than the binary case. One possible explanation for this is that there were no cryptographic applications for the general case. This has recently changed. Indeed, a new signature scheme named Wave was recently proposed in [DST18], based on the difficulty of SD on a ternary alphabet and with large weight. This scheme makes use of the new regime of large weight induced by the asymmetry of the ternary case. Therefore, in addition to the algorithmic interest of studying the general syndrome decoding problem, the results of this study can be applied to a real cryptosystem.

Another reason why the general case $q \ge 3$ has been less studied is that the Hamming weight measure becomes less meaningful as $q$ grows larger. Indeed, the Hamming weight only counts the
<figure><img src="image_2.png" /><figcaption>Fig. 1. Asymptotic complexity of Prange's ISD algorithm for $R := \frac{k}{n} = 0.5$.</figcaption></figure>

number of non-zero elements but not their repartition. Hence, the weight loses a significant amount of information for large values of $q$. Therefore, $q = 3$ seems to be the best candidate to understand the structure of the non-binary case without losing too much information.

## 1.2 State of the Art for $q \ge 3$

Still, there exist some interesting results concerning the SD problem in the general *q*-ary case. Coffey and Goodman [CG90] were the firsts to propose a generalization of Prange's ISD algorithm to $\mathbb{F}_q$. Following this seminal work, most existing ISD algorithms were extended to cover the *q*-ary case. In 2010, Peters [Pet10] generalized Stern's algorithm. In his dissertation thesis, Meurer [Meu17] generalized the BJMM algorithm. Hirose [Hir16] proposed a generalization of Stern's algorithm with May-Ozerov's approach (using nearest neighbors) and showed that for $q \ge 3$ this does not improve the complexity compared to Stern's classical approach. Later, Gueye, Klamti and Hirose [GKH17] extended the BJMM algorithm with May-Ozerov's approach and improved the complexity of the general SD problem. A result from Canto-Torres [CT17] proves that all ISD-based algorithms converge to the same asymptotic complexity when $q \to \infty$. Finally, a recent work [IKR+18] proposed a generalization of the ball-collision decoding over $\mathbb{F}_q$.

All these papers focus solely on the SD problem for relative weight $W < 0.5$. None of them mentions the case of large weight. The claimed worst case complexities in these papers should be understood as the worst case complexity for the SD problem with relative weight $W < 0.5$, but as we can see on Figure 1 the highest complexity is actually reached for large relative weight.

## 1.3 Our Contributions

Our contribution consists in a general study of ternary syndrome decoding with large weight. We first focus on the Wave signature scheme [DST18] and present the best known algorithmic attack on this scheme. We then look more generally at the hardest instances of the ternary syndrome decoding with large weight and show that this problem seems significantly harder than the binary variant, making it a potentially very interesting problem for code-base cryptography.

*The PGE+SS framework.* A first minor contribution consists in a modular description of most ISD-based algorithms. All these algorithms contain two steps. First, performing a partial Gaussian
elimination (PGE), and then, solving a variant of the Subset Sum problem (SS). This was already implicitly used in previous papers but we want to make it explicit to simplify the analysis and hopefully make those algorithms easier to understand for non-specialists.

*Ternary SD with large weight.* We then study specifically the SD problem in the ternary case and for large weights. From our modular description, we can focus only on finding many solutions of a specific instance of the Subset Sum problem. At a high level, we combine Wagner's algorithm [Wag02] and representation techniques [BCJ11, BJMM12] to obtain our algorithm. Our first take-away is that, while representations are very useful to obtain a unique solution (as in [BCJ11]), there are some drawbacks in using them to obtain many solutions. These drawbacks are strongly mitigated in the binary case as in [BJMM12] but it becomes much harder for larger values of $q$. We manage to partially compensate this by changing the moduli size, the place and the number of representations. For instance, for the Wave [DST18] parameters, we derive an algorithm that is a Wagner tree with seven floors where the last two floors have partial representations and the others have none.

*New parameters for Wave.* We then use our algorithms to study the complexity of the Wave signature scheme, for which we significantly improve the original analysis. We show that the key sizes of the original scheme presented for 128 bits of security have to be more than doubled, going roughly from 1Mb to 2.2Mb, to achieve the claimed security. This requires to study the Decode One Out of Many (DOOM) problem, on which Wave actually relies. This problem corresponds to a multiple target SD problem. More precisely, given $N$ syndromes $(\mathbf{s}_1, \dots, \mathbf{s}_N)$ ($N$ can be large, for example $N = 2^{64}$) the goal is to find an error $\mathbf{e}$ of Hamming weight $w$ and an integer $i$ such that $\mathbf{e}\mathbf{H}^T = \mathbf{s}_i$.

*Hardest instances of the ternary SD with large weight.* Next, we look at the hardest instances of the ternary SD with large weight problem. We study the standard ISD algorithms and show that for all of them, the hardest instances occur for $R \approx 0.369$ and $W = 1$ (still in the case $q = 3$). Unsurprisingly, for equivalent code length and dimension, ternary syndrome decoding is harder than its binary counterpart. But this is due to the fact that the input matrix contains more information, since its elements are in $\mathbb{F}_3$, hence the input size is $\log_2(3)$ times larger than a binary matrix with equivalent dimensions.

A more surprising conclusion of our work is that ternary syndrome decoding is significantly harder than the binary case *for equivalent input size*, that is, when normalizing the exponent by a factor $\log_2(q)$. This new result is in sharp contrast with all the previous work on $q$-ary syndrome decoding that showed that the problem becomes simpler as $q$ increases. This is due to the fact that all the previous literature only considered the small weight case while we now take large weights into account.

Table 1 represents the minimum input size for which the underlying syndrome decoding problem offers 128 bits of security, i.e. the associated algorithm needs at least $2^{128}$ operations to solve the problem.

<table><caption>**Table 1.** Minimum input sizes (in kbits) for a time complexity of $2^{128}$.</caption><thead><tr><th>Algorithm</th><th>$q=2$</th><th>$q=3$ and $W > 0.5$</th></tr></thead><tbody><tr><td>Prange</td><td>275</td><td>44</td></tr><tr><td>Dumer/Wagner</td><td>295</td><td>83</td></tr><tr><td>BJMM/Our algorithm</td><td>374</td><td>99</td></tr></tbody></table>

We want to stress again that those input sizes in the ternary case take into account the fact that the matrix elements are in $\mathbb{F}_3$. So the increase in efficiency is quite significant and the ternary SD could efficiently replace its binary counterpart when looking for a hard code-based problem.
## 1.4 Notations

We define here some notations that will be used throughout the paper. The notation $x \triangleq y$ means that $x$ is defined to be equal to $y$. We denote by $\mathbb{F}_q = \{0, 1, \dots, q-1\}$ the finite field of size $q$. Vectors will be written with bold letters (such as $\mathbf{e}$) and uppercase bold letters will be used to denote matrices (such as $\mathbf{H}$). Vectors are in row notation. Let $\mathbf{x}$ and $\mathbf{y}$ be two vectors, we will write $(\mathbf{x}, \mathbf{y})$ to denote their concatenation. Finally, we denote by $\llbracket a, b \rrbracket$ the set $\{\tilde{a}, \tilde{a} + 1, \dots, \tilde{b}\}$ where $\tilde{a} = \lfloor a \rfloor$ and $\tilde{b} = \lfloor b \rfloor$.

# 2 A General Framework for Solving the Syndrome Decoding Problem

## 2.1 The Syndrome Decoding Problem

The goal of this paper is to study the Syndrome Decoding problem, which is at the core of most code-based cryptosystems.

**Problem 1.** [Syndrome Decoding - SD($q, R, W$)]

Instance: $\mathbf{H} \in \mathbb{F}_q^{(n-k) \times n}$ of full rank, $\mathbf{s} \in \mathbb{F}_q^{n-k}$ (usually called the syndrome).

Output: $\mathbf{e} \in \mathbb{F}_q^n$ such that $|\mathbf{e}| = w$ and $\mathbf{e}\mathbf{H}^\top = \mathbf{s}$, where $k \triangleq \lfloor Rn \rfloor$, $w \triangleq \lfloor Wn \rfloor$ and $|\mathbf{e}| = |\{i : \mathbf{e}_i \neq 0\}|$.

The problem SD($q, R, W$) is parametrized by the field size $q$, the rate $R \in [0, 1]$ and the relative weight $W \in [0, 1]$. We are always interested in the average case complexity (as a function of $n$) of this problem, where $\mathbf{H}$ is chosen uniformly at random and $\mathbf{s}$ is chosen uniformly from the set $\{\mathbf{e}\mathbf{H}^\top : |\mathbf{e}| = w\}$. This ensures the existence of a solution for each input and corresponds to the typical situation in cryptanalysis. More generally, the following proposition gives the average expected number of solutions

**Proposition 1.** Let $n, k, w$ be integers with $k \le n$ and $\mathbf{s} \in \mathbb{F}_q^{n-k}$. The expected number of solutions of $\mathbf{e}\mathbf{H}^\top = \mathbf{s}$ in $\mathbf{e}$ of weight $w$ when $\mathbf{H}$ is chosen uniformly at random in $\mathbb{F}_q^{(n-k) \times n}$ is given by:

$$ \frac{\binom{n}{w}(q-1)^w}{q^{n-k}}. $$

*Proof.* This is simple combinatorics. The numerator corresponds to the number of vectors $\mathbf{e}'$ of weight $w$. The denominator corresponds to the inverse of the probability over $\mathbf{H}$ that $\mathbf{e}\mathbf{H}^\top = \mathbf{s}^\top$ for $\mathbf{e} \neq \mathbf{0}$. $\square$

*Remark 1.* The matrix length $n$ is not considered as a parameter of the problem since we are only interested in the asymptotic complexity, that is the coefficient $F(q, R, W)$ (which does not depend on $n$) such that the complexity of the Syndrome Decoding problem for a matrix of size $n$ can be expressed as $2^{n(F(q, R, W)+o(1))}$.

**State of the Art on $\mathbb{F}_2$.** This problem was mostly studied in the case $q = 2$. Depending on the parameters $R$ and $W$, the complexity of the problem can greatly vary. Let us fix a value $R$, and let $W_{GV}$ denote the Gilbert-Varshamov bound, that is $W_{GV} \triangleq h_2^{-1}(1-R)$ where $h_2$ is the binary entropy function restricted to the input space $[0, \frac{1}{2}]$. For $W \in [0, \frac{1}{2}]$, there exist three different regimes.

1. $W \approx W_{GV}$. When $W$ is close to $W_{GV}$, there is on average a small number of solutions. This is the regime where the problem is the hardest and where it is the most studied. To the best of our knowledge, we only know two code-based cryptosystems in this regime, namely the CFS signature scheme [CFS01] and the authentication scheme of Stern [?].
2. $W \gg W_{GV}$. In this case, there are on average exponentially many solutions and this makes the problem simpler. When $W$ reaches $\frac{1-R}{2}$, the problem can be solved in average polynomial time using Prange's algorithm [Pra62]. There is a cryptographic motivation to consider $W$ much larger than $W_{GV}$, for instance to build signatures schemes following the [GPV08] paradigm as it was done in [DST17] but one has to be careful to not make SD too simple.

3. $W \ll W_{GV}$. In this regime, we have with high probability a unique solution. However, the search space, i.e. the set of vectors $\mathbf{e}$ st. $|\mathbf{e}| = [Wn]$ is much smaller than in the other regimes. The original McEliece system [McE78] or the QC-MDPC systems [MTSB12] are in this regime.

**Remark 2.** Solving SD(2, R, W) for $W \in [\frac{1}{2}, 1]$ and the instance $(\mathbf{H}, \mathbf{s})$ can be reduced to one of the above-mentioned cases using SD(2, R, $1-W$) and the instance $(\mathbf{H}, \mathbf{s} + \mathbf{1}\mathbf{H}^\top)$ where $\mathbf{1}$ denotes the vector with all its components equal to 1.

**Remark 3.** Contrary to the binary case, when $q \ge 3$ the case of large relative weight can not be reduced to that of small relative weight using the trick of Remark 2. In fact, the problem has a quite different behavior in small and large weights, see Figure 1.

## 2.2 The PGE+SS Framework in $\mathbb{F}_q$

The SD problem has been extensively studied in the binary case. Most algorithms designed to solve this problem [Dum91, MMT11, BJMM12] follow the same framework:

1. perform a partial Gaussian elimination (PGE);

2. solve the Subset Sum problem (SS) on a reduced instance.

We will see how we can extend this framework to the non-binary case. Our goal here is to describe the PGE+SS framework for solving SD($q, R, W$). Fix $\mathbf{H} \in \mathbb{F}_q^{(n-k) \times n}$ of full rank and $\mathbf{s} \in \mathbb{F}_q^{n-k}$. Recall that we want to find $\mathbf{e} \in \mathbb{F}_q^n$ such that $|\mathbf{e}| = w \triangleq [Wn]$ and $\mathbf{He}^\top = \mathbf{s}^\top$. Let us introduce $\ell$ and $p$, two parameters of the system, that we will consider fixed for now. In this framework, an algorithm for solving SD($q, R, W$) will consist of 4 steps: a permutation step, a partial Gaussian Elimination step, a Subset Sum step and a test step.

1. *Permutation step.* Pick a random permutation $\pi$. Let $\mathbf{H}_\pi$ be the matrix $\mathbf{H}$ where the columns have been permuted according to $\pi$. We now want to solve the problem SD($q, R, W$) on inputs $\mathbf{H}_\pi$ and $\mathbf{s}$.

2. *Partial Gaussian Elimination step.* If the top left square submatrix of $\mathbf{H}_\pi$ of size $n-k-\ell$ is not of full rank, go back to step 1 and choose another random permutation $\pi$. This happens with constant probability. Else, if this submatrix is of full rank, perform a Gaussian elimination on the rows of $\mathbf{H}_\pi$ using the first $n-k-\ell$ columns. Let $\mathbf{S} \in \mathbb{F}_q^{(n-k) \times (n-k)}$ be the invertible matrix corresponding to this operation. We now have two matrices $\mathbf{H}' \in \mathbb{F}_q^{(n-k-\ell) \times (k+\ell)}$ and $\mathbf{H}'' \in \mathbb{F}_q^{\ell \times (k+\ell)}$ such that:

$$ \mathbf{SH}_\pi = \begin{pmatrix} \mathbf{1}_{n-k-\ell} & \mathbf{H}' \\ \mathbf{0} & \mathbf{H}'' \end{pmatrix}. $$

The error $\mathbf{e}$ can be written as $\mathbf{e} = (\mathbf{e}', \mathbf{e}'')$ where $\mathbf{e}' \in \mathbb{F}_q^{n-k-\ell}$ and $\mathbf{e}'' \in \mathbb{F}_q^{k+\ell}$, and one can write $\mathbf{s}\mathbf{S}^\top = (\mathbf{s}', \mathbf{s}'')$ with $\mathbf{s}' \in \mathbb{F}_q^{n-k-\ell}$ and $\mathbf{s}'' \in \mathbb{F}_q^\ell$.

$$ \begin{align} \mathbf{H}_\pi \mathbf{e}^\top = \mathbf{s}^\top &\iff \mathbf{SH}_\pi \mathbf{e}^\top = \mathbf{S}\mathbf{s}^\top \nonumber \\ &\iff \begin{pmatrix} \mathbf{1}_{n-k-\ell} & \mathbf{H}' \\ \mathbf{0} & \mathbf{H}'' \end{pmatrix} \begin{pmatrix} \mathbf{e}'^\top \\ \mathbf{e}''^\top \end{pmatrix} = \begin{pmatrix} \mathbf{s}'^\top \\ \mathbf{s}''^\top \end{pmatrix} \nonumber \\ &\iff \begin{cases} \mathbf{e}'^\top + \mathbf{H}'\mathbf{e}''^\top = \mathbf{s}'^\top \\ \mathbf{H}''\mathbf{e}''^\top = \mathbf{s}''^\top \end{cases} \tag{1} \end{align} $$

To solve the problem, we will try to find a solution $(\mathbf{e}', \mathbf{e}'')$ to the above system such that $|\mathbf{e}''| = p$ and $|\mathbf{e}'| = w - p$.
3. The Subset Sum step. Compute a set $S \subseteq \mathbb{F}_q^{k+\ell}$ of solutions $\mathbf{e}''$ of $\mathbf{H}''\mathbf{e}''^\mathrm{T} = \mathbf{s}''^\mathrm{T}$ such that $|\mathbf{e}''| = p$. We will solve this problem by considering it as a Subset Sum problem as it is described in Subsection 2.4.

4. The test step. Take a vector $\mathbf{e}'' \in S$ and let $\mathbf{e}'^\mathrm{T} = \mathbf{s}'^\mathrm{T} - \mathbf{H}'\mathbf{e}''^\mathrm{T}$. Equation (1) ensures that $\mathbf{H}_\pi(\mathbf{e}', \mathbf{e}'')^\mathrm{T} = \mathbf{s}^\mathrm{T}$. If $|\mathbf{e}'| = w-p$, $\mathbf{e} = (\mathbf{e}', \mathbf{e}'')$ is a solution of SD($q, R, W$) on inputs $\mathbf{H}_\pi$ and $\mathbf{s}$, which can be turned into a solution of the initial problem by permuting the indices, as detailed in Equation (2). Else, try again for other values of $\mathbf{e}'' \in S$. If no element of $S$ gives a valid solution, go back to step 1.

At the end of protocol, we have a vector $\mathbf{e}$ such that $\mathbf{H}_\pi \mathbf{e}^\mathrm{T} = \mathbf{s}^\mathrm{T}$ and $|\mathbf{e}| = w$. Let $\mathbf{e}_{\pi-1}$ be the vector $\mathbf{e}$ where we permute all the coordinates according to $\pi^{-1}$. Hence,

$$ \mathbf{H} \mathbf{e}_{\pi-1}^\mathrm{T} = \mathbf{H}_\pi \mathbf{e}^\mathrm{T} = \mathbf{s}^\mathrm{T} \quad \text{and} \quad |\mathbf{e}_{\pi-1}| = |\mathbf{e}| = w. \qquad (2) $$

Therefore, $\mathbf{e}_{\pi-1}$ is a solution to the problem.

## 2.3 Analysis of the Algorithm

In order to analyse this algorithm, we rely on the following two propositions.

**Notation 1** An important quantity to understand the complexity of this algorithm is the probability of success at step 4. On an input $(\mathbf{H}, \mathbf{s})$ uniformly drawn at random, suppose that we have a solution to the Subset Sum problem, i.e. a vector $\mathbf{e}''$ such that $\mathbf{H}''\mathbf{e}''^\mathrm{T} = \mathbf{s}''^\mathrm{T}$ and $|\mathbf{e}''| = p$. Let $\mathbf{e}'^\mathrm{T} = \mathbf{s}'^\mathrm{T} - \mathbf{H}'\mathbf{e}''^\mathrm{T}$. We will denote:

$$ \mathcal{P}_{p,l} \triangleq \mathbb{P}(|\mathbf{e}'| = w - p \mid |\mathbf{e}''| = p). $$

**Proposition 2.** We have, up to a polynomial factor,

$$ \mathcal{P}_{p,l} = \frac{\binom{n-k-l}{w-p} (q-1)^{w-p}}{\min(q^{n-k-l}, \binom{n}{w}(q-1)^w q^{-l})}. $$

*Proof.* The proof of this statement is simple combinatorics. The numerator corresponds to the number of vectors $\mathbf{e}'$ of weight $w-p$. The denominator corresponds to the inverse of the probability that $\mathbf{e}'^\mathrm{T} = \mathbf{s}'^\mathrm{T} - \mathbf{H}'\mathbf{e}''^\mathrm{T}$. For a typical random behavior, this is equal to $q^{n-k-l}$. But here we know that there is at least one solution. Therefore, we know that the number of vectors of weight $w-p$ is bounded from above by the number of vectors $\mathbf{e}$ such that $\mathbf{H}''\mathbf{e}''^\mathrm{T} = \mathbf{s}''^\mathrm{T}$. This explains the second term of the minimum. $\square$

**Proposition 3.** Assume that we have an algorithm that finds a set $S$ of solutions of the Subset Sum problem in time $T$. The average running time of the algorithm is, up to a polynomial factor,

$$ T \cdot \max \left( 1, \frac{1}{|S| \cdot \mathcal{P}_{p,l}} \right). $$

As we can see, all the parameters are entwined. The success probability $\mathcal{P}_{p,l}$ depends of $p$ and $l$, as well as the time $T$ to find the set $S$ of solutions.

In this work, we will focus on a family of parameters useful in the analysis of the Wave signature scheme [DST18]. More precisely, we will study the following regime:

$$ q = 3 \ ; \ R \in [0.5, 0.9] \ ; \ W \in [0.9, 0.99]. $$

One consequence of working with a very high relative weight $W$ is that our best algorithms will work with:

$$ l = \Theta(n) \ ; \ p = k + l. \qquad (3) $$

Here, $l$ is $\Theta(n)$ for the following reason: if $l = o(n)$ then it is readily verified that, asymptotically in $n$, the average running time of the PGE+SS framework will be bounded from below (up to a polynomial factor) by $1/\mathcal{P}_{p,0}$. This exactly corresponds to the complexity of the simplest generic algorithm to solve SD, namely Prange's ISD algorithm [Pra62].
## 2.4 Reduction to the Subset Sum Problem

In step 3 of the PGE+SS framework, we have a matrix $\mathbf{H}'' \in \mathbb{F}_q^{\ell \times (k+\ell)}$, a vector $\mathbf{s}'' \in \mathbb{F}_q^\ell$ and we want to compute a set $S \subseteq \mathbb{F}_q^{k+\ell}$ of solutions $\mathbf{e}''$ of $\mathbf{H}''\mathbf{e}''^\mathrm{T} = \mathbf{s}''^\mathrm{T}$ such that $|\mathbf{e}''| = p$. At first sight, this looks exactly like a Syndrome Decoding problem with inputs $\mathbf{H}''$ and $\mathbf{s}''$ so we could just recursively apply the best SD algorithm on this subinstance. But the main difference is that, in this case, we want to find many solutions to the problem and not just one. One possibility to solve this problem is to reduce it to the Subset Sum problem on vectors in $\mathbb{F}_q^\ell$.

**Problem 2.** [Subset Sum problem - SS($q, n, m, L, p$)]

Instance: $n$ vectors $\mathbf{x}_i \in \mathbb{F}_q^m$ for $1 \le i \le n$, a target vector $\mathbf{s} \in \mathbb{F}_q^m$.

Output: $L$ solutions $\mathbf{b}^{(j)} = (b_1^{(j)}, \dots, b_n^{(j)}) \in \{0, 1\}^n$ for $1 \le j \le L$, such that for all $j$, $\sum_{i=1}^n b_i^{(j)} \mathbf{x}_i = \mathbf{s}$ and $|\mathbf{b}^{(j)}| = p$.

We can consider the same problem with elements $b$ in $\mathbb{F}_q$ instead of $\{0, 1\}$.

**Problem 3.** [Subset Sum with non-zero characteristic - SSNZC($q, n, m, L, p$)]

Instance: $n$ vectors $\mathbf{x}_i \in \mathbb{F}_q^m$ for $1 \le i \le n$, a target vector $\mathbf{s} \in \mathbb{F}_q^m$.

Output: $L$ solutions $\mathbf{b}^{(j)} = (b_1^{(j)}, \dots, b_n^{(j)}) \in \mathbb{F}_q^n$ for $1 \le j \le L$, such that for all $j$, $\sum_{i=1}^n b_i^{(j)} \mathbf{x}_i = \mathbf{s}$ and $|\mathbf{b}^{(j)}| = p$.

**Notation 2** We will denote SS($q, n, m, L, \emptyset$) (resp. SSNZC) the SS problem (resp. SSNZC problem) without any constraint on the weight.

Again, we will be interested in the average case, where all the inputs are taken uniformly at random. Notice that the problem that needs to be solved at step 3 of the PGE+SS framework reduces exactly to SSNZC($q, k + \ell, \ell, |S|, p$).

There is an extensive literature [HJ10, BCJ11] about the Subset Sum problem for specific parameter ranges, typically when $L = 1, q = 2, n = m$ and $p = \frac{m}{2}$. This is the hardest case where there is on average a single solution. There are several regimes of parameters, each of which lead to different algorithms. For instance, when $m = O(n^\epsilon)$ for $\epsilon < 1$, there are many solutions on average and we are in the high density setting for which we have sub-exponential algorithm [Lyu05]. Table 2 summarizes the complexity of algorithms to solve the Subset Sum problem for some different regimes of parameters when only one solution is required ($L = 1$) and for $q = 2$.

<table><caption>**Table 2.** Complexity of best known algorithms to solve SS(2, n, m, 1, ∅).</caption><thead><tr><td>Value of m</td><td>Complexity</td><td>Reference</td></tr></thead><tbody><tr><td>O(log(n))</td><td>poly(n)</td><td>[GM91, CFG89]</td></tr><tr><td>O(log(n)<sup>2</sup>)</td><td>poly(n)</td><td>[FP05]</td></tr><tr><td>O(n<sup>ε</sup>) for ε &lt; 1</td><td>2<sup>O(n<sup>ε</sup>)</sup></td><td>[Lyu05]</td></tr><tr><td>n</td><td>2<sup>O(n)</sup></td><td>[HJ10, BCJ11]</td></tr></tbody></table>

In our case, $m$ will be a small, but constant, fraction of $n$, which leads to multiple solutions but exponentially complex algorithms to find them. We will be in a moderate density situation. Furthermore, the case $L = 1$ and $L \gg 1$ require quite different algorithms. When $q = 2$, authors of [BJMM12] show how to optimize this whole approach to solve the original Syndrome Decoding problem using better algorithms for the Subset Sum problem.
## 2.5 Application to the PGE+SS Framework with High Weight

There are quite a lot of interesting regimes that could be studied with this approach and have not been studied yet. Indeed, very few papers tackle the case $q \ge 3$ and they only cover a small fraction of the possible parameters. In this work we focus on the problem SSNZC($3, k+\ell, \ell, |S|, k+\ell$) given by the PGE+SS framework for high weights in $\mathbb{F}_3$. The choice of $p = k+\ell$ for large weights is explained in Equation (3). This is quite convenient because this problem is actually equivalent to solving SS($3, k+\ell, \ell, |S|, \emptyset$) as shown by the following lemma.

**Lemma 1.** If we have an algorithm that solves SS($3, k+\ell, \ell, |S|, \emptyset$) then we have an algorithm that solves SSNZC($3, k+\ell, \ell, |S|, k+\ell$) with the same complexity.

*Proof.* Let $\mathcal{A}$ be an algorithm that solves SS($3, k+\ell, \ell, |S|, \emptyset$) and consider an instance $(\mathbf{x}_1, \dots, \mathbf{x}_{k+\ell})$, $\mathbf{s}$ of SSNZC($3, k+\ell, \ell, |S|, k+\ell$). We want to find $b_1, \dots, b_{k+\ell} \in \{1, 2\}$ (see $\mathbb{F}_3 = \{0, 1, 2\}$) such that $\sum_{i=1}^{k+\ell} b_i \mathbf{x}_i = \mathbf{s}$. Let $\mathbf{s}' = 2\mathbf{s} + \sum_i \mathbf{x}_i$ and let us run $\mathcal{A}$ on input $(\mathbf{x}_1, \dots, \mathbf{x}_{k+\ell})$, $\mathbf{s}'$. We obtain $b'_1, \dots, b'_{k+\ell} \in \{0, 1\}$ such that $\sum_{i=1}^{k+\ell} b'_i \mathbf{x}_i = \mathbf{s}'$. Take $b_i = \frac{b_i'-1}{2}$ for $1 \le i \le k+\ell$, where the division is done in $\mathbb{F}_3$ and return $(b_1, \dots, b_{k+\ell})$.

Indeed, this gives a valid solution to the problem: the elements $b_i$ belong to $\{1, 2\}$ and we have:

$$ \sum_{i=1}^{k+\ell} b_i \mathbf{x}_i = \sum_{i=1}^{k+\ell} \frac{b_i' - 1}{2} \mathbf{x}_i = \frac{\mathbf{s}'}{2} - \frac{\sum_{i=1}^{k+\ell} \mathbf{x}_i}{2} = \mathbf{s}. \quad \square $$

Hence, in the context of the PGE+SS framework for solving SD with high weights, it is enough to solve SS($3, k+\ell, \ell, |S|, \emptyset$). However, as explained at the end of Subsection 2.2, we will have to choose $\ell = \Theta(n) = \Theta(k)$ (because $k = \lceil Rn \rceil$). Therefore, we are in a regime where solving the Subset Sum problem requires exponential complexity, as explained in the previous subsection. However, as we will see in the next session, we will be able to choose $\ell$ as a small fraction of $k$. In this case, generic algorithms as Wagner's [Wag02] perform exponentially better compared to Prange's algorithm [Pra62] (case $\ell = 0$) or Subset Sum algorithms [BCJ11] (case $\ell = n-k$).

# 3 Ternary Subset Sum with the Generalized Birthday Algorithm

We show in this section how to solve SS($3, k+\ell, \ell, L, \emptyset$), first with Wagner's algorithm [Wag02]. Parameters $k$ and $\ell$ will be free. We will focus on the values $L$ for which we can find $L$ solutions to SS ($3, k+\ell, \ell, L, \emptyset$) in time $O(L)$. In such a case, we say that we can find solutions in amortized time $O(1)$.

## 3.1 A Brief Description of Wagner's Algorithm

Recall that we are here in the context of the Subset Sum step of the PGE+SS framework described in Subsection 2.2. Given $k+\ell$ vectors $\mathbf{x}_1, \dots, \mathbf{x}_{k+\ell} \in \mathbb{F}_3^\ell$ (columns of the matrix $\mathbf{H}''$) and a target vector $\mathbf{s} \in \mathbb{F}_3^\ell$, our goal is to find $L$ solutions of the form $\mathbf{b}^{(j)} = (b_1^{(j)}, \dots, b_{k+\ell}^{(j)}) \in \{0, 1\}^{k+\ell}$ such that for all $1 \le j \le L$,

$$ \sum_{i=1}^{k+\ell} b_i^{(j)} \mathbf{x}_i = \mathbf{s}. \quad (4) $$

Here, we are interested in the average case, which means that all the vectors $\mathbf{x}_i$ are independent and follow a uniform law over $\mathbb{F}_3^\ell$. In order to apply Wagner's algorithm [Wag02], let $a \in \mathbb{N}^*$ be some integer parameter. For $i \in [1, 2^a]$, denote by $\mathcal{I}_i$ the sets $\mathcal{I}_i \triangleq [1 + \frac{(i-1)(k+l)}{2^a}, \frac{i(k+l)}{2^a}]$. The sets $\mathcal{I}_i$ form a partition of $[1, k+\ell]$.

The first step of Wagner's algorithm is to compute $2^a$ lists $(\mathcal{L}_i)_{1 \le i \le 2^a}$ of size $L$ such that:

$$ \forall i \in [1, 2^a], \quad \mathcal{L}_i \subseteq \left\{ \sum_{j \in \mathcal{I}_i} b_j \mathbf{x}_j : \forall j \in \mathcal{I}_i, b_j \in \{0, 1\} \right\} \quad \text{and} \quad |\mathcal{L}_i| = L. \quad (5) $$
Each list $\mathcal{L}_i$ consists of $L$ random elements of the form $\sum_{j \in \mathcal{I}_i} b_j \mathbf{x}_j$ where the randomness is on $b_j \in \{0, 1\}$. By construction, we make sure that given $\mathbf{y} \in \mathcal{L}_i$ we have access to the coefficients $(b_j)_{j \in \mathcal{I}_i}$ such that $\mathbf{y} = \sum_{j \in \mathcal{I}_i} b_j \mathbf{x}_j$. In other words, we have divided the vectors $\mathbf{x}_1, \dots, \mathbf{x}_{k+\ell}$ in $2^a$ stacks of $(k+\ell)/2^a$ vectors and for each stack we have computed a list of $L$ random linear combinations of the vectors in the stack. The running time to build these lists is $O(L)$. Once we have computed these lists we can use the main idea of Wagner to solve (4). In our case we would like to find solutions in amortized time $O(1)$. For this, Wagner's algorithm requires the lists $\mathcal{L}_i$ to be all of the same size:

$$ \forall i \in [1, 2^a], |\mathcal{L}_i| = L = 3^{\ell/a}. $$

This gives a first constraint on the parameters $k, \ell$ and $a$, namely:

$$ 3^{\ell/a} \le 2^{(k+\ell)/2^a} \quad (\text{number of vectors } \mathbf{b}^{(j)} \text{ in each stack}). $$

which puts a constraint on $a$ since $k, \ell$ are fixed. With these lists at hand, Wagner's idea is to merge the lists in the following way. For every $p \in \{1, 3, \dots, 2^a - 3\}$, create a list $\mathcal{L}_{p,p+1}$ from $\mathcal{L}_p$ and $\mathcal{L}_{p+1}$ such that:

$$ \mathcal{L}_{p,p+1} \triangleq \{\mathbf{y}_p + \mathbf{y}_{p+1} : \mathbf{y}_i \in \mathcal{L}_i \text{ and the last } \ell/a \text{ bits of } \mathbf{y}_p + \mathbf{y}_{p+1} \text{ are 0s.}\}. $$

A list $\mathcal{L}_{2^a-1, 2^a}$ is created from $\mathcal{L}_{2^a-1}$ and $\mathcal{L}_{2^a}$ in the same way except that the last $\ell/a$ bits have to be equal to those of $\mathbf{s}$. As the elements of the lists $\mathcal{L}_p$ are drawn uniformly at random in $\mathbb{F}_3^\ell$, it is easily verified that by merging them on $\ell/a$ bits, the new lists $\mathcal{L}_{p,p+1}$ are typically of size $|\mathcal{L}_i|^2/3^{\ell/a} = (3^{\ell/a})^2/3^{\ell/a} = 3^{\ell/a}$. Therefore, the cost in time and in space of such a merging (by using classical techniques such as hash tables or sorted lists) will be $O(3^{\ell/a})$ on average. This way, we obtain $2^{a-1}$ lists of size $L$. It is readily seen that we can repeat this process $a-1$ times, with each time a cost of $O(3^{\ell/a})$ for merging on $\ell/a$ new bits. After $a$ steps, we obtain a list of solutions to the Equation (4) containing $L = 3^{\ell/a}$ elements on average.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Wagner's algorithm with $a=2$.</figcaption></figure>

Let us summarize the previous discussion with the following theorem.

**Theorem 1.** Fix $k, \ell \in \mathbb{N}^*$ and let $a$ be any non zero integer such that

$$ 3^{\ell/a} \le 2^{(k+\ell)/2^a}. $$

The associated SS(3, $k+\ell$, $\ell$, $3^{\ell/a}$, $\emptyset$) problem can be solved in average time and space $O(3^{\ell/a})$.

This theorem indicates for which value $L$ it is possible to find $L$ solutions in time $O(L)$ using Wagner's approach.
## 3.2 Smoothing of Wagner's Algorithm

Wagner's algorithm as stated above shows how to find $L$ solutions in amortized time $O(1)$ for $L = 3^{\ell/a}$. If we want more than $L$ solutions, we can repeat this algorithm and find all those solutions also in amortized time $O(1)$. So the smaller $L$ is, the better the algorithm performs. So the idea is to take the largest integer $a$ such that $3^{\ell/a} < 2^{(k+\ell)/2^a}$ and take $L = 3^{\ell/a}$, as explained in Theorem 1. But this induces a discontinuity in the optimal value of $L$ and on the complexity: when the input parameters change continuously, the optimal value of $a$ (which has to be an integer) evolves discontinuously, therefore the slope of the complexity curve is discontinuous, as we can see on Figures 8 and 9. We show here a refinement of Theorem 1 that reduces the discontinuity.

**Proposition 4.** Let $a$ be the largest integer such that $3^{\ell/(a-1)} < 2^{(k+\ell)/2^a}$. If $a \ge 3$, the above algorithm can find $2^\lambda$ solutions in time $O(2^\lambda)$ with

$$ \lambda = \frac{\ell \log(3)}{a-2} - \frac{k+\ell}{(a-2)2^{a-1}}. $$

We see that we retrieve the result of Theorem 1 when $3^{\ell/a} = 2^{(k+\ell)/2^a}$. We have not found any statement of this form in the literature, which is surprising because Wagner's algorithm has a variety of applications. We now prove the proposition.

*Proof.* Parameters $k$ and $\ell$ are fixed. Let $a$ be the largest integer such that $3^{\ell/(a-1)} < 2^{(k+\ell)/2^a-1}$ and we suppose that $a \ge 3$. We will consider Wagner's algorithm on $a$ levels but the merging at the bottom of the tree will be performed with a lighter constraint: we want the sums to agree on less than $\ell/a$ bits. Indeed, we consider the following list sizes. At the bottom of the trees, we take lists of size $2^{\frac{k+\ell}{2^a}}$ (the maximal possible size); at all other levels, we want lists of size $2^\lambda$. We run Wagner's algorithm by firstly merging on $m$ bits. In order to obtain lists of size $2^\lambda$ at the second step, we have to choose $m$ such that

$$ \frac{(2^{(k+\ell)/2^a})^2}{3^m} = 2^\lambda \quad \text{i.e.} \quad \frac{2(k+\ell)}{2^a} - m \log_2(3) = \lambda. \qquad (6) $$

The other $(a-1)$ merging steps are designed such that merging two lists of size $2^\lambda$ gives a new list of size $2^\lambda$, which means that we merge on $\lambda/\log_2(3)$ bits. However, in the final list we want to obtain solutions to the problem, which means that in total we have to put a constraint on all bits. Therefore, $\lambda$ and $m$ have to verify:

$$ m + (a-1) \frac{\lambda}{\log_2(3)} = \ell. \qquad (7) $$

By combining Equations (6) and (7) we get:

$$ \lambda = \frac{\ell \log_2(3)}{a-2} - \frac{k+\ell}{(a-2)2^{a-1}}. $$

It is easy to check that under the conditions $3^{\ell/(a-1)} < 2^{(k+\ell)/2^a-1}$ and $a \ge 3$, $\lambda$ and $m$ are positive which concludes the proof. $\square$

# 4 Ternary Subset Sum Using Representations

## 4.1 Basic Idea

In the list tree of Wagner's algorithm (see Figure 2), we split each list in two, according to what is called the *left-right* procedure. This means that if we start from a set $S = \{\sum_{j \in [A,B]} b_j x_j : |b_j| = $
$p\}$, we decompose each element of $\mathbf{y} \in S$ as $\mathbf{y} = \mathbf{y}_1 + \mathbf{y}_2$ where $\mathbf{y}_1 \in S_1$ and $\mathbf{y}_2 \in S_2$, where

$$
\begin{align*}
S_1 &\triangleq \left\{ \sum_{j \in [A, \lfloor \frac{B+A}{2} \rfloor]} b_j \mathbf{x}_j : b_j \in \{0, 1\}, \ |\mathbf{b}| = p/2 \right\} \\
S_2 &\triangleq \left\{ \sum_{j \in [\lfloor \frac{B+A}{2} \rfloor + 1, B]} b_j \mathbf{x}_j : b_j \in \{0, 1\}, \ |\mathbf{b}| = p/2 \right\}.
\end{align*}
$$

Such a decomposition does not always exist, but it exists with probability at least $\frac{1}{p}$. Indeed, the
probability that a vector of weight $p$ can be split this way is

$$
\frac{\binom{n/2}{p/2}}{\binom{n}{p}} \geq \frac{1}{p}.
$$

Wagner's algorithm uses this principle. When looking for vectors **b** containing the same number of 0's and 1's, it looks for **b** in the form **b** = **b**<sub>1</sub> + **b**<sub>2</sub>, where the second half of **b**<sub>1</sub> and the first half of **b**<sub>2</sub> are only zeros. The first half of **b**<sub>1</sub> and the second half of **b**<sub>2</sub> are expected to have the same number of 0s and 1s.

The idea of representations is to follow Wagner's approach of list merging while allowing more
possibilities to write **b** as the sum of two vectors **b** = **b**<sub>1</sub> + **b**<sub>2</sub>. We remove the constraint that
**b**<sub>1</sub> has zeros on its right half and **b**<sub>2</sub> has zeros on its left half. We replace it by a less restrictive
constraint: we fix the number of 0s, 1s and 2s (see F<sub>3</sub> = {0, 1, 2}) in **b**<sub>1</sub> and **b**<sub>2</sub>.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Same vector (1) using left-right split and (2) using representations.</figcaption></figure>

More precisely, we consider the set

$$
S' = \left\{ \sum_{j \in [A, B]} b_j \mathbf{x}_j : b_j \in \mathbb{F}_3, |\{b_j = 1\}| = p_1 \text{ and } |\{b_j = 2\}| = p_2 \right\} \quad (8)
$$

for some weights $p_1$ and $p_2$ and we want to decompose each $\mathbf{y}$ into $\mathbf{y}_1 + \mathbf{y}_2$ such that $\mathbf{y}_1, \mathbf{y}_2 \in S'$.
On the example of Figure 3, we have $p = 4$, $p_1 = 3$ and $p_2 = 1$.

At first sight, this approach may seem unusual. Indeed, except for very specific values of $p_1$ and $p_2$, the sum $\mathbf{y}_1 + \mathbf{y}_2$ will rarely match the desired weight $p$ to be in $S$. Such a sum $\mathbf{y}_1 + \mathbf{y}_2$, which matches the targeted bits for merging but not the weight constraint, will be called *badly-formed*. Those *badly-formed* sums cannot be used for the remaining of the algorithm and must be discarded.
However, the positive aspect is that each element $\mathbf{y} \in S$ accepts many decompositions (the so-called *representations*) $\mathbf{y}_1 + \mathbf{y}_2$ where $\mathbf{y}_1, \mathbf{y}_2 \in S'$. The results from [HJ10, BCJ11, BJMM12] show that this large number of ways to represent each element can compensate the fact that most decompositions do not belong to $S$. One can slightly lower the number of agreement bits when merging the lists, in order to obtain on average the desired number of elements in the merged list.

Notice that in this definition of $S'$, the elements $b_j$ belong to the set $\mathbb{F}_3$ and not $\{0, 1\}$, even though we want to obtain a binary solution. The ternary structure also increases the number of representations as shown in Figure 3. It is actually natural to consider representations of binary strings using three elements $\{0, 1, 2\}$, as in [BCJ11].

## 4.2 Partial Representations

If we relieve too many constraints and allow too many representations of a solution, it may happen that we end up with multiple copies of the same solution. In order to avoid this situation, we use *partial representations*, which is an intermediate approach between *left-right splitting* and using *representations*, as illustrated in Figure 4.

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Decomposing a vector using partial representations.</figcaption></figure>

## 4.3 Presentation of our Algorithm

Plugging representations in Wagner's algorithm can be done in a variety of ways. The way we achieved our best algorithm was mostly done by trial and error. We present here the main features of our algorithm.

- In the regime we consider, the number of floors $a$ varies from 5 to 7. Notice that this is quite larger than in other similar algorithms and is mostly due to the fact that we have many solutions to our Subset Sum problem.

- Because we want to find many solutions, representations become less efficient. Indeed, the fact that we obtain many *badly-formed* elements makes it harder to find solutions in amortized time $O(1)$ (or even just in small time).

- However, we show that representations can still be useful. For most parameters, the optimal algorithm consists of a left-right split at the bottom level of the tree, then 2 layers of partial representation and from there to the top level, left-right splits again.

Figure 5 illustrates an example for $a = 7$. When we increase the number of floors, we just add some left-right splits.

In the next section, we present the different parameters for a particular input to show how our algorithm behaves.
<figure><img src="image_6.png" /><figcaption>Fig. 5. Wagner tree for $a = 7$. Yellow list correspond to representations and blue list to left-right splits.</figcaption></figure>

## 4.4 Application to the Syndrome Decoding Problem

We embedded in the PGE+SS framework the three above-described algorithms, namely the classical Wagner algorithm, the smoothed one in §3 and the one using the representation technique in §4. By using Proposition 2, we derived the exponents given in Figures 8 and 9.

We present here the details of our algorithm for the SD(3, R, W) with $R = 0.676$ and $W = 0.948366$. These are the parameters which are used for the analysis of Wave. For this set of parameters, we claim that the complexity of our algorithm is $2^{0.0176n}$. In the PGE+SS framework (see Section 2.2), we needed to choose to parameters $p$ and $\ell$. We take $\ell = 0.060835n$ and $p = k+\ell$.

The best algorithm we found uses $a = 7$, which means that the associated Wagner tree has 7 levels, and therefore 128 leaves (Figure 5). From level 0 to level 6, the lists have size $L = 2^{0.0176}$ (i.e. equal to the overall complexity of the Subset Sum problem). As we have more than the required number of solutions for 6 levels, but not enough for 7 levels, we use the smoothing method described in section 3.2, which gives a size of the leaves equal to $2^{0.01039}$.

We present below in more detail how we construct the different lists of the Wagner tree.

- Levels 1 to 4 consist of left-right splits. For instance, at level 4, we have 16 lists

$$ \forall i \in [1, 16], \mathcal{L}_i \subseteq \left\{ \sum_{j \in \mathcal{I}_i} b_j \mathbf{x}_j : \forall j \in \mathcal{I}_i, b_j \in \{0, 1\} \right\} \text{ and } |\mathcal{L}_i| = L. $$

$$ \text{with } \mathcal{I}_i \triangleq [1 + \frac{(i-1)(k+\ell)}{16}, \frac{i(k+\ell)}{16}]. $$

- In levels 5 and 6, we use partial representations. Going from level 4 to level 5, on a proportion $\lambda_1 = 0.7252$ of the vector, we use representations for level 5 and left-right split for level 6. On the remaining fraction of the vector, we use representations on both levels. More precisely, for each interval $\mathcal{I}_i$, we split it in 2 according to Figure 6. For each part, we use Equation 8 with the following densities:

* for the part with only one level of representations, $\rho_1$ consists on 74.8% of 0s, 25.1% of 1s and 0.1% of 2s;

* for the part with two levels of representations, we have $\rho_2$, composed of 74.2% of 0s, 25.4% of 1s and 0.4% of 2s for level 5, and $\rho_3$ composed of 86.9% of 0s, 13.1% of 1s and 0.0% of 2s for level 6.

- In order to construct level 7, we start from each list of level 6 and perform again a left-right split.

The choice of the densities and all the calculi related to the representations can be quite complicated. We perform a full analysis in Appendix A, see in particular Proposition 5.

As explained in section 4.1, most of the elements we build at floors 5 and 4 are badly-formed and do not match the desired densities of 0s, 1s and 2s. We only keep the well-formed elements and lower the number of bits on which we merge, so that the merged lists have again $L$ elements. In our case, as the expected number of well-formed elements in level-4 lists is $2^{0.0116n}$, we merge on $2^{0.0055n}$ bits to compute the level-3 lists (instead of $2^{0.0176n}$ bits.) Similarly, we merge on $2^{0.0173n}$ bits.
<figure><img src="image_7.png" /><figcaption>Fig. 6. Detail of the floors where we use partial representations.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7. Detail of the bottom floors. Red elements are badly-formed elements.</figcaption></figure>

bits to compute the level-4 lists because level-5 lists have $2^{0.0174n}$ well-formed elements. This is represented in Figure 7.

Finally, level 7 is a left-right split with smaller lists (because of the smoothing). The leaves have size $2^{0.01039n}$, so we merge on $2^{0.0032n}$ bits to build the level-6 lists.

The numbers of well-formed elements per list are thus (from level 0 to level 7):

$$2^{0.0176n}, 2^{0.0176n}, 2^{0.0176n}, 2^{0.0176n}, 2^{0.0116n}, 2^{0.0174n}, 2^{0.0176n}, 2^{0.01039n},$$

and the numbers of bits we merge on:

$$2^{0.0176n}, 2^{0.0176n}, 2^{0.0176n}, 2^{0.0055n}, 2^{0.0173n}, 2^{0.0176n}, 2^{0.0032n}.$$

One can check that we merge on a total of $2^{0.0964n}$ bits, which is exactly equal to $2^{\ell \log_2(3)}$, meaning that the level-0 list is entirely composed of solutions of the Subset Sum problem.

One can also check that the Subset Sum problem has $2^{\ell+k-\ell \log_2(3)} = 2^{0.6404n}$ solutions, that one solution has $2^{0.4915n}$ representations (see appendix A), and that the merging constraints waste $2^{1.1143n}$ solution representations. We are thus left with $2^{0.0176n}$ solutions, which are exactly the solutions we get on the level-0 list.
## 4.5 Summary of our Results

We present here 2 plots that illustrate the performance of our different algorithms. What we show is that, in this parameter range, the gain obtained by using representations is relatively small. This is quite surprising because, in the binary case, representations are very efficient. One explanation we have is that, in a regime where there are naturally many solutions, Wagner's algorithm is very efficient while the representation technique has difficulties in finding solutions in amortized time $O(1)$. In Section 6, we study the hardest instances, and show that representations turn out to be more efficient.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Comparison of the exponent complexities for $R = 0.5$</figcaption></figure>

# 5 New Parameters for the WAVE Signature Scheme

Wave is a new code-based signature scheme proposed in [DST18]. It uses a hash-and-sign approach and follows the GPV paradigm [GPV08] with the instantiation of a code-based preimage sampleable family of functions.

Forging a signature in the Wave scheme amounts to solving the SD problem. Roughly speaking, the public key is a specific pseudo-random parity-check matrix **H** of size $(n-k) \times n$ and the signature of a message **m** is an error **e** of weight *w* such that $\mathbf{eH}^\mathrm{T} = h(\mathbf{m})$ with *h* a hash function. However, instead of trying to forge a signature for one message of our choice, a natural idea is to try to forge one message among a selected set of messages. This context leads directly to a slight variation of the classical SD problem. Instead of having one syndrome, there is a list of possible syndromes and the goal is to decode one of them. This problem is known as the *Decoding One Out of Many* (DOOM) problem.

*Problem 4.* [Decoding One Out of Many - DOOM(n, z, q, R, W)]

Instance: $\mathbf{H} \in \mathbb{F}_q^{(n-k) \times n}$ of full rank,
$\mathbf{s}_1, \dots, \mathbf{s}_z \in \mathbb{F}_q^{n-k}$.

Output: $\mathbf{e} \in \mathbb{F}_q^n$ and $i \in [1, z]$ such that $|\mathbf{e}| = w$ and $\mathbf{e}\mathbf{H}^\mathrm{T} = \mathbf{s}_i$, where $k \triangleq [Rn]$, $w \triangleq [Wn]$ and $|\mathbf{e}| = |\{i: \mathbf{e}_i \neq \mathbf{0}\}|$.
<figure><img src="image_10.png" /><figcaption>Fig. 9. Comparison of the exponent complexities for $R = 0.676$</figcaption></figure>

This problem was first considered in [JJ02] and later analyzed for the binary case ($q = 2$) in [Sen11, DST17]. These papers show that one can solve the DOOM problem with an exponential speed-up compared to the SD problem with equivalent parameters.

The difference induced by DOOM on the PGE+SS framework is that it increases the search space. Namely, instead of searching a solution $\mathbf{e}$ of weight $w$ in the space $\{\mathbf{e} : \mathbf{e}\mathbf{H}^\top = \mathbf{s}\}$ we search in $\cup_{i=1}^z \{\mathbf{e} : \mathbf{e}\mathbf{H}^\top = \mathbf{s}_i\}$.

The idea to solve this problem with Wagner's approach is to take $z \ge 3^{\ell/a}$ and replace the bottom-right list of the tree $\mathcal{L}_{2^a}$ by a list containing all the syndromes. Hence, there are only $2^a - 1$ lists to generate from the search space. Therefore, the constraint of Theorem 1 becomes

$$3^{\ell/a} \le 2^{(k+\ell)/(2^a-1)}.$$

For the practical parameters, we have $a = 6$ or $a = 7$ so the change from $2^a$ to $2^a - 1$ has a negligible impact when we adapt the representation technique to the DOOM problem.

The DOOM parameters stated in [DST18] are derived from the complexity of a key attack detailed in the Wave paper. Our result stated in Section 4.4 provides another attack to consider. We computed the minimal parameters for the Wave scheme so that both attacks would have a time complexity of at least $2^{128}$. They are stated in Table 3 where $n$ is the length of code used in Wave, $k$ its dimension and $w$ the weight of the signature. These should be considered as the new parameters to use for the Wave scheme.

<table><caption>Table 3. New parameters of the Wave signature scheme for 128 bits of security.</caption><thead><tr><td>$(n, k, w)$</td><td>Public key size (in MB)</td><td>Signature length (in kB)</td></tr></thead><tbody><tr><td>(7236,4892,6862)</td><td>2.27</td><td>1.434</td></tr></tbody></table>
## 6 Hardest Instances of Ternary Syndrome Decoding

In the previous sections, we tried to optimize our algorithms for the regime of parameters used in the Wave signature scheme. The corresponding Syndrome Decoding problem uses $R = 0.676$ and $w \approx 0.948$. This corresponds to a regime where there are many solutions to the problem and hence Wagner's algorithm with a large number of floors was efficient. However, this is not the setting where the problem is the hardest.

We will now look at the hardest instances of the ternary Syndrome Decoding in large weight. As we already pointed out in the introduction, ternary SD is much harder in large weights than in small weights. In the two examples we considered, namely $R = 0.5$ and $R = 0.676$, the problem was the hardest for $W = 1$. As we will see, there are some lower rates for which the complexity of the Syndrome Decoding problem is maximal for $W < 1$.

Consider an instance $(\mathbf{H}, \mathbf{s})$ of SD(3, $R$, $W$) with $W \ge \frac{2}{3}$. We have $\mathbf{H} \in \mathbb{F}_3^{(n-k) \times n}$ of full rank and $\mathbf{s} \in \mathbb{F}_3^{n-k}$. As in the binary case, the problem is the hardest when it has a unique solution on average, if such a regime exists.

Let $R_{\max} \triangleq \frac{\log_2(3)-1}{\log_2(3)} \approx 0.36907$. For $R \in [0, R_{\max}]$, we define $W_{GV}^{\text{high}}(R)$ as the only value in $[2/3, 1]$ such that

$$W_{GV}^{\text{high}}(R) + h_2(W_{GV}^{\text{high}}(R)) = (1 - R) \log_2(3),$$

$$\text{where } h_2(x) \triangleq -x \log_2(x) - (1-x) \log_2(1-x).$$

The rate $R_{\max}$ was defined such that $W_{GV}^{\text{high}}(R_{\max}) = 1$, while this quantity is not defined for $R > R_{\max}$. This is why Figures 8 and 9 do not show a high peak for $R = 0.5$ and $R = 0.676$ but an increasing function up to $W = 1$. By Proposition 1, quantity $W_{GV}^{\text{high}}(R)$ corresponds to the relative weight for which we expect one solution to SD with rate $R$ and $q = 3$.

In order to study the above problem for hard high weight instances, we compared the performance of 3 standard algorithms: Prange's algorithm, Dumer's algorithm and the BJMM algorithm.

We performed a case study and showed that, for all the above-mentioned algorithms, the hardest case is reached for $R = R_{\max} \approx 0.36907$ and $W = 1$. We obtain the following results.

<table><caption>**Table 4.** Best exponents with associated rates.</caption><thead><tr><th>Algorithm</th><th>$q=2$</th><th>$q=3$ and $W > 0.5$</th></tr></thead><tbody><tr><td>Prange</td><td>0.121 ($R = 0.454$)</td><td>0.369 ($R = 0.369$)</td></tr><tr><td>Dumer/Wagner</td><td>0.116 ($R = 0.447$)</td><td>0.269 ($R = 0.369$)</td></tr><tr><td>BJMM/our algorithm</td><td>0.102 ($R = 0.427$)</td><td>0.247 ($R = 0.369$)</td></tr></tbody></table>

In both the binary and the ternary case, we can see that Prange's algorithm performs very poorly, but that Dumer's algorithm already gives much better results and that BJMM's Subset Sum techniques, using representations, increases the gain. The analysis of Prange and Dumer for $q = 3$ is quite straightforward and follows closely the binary case. For BJMM (i.e. Wagner's algorithm with representations), the exponent 0.247 comes from a 2-levels Wagner tree that includes 1 layer of representations. We tried using a larger Wagner trees but this did not give any improvement.

The ternary SD appears significantly harder than its binary counterpart. This was expected to some extent because in the ternary case, the input matrices have elements in $\mathbb{F}_3$ and not $\mathbb{F}_2$, which means that matrices of the same dimension contain more information.

In order to confirm this idea, we define the following metric: what is the smallest input size for which the algorithms need at least $2^{128}$ operations to decode? We use the value 128, as 128 security bits is a cryptographic standard. The input matrix $\mathbf{H} \in \mathbb{F}_3^{(1-R) \times n}$ is represented in systematic form. This means that we write

$$\mathbf{H} = (\mathbf{1}_{n(1-R)} \mathbf{H}') .$$
The only relevant part that needs to be specified is $\mathbf{H}'$. This requires $R(1-R)n^2 \log_2(q)$ bits. We show that, even in this metric, the ternary syndrome decoding problem is much harder, i.e. requires $2^{128}$ operations to decode inputs of much smaller sizes. Our results are summarized in the table below.

<table><caption>**Table 5.** Minimum input sizes (in Kbits) for a time complexity of $2^{128}$.</caption><thead><tr><th>Algorithm</th><th>$q=2$</th><th>$q=3$ and $W > 0.5$</th></tr></thead><tbody><tr><td>Prange</td><td>275 ($R=0.384$)</td><td>44 ($R=0.369$)</td></tr><tr><td>Dumer/Wagner</td><td>295 ($R=0.369$)</td><td>83 ($R=0.369$)</td></tr><tr><td>BJMM/our algorithm</td><td>374 ($R=0.326$)</td><td>99 ($R=0.369$)</td></tr></tbody></table>

Notice that in this metric, in the binary case, it is worth reducing the rate $R$, as this reduces the input size. But in the ternary case, we do not observe this behavior, which shows that the problem quickly becomes simpler, as $R$ decreases.

The work we present here is very preliminary but opens many new perspectives. It seems there are many cases in code-based cryptography, from encryption schemes to signatures, where this problem could replace the binary Syndrome Decoding problem to get smaller key sizes.

# 7 Conclusion

In this work, we stressed a strong difference between the cases $q=2$ and $q \ge 3$ of the Syndrome Decoding problem. Namely, the symmetry between the small weight and the large weight cases, which occurs in the binary case, is broken for larger values of $q$. The large weight case of the general Syndrome Decoding problem had never been studied before. We proposed two algorithms to solve the Syndrome Decoding problem in this new regime in the context of the *Partial Gaussian Elimination and Subset Sum* framework. Our first algorithm uses a $q$-ary version of Wagner's approach to solve the underlying Subset Sum problem. We proposed a second algorithm making use of representations as in the BJMM approach. We studied both algorithms and proposed a first application for cryptographic purposes, namely for the Wave signature scheme. Considering our complexity analysis, we proposed new parameters for this scheme. Furthermore, we showed that the worst case complexity of Syndrome Decoding in large weight is higher than in small weight. This implies that it should be possible to develop new code-based cryptographic schemes using this regime of parameters that reach the same security level with smaller key sizes.
References

[ABB<sup>+</sup>17] Nicolas Aragon, Paulo Barreto, Slim Bettaieb, Loïc Bidoux, Olivier Blazy, Jean-Christophe Deneuville, Phillipe Gaborit, Shay Gueron, Tim Güneysu, Carlos Aguilar Melchor, Rafael Misoczki, Edoardo Persichetti, Nicolas Sendrier, Jean-Pierre Tillich, and Gilles Zémor. BIKE, December 2017. NIST Round 1 submission for Post-Quantum Cryptography.

[ACP<sup>+</sup>17] Martin Albrecht, Carlos Cid, Kenneth G. Paterson, Cen Jung Tjhai, and Martin Tomlinson. NTS-KEM. first round submission to the NIST post-quantum cryptography call, December 2017.

[AMAB<sup>+</sup>17] Carlos Aguilar Melchor, Nicolas Aragon, Slim Bettaieb, Loïc Bidoux, Olivier Blazy, Jean-Christophe Deneuville, Philippe Gaborit, Edoardo Persichetti, and Gilles Zémor. HQC, December 2017. NIST Round 1 submission for Post-Quantum Cryptography.

[BBC<sup>+</sup>19] Marco Baldi, Alessandro Barenghi, Franco Chiaraluce, Gerardo Pelosi, and Paolo Santini. LEDAcrypt. second round submission to the NIST post-quantum cryptography call, January 2019.

[BCJ11] Anja Becker, Jean-Sébastien Coron, and Antoine Joux. Improved generic algorithms for hard knapsacks. In *Advances in Cryptology - EUROCRYPT 2011 - 30th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tallinn, Estonia, May 15-19, 2011. Proceedings*, pages 364–385, 2011.

[BCL<sup>+</sup>17] Daniel J. Bernstein, Tung Chou, Tanja Lange, Ingo von Maurich, Ruben Niederhagen, Edoardo Persichetti, Christiane Peters, Peter Schwabe, Nicolas Sendrier, Jakub Szefer, and Wang Wen. Classic McEliece: conservative code-based cryptography. https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/round-1/submissions/Classic_McEliece.zip, November 2017. First round submission to the NIST post-quantum cryptography call.

[BJMM12] Anja Becker, Antoine Joux, Alexander May, and Alexander Meurer. Decoding random binary linear codes in $2^{n/20}$: How $1+1=0$ improves information set decoding. In *Advances in Cryptology - EUROCRYPT 2012*, LNCS. Springer, 2012.

[BMvT78] Elwyn Berlekamp, Robert McEliece, and Henk van Tilborg. On the inherent intractability of certain coding problems. *IEEE Trans. Inform. Theory*, 24(3):384–386, May 1978.

[CFG89] Mark Chaimovich, Gregory Freiman, and Zvi Galil. Solving dense subset-sum problems by using analytical number theory. *J. Complexity*, 5(3):271–282, 1989.

[CFS01] Nicolas Courtois, Matthieu Finiasz, and Nicolas Sendrier. How to achieve a McEliece-based digital signature scheme. In *Advances in Cryptology - ASIACRYPT 2001*, volume 2248 of LNCS, pages 157–174, Gold Coast, Australia, 2001. Springer.

[CG90] John T Coffey and Rodney M Goodman. The complexity of information set decoding. *IEEE Transactions on Information Theory*, 36(5):1031–1037, 1990.

[CT17] Rodolfo Canto Torres. Asymptotic analysis of ISD algorithms for the *q*-ary case. In *Proceedings of the Tenth International Workshop on Coding and Cryptography WCC 2017*, September 2017.

[DST17] Thomas Debris-Alazard, Nicolas Sendrier, and Jean-Pierre Tillich. Surf: a new code-based signature scheme. preprint, September 2017. arXiv:1706.08065v3.

[DST18] Thomas Debris-Alazard, Nicolas Sendrier, and Jean-Pierre Tillich. Wave: A new code-based signature scheme. Cryptology ePrint Archive, Report 2018/996, October 2018. https://eprint.iacr.org/2018/996.

[Dum91] Ilya Dumer. On minimum distance decoding of linear codes. In *Proc. 5th Joint Soviet-Swedish Int. Workshop Inform. Theory*, pages 50–52, Moscow, 1991.

[FP05] Abraham Flaxman and Bartosz Przydatek. Solving medium-density subset sum problems in expected polynomial time. In *STACS 2005, 22nd Annual Symposium on Theoretical Aspects of Computer Science, Stuttgart, Germany, February 24-26, 2005, Proceedings*, pages 305–314, 2005.

[GKH17] Cheikh Thiécoumba Gueye, Jean Belo Klamti, and Shoichi Hirose. Generalization of BJMM-ISD using may-ozerov nearest neighbor algorithm over an arbitrary finite field $\mathbb{F}_{q}$. In *Codes, Cryptology and Information Security - Second International Conference, C2SI 2017, Rabat, Morocco, April 10-12, 2017, Proceedings - In Honor of Claude Carlet*, pages 96–109, 2017.

[GM91] Zvi Galil and Oded Margalit. An almost linear-time algorithm for the dense subset-sum problem. *SIAM J. Comput.*, 20(6):1157–1189, 1991.
[GPV08] Craig Gentry, Chris Peikert, and Vinod Vaikuntanathan. Trapdoors for hard lattices and new cryptographic constructions. In *Proceedings of the fortieth annual ACM symposium on Theory of computing*, pages 197–206. ACM, 2008.

[Hir16] Shoichi Hirose. May-ozerov algorithm for nearest-neighbor problem over $\mathcal{U}_q$ and its application to information set decoding. In *Innovative Security Solutions for Information Technology and Communications - 9th International Conference, SECITC 2016, Bucharest, Romania, June 9-10, 2016, Revised Selected Papers*, pages 115–126, 2016.

[HJ10] Nicholas Howgrave-Graham and Antoine Joux. New generic algorithms for hard knapsacks. In Henri Gilbert, editor, *Advances in Cryptology - EUROCRYP 2010*, volume 6110 of LNCS. Springer, 2010.

[IKR$^+$18] Carmelo Interlando, Karan Khathuria, Nicole Rohrer, Joachim Rosenthal, and Violetta Weger. Generalization of the ball-collision algorithm. *arXiv preprint arXiv:1812.10955*, 2018.

[JJ02] Thomas Johansson and Fredrik Jönsson. On the complexity of some cryptographic problems based on the general decoding problem. *IEEE Trans. Inform. Theory*, 48(10):2669–2678, October 2002.

[Lyu05] Vadim Lyubashevsky. On random high density subset sums. *Electronic Colloquium on Computational Complexity (ECCC)*, 1(007), 2005.

[McE78] Robert J. McEliece. *A Public-Key System Based on Algebraic Coding Theory*, pages 114–116. Jet Propulsion Lab, 1978. DSN Progress Report 44.

[Meu17] Alexander Meurer. *A Coding-Theoretic Approach to Cryptanalysis*. PhD thesis, Ruhr University Bochum, November 2017.

[MMT11] Alexander May, Alexander Meurer, and Enrico Thomae. Decoding random linear codes in $O(2^{(0.054n)})$. In Dong Hoon Lee and Xiaoyun Wang, editors, *Advances in Cryptology - ASIACRYPT 2011*, volume 7073 of LNCS, pages 107–124. Springer, 2011.

[MTSB12] Rafael Misoczki, Jean-Pierre Tillich, Nicolas Sendrier, and Paulo S. L. M. Barreto. MDPC-McEliece: New McEliece variants from moderate density parity-check codes. *IACR Cryptology ePrint Archive, Report2012/409*, 2012, 2012.

[Pet10] Christiane Peters. Information-set decoding for linear codes over $F_q$. In *Post-Quantum Cryptography 2010*, volume 6061 of LNCS, pages 81–94. Springer, 2010.

[Pra62] Eugene Prange. The use of information sets in decoding cyclic codes. *IRE Transactions on Information Theory*, 8(5):5–9, 1962.

[Sen11] Nicolas Sendrier. Decoding one out of many. In *Post-Quantum Cryptography 2011*, volume 7071 of LNCS, pages 51–67, 2011.

[Wag02] David Wagner. A generalized birthday problem. In Moti Yung, editor, *Advances in Cryptology - CRYPTO 2002*, volume 2442 of LNCS, pages 288–303. Springer, 2002.
# A Appendix: Ternary Representations

In this section, we explain how to compute the number of representations as well as the number of badly-formed vectors when using ternary representations.

## A.1 Notations

The notation $\binom{n}{k_1, \dots, k_i}$ will denote the multinomial coefficient $\frac{n!}{k_1! \dots k_i!}$, assuming that $n = k_1 + \dots + k_i$.

Let us denote $g: (n, k_1, k_2) \to n \log_2(n) - k_1 \log_2(k_1) - k_2 \log_2(k_2) - (n - k_1 - k_2) \log_2(n - k_1 - k_2)$. We have:

$$ \binom{n}{k_1, k_2, n - k_1 - k_2} = \tilde{O}\left(2^{g(n, k_1, k_2)}\right). $$

The function $g$ satisfies:

$$ - g(n, k_1, k_2) = g(n, k_2, k_1), $$

$$ - g(n, k_1, k_2) = g(n, k_1, n - k_1 - k_2), $$

$$ - g(\lambda n, \lambda k_1, \lambda k_2) = \lambda g(n, k_1, k_2), $$

$$ - g(n, k_1, k_2) = nh_2\left(\frac{k_1+k_2}{n}\right) + (k_1+k_2)h_2\left(\frac{k_1}{k_1+k_2}\right), \text{ where } h_2 \text{ stands for the binary entropy.} $$

We denote by $T(n, \alpha, \beta)$ the set of all vectors of length $n$ composed with $\alpha n$ 1s, $\beta n$ 2s and $(1 - \alpha - \beta)n$ 0s. There exist $\binom{n}{\alpha n, \beta n, (1 - \alpha - \beta)n} = \tilde{O}(2^{ng(1, \alpha, \beta)})$ such vectors.

## A.2 Main Result

The goal of this section is to prove the following result.

**Proposition 5.** For $b \in T(n, \alpha_0, \beta_0)$, the number of ways one can decompose $b$ as the sum of two vectors from $T(n, \alpha_1, \beta_1)$ is given by:

$$ \tilde{O}\left(2^{n(g(1-\alpha_0-\beta_0, \bar{x}_{12}, \bar{x}_{12})+g(\alpha_0, \bar{x}_{01}, \bar{x}_{01})+g(\beta_0, \bar{x}_{02}, \bar{x}_{02}))}\right), $$

where

$$ \begin{aligned} \bar{x}_{01} &= \frac{2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1}{3} + z \\ \bar{x}_{02} &= \frac{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1}{3} + z \\ \bar{x}_{12} &= z \end{aligned} $$

and $z$ is the real root of

$$ \frac{(2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1 + 3z)(\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1 + 3z)z}{(1 - \alpha_0 - \beta_0 - 2z)(-2\alpha_0 - \beta_0 + 4\alpha_1 + 2\beta_1 - 6z)(-\alpha_0 - 2\beta_0 + 2\alpha_1 + 4\beta_1 - 6z)} = 1. $$

## A.3 A Simple Example

Let us consider a very simple case: we want to decompose a balanced vector of size $n$ (the number of 0s, 1s and 2s is $n/3$) in two balanced vectors (i.e. $\alpha_0 = \beta_1 = \alpha_1 = \beta_1 = 1/3$). There are several ways to achieve this. One solution is that each 0 is obtained by $0+0$, each 1 by $2+2$, and each 2 by $1+1$. There is exactly only one way to build the vector in this way. Another possibility is that each case $(0+0, 1+2, 2+1, 1+0, 2+2, 0+1, 2+0, 0+2$ and $1+1$) happens $n/9$ times. This is the scenario admitting the maximal number of decompositions: $\tilde{O}(3^n)$. There are many more possibilities.
The number of representations is the sum of all decompositions for all the possible scenarios. There are only a polynomial number of different scenarios. The total number of representations (which is what we want to determine) is determined, up to a polynomial factor, by the scenario which gives the maximal number of decompositions. In this case, there are $\tilde{O}(3^n)$ representations.

Let us check that this is the result given by Proposition 5. Indeed, in this case, $z$ must satisfy the equation

$$ \frac{(3z)(3z)z}{(1/3 - 2z)(1 - 6z)(1 - 6z)} = 1, \text{ or equivalently } 27z^3 = (1 - 6z)^3. $$

The real root of this equation is $1/9$. Thus we obtain $\bar{x}_{01} = \bar{x}_{02} = \bar{x}_{12} = 1/9$. Finally, the number of representations is

$$ \tilde{O}\left(2^{3ng(1/3,1/9,1/9)}\right) = \tilde{O}\left(2^{3n \times (\log_2(3)/3)}\right) = \tilde{O}(3^n). $$

### A.4 Typical Case

In general, we have a vector $b \in T(n, \alpha_0, \beta_0)$. We want to decompose it into two vectors of $T(n, \alpha_1, \beta_1)$. Let us call $x_{00}, \dots, x_{22}$ the density of the nine cases $(0+0, 0+1, \dots, 2+2)$ as shown in the following table :

<table><thead><tr><th></th><th>0</th><th>1</th><th>2</th></tr></thead><tbody><tr><th>0</th><td>$x_{00}$</td><td>$x_{01}$</td><td>$x_{02}$</td></tr><tr><th>1</th><td>$x_{10}$</td><td>$x_{11}$</td><td>$x_{12}$</td></tr><tr><th>2</th><td>$x_{20}$</td><td>$x_{21}$</td><td>$x_{22}$</td></tr></tbody></table>

We denote by $\mathcal{A}$ the set of possible such tuples $x_{00}, \dots, x_{22}$.

Given the target vector $b$, there are $\binom{n(1 - \alpha_0 - \beta_0)}{nx_{00}, nx_{12}, nx_{21}} \binom{n\alpha_0}{nx_{01}, nx_{10}, nx_{22}} \binom{n\beta_0}{nx_{02}, nx_{11}, nx_{20}}$ ways of decomposing this $b$ according to $(x_{00}, \dots, x_{22})$. Indeed, a 0 in $b$ can be decomposed as $0+0$ (this happens $nx_{00}$ times), $1+2$ ($nx_{12}$ times) or $2+1$ ($nx_{21}$ times). As the number of 0s in $b$ is $n(1 - \alpha_0 - \beta_0)$, there are $\binom{n(1 - \alpha_0 - \beta_0)}{nx_{00}, nx_{12}, nx_{21}}$ ways to choose the decomposition of each 0 of $b$. The choices of the decompositions of the 1s and the 2s give the other two factors.

For given $\alpha_0, \beta_0, \alpha_1$ and $\beta_1$, the number of possible decompositions is

$$ \sum_{(x_{00}, \dots, x_{22}) \in \mathcal{A}} \binom{n(1 - \alpha_0 - \beta_0)}{nx_{00}, nx_{12}, nx_{21}} \binom{n\alpha_0}{nx_{01}, nx_{10}, nx_{22}} \binom{n\beta_0}{nx_{02}, nx_{11}, nx_{20}}. $$

Up to a polynomial factor this is equal to

$$ \sum_{(x_{00}, \dots, x_{22}) \in \mathcal{A}} 2^{n(g(1-\alpha_0-\beta_0, x_{21}, x_{12}) + g(\alpha_0, x_{01}, x_{10}) + g(\beta_0, x_{02}, x_{20})).} $$

The largest term (or eventually one of the largest terms) of this sum is

$$ 2^{n(g(1-\alpha_0-\beta_0, \bar{x}_{21}, \bar{x}_{12}) + g(\alpha_0, \bar{x}_{01}, \bar{x}_{10}) + g(\beta_0, \bar{x}_{02}, \bar{x}_{20})),} $$

where $(\bar{x}_{00}, \dots, \bar{x}_{22})$ is called the typical case.

We are interested in this typical case because it gathers a polynomial fraction of all the possible decompositions. The asymptotic exponent of the total number of representations is then simply given by the exponent of the typical case.
### A.5 Computation of the Typical Case

Given $\alpha_0, \beta_0, \alpha_1$ and $\beta_1$, the following constraints exist on $x_{00}, \dots, x_{22}$.

$$ 
\begin{aligned}
x_{00} + x_{01} + x_{02} &= 1 - \alpha_1 - \beta_1 \\
x_{10} + x_{11} + x_{12} &= \alpha_1 \\
x_{20} + x_{21} + x_{22} &= \beta_1
\end{aligned}
 $$

$$ 
\begin{aligned}
x_{00} + x_{10} + x_{20} &= 1 - \alpha_1 - \beta_1 \\
x_{01} + x_{11} + x_{21} &= \alpha_1 \\
x_{02} + x_{12} + x_{22} &= \beta_1
\end{aligned}
 $$

$$ 
\begin{aligned}
x_{00} + x_{12} + x_{21} &= 1 - \alpha_0 - \beta_0 \\
x_{01} + x_{10} + x_{22} &= \alpha_0 \\
x_{02} + x_{11} + x_{20} &= \beta_0
\end{aligned}
 $$

However, these equations are not independent. Each of the three sets of three equations implies $x_{00} + \dots + x_{22} = 1$. We are actually left with two degrees of freedom, and any solution can be written as

$$ 
\begin{aligned}
x_{00} &= 1 - \alpha_0 - \beta_0 - 2z \\
x_{01} &= \frac{2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1}{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1} + z + w \\
x_{02} &= \frac{3}{2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1} + z - w \\
x_{10} &= \frac{3}{2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1} + z - w \\
x_{11} &= \frac{-2\alpha_0 - \beta_0 + 4\alpha_1 + 2\beta_1}{3} - 2z \\
x_{12} &= \frac{0}{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1} + z + w \\
x_{20} &= \frac{3}{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1} + z + w \\
x_{21} &= \frac{0}{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1} + z - w \\
x_{22} &= \frac{-\alpha_0 - 2\beta_0 + 2\alpha_1 + 4\beta_1}{3} - 2z.
\end{aligned}
 $$

Thus, $\mathcal{A} = \{(x_{00}(w, z), \dots, x_{22}(w, z)) \mid \forall (i, j), x_{ij} \ge 0\}$.

**Determining w.** In a first step, we will show that the typical case must be symmetric (i.e. $\bar{x}_{01} = \bar{x}_{10}$, $\bar{x}_{02} = \bar{x}_{20}$ and $\bar{x}_{12} = \bar{x}_{21}$), which means that $w$ must be 0. To do so, we consider a pair $(w, z)$ such that the corresponding $(x_{00}, \dots, x_{22})$ is in $\mathcal{A}$, and we call $(\tilde{x}_{00}, \dots, \tilde{x}_{22})$ the solution with the same $z$ but 0 instead of $w$.

As $(x_{00}, \dots, x_{22})$ is in $\mathcal{A}$, all $x_{ij}$ are positive or zero. This implies that all $\tilde{x}_{ij}$ are positive or zero. For example, for $\tilde{x}_{01}$ we have:

$$ 0 \le \min(x_{01}, x_{10}) = \tilde{x}_{01} - \text{abs}(w) \le \tilde{x}_{01}. $$

Therefore, $(\tilde{x}_{00}, \dots, \tilde{x}_{22})$ is in $\mathcal{A}$ and we obtain

$$ \frac{\text{Nrep}(\tilde{\mathbf{x}})}{\text{Nrep}(\mathbf{x})} = \frac{2^{n(g(1-\alpha_0-\beta_0, \tilde{x}_{21}, \tilde{x}_{12}) + g(\alpha_0, \tilde{x}_{01}, \tilde{x}_{10}) + g(\beta_0, \tilde{x}_{02}, \tilde{x}_{20}))}}{2^{n(g(1-\alpha_0-\beta_0, x_{21}, x_{12}) + g(\alpha_0, x_{01}, x_{10}) + g(\beta_0, x_{02}, x_{20}))}}. \quad (9) $$

But we have the following equality.

$$ 
\begin{aligned}
g(1 - \alpha_0 - \beta_0, x_{21}, x_{12}) &= g(1 - \alpha_0 - \beta_0, \tilde{x}_{12} + w, \tilde{x}_{12} - w) \\
&= g(1 - \alpha_0 - \beta_0, \tilde{x}_{12}, \tilde{x}_{12}) + 2\tilde{x}_{21}(h(1/2 + w/\tilde{x}_{12}) - h(1/2)).
\end{aligned}
 $$

Similarly, we obtain two other formulas.

$$ g(\alpha_0, x_{01}, x_{10}) = g(\alpha_0, \tilde{x}_{01}, \tilde{x}_{10}) + 2\tilde{x}_{01}(h(1/2 + w/\tilde{x}_{01}) - h(1/2)), $$
$$g(\beta_0, x_{02}, x_{20}) = g(\beta_0, \tilde{x}_{02}, \tilde{x}_{20}) + 2\tilde{x}_{02} \left(h\left(\frac{1}{2} + \frac{w}{\tilde{x}_{02}}\right) - h\left(\frac{1}{2}\right)\right).$$

Therefore, we can reduce Equation 9 to

$$\frac{\mathrm{Nrep}(\tilde{\mathbf{x}})}{\mathrm{Nrep}(\mathbf{x})} = 2^{2n} \left( \tilde{x}_{01} \left( 1 - h\left( \frac{1}{2} + \frac{w}{\tilde{x}_{01}} \right) \right) + \tilde{x}_{02} \left( 1 - h\left( \frac{1}{2} + \frac{w}{\tilde{x}_{02}} \right) \right) + \tilde{x}_{12} \left( 1 - h\left( \frac{1}{2} + \frac{w}{\tilde{x}_{12}} \right) \right) \right).$$

So $\mathrm{Nrep}(\mathbf{x}) \le \mathrm{Nrep}(\tilde{\mathbf{x}})$ and these two quantities are equal if and only if $w = 0$, i.e. $\mathbf{x} = \tilde{\mathbf{x}}$.

**Determining z.** To get the typical case, we now have to find the value of z that maximises the expression

$$g(1 - \alpha_0 - \beta_0, x_{21}, x_{12}) + g(\alpha_0, x_{01}, x_{10}) + g(\beta_0, x_{02}, x_{20}).$$

Notice that this expression is equivalent, up to an additive constant, to $-\sum_{i,j} x_{ij} \log_2(x_{ij})$.

This function is concave and thus admit a single maximum. The differentiation of this function with respect to z gives

$$-2 \log_2 \left( \frac{\left( \frac{2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1}{3} + z \right) \left( \frac{\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1}{3} + z \right) z}{(1 - \alpha_0 - \beta_0 - 2z) \left( \frac{-2\alpha_0 - \beta_0 + 4\alpha_1 + 2\beta_1}{3} - 2z \right) \left( \frac{-\alpha_0 - 2\beta_0 + 2\alpha_1 + 4\beta_1}{3} - 2z \right)} \right),$$

which is equal to zero if and only if

$$\frac{(2\alpha_0 + \beta_0 - \alpha_1 - 2\beta_1 + 3z)(\alpha_0 + 2\beta_0 - 2\alpha_1 - \beta_1 + 3z)z}{(1 - \alpha_0 - \beta_0 - 2z)(-2\alpha_0 - \beta_0 + 4\alpha_1 + 2\beta_1 - 6z)(-\alpha_0 - 2\beta_0 + 2\alpha_1 + 4\beta_1 - 6z)} = 1.$$

This explains why z is the root of a polynomial of degree 3.

## A.6 Number of Representations and Badly-formed Elements

There are $\tilde{O}(2^{ng(1,\alpha_0,\beta_0)})$ vectors in $T(n, \alpha_0, \beta_0)$. For each of these vectors, there are by definition $\mathrm{Nrep}(\alpha_0, \beta_0, \alpha_1, \beta_1)$ ways of decomposing it as the sum of two vectors of $T(n, \alpha_1, \beta_1)$. Moreover, the number of vectors in $T(n, \alpha_1, \beta_1)$ is $\tilde{O}(2^{ng(1,\alpha_1,\beta_1)})$. There are then $\tilde{O}(2^{2ng(1,\alpha_1,\beta_1)})$ pairs of vectors of $T(n, \alpha_1, \beta_1)$, but only $\tilde{O}(\mathrm{Nrep}(\alpha_0, \beta_0, \alpha_1, \beta_1)2^{ng(1,\alpha_0,\beta_0)})$ of these pairs give a valid representation of a vector of $T(n, \alpha_0, \beta_0)$. All the other pairs give badly-formed elements. Thus, when we merge two L-sized lists of elements of $T(n, \alpha_1, \beta_1)$ on L bits, we obtain $\tilde{O}(L \mathrm{Nrep}(\alpha_0, \beta_0, \alpha_1, \beta_1)2^{ng(1,\alpha_0,\beta_0)-2ng(1,\alpha_1,\beta_1)})$ vectors of $T(n, \alpha_0, \beta_0)$, the remaining consisting on badly-formed vectors.