<figure><img src="image_1.png" /></figure>

HAL
open science

# Polynomial time key-recovery attack on high rate random alternant codes

Magali Bardet, Rocco Mora, Jean-Pierre Tillich

► To cite this version:

Magali Bardet, Rocco Mora, Jean-Pierre Tillich. Polynomial time key-recovery attack on high rate random alternant codes. IEEE Transactions on Information Theory, 2024, 70 (6), pp.4492-4511. 10.1109/TIT.2023.3334592. hal-04276519

HAL Id: hal-04276519

https://inria.hal.science/hal-04276519v1

Submitted on 9 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
POLYNOMIAL TIME KEY-RECOVERY ATTACK ON HIGH RATE RANDOM ALTERNANT CODES

MAGALI BARDET, ROCCO MORA, AND JEAN-PIERRE TILLICH

ABSTRACT. A long standing open question is whether the distinguisher of high rate alternant codes or Goppa codes [FGO$^+$11] can be turned into an algorithm recovering the algebraic structure of such codes from the mere knowledge of an arbitrary generator matrix of it. This would allow to break the McEliece scheme as soon as the code rate is large enough and would break all instances of the CFS signature scheme. We give for the first time a positive answer for this problem when the code is a *generic alternant code* and when the code field size $q$ is small : $q \in \{2, 3\}$ and for all regime of other parameters for which the aforementioned distinguisher works. This breakthrough has been obtained by two different ingredients :

(i) a way of using code shortening and the component-wise product of codes to derive from the original alternant code a sequence of alternant codes of decreasing degree up to getting an alternant code of degree 3 (with a multiplier and support related to those of the original alternant code);

(ii) an original Gröbner basis approach which takes into account the non standard constraints on the multiplier and support of an alternant code which recovers in polynomial time the relevant algebraic structure of an alternant code of degree 3 from the mere knowledge of a basis for it.

# 1. INTRODUCTION

**The McEliece scheme.** The McEliece encryption scheme [McE78], which dates back to 1978 and which is almost as old as RSA [RSA78], is a code-based cryptosystem built upon the family of binary Goppa codes. We will denote this system by McEliece-binary Goppa from now on since we will be interested in variations of the McEliece cryptosystem obtained by changing the underlying code family. It is equipped with very fast encryption and decryption algorithms and has very small ciphertexts but large public keysize. Contrarily to RSA which is broken by quantum computers [Sho94], it is also widely viewed as a viable quantum safe cryptosystem. A variation of this public key cryptosystem intended to be IND-CCA secure and an associated key exchange protocol [ABC$^+$20] are now in the process of being standardized by the NIST as quantum safe cryptosystems. It should be noted that the best quantum algorithm for breaking it [KT17] has exponential complexity and the corresponding exponent barely improves the exponent of the best classical algorithm [BM17] by about 40 percent.

The attacks on the McEliece scheme can be put in two classes: message-recovery attacks on one hand and key-recovery attacks on the other hand. The first one consists in inverting the McEliece encryption without finding a trapdoor and makes use of generic decoding algorithms. Despite considerable improvements [Ste88, CC98, MMT11, BJMM12, MO15, BM17], all these algorithms have exponential complexity. The second one consists in trying to recover the private key or an
equivalent private key which allows to decipher as efficiently as a legitimate user. The most efficient attack of this kind was given in [LS01] and essentially consists in trying all Goppa polynomials and all possible supports, where the verification consists in solving a code equivalence problem which is often easy with the help of the support splitting algorithm [Sen00]. The complexity of the second attack is also exponential with an exponent which is much bigger than the one obtained for the first attack. This is why the first kind of attack is considered as the main threat against McEliece-Goppa, and consequently the parameters of all those schemes based on Goppa codes have been chosen to thwart the first attack.

**Towards efficient key recovery attacks.** This picture began to change due on one hand to variations on the original McEliece proposal based on binary Goppa codes of rate close to $\frac{1}{2}$ by either considering very high rate binary Goppa codes for devising signature schemes [CFS01], or by moving from binary Goppa codes to nonbinary Goppa codes over large alphabets [BLP10, BLP11], thus decreasing significantly the extension degree $m$ over which the (secret) support of the Goppa code is defined. Recall here that the Goppa code is defined over some finite field $\mathbb{F}_q$ whereas the support of the Goppa code is defined over an extension field $\mathbb{F}_{q^m}$ (see Definition 6 which follows). On the other hand, more structured versions of the McEliece system also appeared, based on quasi-cyclic codes such as [BCGO09, CBB$^+$17] or on quasi-dyadic codes such as [MB09, BLM11, BBB$^+$17].

The quasi-cyclic or quasi-dyadic Goppa codes could be attacked by an algebraic modeling [FOPT10, GUL09] for the secret key which could be efficiently solved with Gröbner bases techniques because the added structure allowed to reduce drastically the number of unknowns of the algebraic system. By trying to solve the same algebraic system in the case of high rate Goppa codes it was also found that Gröbner bases techniques behaved very differently when the system corresponds to a Goppa code instead of a random linear code of the same length and dimension. This approach lead to [FGO$^+$11] that gave a way to distinguish high rate Goppa codes from random codes. It is based on the kernel of a linear system related to the aforementioned algebraic system. Indeed, it was shown to have an unexpectedly high dimension when instantiated with Goppa codes or the more general family of alternant codes rather than with random linear codes. This distinguisher was later on given another interpretation in [MP12], where it was proved that this dimension is related to the dimension of the square of the dual of the Goppa code. Very recently, [MT21] revisited [FGO$^+$11] and gave rigorous bounds for the dimensions of the square codes of Goppa or alternant codes and a better insight on the algebraic structure of these squares. Recall here that the component-wise product of codes (also called the Schur product) is defined from the component-wise product of vectors $\mathbf{a} = (a_i)_{1 \le i \le n}$ and $\mathbf{b} = (b_i)_{1 \le i \le n}$

$$ \mathbf{a} \star \mathbf{b} \stackrel{\text{def}}{=} (a_1 b_1, \dots, a_n b_n) $$

by

**Definition 1.** The component-wise product of codes $\mathcal{C}, \mathcal{D}$ over $\mathbb{F}$ with the same length $n$ is defined as

$$ \mathcal{C} \star \mathcal{D} \stackrel{\text{def}}{=} \langle \mathbf{c} \star \mathbf{d} \mid \mathbf{c} \in \mathcal{C}, \mathbf{d} \in \mathcal{D} \rangle_{\mathbb{F}}. $$

If $\mathcal{C} = \mathcal{D}$, we call $\mathcal{C}^{*2} \stackrel{\text{def}}{=} \mathcal{C} \star \mathcal{C}$ the square code of $\mathcal{C}$.
**Square code and cryptanalysis.** Interestingly enough, it was proved in [CGG$^{+}$14] that square code considerations could also be used to mount an attack on McEliece or Niederreiter schemes based on Generalized Reed-Solomon (GRS) codes. Recall that this scheme was proposed in [Nie86] and was subsequently broken in [SS92]. Note that when the extension degree of the Goppa code is 1 (i.e. the support of the Goppa code is defined over the same field as the Goppa code itself), a Goppa code is indeed a GRS code, so a McEliece scheme based on a Goppa code of extension degree 1 can be attacked with the [SS92] attack. However, this does not seem to generalize to higher extension degrees; i.e. on McEliece schemes based on Goppa codes in general. The point of the new attack [CGG$^{+}$14], is that it uses arguments on square codes for which there is hope that they could be applied to a much broader class of Goppa codes. It is insightful here to recall in a simplified way the attack obtained in [CGG$^{+}$14, §5].

Indeed, GRS codes turn out to display a very peculiar property with respect to the square of codes. It is readily seen that $\dim \mathcal{C}^{*2} \le \min(n, \binom{k+1}{2})$ where $k$ and $n$ are respectively the dimension and length of $\mathcal{C}$. It turns out that for random codes the upper-bound is almost always an equality [CCMZ15], whereas the situation for GRS codes is completely different: in this case, we namely have

$$ (1) \qquad \dim \mathcal{C}^{*2} = \min(n, 2k-1). $$

The reason of this peculiar behavior comes from the fact that GRS codes are polynomial evaluation codes. Indeed, recall that a GRS code is given by

**Definition 2 (GRS code).** Let $\mathbf{x} = (x_1, \dots, x_n) \in \mathbb{F}^n$ be a vector of pairwise distinct entries and $\mathbf{y} = (y_1, \dots, y_n) \in \mathbb{F}^n$ a vector of nonzero entries. The $[n, k]$ generalized Reed-Solomon (GRS) code with support $\mathbf{x}$ and multiplier $\mathbf{y}$ is

$$ \mathbf{GRS}_k(\mathbf{x}, \mathbf{y}) \stackrel{\text{def}}{=} \{(y_1 P(x_1), \dots, y_n P(x_n)) \mid P \in \mathbb{F}[z], \deg P < k\}. $$

Since the Schur product of two polynomial evaluations of degree $\deg P \le k-1$ and $\deg Q \le k-1$ respectively is itself a polynomial evaluation of degree $\deg(P \cdot Q) = \deg P + \deg Q < 2k-1$: $(y_i P(x_i))_i \star (y_i Q(x_i))_i = (y_i^2 P \cdot Q(x_i))_i$, it is readily seen that

$$ (2) \qquad \mathbf{GRS}_k(\mathbf{x}, \mathbf{y})^{*2} = \mathbf{GRS}_{2k-1}(\mathbf{x}, \mathbf{y} \star \mathbf{y}), $$

which explains (1). In a sense, the square code construction “sees” the polynomial structure of the GRS code. A key recovery attack could be mounted as follows. Recall that it amounts here to recover from an arbitrary generator matrix of a GRS code $\mathcal{C} = \mathbf{GRS}_k(\mathbf{x}, \mathbf{y})$ a pair $(\mathbf{x}', \mathbf{y}')$ satisfying $\mathcal{C} = \mathbf{GRS}_k(\mathbf{x}', \mathbf{y}')$. Let us define $\mathcal{C}(i)$ as the subcode of the GRS code $\mathcal{C}$ given by

$$ \mathcal{C}(i) = \{(y_i P(x_j))_{1 \le j \le n} : \deg P < k, x_1 \text{ is a zero of order } \ge i \text{ of } P\}, $$

then

(i) $\mathcal{C}(1)$ can be readily computed from $\mathcal{C}$ since it is the shortened code of $\mathcal{C}$ in the first position.

(ii) We have in general the equality

$$ (3) \qquad \mathcal{C}(i-1) \star \mathcal{C}(i+1) = \mathcal{C}(i)^{*2} $$

coming from the fact that the product of two polynomials which have a zero of order $i$ at $x_1$ gives a polynomial with a zero of order $2i$ in $x_1$ and
so does the product of a polynomial with a zero of order $i-1$ in $x_1$ with a polynomial which has a zero of order $i+1$ at the same place.

(iii) Solving the equation $\mathcal{X} \star \mathcal{A} = \mathcal{B}$ for two known linear codes $\mathcal{A}$ and $\mathcal{B}$ amounts to solve a linear system in the case where $\mathcal{X}$ is the maximal code satisfying $\mathcal{X} \star \mathcal{A} \subseteq \mathcal{B}$. This is indeed the case here for $\mathcal{A} = \mathcal{C}(i-1)$ and $\mathcal{B} = \mathcal{C}(i)^*$. $\mathcal{X}$ corresponds in such a case to the conductor of $\mathcal{A}$ into $\mathcal{B}$, which can be computed by solving a linear system (see below).

**Definition 3.** Let $\mathcal{C}, \mathcal{D} \subseteq \mathbb{F}^n$ be two codes. The conductor of $\mathcal{C}$ into $\mathcal{D}$ is

$$ \mathrm{Cond}(\mathcal{C}, \mathcal{D}) \stackrel{\mathrm{def}}{=} \{u \in \mathbb{F}_q^n \mid u \star \mathcal{C} \subseteq \mathcal{D}\}, $$

where $u \star \mathcal{C} \stackrel{\mathrm{def}}{=} \{u \star c \mid c \in \mathcal{C}\}$.

**Proposition 4** ([CMCP17, Lemma 7]). The conductor can be computed with

$$ \mathrm{Cond}(\mathcal{C}, \mathcal{D}) = (\mathcal{C} \star \mathcal{D}^{\perp})^{\perp}. $$

The first two points show that we can therefore compute $\mathcal{C}(2)$ in polynomial time, because $\mathcal{C}(1)$ and $\mathcal{C}(0)$ are known (the first is the shortened code and the second is the code $\mathcal{C}$ itself). We can iterate this process and compute recursively the decreasing set of codes $\mathcal{C}(3), \mathcal{C}(4), \dots$ and stop when we get a code of dimension 1 (i.e. $\mathcal{C}(k-1)$) which basically reveals a great deal of information about the multiplier $y$. It is then straightforward with this approach to finish the attack to recover the whole algebraic structure of $\mathcal{C}$. Note that we have basically computed a decreasing set of codes

$$ \mathcal{C} = \mathcal{C}(0) \supset \mathcal{C}(1) \supset \mathcal{C}(2) \dots \supset \mathcal{C}(k-1) $$

that we will call a filtration in what follows.

This approach works basically like this to attack a McEliece scheme based on GRS codes [CGG+14], but interestingly enough it also applies to Wild Goppa codes of extension degree 2 as shown in [COT14]. Such schemes were indeed proposed in [BLP10]. This extension degree corresponds to the largest extension degree where we can expect the Goppa code to behave differently from a random linear code with respect to the square code dimension. Roughly speaking in this case, even if Goppa codes are subfield subcodes of GRS codes, the equality (3) “almost” holds and this is sufficient to mount a similar attack. However as explained in [COT14], this approach is bound to fail when the extension degree $m$ is bigger than 2. However as observed in [MP12], even when $m > 2$, the square code $\mathcal{C}^{*2}$ can also be of unusually small dimension when the rate of the Goppa code is close to 1, but this time not by taking $\mathcal{C}$ to be the Goppa code itself, but by choosing $\mathcal{C}$ to be the dual of the Goppa code. This strongly suggests that an approach similar to [CGG+14, COT14] could be followed to attack McEliece schemes based on very high rate Goppa codes. Even if the parameters of the McEliece schemes proposed in the literature are never in the regime where the dimension of the square of the dual of the Goppa code behaves differently from a random linear code, there is the notable exception of the code-based signature scheme [CFS01], which is based in a crucial way on high rate Goppa codes, and which similarly to the McEliece scheme would be broken, if we can recover the unknown support of the Goppa code from an arbitrary generator matrix for it. However, the fact that the dual code is actually the trace code of a GRS code but not a subfield subcode of a GRS code loses a lot of the original polynomial structure and seems to complicate very significantly this
approach. This is still an open problem since the problem was explicitly raised in [FGO+11].

**Our contribution.** In the present article, we make what we consider to be a significant step in this direction. We will namely show that somewhat unexpectedly, an equality related to (3) holds, when taking duals of (generic) high rate alternant codes, but not when we take Goppa codes. This is extremely surprising because Goppa codes are just alternant codes with a peculiar structure. To explain the phenomenon we are witnessing, let us recall the definition of an alternant code which is a subfield subcode of a GRS code:

**Definition 5 (alternant code).** Let $n \le q^m$, for some positive integer $m$. Let $\mathbf{GRS}_r(\mathbf{x}, \mathbf{y})$ be the GRS code over $\mathbb{F}_{q^m}$ of dimension $r$ with support $\mathbf{x} \in \mathbb{F}_{q^m}$ and multiplier $\mathbf{y} \in (\mathbb{F}_{q^m} \setminus \{0\})^n$. The *alternant code* with support $\mathbf{x}$ and multiplier $\mathbf{y}$ and degree $r$ over $\mathbb{F}_q$ is

$$ \mathcal{A}_r(\mathbf{x}, \mathbf{y}) \stackrel{\text{def}}{=} \mathbf{GRS}_r(\mathbf{x}, \mathbf{y})^{\perp} \cap \mathbb{F}_q^n = \mathbf{GRS}_{n-r}(\mathbf{x}, \mathbf{y}^{\perp}) \cap \mathbb{F}_q^n, $$

where

$$ \mathbf{y}^{\perp} \stackrel{\text{def}}{=} \left( \frac{1}{\pi'_{\mathbf{x}}(x_1)y_1}, \dots, \frac{1}{\pi'_{\mathbf{x}}(x_n)y_n} \right) $$

and $\pi'_{\mathbf{x}}$ is the derivative of $\pi_{\mathbf{x}}$. The integer $m$ is called *extension degree* of the alternant code.

In other words, an alternant code is the dual (over $\mathbb{F}_q$) of a GRS code defined over $\mathbb{F}_{q^m}$. It is also a subfield subcode of a GRS code since the dual of a GRS code is a GRS code: $\mathbf{GRS}_r(\mathbf{x}, \mathbf{y})^{\perp} = \mathbf{GRS}_{n-r}(\mathbf{x}, \mathbf{y}^{\perp})$, see [MS86, Theorem 4, p. 304]. We also recall that Goppa codes form a particular family of alternant codes

**Definition 6 (Goppa code).** Let $\mathbf{x} \in \mathbb{F}_{q^m}^n$ be a support vector and $\Gamma \in \mathbb{F}_{q^m}[z]$ a polynomial of degree $r$ such that $\Gamma(x_i) \ne 0$ for all $i \in \{1, \dots, n\}$. The *Goppa code* of degree $r$ with support $\mathbf{x}$ and *Goppa polynomial* $\Gamma$ is defined as

$$ \mathcal{G}(\mathbf{x}, \Gamma) \stackrel{\text{def}}{=} \mathcal{A}_r(\mathbf{x}, \mathbf{y}), $$

where $\mathbf{y} \stackrel{\text{def}}{=} \left( \frac{1}{\Gamma(x_1)}, \dots, \frac{1}{\Gamma(x_n)} \right)$.

The very unusual behavior we observe in the case of a generic alternant code $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ is that in a certain high rate regime, if we shorten its *dual* in one position $i$ and take its square to get $\mathcal{B} \stackrel{\text{def}}{=} (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$, where $\mathbf{Sh}_i(\mathcal{C})$ denotes the code $\mathcal{C}$ shortened in position $i$, then the maximal code $\mathcal{X}$ satisfying

$$ \mathcal{X} \star \mathcal{A} \subseteq \mathcal{B} $$

is the dual of a certain alternant code of degree $r-1$:

$$ \mathcal{X} = \mathcal{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}. $$

where $\mathbf{x}_{\tilde{i}}$ denotes the vector $\mathbf{x}$ where we have dropped the index $i$ and $\mathcal{A}$ is the dual of the shortened alternant code in position $i$, i.e. $(\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})))^{\perp}$. We also use the short notation $\mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i)$ for $\mathbf{y}_{\tilde{i}} \star (x_j - x_i)_{j \ne i}$. Note that this code is actually the dual of an alternant code since $(\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})))^{\perp} = \mathcal{A}_r(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}})^{\perp}$ (see Proposition 12). In other words

$$ (4) \quad \text{Cond} \left( \mathcal{A}_r(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}), (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2} \right) = \mathcal{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}. $$
This means that starting from a generic alternant code $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ of degree $r$, we can derive in polynomial time, by first computing two auxiliary codes $\mathcal{A} = (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})))^{\perp}$ and $\mathcal{B} = (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$, and then computing the conductor $\mathbf{Cond}(\mathcal{A}, \mathcal{B})$ of $\mathcal{A}$ into $\mathcal{B}$, an alternant code of degree $r-1$. As we will see, there are only two conditions to be met for performing this task:

(i) $r \ge q+1$ where $q$ is the alphabet size of the alternant code,

(ii) $(\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$ is not the full code $\mathbb{F}_q^{n-1}$ where $n$ is the codelength of the alternant code.

By iterating this process, we can compute in polynomial time some kind of “filtra-
tion” of duals of alternant codes of decreasing degree

$$
(5) \quad \mathcal{A}_r^{\perp} = \mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp} \stackrel{\mathbf{Sh}_{i_1}}{\supseteq} \mathcal{A}_{r-1}^{\perp} \stackrel{\mathbf{Sh}_{i_2}}{\supseteq} \dots \stackrel{\mathbf{Sh}_{i_{r-q}}}{\supseteq} \mathcal{A}_q^{\perp}
$$

with multipliers and support which are related to the original support and multiplier
(and from which the original support and multiplier can be easily recovered). Here
the notation $\mathcal{A} \stackrel{\mathbf{Sh}_i}{\supseteq} \mathcal{B}$ means that

$$
\mathbf{Sh}_i(\mathcal{A}) \supseteq \mathcal{B}.
$$

What can we do with this sequence? The point is that if the degree of the alternant
code is small enough, we can compute its support and multiplier by solving a low
degree algebraic system related to the algebraic systems considered in [FOPT10,
FGO+13]. We will detail this in the particular case where $r=3$ and show that in
this case, solving the system can be performed in polynomial time with Gröbner
basis techniques. Roughly speaking the reason for this is that we have a conjunction
of factors : a very overdetermined and highly structured system which gives during
the Gröbner basis computation many new very low degree equations. We will also
show that it is possible to speed up significantly the system solving process by
introducing in the algebraic modeling new low degree polynomial equations which
are not in the ideal of the original algebraic equations from [FGO+13] and which
express the fact that the multiplier vector has only non zero entries and the support
vector has only distinct entries. This will result in the end in a very efficient system
solving procedure. Note that the aforementioned procedure reaches an alternant
code $\mathcal{A}_3$ of degree 3 when the field size $q$ is either equal to 2 or 3. In other words,
we have at the end a way to break a McEliece scheme based on binary or ternary
alternant codes as soon as $(\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$ is not the full code $\mathbb{F}_q^{n-1}$. By using
the formula given in [MT21], this is the case when

$$
(6) \quad n-1 > \left( \binom{rm+1}{2} - \frac{m}{2}(r-1) \left( (2e+1)r - 2 \frac{q^e-1}{q-1} \right) \right),
$$
$$
\text{where } e \stackrel{\text{def}}{=} \max\{i \in \mathbb{N} \mid r \ge q^i + 1\} = \lfloor \log_q(r-1) \rfloor
$$

We give in Table 1 the known cases where it is possible to attack a McEliece scheme based on alternant codes together with the new attack proposed here:

In a nutshell, our contribution can be summarized as follows.

Breaking the $m = 2$ barrier. It has been a long standing open problem after the [SS92] attack on McEliece-GRS whether it is also possible to attack subfield subcodes of GRS codes, i.e. attack McEliece-alternant or McEliece-Goppa. A first step in this direction was made in [CGG$^{+}$14] where a new attack on McEliece-GRS was derived with a hope to generalize it to McEliece-alternant or McEliece-Goppa
<table><caption>TABLE 1. Summary of polynomial time attacks on McEliece schemes based on alternant codes with the conditions to apply them.</caption><thead><tr><th>paper</th><th>restriction</th></tr></thead><tbody><tr><td>[SS92, CGG<sup>+</sup>14]</td><td>m = 1</td></tr><tr><td>[COT14]</td><td>m = 2 + Wild Goppa code</td></tr><tr><td>this paper</td><td>q = 2 or q = 3, m arbitrary + high rate condition (6)<br>(does not apply in the particular case of Goppa codes)</td></tr></tbody></table>

because it is in essence only based on the fact that certain alternant or Goppa codes behave differently from random codes with respect to the dimension of the square code. This was confirmed in [COT14] by attacking McEliece-wild Goppa in the particular case where the extension degree $m$ is 2, but the method used there which uses squares of the (shortened) Goppa code is bound to fail for higher extension degrees. Here we break for the first time the $m = 2$ barrier, which was even conjectured at some point to be the ultimate limit for such algebraic attacks to work in polynomial time and show that we can actually attack McEliece-alternant for any extension degree $m$ provided that the rate of the alternant code is sufficiently large (6) and the field size sufficiently low $q = 2$ or $q = 3$. Our attack is also based on square code considerations, but this time on the dual of the alternant code. The point is that in this case the square of the dual can also be distinguished from a random code in this regime [FGO<sup>+</sup>11, MP12]. The attack is however much more involved in this case, because the dual loses the simple polynomial evaluation formulation of the Goppa code, since it is in this case the trace of a GRS code and not subfield subcode of a GRS code. Understanding the structure of the square is more complicated as was already apparent in [MT21] which started such a task.

*Opening the road for attacking the CFS scheme.* Interestingly our attack does not work at all when the alternant code has the additional structure of being a Goppa code. However this work could open the road for also attacking this subcase, in which case we could hope to break the CFS scheme [CFS01] which operates precisely in the high rate regime where the square of the dual of the Goppa code behaves abnormally.

*New algebraic attack.* Our attack consists in two phases, the first phase computes a filtration of the dual of the alternant code by computing iteratively conductors and the second phase solves with Gröbner bases techniques a variation of the algebraic system considered in [FGO<sup>+</sup>13] and recovers the support and the multiplier from the dual of the alternant code of degree 3 we have at the end of the filtration when $q = 2$ or $q = 3$. We improve rather significantly upon the complexity of solving this system by adding new equations expressing the constraints on the support (all elements are distinct) and the multipliers (all elements are nonzero). By using certain heuristics that we confirmed experimentally we are able to prove that the Gröbner basis computation takes polynomial time and give a complete algebraic explanation of each step of the computation. It is likely that this analysis could be carried over for larger constant degree alternant codes. This would allow to break McEliece-alternant for larger field size than 3.

A proof-of-concept implementation in MAGMA of the whole attack can be found at https://github.com/roccomora/HighRateAlternant.
## 2. NOTATION AND PREREQUISITES

This section fixes the notation and reviews some tools and constructions needed for understanding the filtration attack and the rest of the article. These include the notion of subfield subcodes, trace codes, puncturing and shortening operators.

### 2.1. Notation.

*Finite fields.* We denote by $\mathbb{F}$ a generic finite field and we will use it whenever it is not important to specify the field size. We will extensively consider the finite field extension $\mathbb{F}_q \subseteq \mathbb{F}_{q^m}$, where $\mathbb{F}_q$ and $\mathbb{F}_{q^m}$ are two finite fields with $q$ and $q^m$ elements respectively, for a prime power $q$ and a positive integer $m$.

*Vector and matrix notation.* Vectors are indicated by lowercase bold letters $\mathbf{x}$ and matrices by uppercase bold letters $M$. By convention, vectors coordinates are indexed starting from 1. Moreover, $\llbracket a, b \rrbracket$ indicates the closed integer interval between $a$ and $b$. Given a function $f$ acting on $\mathbb{F}$ and a vector $\mathbf{x} = (x_i)_{1 \le i \le n} \in \mathbb{F}$, the expression $f(\mathbf{x})$ is the component-wise mapping of $f$ on $\mathbf{x}$, i.e. $f(\mathbf{x}) = (f(x_i))_{1 \le i \le n}$. We will even apply this with functions $f$ acting on $\mathbb{F} \times \mathbb{F}$: for instance for two vectors $\mathbf{x}$ and $\mathbf{y}$ in $\mathbb{F}^n$ and two positive integers $a$ and $b$ we denote by $\mathbf{x}^a \mathbf{y}^b$ the vector $(x_i^a y_i^b)_{1 \le i \le n}$. It will also be convenient to use a notation to say that we drop a few positions in a vector. If $\mathbf{x} = (x_i)_{i \in \llbracket 1, n \rrbracket}$ and $\mathcal{I}$ is a subset of positions, we denote by $\mathbf{x}_{\mathcal{I}}$ the vector $\mathbf{x}_{\mathcal{I}} \stackrel{\text{def}}{=} (x_i)_{i \in \llbracket 1, n \rrbracket \setminus \mathcal{I}}$. When there is just one position $i$ in $\mathcal{I}$ we simply write $\mathbf{x}_{\tilde{i}}$ in this case (for instance, if $\mathbf{x} = (x_1, x_2, x_3)$ then $\mathbf{x}_{\tilde{2}} = (x_1, x_3)$).

*Vector spaces.* Vector spaces are indicated by $\mathcal{C}$. For two vector spaces $\mathcal{C}$ and $\mathcal{D}$, the notation $\mathcal{C} \oplus \mathcal{D}$ means that the two vector spaces are in direct sum, i.e. that $\mathcal{C} \cap \mathcal{D} = \{0\}$.

### 2.2. Trace codes and subfield subcodes.
A useful $\mathbb{F}_q$-linear map from $\mathbb{F}_{q^m}$ to its subfield $\mathbb{F}_q$ is the trace operator:

**Definition 7.** Given the finite field extension $\mathbb{F}_{q^m}/\mathbb{F}_q$, we define the field trace $\text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}$ for all $x \in \mathbb{F}_{q^m}$ as the $\mathbb{F}_q$-linear map from $\mathbb{F}_{q^m}$ to $\mathbb{F}_q$ such that

$$ \text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(x) = \sum_{i=0}^{m-1} x^{q^i}. $$

The definition extends to vectors $\mathbf{x} \in \mathbb{F}_{q^m}$ so that the trace acts component-wise:

$$ \text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(\mathbf{x}) = (\text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(x_1), \dots, \text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(x_n)) $$

and hence to codes $\mathcal{C}$ over $\mathbb{F}_{q^m}$

$$ \text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(\mathcal{C}) = \{\text{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(\mathbf{c}) \mid \mathbf{c} \in \mathcal{C}\}. $$

We will omit the subscript $\mathbb{F}_{q^m}/\mathbb{F}_q$ and write more precisely $\text{Tr}$ from now on, whenever the extension field is clear from the context. Another standard construction to move from codes over $\mathbb{F}_{q^m}$ to codes over a subfield $\mathbb{F}_q$ is the one of subfield subcode.

**Definition 8.** Let $\mathcal{C} \subseteq \mathbb{F}_{q^m}^n$ be a code. The code

$$ \mathcal{C}_{\mathbb{F}_q} \stackrel{\text{def}}{=} \mathcal{C} \cap \mathbb{F}_q^n = \{\mathbf{c} \in \mathcal{C} \mid \forall i \in \llbracket 1, n \rrbracket, c_i \in \mathbb{F}_q\} $$

is called subfield subcode over $\mathbb{F}_q$ of $\mathcal{C}$.
It is easy to see that $\mathscr{C}_{\mathbb{F}_q}$ is an $\mathbb{F}_q$-linear subspace of $\mathbb{F}_{q^m}^n$, where $n$ is the length of $\mathscr{C}$. We conclude this subsection by stating the main classical result linking trace codes to subfield subcodes, i.e. Delsarte's theorem.

**Theorem 9.** (Delsarte's Theorem [Del75]) Let $\mathscr{C}$ be a code over $\mathbb{F}_{q^m}$. Then

$$ (\mathscr{C}_{\mathbb{F}_q})^{\perp} = \mathrm{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}(\mathscr{C}^{\perp}). $$

## 2.3. Shortening and Puncturing.

**Definition 10.** Given a code $\mathscr{C} \subseteq \mathbb{F}^n$ and a subset $\mathcal{I} \subseteq [1, n]$, the punctured code $\mathrm{Pct}_{\mathcal{I}}(\mathscr{C})$ and the shortened code $\mathrm{Sh}_{\mathcal{I}}(\mathscr{C})$ over $\mathcal{I}$ are defined respectively as

$$ \mathrm{Pct}_{\mathcal{I}}(\mathscr{C}) = \{(c_i)_{i \in [1,n]} \setminus \mathcal{I} \mid c \in \mathscr{C}\}, $$

$$ \mathrm{Sh}_{\mathcal{I}}(\mathscr{C}) = \{(c_i)_{i \in [1,n]} \setminus \mathcal{I} \mid \exists c = (c_i)_{i \in [1,n]} \in \mathscr{C} \text{ s.t. } \forall i \in \mathcal{I}, c_i = 0\}. $$

For the sake of simplicity, when $\mathcal{I} = \{i\}$, we denote the punctured and the shortened codes in $\mathcal{I}$ with $\mathrm{Pct}_i(\mathscr{C})$ and $\mathrm{Sh}_i(\mathscr{C})$ respectively.

Since we will frequently move from a code to its dual, it is worthwhile to mention how the shortening and puncturing combine with the dual operator:

**Proposition 11.** [HP03, Theorem 1.5.7] Let $\mathscr{C}$ be a linear code of length $n$ and $\mathcal{I} \subset [1, n]$. Then

$$ \mathrm{Sh}_{\mathcal{I}}(\mathscr{C}^{\perp}) = \mathrm{Pct}_{\mathcal{I}}(\mathscr{C})^{\perp} \quad \text{and} \quad \mathrm{Pct}_{\mathcal{I}}(\mathscr{C}^{\perp}) = \mathrm{Sh}_{\mathcal{I}}(\mathscr{C})^{\perp}. $$

Finally we recall the well known fact that a shortened alternant codes is itself an alternant code

**Proposition 12.** Let $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be an alternant code of length $n$ and $\mathcal{I} \subseteq \{1, \dots, n\}$. Then

$$ \mathrm{Sh}_{\mathcal{I}}(\mathcal{A}_r(\mathbf{x}, \mathbf{y})) = \mathcal{A}_r(\mathbf{x}_{\tilde{\mathcal{I}}}, \mathbf{y}_{\tilde{\mathcal{I}}}). $$

## 2.4. Base field extension and alternant codes.
It will be convenient to consider for a code defined over $\mathbb{F}_q$ its extension by scalars over $\mathbb{F}_{q^m}$, meaning the following.

**Definition 13** (extension of a code over a field extension). Let $\mathscr{C}$ be a linear code over $\mathbb{F}_q$. We denote by $\mathscr{C}_{\mathbb{F}_{q^m}}$ the $\mathbb{F}_{q^m}$-linear span of $\mathscr{C}$ in $\mathbb{F}_{q^m}^n$.

It could seem that this is a somewhat trivial object, but this notion comes very handy in our case, since the extension to $\mathbb{F}_{q^m}$ of the dual of an alternant code defined over $\mathbb{F}_q$ is a sum of $m$ GRS codes as shown by

**Proposition 14.** Let $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be an alternant code over $\mathbb{F}_q$. Then

$$ (\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}} = \sum_{j=0}^{m-1} \mathrm{GRS}_r(\mathbf{x}, \mathbf{y})^{q^j}. $$

where $\mathscr{C}^{q^j} \stackrel{\mathrm{def}}{=} \{\mathbf{c}^{q^j} = (c_i^{q^j})_i : \mathbf{c} \in \mathscr{C}\}$ is readily seen to be an $\mathbb{F}_{q^m}$-linear code of the same dimension as $\mathscr{C}$ when $\mathscr{C}$ is itself an $\mathbb{F}_{q^m}$-linear code

*Proof.* It follows directly from Proposition 9 by taking $\mathscr{C} \stackrel{\mathrm{def}}{=} \mathrm{GRS}_r(\mathbf{x}, \mathbf{y})$ and the fact that we have for any nonnegative integer $j$ and any $\mathbb{F}_{q^m}$-linear code $\mathscr{C}$ (see Proposition 41 in Appendix A).

$$ \mathrm{Tr}(\mathscr{C})_{\mathbb{F}_{q^m}} = \mathscr{C} + \mathscr{C}^q + \dots + \mathscr{C}^{q^{m-1}}. $$

□
We will also need the following straightforward properties

**Lemma 15.** [Ran15, Lemma 2.23] Let $\mathcal{C}, \mathcal{D} \subseteq \mathbb{F}_q^n$ be two codes. Then

$$ (1) \quad (\mathcal{C}^{\perp})_{\mathbb{F}_{q^m}} = (\mathcal{C}_{\mathbb{F}_{q^m}})^{\perp} \subseteq \mathbb{F}_{q^m}^n. $$

$$ (2) \quad \mathcal{C} \subseteq \mathcal{D} \iff \mathcal{C}_{\mathbb{F}_{q^m}} \subseteq \mathcal{D}_{\mathbb{F}_{q^m}}. $$

$$ (3) \quad (\mathcal{C} + \mathcal{D})_{\mathbb{F}_{q^m}} = \mathcal{C}_{\mathbb{F}_{q^m}} + \mathcal{D}_{\mathbb{F}_{q^m}} \quad \text{and} \quad (\mathcal{C} \oplus \mathcal{D})_{\mathbb{F}_{q^m}} = \mathcal{C}_{\mathbb{F}_{q^m}} \oplus \mathcal{D}_{\mathbb{F}_{q^m}}. $$

$$ (4) \quad (\mathcal{C} \cap \mathcal{D})_{\mathbb{F}_{q^m}} = \mathcal{C}_{\mathbb{F}_{q^m}} \cap \mathcal{D}_{\mathbb{F}_{q^m}}. $$

$$ (5) \quad (\mathcal{C} \star \mathcal{D})_{\mathbb{F}_{q^m}} = \mathcal{C}_{\mathbb{F}_{q^m}} \star \mathcal{D}_{\mathbb{F}_{q^m}}. $$

**2.5. A few useful notions for algebraic system solving.** We will use here techniques inspired by Gröbner bases computations and it will be useful to recall a few useful notions about this topic. We will consider here ideals generated by polynomials over some field $\mathbb{F}$ in a certain number of variables, say $X_1, \dots, X_n$. We will use two orders on the monomials of $\mathbb{F}[X_1, \dots, X_n]$. The first is just the usual lexicographic order where the single variables are ordered as $X_1 > X_2 > \dots > X_n$ and two monomials $X^\alpha \stackrel{\text{def}}{=} X_1^{\alpha_1} \cdots X_n^{\alpha_n}$ and $X^\beta \stackrel{\text{def}}{=} X_1^{\beta_1} \cdots X_n^{\beta_n}$ are ordered as follows: $X^\alpha > X^\beta$ if and only if there exists $i \in [1, n]$ such that $\alpha_i > \beta_i$ and $\alpha_j = \beta_j$ for $j < i$. We will also use the graded reverse lexicographical order (grevlex) meaning that $X^\alpha > X^\beta$ if and only if there exists $i \in [1, n]$ such that $\alpha_i < \beta_i$ and $\alpha_j = \beta_j$ for $j > i$. We will also need the notion of leading monomial. The leading monomial $\text{LM}(P)$ of a polynomial (for a given order on the monomials) is the largest monomial appearing in $P$. In other words for $P = \sum_{\alpha \in \mathbb{Z}^n} c_\alpha X^\alpha$ with $P \in \mathbb{F}[X_1, \dots, X_n]$ where we use the notation $X^\alpha = X_1^{\alpha_1} \cdots X_n^{\alpha_n}$ we have $\text{LM}(P) = \max\{X^\alpha \mid c_\alpha \neq 0\}$ where the maximum is taken with respect to the order on the monomials which is chosen.

Another notion will be useful for manipulating an algebraic system, namely

**Definition 16 (Macaulay matrix).** The Macaulay $M^{\text{acaulay}}(f_1, \dots, f_k)$ matrix associated to an algebraic system (and with respect to a certain order of the monomials) specified by the polynomials $(f_1, \dots, f_k)$ is the $k \times M$ matrix where $M$ is the number of different monomials $m_1 > m_2 > \dots > m_M$ involved in the $f_i$'s where the entry $m_{ij}$ is the coefficient of the monomial $m_j$ in $f_i$.

### 3. THE FILTRATION

Assume that $r \ge q+1$. We are going to explain in this section how, starting with the code $\mathcal{A}_r = \mathcal{A}_r(\mathbf{x}, \mathbf{y})$, we are (generally) able to compute in polynomial time a sequence of alternant codes such that

$$ (7) \qquad \mathcal{A}_r^{\perp} = \mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp} \supseteq \mathcal{A}_{r-1}^{\perp} \supseteq \dots \supseteq \mathcal{A}_q^{\perp}, $$

where all the alternant codes have a support which is easily derived from the support of $\mathcal{A}_r$, since it just amounts to drop some positions. This is instrumental for recovering efficiently the algebraic structure of the alternant code (i.e. the support $\mathbf{x}$ and the multiplier $\mathbf{y}$) in what follows. The core of this attack is the following theorem.

**Theorem 17.** Let $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be an alternant code such that $r \ge q+1$. Let $\mathcal{C} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})))^{\perp}, \mathcal{D} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$, for an arbitrary position $i$. Then

$$ \mathcal{A}_{r-1}(\tilde{\mathbf{x}}_i, \tilde{\mathbf{y}}_i(\tilde{\mathbf{x}}_i - \tilde{x}_i))^{\perp} \star \mathcal{C} \subseteq \mathcal{D}, $$

or, equivalently,

$$ \text{Cond}(\mathcal{C}, \mathcal{D}) \supseteq \mathcal{A}_{r-1}(\tilde{\mathbf{x}}_i, \tilde{\mathbf{y}}_i(\tilde{\mathbf{x}}_i - \tilde{x}_i))^{\perp}. $$
Experimentally it turns out that we actually have equality here **Cond**($\mathscr{C}$, $\mathscr{D}$) = $\mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}$ when choosing a random alternant code. It is however possible to build artificial examples where equality does not hold. Notably, we also found that the subfamily of Goppa codes does not meet this property either. However, if $\mathbf{x}$ and $\mathbf{y}$ are sampled at random, we never met a case in our experiments where equality does not hold. This leads us to state the following conjecture.

**Conjecture 18.** Let $\mathscr{A}_r(\mathbf{x}, \mathbf{y})$ be a random alternant code over $\mathbb{F}_q$, such that $r \ge q+1$ and $(\mathscr{A}_r(\mathbf{x}, \mathbf{y})^{\perp})^{*2}$ is not the full code. Let $\mathscr{C} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathscr{A}_r(\mathbf{x}, \mathbf{y})))^{\perp}$ and $\mathscr{D} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathscr{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))^{*2}$, for an arbitrary position $i$. Then, we expect that

$$ \text{Cond}(\mathscr{C}, \mathscr{D}) = \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}. $$

with probability $1-o(1)$ as $n \to \infty$.

It is clear that this conjecture, if true, allows to compute in polynomial time the filtration (7), since computing conductors just amounts to solve a linear system. Moreover, if $\mathbf{x}, \mathbf{y}$ are taken at random, then we expect a random behaviour for $\mathbf{x}_{\tilde{i}}$ and $\mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i)$ as well. The fact that when taking a random alternant code, the whole filtration can be computed has indeed been verified experimentally. The first conductor $\mathscr{A}_{r-1} = \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}_1}, \mathbf{y}_{\tilde{i}_1}(\mathbf{x}_{\tilde{i}_1} - x_{i_1}))^{\perp}$ is computed by using directly Conjecture 18 and we iterate the process by choosing a sequence of positions $i_1, i_2, \dots, i_{r_q}$ by which we shorten. We let $\mathcal{I}_s = \{i_1, \dots, i_s\}$. It is readily seen that we compute iteratively from $\mathscr{A}_{r-s+1}^{\perp} \stackrel{\text{def}}{=} \mathscr{A}_{r-s+1}(\mathbf{x}_{\overline{\mathcal{I}_{s-1}}}, \mathbf{y}_{\overline{\mathcal{I}_{s-1}}}, \prod_{j=1}^{s-1}(\mathbf{x}_{\overline{\mathcal{I}_{s-j}}}-x_{i_j}))^{\perp}$ the code

$$ \mathscr{A}_{r-s}^{\perp} \stackrel{\text{def}}{=} \mathscr{A}_{r-s}(\mathbf{x}_{\overline{\mathcal{I}_s}}, \mathbf{y}_{\overline{\mathcal{I}_s}} \prod_{j=1}^{s} (\mathbf{x}_{\overline{\mathcal{I}_s}} - x_{i_j}))^{\perp}. $$

This allows to decrease the degree of the alternant one by one. The last step ends by using the conjecture with $r = q+1$ and ends with the conductor $\mathscr{A}_q^{\perp}$. Let us now prove Theorem 17.

**3.1. Proof of Theorem 17.** It will be convenient to prove a slightly stronger result which implies Theorem 17. It is based on the observation that $\mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp} \subseteq \text{Sh}_i(\mathscr{A}_r(\mathbf{x}, \mathbf{y})^{\perp})$. Theorem 17 is indeed implied by the following slightly stronger result:

**Theorem 19.** Let $\mathscr{A}_r(\mathbf{x}, \mathbf{y})$ be an alternant code such that $r \ge q+1$. Let $\mathscr{C} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathscr{A}_r(\mathbf{x}, \mathbf{y})))^{\perp}$ and $\mathscr{D}' \stackrel{\text{def}}{=} \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}_1}, \mathbf{y}_{\tilde{i}_1}(\mathbf{x}_{\tilde{i}_1} - x_{i_1}))^{\perp} \star \text{Sh}_i(\mathscr{A}_r(\mathbf{x}, \mathbf{y})^{\perp})$, for an arbitrary position $i$. Then

$$ \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp} \star \mathscr{C} \subseteq \mathscr{D}', $$

or, equivalently,

$$ \text{Cond}(\mathscr{C}, \mathscr{D}') \supseteq \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}. $$

To prove this theorem, it will be convenient to consider the extensions of all these codes to $\mathbb{F}_{q^m}$, in other words we are going to prove that

$$ (8) \qquad \mathcal{B}_{\mathbb{F}_{q^m}} \star \mathcal{C}_{\mathbb{F}_{q^m}} \subseteq \mathcal{D}'_{\mathbb{F}_{q^m}}, $$

where $\mathcal{B} \stackrel{\text{def}}{=} \mathscr{A}_{r-1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}}(\mathbf{x}_{\tilde{i}} - x_i))^{\perp}$. The point of doing this, is that (i) it is equivalent to prove (8) because of the points 2 and 5 of Lemma 15, (ii) the extended
codes can be expressed as a sum of GRS codes due to Proposition 14. The proof of Theorem 19 will proceed by following the steps below.

**Step 1:** We first observe that the code $\mathcal{C}_{\mathbb{F}_{q^m}} = ( (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})))^\perp )_{\mathbb{F}_{q^m}} = (\mathcal{A}_r(\mathbf{x}_i^\tilde{,} \mathbf{y}_i^\tilde{,}))^\perp)_{\mathbb{F}_{q^m}}$ (where the last equality follows from Proposition 12) decomposes as

$$ (\mathcal{A}_r(\mathbf{x}_i^\tilde{,} \mathbf{y}_i^\tilde{,}))^\perp)_{\mathbb{F}_{q^m}} = (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \oplus \langle \mathbf{y}_i^\tilde{,} \rangle_{\mathbb{F}_{q^m}}. $$

This is Lemma 20 below. This implies that

$$ \mathcal{B}_{\mathbb{F}_{q^m}} \star \mathcal{C}_{\mathbb{F}_{q^m}} = \underbrace{\mathcal{B}_{\mathbb{F}_{q^m}} \star (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}}}_{\mathcal{B}_{\mathbb{F}_{q^m}}^\prime} + \mathcal{B}_{\mathbb{F}_{q^m}} \star \langle \mathbf{y}_i^\tilde{,} \rangle_{\mathbb{F}_{q^m}}. $$

Therefore in order to prove (8) it will be enough to prove the inclusion

$$ (9) \qquad \mathcal{B}_{\mathbb{F}_{q^m}} \star \langle \mathbf{y}_i^\tilde{,} \rangle_{\mathbb{F}_{q^m}} \subseteq \mathcal{B}_{\mathbb{F}_{q^m}}^\prime. $$

**Step 2:** To achieve this purpose, we then prove that the extended shortened code $(\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}}$ contains as a subcode $\mathcal{B}_{\mathbb{F}_{q^m}} = (\mathcal{A}_{r-1}(\mathbf{x}_i^\tilde{,}, (\mathbf{x}_i^\tilde{,} - x_i)\mathbf{y}_i^\tilde{,}))_{\mathbb{F}_{q^m}}$ (Lemma (21)) on one hand and $\mathcal{C}_0 \stackrel{\text{def}}{=} \langle y_i^{q^u} \mathbf{y}_i^{q^v} - y_i^{q^v} \mathbf{y}_i^{q^u} : u, v \in [0, m-1] \rangle_{\mathbb{F}_{q^m}}$ on the other hand. Actually more is true, namely that the extended shortened code is a sum of these two subcodes but we will not need this.

**Step 3:** By using this, in order to prove (9) we will start with an element in $\mathcal{B}_{\mathbb{F}_{q^m}} \star \langle \mathbf{y}_i^\tilde{,} \rangle_{\mathbb{F}_{q^m}}$ and by adding suitable elements of $\mathcal{B}_{\mathbb{F}_{q^m}} \star \mathcal{C}_0$ and $\mathcal{B}_{\mathbb{F}_{q^m}}^{\star 2}$ we will show that we end with an element in $\mathcal{D}_{\mathbb{F}_{q^m}}^\prime = \mathcal{B}_{\mathbb{F}_{q^m}} \star (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}}$.

Let us now state and prove the lemmas we have mentioned above.

**Lemma 20.**

$$ (\mathcal{A}_r(\mathbf{x}_i^\tilde{,} \mathbf{y}_i^\tilde{,}))^\perp)_{\mathbb{F}_{q^m}} = (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \oplus \langle \mathbf{y}_i^\tilde{,} \rangle_{\mathbb{F}_{q^m}}. $$

*Proof.* Note that $(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp)_{\mathbb{F}_{q^m}}$ decomposes as the set of codewords $\mathcal{A}_0$ that are zero in $i$ (this is $(\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}}$ where add an extra-position at $i$ which is always 0) plus a space of dimension 1 generated by an element of $(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp)_{\mathbb{F}_{q^m}}$ which is not equal to 0 at position $i$. $\mathbf{y}$ is clearly such an element and we can write

$$ (\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp)_{\mathbb{F}_{q^m}} = \mathcal{A}_0 \oplus \langle \mathbf{y} \rangle_{\mathbb{F}_{q^m}}. $$

By puncturing these codes at $i$ we get our lemma.

<figure><img src="image_3.png" /></figure>

**Lemma 21.** We have for any position $i$

$$ (10) \quad (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \supseteq (\mathcal{A}_{r-1}(\mathbf{x}_i^\tilde{,}, (\mathbf{x}_i^\tilde{,} - x_i)\mathbf{y}_i^\tilde{,}))^\perp)_{\mathbb{F}_{q^m}} $$

$$ (11) \quad (\mathrm{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \supseteq \mathcal{C}_0 \quad \text{where} $$

$$ (12) \qquad \mathcal{C}_0 \stackrel{\text{def}}{=} \langle y_i^{q^u} \mathbf{y}_i^{q^v} - y_i^{q^v} \mathbf{y}_i^{q^u} \mid u, v \in [0, m-1] \rangle_{\mathbb{F}_{q^m}}. $$

*Proof.* By using Proposition 14 we know that

$$ (\mathcal{A}_r(\mathbf{x}, \mathbf{y})^\perp)_{\mathbb{F}_{q^m}} = \langle \mathbf{x}^{aq^\ell} \mathbf{y}^{q^\ell}; a \in [0, r-1], \ell \in [0, m-1] \rangle_{\mathbb{F}_{q^m}}, $$

$$ (\mathcal{A}_{r-1}(\mathbf{x}_i^\tilde{,}, (\mathbf{x}_i^\tilde{,} - x_i)\mathbf{y}_i^\tilde{,}))^\perp)_{\mathbb{F}_{q^m}} = \langle \mathbf{x}_i^{aq^\ell} (\mathbf{x}_i^\tilde{,} - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell}; a \in [0, r-2], \ell \in [0, m-1] \rangle_{\mathbb{F}_{q^m}}. $$
Observe now that

$$ (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))_{\mathbb{F}_{q^m}} = \mathbf{Sh}_i((\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}}). $$

Clearly $\mathbf{x}^{aq^\ell}(\mathbf{x}-x_i)^{q^\ell}\mathbf{y}^{q^\ell} = (\mathbf{x}^a(\mathbf{x}-x_i))^{q^\ell}\mathbf{y}^{q^\ell}$ vanishes at $i$ and belongs to $(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}}$ for $a \in [0, r-2]$. Therefore $\mathbf{x}_i^{aq^\ell}(\mathbf{x}_i-x_i)^{q^\ell}\mathbf{y}_i^{q^\ell}$ belongs to $\mathbf{Sh}_i((\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}})$. This proves (10). Similarly $\mathbf{y}_i^{q^u}\mathbf{y}_i^{q^v} - \mathbf{y}_i^{q^v}\mathbf{y}_i^{q^u}$ belongs clearly to $(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}}$ and vanishes at $i$. Hence $\mathbf{y}_i^{q^u}\mathbf{y}_i^{q^v} - \mathbf{y}_i^{q^v}\mathbf{y}_i^{q^u}$ belongs to $\mathbf{Sh}_i((\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})_{\mathbb{F}_{q^m}})$. This proves (12). $\square$

**Lemma 22.** Let $\mathcal{B} \stackrel{\text{def}}{=} \mathcal{A}_{r-1}(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i-x_i))^{\perp}$ and $\mathcal{D}' \stackrel{\text{def}}{=} \mathcal{B} \star \mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})$, then if $r \ge q+1$:

$$ \mathcal{B}_{\mathbb{F}_{q^m}} \star \langle \mathbf{y}_i \rangle_{\mathbb{F}_{q^m}} \subseteq \mathcal{D}'. $$

*Proof.* We use the same notation as in Lemma 21 and observe that $\mathcal{D}_0$ and $\mathcal{D}_1$ defined below are both subcodes of $\mathcal{D}'_{\mathbb{F}_{q^m}}$:

$$ \begin{aligned} \mathcal{D}_0 \stackrel{\text{def}}{=} \mathcal{B}_{\mathbb{F}_{q^m}} \star \mathcal{C}_0 &= \left\langle \mathbf{x}_i^{aq^\ell} (\mathbf{x}_i - x_i)^{q^\ell} (\mathbf{y}_i^{q^u} \mathbf{y}_i^{q^v} + \mathbf{y}_i^{q^v} \mathbf{y}_i^{q^u}) - \mathbf{y}_i^{q^v} \mathbf{y}_i^{q^u} \right\rangle_{\mathbb{F}_{q^m}} \quad \forall a \in [0, r-2], \ell, u, v \in [0, m-1] \\ \mathcal{D}_1 \stackrel{\text{def}}{=} \mathcal{B}_{\mathbb{F}_{q^m}}^{*2} &= \left\langle \mathbf{x}_i^{aq^j+bq^\ell} (\mathbf{x}_i - x_i)^{q^j+q^\ell} \mathbf{y}_i^{q^j+q^\ell} \right\rangle_{\mathbb{F}_{q^m}} \quad \forall a, b \in [0, r-2], j, \ell \in [0, m-1] \end{aligned} $$

This is a direct consequence of $\mathcal{D}'_{\mathbb{F}_{q^m}} = \mathcal{B}_{\mathbb{F}_{q^m}} \star (\mathbf{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp}))_{\mathbb{F}_{q^m}}$ (definition of $\mathcal{D}'$ and Point 5 in Lemma 15) and Lemma 21. We also observe that

$$ \mathcal{B}_{\mathbb{F}_{q^m}} \star \langle \mathbf{y}_i \rangle_{\mathbb{F}_{q^m}} = \left\langle (\mathbf{x}_i^{q^\ell})^{q^\ell} (\mathbf{x}_i - x_i)^{q^\ell} (\mathbf{y}_i^{q^\ell+1}) \mid a \in [0, r-2], l \in [0, m-1] \right\rangle_{\mathbb{F}_{q^m}}. $$

Our proof strategy is to start with an element appearing in the vector span above and by suitable additions of elements of $\mathcal{D}_i$ (where $i \in \{0, 1\}$), and possibly also by multiplying by some elements in $\mathbb{F}_{q^m}$, results at the end in an element in $\mathcal{D}_i$. This will prove our lemma. We will use here the notation

$$ u \xrightarrow{\mathcal{D}_i} v $$

to write that $v$ can be obtained from $u$ by adding a suitable element of $\mathcal{D}_i$ and multiplying by some element in $\mathbb{F}_{q^m}$, i.e. this is equivalent to $u - \lambda v \in \mathcal{D}_i$ for a suitable element $\lambda$ in $\mathbb{F}_{q^m}$. It is readily seen that for any $a \in [0, r-2]$, $u$ and $v$ in $[0, m-1]$ and any polynomial $P$ in $\mathbb{F}_{q^m}[X]$ of degree $\le r-2$ we have

$$ (13) \qquad P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell+q^u} \xrightarrow{\mathcal{D}_0} P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell+q^v} $$

$$ (14) \qquad P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{2q^\ell} \xrightarrow{\mathcal{D}_1} (\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{2q^\ell}. $$

The first reduction follows by noticing that

$$ \begin{aligned} P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell+q^u} &= P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell} \mathbf{y}_i^{-q^v} (\mathbf{y}_i^{q^v} \mathbf{y}_i^{q^u} - \mathbf{y}_i^{q^u} \mathbf{y}_i^{q^v} + \mathbf{y}_i^{q^u} \mathbf{y}_i^{q^v}) \\ &= d + \mathbf{y}_i^{q^u-q^v} P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell+q^v} \end{aligned} $$

where $d = P(\mathbf{x}_i^{q^\ell})(\mathbf{x}_i - x_i)^{q^\ell} \mathbf{y}_i^{q^\ell} \mathbf{y}_i^{-q^v} (\mathbf{y}_i^{q^v} \mathbf{y}_i^{q^u} - \mathbf{y}_i^{q^u} \mathbf{y}_i^{q^v})$ clearly belongs to $\mathcal{D}_0$. The second reduction follows by performing the Euclidean division of $P(X)$ by $(X - x_i)$.
We can namely write $P(X) = (X - x_i)Q(X) + P(x_i)$ for a polynomial $Q$ of degree $\deg P - 1$. Therefore

$$ (15) \qquad \begin{aligned} P(\tilde{x}_i)^{q\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} &= ((\tilde{x}_i - x_i)Q(\tilde{x}_i) + P(x_i))^{q\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} \\ &= ((\tilde{x}_i - x_i)^{q\ell} Q(\tilde{x}_i)^{q\ell} + P(x_i)^{q\ell}) (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} \\ &= \mathbf{d} + P(x_i)^{q\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} \end{aligned} $$

where (15) follows from the $\mathbb{F}_q$-linearity of the Frobenius action $x \mapsto x^{q\ell}$ and $\mathbf{d} = ((\tilde{x}_i - x_i)^{q\ell} Q(\tilde{x}_i)^{q\ell}) (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell}$ belongs obviously to $\mathscr{D}_1$.

Let us show the inclusion by performing for a generator $\tilde{x}_i^{aq\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{q\ell+1}$ of $\mathscr{D}_{\mathbb{F}_{q^m}} \star \langle \tilde{y}_i \rangle_{\mathbb{F}_{q^m}}$ a sequence of reductions

$$ \begin{aligned} \tilde{x}_i^{aq\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{q\ell+1} &\xrightarrow{\mathscr{D}_0} \tilde{x}_i^{aq\ell} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} \\ &\xrightarrow{\mathscr{D}_1} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} \end{aligned} $$

The crucial argument is now the simple observation that

$$ (\tilde{x}_i - x_i)^{q\ell} = ((\tilde{x}_i - x_i)^q)^{q\ell-1} $$

where $\ell^- \stackrel{\text{def}}{=} \ell - 1$ if $\ell > 0$ and $\ell^- \stackrel{\text{def}}{=} m - 1$ if $\ell = 0$. This is a consequence of the fact that the entries of $\mathbf{x}$ are in $\mathbb{F}_{q^m}$. This suggests the following sequence of reductions

$$ \begin{aligned} (\tilde{x}_i - x_i)^{q\ell} \tilde{y}_i^{2q\ell} &\xrightarrow{\mathscr{D}_0} ((\tilde{x}_i - x_i)^q)^{q\ell-1} \tilde{y}_i^{q\ell+q\ell-1} = ((\tilde{x}_i - x_i)^{q-1})^{q\ell-1} (\tilde{x}_i - x_i)^{q\ell-1} \tilde{y}_i^{q\ell-1+q\ell} \\ &\xrightarrow{\mathscr{D}_0} ((\tilde{x}_i - x_i)^{q-1})^{q\ell-1} (\tilde{x}_i - x_i)^{q\ell-1} \tilde{y}_i^{2q\ell-1} = ((\tilde{x}_i - x_i)^{q-2})^{q\ell-1} (\tilde{x}_i - x_i)^{2q\ell-1} \tilde{y}_i^{2q\ell-1}. \end{aligned} $$

Note that the last reduction could be performed because the degree of the polynomial $(\tilde{x}_i - x_i)^{q-1}$, which is $q-1$, is less than or equal to $r-2$ by assumption on $r$. For the very same reason $(r \ge q+1)$ we observe that the right-hand term $((\tilde{x}_i - x_i)^{q-2})^{q\ell-1} (\tilde{x}_i - x_i)^{2q\ell-1} \tilde{y}_i^{2q\ell-1}$ belongs to $\mathscr{D}_1$ which finishes the proof. $\square$

We are ready now to prove Theorem 19.

*Proof of Theorem 19.* From Lemma 20, we know that $\mathscr{C}_{\mathbb{F}_{q^m}} = (\mathscr{A}_r(\tilde{x}_i, \tilde{y}_i)^\perp)_{\mathbb{F}_{q^m}}$ can be decomposed as

$$ \mathscr{C}_{\mathbb{F}_{q^m}} = (\mathbf{Sh}_i (\mathscr{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \oplus \langle \tilde{y}_i \rangle_{\mathbb{F}_{q^m}}. $$

This implies that

$$ \begin{aligned} \mathscr{B}_{\mathbb{F}_{q^m}} \star \mathscr{C}_{\mathbb{F}_{q^m}} &= \underbrace{\mathscr{B}_{\mathbb{F}_{q^m}} \star (\mathbf{Sh}_i (\mathscr{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}}}_{\mathscr{D}_{\mathbb{F}_{q^m}}^\perp} + \mathscr{B}_{\mathbb{F}_{q^m}} \star \langle \tilde{y}_i \rangle_{\mathbb{F}_{q^m}} \\ &= \mathscr{B}_{\mathbb{F}_{q^m}} \star (\mathbf{Sh}_i (\mathscr{A}_r(\mathbf{x}, \mathbf{y})^\perp))_{\mathbb{F}_{q^m}} \quad \text{(by Lemma 22)} \\ &= \mathscr{D}_{\mathbb{F}_{q^m}}^\perp. \end{aligned} $$

The equality of the extended codes over $\mathbb{F}_{q^m}$ implies the equalities of the codes over $\mathbb{F}_q$ which ends the proof. $\square$
3.2. **What is wrong with Goppa codes?** Before moving to the second part of the attack, we make a short digression on how the arguments explained so far (do not) apply to the Goppa case. The discussion below does not represent a proof that computing a filtration is impossible for Goppa codes, but rather an intuition about what hampers it. Goppa codes behave differently from random alternant codes and provide counterexamples to Heuristic 18. The latter should be replaced by

**Heuristic 23.** Let $\mathcal{G}(\mathbf{x}, \Gamma) \stackrel{\text{def}}{=} \mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be a random Goppa code of degree $r$, with $r \ge q-1$ and $(\mathcal{G}(\mathbf{x}, \Gamma))^{\star 2}$ being different from the full code. Choose an arbitrary code position $i$ and let $\mathcal{C} \stackrel{\text{def}}{=} (\text{Sh}_i(\mathcal{G}(\mathbf{x}, \Gamma)))^{\perp}$ and $\mathcal{D} \stackrel{\text{def}}{=} \text{Sh}_i(\mathcal{A}_r(\mathbf{x}, \mathbf{y})^{\perp})^{\star 2}$. Then, with high probability,

$$ \text{Cond}(\mathcal{C}, \mathcal{D}) = \mathcal{A}_r(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i - \mathbf{x}_i))^{\perp}. $$

This is unfortunate, since our approach heavily builds upon the fact that the degree of the conductor decreases. Moreover, it will turn out that the code we obtain as a conductor could have been obtained directly by shortening a suitable code. Actually, it will turn out that for Goppa codes, there are several codes which are very close to each other and which are obtained by various shortenings. This is summarized by the following proposition, whose proof will be given in the appendix. We will explain in what follows why this phenomenon is the main obstacle for applying our conductor approach.

**Proposition 24.** Let $\mathcal{G}(\mathbf{x}, \Gamma) \stackrel{\text{def}}{=} \mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be a Goppa code of degree $r$. We have for any code positions $i$ and $j$ with $i \neq j$:

$$ (16) \qquad \mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp} = \mathcal{G}(\mathbf{x}, \Gamma) + \langle \mathbf{1} \rangle_{\mathbb{F}_q} \quad (\text{from [Ber00, prop. 1]}) $$

$$ (17) \qquad \mathcal{A}_r(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i - \mathbf{x}_i))^{\perp} = \text{Sh}_i(\mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp}) $$

$$ (18) \quad \text{Sh}_j(\mathcal{A}_r(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i - \mathbf{x}_i)))^{\perp} = \text{Sh}_i(\mathcal{A}_r(\mathbf{x}_j, \mathbf{y}_j(\mathbf{x}_j - \mathbf{x}_j)))^{\perp}. $$

We can summarize these relationships with the diagram below, where arrows mean an inclusion of the lower code into the upper code and two arrows pointing at same code represent the intersection. The typical code dimensions are shown too.

<figure><img src="image_4.png" /></figure>

## 4. ALGEBRAIC CRYPTANALYSIS

The previous section explains how to obtain, under some conditions, the alternant code $\mathcal{A}_3(\mathbf{x}', \mathbf{y}')$ with support $\mathbf{x}' = \mathbf{x}_{\tilde{\mathcal{I}}}$ and multiplier $\mathbf{y}' = \mathbf{y}_{\tilde{\mathcal{I}}} (\prod_{i \in \mathcal{I}} (\mathbf{x}_{\tilde{\mathcal{I}}} - \mathbf{x}_i))$ for some $\mathcal{I} \subseteq [1, n]$ such that $|\mathcal{I}| = r-3$, and with length $n' = n-r+3$ and degree 3, starting from the knowledge of the length-$n$ public code $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$. For the sake
of clarity, in this section we perform algebraic cryptanalysis on the alternant code $\mathcal{A}_3(\mathbf{x}, \mathbf{y})$ of length $n$. Essentially, we can ignore the structure of $\mathbf{y}'$ and the decreased length because the filtration preserves the support and multiplier randomness and the code distinguishability. In Section 4.8, we will see how to get back a support and a multiplier defining $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ (not necessarily $\mathbf{x}$ and $\mathbf{y}$) from a support and a multiplier defining $\mathcal{A}_3(\mathbf{x}', \mathbf{y}')$ (not necessarily $\mathbf{x}'$ and $\mathbf{y}'$). Moreover, we will focus on the case $r = 3$ for the system resolution, but the algebraic modeling we describe in Sections 4.1 and 4.2 is more general and makes sense for any $r \ge 3$. The full attack needs the filtration to reach degree 3, and therefore works specifically for $q = 2$ or $q = 3$. However, the specific modeling for $r = 3$ described in Section 4.3 is valid for any field size. We describe in Sections 4.4 and 4.5 a polynomial time attack on alternant codes of degree 3 for any $q > 2$. This result is original, and to the best of our knowledge, no polynomial time attack was known on non-structured alternant or Goppa codes even for $r = 3$. Section 4.6 contains the theoretical proofs and experimental validation for our algorithm.

For $q = 2$, we also have a polynomial time attack on random alternant codes of degree 3. However, the algorithm and the theoretical justifications are quite different from the $q > 2$ case, they involve algebraic properties of binary alternant codes and are beyond the scope of this article. We just sketch the solving algorithm in Section 4.7.

4.1. The algebraic modeling from [FGO$^{+}$13], $r \ge 3$. We give an alternative definition of alternant codes which is more suitable for this section. To this end we first need to introduce a rectangular Vandermonde-like matrix:

$$V_r(\mathbf{x}, \mathbf{y}) \stackrel{\text{def}}{=} \begin{bmatrix} y_1 & \cdots & y_n \\ y_1 x_1 & \cdots & y_n x_n \\ \vdots & \ddots & \vdots \\ y_1 x_1^{r-1} & \cdots & y_n x_n^{r-1} \end{bmatrix},$$

where $\mathbf{x} = (x_1, \dots, x_n) \in \mathbb{F}_{q^m}^n$ and $\mathbf{y} = (y_1, \dots, y_n) \in \mathbb{F}_{q^m}^n$. It is readily seen that an alternant code $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ can be expressed as the code with parity-check matrix $V_r(\mathbf{x}, \mathbf{y})$:

$$\mathcal{A}_r(\mathbf{x}, \mathbf{y}) \stackrel{\text{def}}{=} \{ \mathbf{c} \in \mathbb{F}_q^n \mid V_r(\mathbf{x}, \mathbf{y}) \mathbf{c}^T = \mathbf{0} \}$$

We will adopt the notation and follow the description of the algebraic model presented in [FGO$^{+}$13]. We denote with $\mathbf{G} = (g_{i,j}) \in \mathbb{F}_q^{k \times n}$ the $k \times n$ generator matrix of $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$. The definition of alternant codes given above implies that

$$V_r(\mathbf{x}, \mathbf{y}) \mathbf{G}^T = \mathbf{0}_{k \times n}.$$

This matrix equation translates into the following system of polynomial equations:

$$\left\{ \sum_{j=1}^{n} g_{i,j} Y_j X_j^e = 0 \mid i \in [1, k], e \in [0, r-1] \right\},$$

where $\mathbf{X} \stackrel{\text{def}}{=} \{X_1, \dots, X_n\}$ and $\mathbf{Y} \stackrel{\text{def}}{=} \{Y_1, \dots, Y_n\}$ are two blocks of $n$ unknowns, each corresponding to the support and multiplier coordinates respectively. Observe that the sought vectors $\mathbf{x}$ and $\mathbf{y}$ satisfy indeed the polynomial system.

We can assume, up to a permutation of columns, that $\mathbf{G}$ is in systematic form, i.e. $\mathbf{G} = (\mathbf{I}_k | \mathbf{P})$, where $\mathbf{I}_k$ is the identity matrix of size $k$ and $\mathbf{P} = (p_{i,j})$ for
$i \in [1, k]$, $j \in [k+1, n]$. The polynomial system can be therefore rewritten as

$$ (19) \qquad \left\{ Y_i X_i^e = - \sum_{j=k+1}^{n} p_{i,j} Y_j X_j^e \mid i \in [1, k], e \in [0, r-1] \right\}. $$

As explained in [FGO+13], thanks to the systematic form assumption, we can get rid of several variables and consider an algebraic system in only $2(n-k)$ unknowns. Indeed it is possible to exploit the trivial identity for $i \in [1, k]$

$$ Y_i(Y_i X_i^2) = (Y_i X_i)^2 $$

and replace the term $Y_i X_i^a$ for $a \in [0, 2]$ by $-\sum_{j=k+1}^{n} p_{i,j} Y_j X_j^a$ to obtain the equation $(\sum_{j=k+1}^{n} p_{i,j} Y_j) (\sum_{j=k+1}^{n} p_{i,j} Y_j X_j^2) = (\sum_{j=k+1}^{n} p_{i,j} Y_j X_j)^2$. After putting all terms on one side and expanding the products, we obtain that the polynomial

$$ f_i \stackrel{\text{def}}{=} \sum_{j,j' \in [k+1, n]} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_{j'}^2 - X_j X_{j'}) $$

must be zero on the support $\boldsymbol{x}$ and multiplier $\boldsymbol{y}$. Since

$$ f_i = \sum_{k+1 \le j < j' \le n} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_j^2 + X_{j'}^2 - 2X_j X_{j'}) $$

$$ = \sum_{k+1 \le j < j' \le n} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_j - X_{j'})^2 $$

we finally obtain the following algebraic system

$$ (20) \qquad S \stackrel{\text{def}}{=} \left\{ \sum_{k+1 \le j < j' \le n} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_j - X_{j'})^2 \mid i \in [1, k] \right\}. $$

**4.2. Reducing the number of solutions.** The ideal generated by $S$ is not zero-dimensional. It will be convenient here to reduce to this case by specializing appropriately some variables. The positive dimension of this ideal is due in the first instance to the degrees of freedom for the support and multiplier coordinates. In essence this is due to the fact that a homography $z \mapsto \frac{az+b}{cz+d}$ maps the support $\boldsymbol{x}$ of an alternant code to another support describing the same alternant code (but possibly with a different multiplier) at the condition that $cx_i + d$ never vanishes. When there exists a value $x_i$ of the support of the alternant code for which $cz_i + d = 0$, the resulting code is not an alternant code, but belongs to a slightly larger family of codes: it will be a subfield subcode of a Cauchy code. Let us recall its definition taken from [Dür87]. Given a field $\mathbb{F}$ we can identify the projective line $\mathcal{P}^1(\mathbb{F})$ with $\bar{\mathbb{F}} \stackrel{\text{def}}{=} \mathbb{F} \cup \{\infty\}$, where the symbol $\infty$ is called point at infinity, through the map $\phi: \bar{\mathbb{F}} \to \mathbb{F}^2 \setminus \{0\}$, $\phi(e) = (e, 1)$ if $e \in \mathbb{F}$ and $\phi(\infty) = (1, 0)$. Moreover, let $\mathbb{F}[W, Z]^H_l$ be the set of homogeneous polynomials of degree $l$ in two variables $W, Z$. Given $P \in \mathbb{F}[W, Z]^H_l$ and $e \in \bar{\mathbb{F}}$, we define $P(e) \stackrel{\text{def}}{=} P(\phi(e))$. Then

**Definition 25 (Cauchy code).** Let $\boldsymbol{x} \stackrel{\text{def}}{=} (x_1, \dots, x_n) \in \bar{\mathbb{F}}^n$ be a vector of distinct elements and $\boldsymbol{y} \stackrel{\text{def}}{=} (y_1, \dots, y_n) \in \mathbb{F}^n$ be a vector of nonzero elements. Let $r \in [0, n]$. The **Cauchy code** $\mathcal{C}_r(\boldsymbol{x}, \boldsymbol{y})$ is defined as

$$ \mathcal{C}_r(\boldsymbol{x}, \boldsymbol{y}) \stackrel{\text{def}}{=} \{(y_1 P(x_1), \dots, y_n P(x_n)) \mid P \in \mathbb{F}[W, Z]_{r-1}^H\}. $$
As in the case of generalized Reed-Solomon codes, $\boldsymbol{x}$ is called a *support* and $\boldsymbol{y}$ a *multiplier* of the Cauchy code.

If we assume $x_n = \infty$, a generator matrix of $\mathscr{C}_r(\boldsymbol{x}, \boldsymbol{y})$ is given by

$$ (21) \qquad \begin{bmatrix} y_1 & \cdots & y_{n-1} & 0 \\ y_1 x_1 & \cdots & y_{n-1} x_{n-1} & 0 \\ \vdots & \ddots & \vdots & \\ y_1 x_1^{r-1} & \cdots & y_{n-1} x_{n-1}^{r-1} & y_n \end{bmatrix}. $$

On the other hand, when $\boldsymbol{x} \in \mathbb{F}^n$, i.e. when all the $x_i$'s are different from $\infty$, the Cauchy code $\mathscr{C}_r(\boldsymbol{x}, \boldsymbol{y})$ can be easily seen as $\mathbf{GRS}_r(\boldsymbol{x}, \boldsymbol{y})$. They are also MDS codes. In some sense, Cauchy code are the projective linearization of GRS codes. Analogously subfield subcodes of Cauchy codes generalize subfield subcodes of GRS codes, i.e. alternant codes.

One of the main result of [Dür87] was to characterize the possible supports and multipliers of Cauchy codes. In particular, it is proven there that

**Theorem 26.** [Dür87] Let $r \in [2, n-2]$. Then $\mathscr{C}_r(\boldsymbol{x}, \boldsymbol{y}) = \mathscr{C}_r(\boldsymbol{x}', \boldsymbol{y}')$ if and only if there exists a homography $f(z) = \frac{az+b}{cz+d}$ ($a, b, c, d \in \mathbb{F}$, $ad - bc \neq 0$) such that $\boldsymbol{x}' = f(\boldsymbol{x})$ and $\boldsymbol{y}' = \lambda\theta(\boldsymbol{x})^{r-1}\boldsymbol{y}$ where $\lambda \in \mathbb{F}\setminus\{0\}$ and

$$ \theta(z) = cz + d \quad \text{if } z \in \mathbb{F} \text{ and } cz + d \neq 0, $$

$$ \theta(z) = (ad - bc)/(-c) \quad \text{if } z \in \mathbb{F} \text{ and } cz + d = 0, $$

$$ \theta(\infty) = c \quad \text{if } c \neq 0, $$

$$ \theta(\infty) = a \quad \text{if } c = 0. $$

Since the elements $a, b, c, d$ in $f(z) = \frac{az+b}{cz+d}$ are defined up to a multiplication by a nonzero scalar, Theorem 26 pragmatically implies that we are allowed to fix three variables in block $\boldsymbol{X}$ and one in block $\boldsymbol{Y}$. The price to pay for this additional specialization is that now we have to eventually handle the point at infinity. We have seen that the column corresponding to the point at infinity in the generator matrix of a Cauchy code has a special form and this changes for this coordinate the form of the system $\boldsymbol{S}$ given in (20). The problem is that we do not know a priori which $x_i$ will be infinite. To circumvent this problem, we choose the value $x_i$ that will be set to infinity, say $x_n$.

It is also convenient to make the other choices to belong to the subfield $\mathbb{F}_q$ over which the alternant code is defined: all the Gröbner bases computations will stay in the subfield and this results in slightly improved computation times. Finally we make the following choice (which also simplifies slightly the analysis of the Gröbner basis computations):

$$ (22) \qquad X_{n-2} = 0, \quad X_{n-1} = 1, \quad X_n = \infty, \quad Y_n = 1, $$

From now on, we denote with a prime the systems where the previous values have been specialized. For instance $\boldsymbol{S}'$ corresponds to $\boldsymbol{S}$ with $X_{n-2} = 0, X_{n-1} = 1, X_n = \infty, Y_n = 1$. For the particular specialization $X_n = \infty$, the last column of the Vandermonde matrix is $(0 \cdots 0 \ 1)^\top$ (see (21)), and the shape of $\boldsymbol{S}'$ is a little bit different of the one of $\boldsymbol{S}$ (see next section).

However, the set of solutions of the system $\boldsymbol{S}'$ still contains a component of positive dimension $n-k-1 = rm-1$, that corresponds to the solutions of $\{Y_j =$
$0 | k + 1 \le j \le n - 1\}$. The classical way to deal with the parasite solutions $Y_j = 0$ is to introduce to the system a new variable $T_j$ together with the polynomial $T_j Y_j - 1$. This ensures that $Y_j = 0$ is not a solution to the system. However, this also adds variables to the system, and increases the degree of the polynomials during a Gröbner basis computation. A similar phenomenon occurs with the constraints $X_j - X_{j'} \ne 0$. We solve these problems in an easier way in Steps (2) and (3) of the algorithm presented in Section 4.4.

If we substitute (22) into $\mathcal{S}$ for $r \ge 4$, we just get the same system as if we had specialized $Y_n = 0$. In the $r = 3$ case it is slightly different.

### 4.3. Specialized system $\mathcal{S}'$ in the $r = 3$ case.

**Proposition 27.** For $r = 3$, we can choose part of the support and multiplier as $X_{n-2} = 0$, $X_{n-1} = 1$, $X_n = \infty$, $Y_n = 1$ and obtain the following algebraic system (23)

$$ \mathcal{S}' \stackrel{\text{def}}{=} \left\{ \sum_{k+1 \le j < j' \le n-1} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_j - X_{j'})^2 + \sum_{j=k+1}^{n-1} p_{i,j} p_{i,n} Y_j \mid i \in [1, k] \right\}. $$

*Proof.* The fact that we can choose the support and the multiplier in this way follows from the fact that homographies act 3-transitively on the projective plane and from Theorem 26. To obtain the algebraic system we proceed similarly to what was done to obtain the algebraic system $\mathcal{S}$: this time the Vandermonde matrix is

$$ V_3(\mathbf{X}, \mathbf{Y}) \stackrel{\text{def}}{=} \left[ \begin{array}{ccc|ccc} Y_1 & \cdots & Y_{n-3} & Y_{n-2} & Y_{n-1} & 0 \\ Y_1 X_1 & \cdots & Y_{n-3} X_{n-3} & 0 & Y_{n-1} & 0 \\ Y_1 X_1^2 & \cdots & Y_{n-3} X_{n-3}^2 & 0 & Y_{n-1} & 1 \end{array} \right], $$

so that we get the relations

$$ Y_i = \sum_{k+1 \le j \le n-1} p_{i,j} Y_j $$

$$ Y_i X_i = \sum_{k+1 \le j \le n-1} p_{i,j} Y_j X_j $$

$$ Y_i X_i^2 = \sum_{k+1 \le j \le n-1} p_{i,j} Y_j X_j^2 + p_{i,n}. $$

Substituting them into the relations $Y_i(Y_i X_i^2) = (Y_i X_i)^2$ for $i \in [1, k]$ gives $\mathcal{S}'$. $\square$

Our purpose is to solve the algebraic system $\mathcal{S}'$ that is given by $k$ polynomials of bidegree $(2, 2)$ in the two blocks of variables $\mathbf{X}$ and $\mathbf{Y}$ and involves $2(n-k)$ variables. The polynomials can be expressed in terms of the $\binom{n-k}{2}$ variables

$$ (24) \qquad Z_{j,j'} = \begin{cases} Y_j Y_{j'} (X_j - X_{j'})^2 & \text{for } k+1 \le j < j' \le n-1, \\ Y_j & \text{for } j' = n, \text{ as } X_n = \infty \text{ and } Y_n = 1. \end{cases} $$

The classical way to solve such system is to first try to linearized the system using the variables $Z_{j,j'}$, for that purpose it is important to know the rank of the associated linearized system.

Its rank is trivially upper bounded by the number of expressions $Z_{j,j'}$, i.e. by $\binom{n-k}{2}$. However, in the high rate regime, [FGO+13] proposed an algebraic heuristic explaining why there is a tighter upper bound. This heuristic was confirmed by intensive computations which showed that the upper-bound was indeed tight. In
[MT21] a proof of the upper bound was given. These results lead us to make the following assumptions, given for the case $r = 3$ of interest, that will be satisfied for a random alternant code.

**Assumption 28 (Random alternant code).** We assume that $\mathcal{A}_3(\mathbf{x}, \mathbf{y})$ is in standard form, and that its dimension satisfies $k = n - rm = n - 3m$.

**Assumption 29 (High rate regime).** If $q \ge 3$, we assume that

$$ \mathrm{Rank}(S) = \mathrm{Rank}(S') = \binom{3m}{2} - m \le k. $$

Note that for $q = 2$, the rank is smaller:

**Assumption 30 (High rate regime).** If $q = 2$, we assume that

$$ \mathrm{Rank}(S) = \mathrm{Rank}(S') = \binom{3m}{2} - 3m \le k. $$

This implies that, even after the change of variables $Z_{j,j'}$, the number of unknowns is larger than the number of independent polynomials, and linearization techniques are not enough to solve the system. Therefore, in the following we are going to explain how to tackle this problem with more advanced techniques, namely Gröbner basis.

Note that for any solution $(\mathbf{x}, \mathbf{y})$ to the system, the vector $(\mathbf{x}^q, \mathbf{y}^q)$ defined by taking the $q$-st power coordinate for each coordinate is still a solution to the system. This means that we can expect at least $m$ solutions.

### 4.4. The algorithm for $q$ odd.
Generic Gröbner basis algorithms are not expected to solve efficiently systems with the same degree and same number of unknowns and equations as the one described before. Here however, some expedients can be taken into account to exploit the very strong algebraic structure and specific shape of the equations involved.

We present here an ad hoc algorithm based on Gröbner basis techniques, that recovers the secret key in polynomial time. We start from the system $S'$ given in (23), under Assumptions 28 and 29, and we agglomerate in our strategy the constraints $Y_j \ne 0$ and $X_j - X_{j'} \ne 0$. This section deals with the odd case (i.e. when $q$ is the power of an odd prime), and next one with the even case $q = 2^s$ with $s > 1$. This covers for instance the case $q = 3$, for which a full key recovery can be achieved, thanks to the filtration of alternant codes. The proofs and detailed explanations are postponed in Section 4.6. Our algorithm consists of the following steps:

(1) **Echelonizing system $S'$** We compute a basis of the $\mathbb{F}_q$-vector space $\mathcal{S}'$ generated by $S'$. It has the shape

$$ (25) \qquad \left\{ \begin{array}{ll} Y_j Y_{j'} (X_j - X_{j'})^2 + L_{j,j'} (Y_\ell : \ell \in I) & \forall k+1 \le j < j' \le n-1 \\ Y_{n-2} Y_{n-1} + L_{n-2,n-1} (Y_\ell : \ell \in I) & \\ Y_j - L_{j,n} (Y_\ell : \ell \in I) & \forall k+1 \le j \le n-1, j \notin I. \end{array} \right. $$

where $I \subset [k+1, n-1]$ has size $m$, and the $L_{j,j'}$'s are linear functions of the $Y_\ell$'s, $\ell \in I$. In particular, it contains $2m-1$ homogeneous linear polynomials in $Y$, that come from the choice $X_n = \infty$ and $Y_n = 1$ (see Proposition 32 for the proof). This can be done in $O(m^{2\omega})$ operations in $\mathbb{F}_q$, where $\omega$ is the exponent of linear algebra.
(2) **Removing the $Y_j = 0$ component** For each $j \in [k+1, n-1]$, we prove that there exists a set of $2m-1$ linearly independent polynomials in $\mathcal{S}'$ that are multiple of $Y_j$. As we know that our solution satisfies $Y_j \neq 0$, we add to the system the set $\mathcal{V}_j$ of these polynomials divided by $Y_j$ (see Proposition 33 for details and proof). This has the effect to add $(2m-1)(3m-1)$ linearly independent polynomials of degree 3 to the system, and to remove the non-zero-dimensional component from the solution set. Note that Step (1) corresponds to the computation of $\mathcal{V}_n$, as $Y_n = 1$. The cost for all $j$ is $O(m^{\omega+1})$.

(3) **Adding bilinear polynomials** For each $j \in [k+1, n-1] \setminus \{n-2\}$, we consider the vector spaces $\mathcal{U}_{j,n-2}$ formed by the polynomials $p$ such that $X_j p \in \mathcal{Y}_j + \mathcal{Y}_{n-2}$, where $\mathcal{Y}_j$ is the $\mathbb{F}_q$-vector space generated by $\mathcal{V}_j$. We prove in Proposition 34 that $\dim_{\mathbb{F}_q}(\mathcal{U}_{j,n-2}) \ge m$. Experimentally, this set has dimension exactly $m$. As we know that our solution satisfies $x_j \neq 0 = x_{n-2}$, we add to the system a basis $\mathcal{U}_{j,n-2}$ of $\mathcal{U}_{j,n-2}$. This has the effect to add (at least) $m(3m-2)$ linearly independent polynomials of degree 2 to the system. The cost for all $j$ is $O(m^{\omega+1})$.

(4) **Eliminating $2m-1$ variables $Y_j$ using the linear polynomials from Step (1)** We now eliminate $2m-1$ variables $Y$ from the polynomials in $\mathcal{U}_{j,n-2}$ using the $2m-1$ homogeneous linear polynomials in $Y$ from Step (1). This step is heuristic, and verified experimentally: the resulting system admits a basis with the shape:

$$ \left\{ \begin{array}{ll} Y_j X_{j'} + f_{j,j'} (Y_\ell : \ell \in I, 1), & \text{for all } j \in I, j' \in [k+1, n-3], \\ \sum_{j \in I} a_j Y_j - 1, & \text{with } a_j \in \mathbb{F}_q. \end{array} \right. $$

where the $f_{j,j'}$ are linear functions of the $Y_\ell$'s. The basis contains an affine linear polynomial in $Y$, so that we get in total all $2m$ linearly independent linear polynomials in $Y_j$ that we can expect (see Proposition 31). The cost of this step is $O(m^{2\omega})$.

(5) **Computing linear polynomials in the $X$ variables** Provided that Step (4) occurred as described, we deduce for each $j$ one affine linear polynomial expressing $X_j$ in terms of the $Y$'s (see Proposition 38):

$$ \left\{ X_{j'} + \sum_{j \in I} a_j f_{j,j'} (Y_\ell : \ell \in I, 1) \mid j' \in [k+1, n-3] \right\}. $$

The cost is $O(m^2)$.

(6) **Computing the Gröbner basis** By eliminating the $X_i$'s from the polynomials $Y_j X_{j'} + f_{j,j'} (Y_\ell : \ell \in I, 1)$, we get the final grevlex Gröbner basis of the system, that is experimentally

$$ \left\{ \begin{array}{ll} Y_j Y_{j'} - L'_{j,j'} (Y_i : i \in I_1, 1) & j, j' \in I_1 \\ X_j - L'_{X_j} (Y_i : i \in I_1, 1) & j \in [k+1, n-3] \\ Y_j - L'_{Y_j} (Y_i : i \in I_1, 1) & j \notin I_1 \end{array} \right. $$

for a set $I_1 \subset [k+1, n-1]$ of size $m-1$, where the functions $L'$ are affine linear function. This describes a variety of dimension 0 with $m$ solutions, that are exactly the $m$ solutions obtained by applying the Frobenius morphism. The cost is $O(m^{2\omega})$.
(7) **Computing the solutions** The lex basis can be obtained using the FGLM Algorithm from [FGLM93] with $O(m^4)$ operations in $\mathbb{F}_q$, and allows to retrieve the $m$ solutions by factorization of a polynomial over $\mathbb{F}_q$ of degree $\le m$.

(8) The final step consists in retrieving separately the values for $Y_1, \dots, Y_k$ and $X_1, \dots, X_k$ from (19) for $e=0$ and $e=1$. This costs $O(nm)$ operations in $\mathbb{F}_q$. This needs to be done only once, since with the chosen specialization any of the $m$ solutions is a valid pair of support and multiplier coordinates, thus we can choose arbitrarily any of them.

Note that all steps are just linear algebra, and the total complexity is polynomial in $m$ and $n$, the global cost being $O(m^{2\omega} + nm)$ as $m, n \to \infty$.

**4.5. The algorithm for $q$ even, $q > 2$.** We present here the differences for the case of characteristic 2. We still assume Assumptions 28 and 29. Note that the case $q=2$ is very different, we will consider this case later in Section 4.7. We assume here that $q = 2^s$ with $s > 1$.

First of all, in this case the system $S'$ can be rewritten as

$$S' = \left\{ \sum_{k+1 \le j < j' \le n-1} p_{i,j} p_{i,j'} Y_j Y_{j'} (X_j^2 + X_{j'}^2) + \sum_{j=k+1}^{n-1} p_{i,j} p_{i,n} Y_j \mid i \in [1, k] \right\}.$$

Since the $X_j$'s variables appear in the system with power 2 only, we can perform a change of variables by defining $W_j \stackrel{\text{def}}{=} X_j^2$, so that the system becomes

$$(26) \quad S_2' = \left\{ \sum_{k+1 \le j < j' \le n-1} p_{i,j} p_{i,j'} Y_j Y_{j'} (W_j + W_{j'}) + \sum_{j=k+1}^{n-1} p_{i,j} p_{i,n} Y_j \mid i \in [1, k] \right\}.$$

Therefore, polynomials in $S_2'$ have bidegree (1,2) in $W \stackrel{\text{def}}{=} \{W_1, \dots, W_n\}$ and $Y$ respectively. This simple trick provides an effective speed up to the resolution.

Steps (1)-(2) are identical, we compute for each $j \in [k+1, n-1]$ the vector space $\mathcal{V}_j$ that has dimension $2m-1$. The difference is that the polynomials in $\mathcal{V}_j$ for $j \le n-1$ are linear combination of the monomials $Y_{j'}(W_j + W_{j'})$, $j' \in [k+1, n-1]$, $j' \ne j$, and 1, hence already bilinear. Step (3) is unnecessary.

Step (4): using the linear polynomials in $\mathcal{V}_n$, we can directly eliminate $2m-1$ variables $Y_j$ from the $(3m-1)(2m-1)$ bilinear polynomials in $\cup_{j=k+1}^{n-1} \mathcal{V}_j$. We obtain experimentally a basis of the shape

$$(27) \quad \begin{cases} Y_j W_{j'} + g_{j,j'} (Y_\ell : \ell \in I, 1), & \text{for all } j \in I, j' \in [k+1, n-3], \\ \sum_{j \in I} a_j Y_j - 1, & \text{with } a_j \in \mathbb{F}_q. \end{cases}$$

Steps (5)-(8) are identical to the odd case. Remark that retrieving the value $x_j$ from the values $w_j$ is just a square root computation in characteristic 2.

The final asymptotic cost is the same, $O(m^{2\omega} + nm)$ as $n, m \to \infty$.

**4.6. Theoretical and experimental validation of the algebraic algorithm.** We start with a property that will be useful to determine the number of linearly independent linear polynomials in $S'$, for any $q$ and any $r \ge 3$. Recall that we assumed without loss of generality that $y_n = 1$.
**Proposition 31.** Let $\mathcal{C}$ be the $\mathbb{F}_{q^m}$ linear code generated by $(y_{k+1}, \dots, y_{n-1})$ in $\mathbb{F}_{q^m}^{mr-1}$. Then, under Assumption 28, we have

$$ \dim_{\mathbb{F}_q}(\mathrm{Tr}(\mathcal{C})) = m, \\ \dim_{\mathbb{F}_q}(\mathrm{Tr}(\mathcal{C})^{\perp}) = (r-1)m-1. $$

As any $\mathbb{F}_q$-linear combination of the $y_j$'s that is equal to zero provides a codeword in $\mathrm{Tr}(\mathcal{C})^{\perp}$, therefore there cannot be more than $(r-1)m-1$ linearly independent homogeneous $\mathbb{F}_q$-linear polynomials in $Y_{k+1}, \dots, Y_{n-1}$ which cancel on $y_{k+1}, \dots, y_{n-1}$, and no more than $(r-1)m$ linearly independent affine $\mathbb{F}_q$-linear polynomials in $Y_{k+1}, \dots, Y_{n-1}$ that cancel on $(y_{k+1}, \dots, y_{n-1}, 1)$.

Equivalently, for all $j \in [k+1, n-1]$, the code $\mathcal{C}'_j \subset \mathbb{F}_{q^m}^{mr-1}$ generated by $(y_{k+1}(x_{k+1} - x_j)^{r-1}, \dots, y_{n-1}(x_{n-1} - x_j)^{r-1}, 1) \in \mathbb{F}_{q^m}^{mr}$ and punctured in position $j-k$ satisfies

$$ \dim_{\mathbb{F}_q}(\mathrm{Tr}(\mathcal{C}'_j)) = m, \\ \dim_{\mathbb{F}_q}(\mathrm{Tr}(\mathcal{C}'_j)^{\perp}) = (r-1)m-1. $$

*Proof.* If the code $\mathcal{A}_r(\mathbf{x}, \mathbf{y})$ has dimension $k = n-mr$ and is in standard form, then the last $n-k = mr$ columns of its parity-check matrix must be an information set, i.e. the last $mr$ columns of $V_r(\mathbf{x}, \mathbf{y})$ must generate a trace code with dimension $mr$. This means in particular that the first row $(y_{k+1}, \dots, y_{n-1}, 0)$ must have rank weight $m$, and this is the same for all $r$ rows. Then, by elementary combination of rows, the trace code of the following code must still have dimension $m$:

$$ (0 \quad y_{k+2}(x_{k+2} - x_{k+1})^{r-1} \quad \dots \quad y_{n-1}(x_{n-1} - x_{k+1})^{r-1} \quad 1) $$

and this can be done for any $x_j$ instead of $x_{k+1}$, hence the proposition. $\square$

<figure><img src="image_5.png" /></figure>

We now give details for our algorithm for $r = 3, q \ge 3$.

*Step (1): Echelonizing system $S'$.* We linearize the set of polynomials (23), by replacing “polynomials” by variables, instead of classically replacing any monomial by a new variable. The variables we consider are:

$$ Z_{j,j'} = \begin{cases} Y_j Y_{j'} (X_j - X_{j'})^2 & \text{for } k+1 \le j < j' \le n-1 \\ Y_j & \text{for } j \in [k+1, n-1], j' = n. \end{cases} $$

**Proposition 32.** Assume that $q \ge 3$. Under Assumptions 28 and 29, for any set $I \subset [k+1, n-1]$ of size $m$ such that $\dim_{\mathbb{F}_q}(\langle y_\ell : \ell \in I \rangle_{\mathbb{F}_q}) = m$, a basis of $S'$ is given by

$$ \left\{ \begin{array}{ll} Y_j Y_{j'} (X_j - X_{j'})^2 + L_{j,j'} (Y_\ell : \ell \in I) & j, j' \in [k+1, n-1], j < j' \\ Y_{n-2} Y_{n-1} + L_{n-2,n-1} (Y_\ell : \ell \in I) & \\ Y_j - L_{j,n} (Y_\ell : \ell \in I) & \forall j \in [k+1, n-1] \setminus I. \end{array} \right. $$

where the $L_{j,j'}$ are linear functions of the $Y_\ell$'s, $\ell \in I$ (note that $L_{j,j'}$ implicitly depends on $I$). This basis can be computed in time $O(m^{2\omega})$ where $\omega$ is the constant of linear algebra.

For $q$ even, we get the same basis for $S'_2$ with the terms $(X_j - X_{j'})^2$ replaced by $W_j + W_{j'}$.
**Proof.** We have $\binom{3m}{2} - m$ polynomials in $\binom{3m}{2}$ variables. Among the variables, $3m-1$ are of degree 1 (the $Y_j$'s for $k+1 \le j \le n-1$), one is of degree 2 ($Z_{n-2,n-1} = -Y_{n-2}Y_{n-1}$, as $X_{n-2} = 0$ and $X_{n-1} = 1$) and the last $\binom{3m}{2} - 3m$ are of degree 4. We can eliminate from the system all terms of degree 4 and 2. As the polynomials are linearly independent, we get at least $2m-1$ linear polynomials in the $Y_j$'s.

As by Proposition 31, we have at most $2m-1$ linear relations between the $Y_{k+1}, \dots, Y_{n-1}$, hence we have exactly $2m-1$ linear polynomials in the $Y_j$'s expressing any $Y_j$ in terms of the $\{Y_\ell : \ell \in I\}$ for some $I \subset [k+1, n-1]$ of size $m$, and all other polynomials express the terms of degree $\ge 2$ in terms of the $\{Y_\ell : \ell \in I\}$.

To compute the basis it is enough to compute an echelon form of a matrix of size $\binom{3m}{2} - m \times \binom{3m}{2}$, the cost is $O(m^{2\omega})$. $\square$

**Step (2): removing the $Y_j = 0$ component.** The linear polynomials we get come from the fact that we have specialized the $n$-th component to $x_n = \infty$ and $y_n = 1$. Here we show that it is equivalently possible to introduce the constraint $Y_j \ne 0$ for all $j \in [k+1, n-1]$. Let $\mathbf{X} = (X_{k+1}, \dots, X_{n-3})$ and $\mathbf{Y} = (Y_{k+1}, \dots, Y_{n-1})$. We define the vector spaces

$$ (28) \qquad \mathcal{Y}_j = \frac{1}{Y_j} (\mathcal{S}' \cap Y_j \cdot \mathbb{F}_q[\mathbf{X}, \mathbf{Y}]_{\le 3}), \qquad j \in [k+1, n-1] $$

that is $\mathcal{Y}_j \stackrel{\text{def}}{=} \langle \frac{h_1}{Y_j}, \dots, \frac{h_\ell}{Y_j} \rangle_{\mathbb{F}_q}$ where $\{h_1, \dots, h_\ell\}$ is a basis of $\mathcal{S}' \cap (Y_j \cdot \mathbb{F}_q[\mathbf{X}, \mathbf{Y}]_{\le 3})$. We also define

$$ (29) \qquad \mathcal{Y}_n = \langle Y_j - L_{j,n}(Y_\ell, \ell \in I) \rangle_{j \in \{k+1, \dots, n-1\}, j \notin I}. $$

**Proposition 33.** Under Assumptions (28) and (29), for any $q \ge 3$,

$$ (30) \qquad \dim_{\mathbb{F}_q}(\mathcal{Y}_j) = 2m-1 \qquad \forall j \in [k+1, n] $$

and any polynomial in $\mathcal{Y}_j$ is a linear combination of the $3m-1$ terms

$$ \left\{ \begin{array}{l} Y_{j'}(X_j - X_{j'})^2, \quad j' \in [k+1, n-1], j' \ne j \\ 1 \end{array} \right. $$

We also have, for each $j_1, j_2 \in [k+1, n-1]$ with $j_1 \ne j_2$:

$$ \dim_{\mathbb{F}_q}(\mathcal{Y}_{j_1} + \mathcal{Y}_{j_2}) = 4m-2. $$

A basis $\mathcal{V}_j$ of $\mathcal{Y}_j$ can be computed in time $O(m^\omega)$ from the basis (25) of $\mathcal{S}'$ and the set of all $\mathcal{V}_j$'s can be computed in time $O(m^{\omega+1})$.

**Proof.** Choose any set $I \subset [k+1, n-1]$ of size $m$ such that $\dim_{\mathbb{F}_q}(\langle (y_\ell)_{\ell \in I} \rangle_{\mathbb{F}_{q^m}}) = m$.

a) Consider first the case where $j \notin I$, and $j \le n-3$. To compute $\mathcal{Y}_j$, we just take the polynomials in (25) that contain $Y_j$, they are the $3m-1$ polynomials:

$$ \left\{ \begin{array}{l} Y_j Y_{j'} (X_j - X_{j'})^2 + L_{j,j'} (Y_\ell : \ell \in I) \quad \text{for } j' \in [k+1, n-1], j' \ne j \\ Y_j - L_{j,n} (Y_\ell : \ell \in I). \end{array} \right. $$

We have $3m-1$ linearly independent polynomials in $m$ variables $Y_\ell : \ell \in I$ and $3m-1$ variables $Y_j Y_{j'} (X_j - X_{j'})^2$ and $Y_j$. By eliminating the $Y_\ell : \ell \in I$ we get at least $2m-1$ polynomials that are multiple of $Y_j$.
b) If $j \in I$, $j \le m-3$, we have $\dim_{\mathbb{F}_q} (\langle (y_i)_{i \in [k+1,n-1]} \rangle_{\mathbb{F}_{q^m}}) \in \{m-1, m\}$. If the dimension over $\mathbb{F}_q$ is $m$, then we can take a different set $I$ that generates a $\mathbb{F}_q$-vector space $\langle (y_i)_{i \in I} \rangle_{\mathbb{F}_{q^m}}$ of dimension $m$ such that $j \notin I$. If the dimension over $\mathbb{F}_q$ is $m-1$, then for any $j' \in [k+1, n-1] \setminus I$, the linear polynomials $Y_{j'} - L_{j',n} (Y_\ell : \ell \in I)$ does not involve $Y_j$ (or we would have a linear polynomial expressing $Y_j$ as a $\mathbb{F}_q$-linear combination of the terms $Y_{j'}$ and $Y_\ell : \ell \in I \setminus \{j\}$, which is impossible considering that $\dim_{\mathbb{F}_q} (\langle (y_i)_{i \in [k+1,n-1]} \rangle_{\mathbb{F}_{q^m}}) = m$). In this case, we take the $3m-2$ polynomials involving $(Y_j Y_{j'} (X_j - X_{j'}))^2$ for all $j' \neq j$, they contains those $3m-2$ terms, the variable $Y_j$ and $m-1$ variables $Y_\ell : \ell \in I, \ell \neq j$. By eliminating the $Y_\ell, \ell \in I, \ell \neq j$ we get at least $2m-1$ linear polynomials multiple of $Y_j$.

c) For $j \in [n-2, n-1]$, it is exactly the same as a) and b), but with one more polynomial involving one more variable $Y_{n-2}Y_{n-1}$.

In any case, after dividing by $Y_j$, we get at least $2m-1$ polynomials involving the monomials $Y_{j'}(X_{j'} - X_j)^2$ for $j' \in [k+1, n-1] \setminus \{j\}$ and $1$, and all those polynomials evaluate to zero on the support and multiplier of the code. Now, according to Proposition 31 with $r=3$, $\dim_{\mathbb{F}_q}(\mathrm{Tr}(\mathcal{C}'_j)) = m$, where $\mathcal{C}'_j$ is the code of length $3m-1$ generated by $(y_{k+1}(x_{k+1} - x_j)^2, \dots, y_{n-1}(x_{n-1} - x_j)^2, 1)$ and punctured in position $j-k$ (where its value is 0). Hence there can be at most $2m-1$ linear polynomials between the $3m-1$ terms 1 and $Y_{j'}(X_{j'} - X_j)^2$, $j' \in [k+1, n-1]$, $j' \neq j$.

Finally, the polynomials in two different $\mathcal{Y}_j$'s are linearly independent, as the only common polynomial that could belong simultaneously to two different $\mathcal{Y}_j$'s is 1, and the ideal is not generated by 1. $\square$

**Step (3): adding bilinear polynomials.** The system given by the union of $S'$ and the $(3m-1)(2m-1)$ cubic polynomials $\mathcal{V}_j, j \in [k+1, n-1]$ determined at Step (2) generates a zero-dimensional ideal, whose variety contains exactly $m$ solutions. It would be enough to run a Gröbner basis for this new system, in order to retrieve the support and multiplier. However, specifically for the $q$ odd case, we are able to deepen the analysis and use efficiently the constraints about support coordinates, i.e. $X_{j_1} - X_{j_2} \neq 0$, by computing efficiently a set of bilinear polynomials. The latter do not refine the variety, this one being already finite, but their prediction allows to speed up the computation.

**Proposition 34.** The vector space $\mathcal{U}_{n-2,n-1} \stackrel{\text{def}}{=} (\mathcal{Y}_{n-2} + \mathcal{Y}_{n-1}) \cap \mathbb{F}_q[\mathbf{X}, \mathbf{Y}]_{\le 2}$ contains more than $m$ linearly independent polynomials of degree 2, that are linear combination of the terms $Y_j(2X_j - 1)$ for $j \in [k+1, n-1]$. We denote by $\mathcal{U}_{n-2,n-1}$ this set of $m$ polynomials.

Moreover, $\mathcal{Y}_{n-2} + \mathcal{Y}_{n-1}$ contains an additional polynomial of degree 2 expressing the monomial 1 in terms of the $Y_j(2X_j - 1)$ for $j \in [k+1, n-1]$, denoted by $u_{n-2,n-1}$.

If the characteristic of the field is 2, the $m+1$ polynomials are linear in $Y_j$ and 1. The first $m$ polynomials in $\mathcal{U}_{n-2,n-1}$ already belong to $\mathcal{Y}_n$, but the additional polynomial $u_{n-2,n-1}$ generates together with $\mathcal{V}_n$ a vector space of dimension $2m$ (hence one more linear polynomial than the ones in $\mathcal{V}_n$).

*Proof.* The terms appearing in the polynomials in $\mathcal{V}_{n-2}$ are $1, Y_{n-1}$ and $Y_j X_j^2$ for $j \in [k+1, n-3]$. The ones in $\mathcal{V}_{n-1}$ are $1, Y_{n-2}$ and $Y_j X_j^2 + Y_j(1-2X_j)$, $j \in [k+1, n-3]$. This means that the polynomials in $\mathcal{W}_{n-2,n-1}$ can all be expressed as linear combination of the $3m-3$ monomials $Y_j X_j^2$ of degree 3, the monomial
1 of degree 0, plus $3m-1$ terms of degree at most 2: the monomials $Y_{n-1}$, $Y_{n-2}$ and the $Y_j(2X_j - 1)$'s, $j \in \llbracket k+1, n-3 \rrbracket$. The dimension of the vector space $\mathcal{U}_{n-2,n-1}$ is $4m-2$, so that we get at least $m$ linearly independent polynomials of degree 2 in $\mathcal{U}_{n-2,n-1}$ that are combination of $3m-1$ terms ($Y_{k+1}(2X_{k+1} - 1), \dots, Y_{n-3}(2X_{n-3} - 1), Y_{n-2}, Y_{n-1}$). If the characteristic of the field is 2, then we get $m$ linear polynomials in $Y$, that are already in $\mathcal{Y}_n$ according to Proposition 31.

In all cases, we also get an additional polynomial of degree 2 involving these $3m-1$ terms and the monomial 1, and this gives in characteristic 2 an additional affine linear polynomial in $Y$. $\square$

This can be generalized to the following vector spaces. For any $k+1 \le j_1 < j_2 \le n-1$, define the vector space

$$ (31) \qquad \mathcal{U}_{j_1, j_2} = \frac{1}{X_{j_1} - X_{j_2}} ((\mathcal{Y}_{j_1} + \mathcal{Y}_{j_2}) \cap (X_{j_1} - X_{j_2}) \cdot \mathbb{F}_q[\mathbf{X}, \mathbf{Y}]_{\le 2}) $$

that consists of the polynomials $p$ such that $p(X_{j_1} - X_{j_2}) \in \mathcal{Y}_{j_1} + \mathcal{Y}_{j_2}$.

**Proposition 35.** For any $k+1 \le j_1 < j_2 \le n-1$, $(j_1, j_2) \ne (n-2, n-1)$, we have $\dim(\mathcal{U}_{j_1, j_2}) \ge m$, and the polynomials in $\mathcal{U}_{j_1, j_2}$ are linear combination of the following terms:

$$ Y_j(2X_j - X_{j_1} - X_{j_2}), \qquad j \in \llbracket k+1, n-1 \rrbracket. $$

*Proof.* The $2m-1$ polynomials in $\mathcal{V}_{j_1}$ contains the following $3m-1$ terms:

$$ \begin{cases} Y_j(X_j - X_{j_1})^2 & j \in \llbracket k+1, n-1 \rrbracket, j \ne j_1 \\ 1 & \end{cases} $$

It is the same for $\mathcal{V}_{j_2}$, but we can rewrite, for $j \in \llbracket k+1, n-1 \rrbracket, j \ne j_1, j_2$:

$$ Y_j(X_j - X_{j_2})^2 = Y_j(X_j - X_{j_1})^2 + Y_j(2X_j - X_{j_1} - X_{j_2})(X_{j_1} - X_{j_2}), $$

so that the $4m-2$ polynomials generating $\mathcal{Y}_{j_1} + \mathcal{Y}_{j_2}$ can be written in terms of the following terms:

$$ \left\{ \begin{array}{l} Y_j(X_j - X_{j_1})^2 \\ 1 \\ Y_{j_1}(X_{j_1} - X_{j_2})^2 \\ Y_{j_2}(X_{j_1} - X_{j_2})^2 \\ Y_j(2X_j - X_{j_1} - X_{j_2})(X_{j_1} - X_{j_2}), \quad j \in \llbracket k+1, n-1 \rrbracket, j \ne j_1, j_2. \end{array} \right. \qquad j \in \llbracket k+1, n-1 \rrbracket, j \ne j_1, j_2, $$

If we eliminate the $3m-2$ first terms that are not multiple of $X_{j_1} - X_{j_2}$, we get at least $m$ linearly independent polynomials that are multiple of $X_{j_1} - X_{j_2}$. After division by $X_{j_1} - X_{j_2}$, the polynomials are linear combination of the $3m-1$ terms $Y_j(2X_j - X_{j_1} - X_{j_2})$ for $j \in \llbracket k+1, n-1 \rrbracket$. $\square$

*Remark 36.* If the characteristic of the field is 2, then we can divide directly by $X_{j_1}^2 + X_{j_2}^2$ to get $m$ linear polynomials in the $Y_j$'s. As we cannot have more than $2m-1$ homogeneous linear polynomials in the $Y_j$'s if the multipliers $y_j$ generate $\mathbb{F}_{q^m}$ over $\mathbb{F}_q$, then all the new polynomials are linearly dependent from the previous ones.

Despite the existence of a quadratic number of vector spaces $\mathcal{U}_{j_1, j_2}$, in practice it is sufficient to exploit the polynomials derived from the $3m-2$ subspaces $\mathcal{U}_{\ell, n-2}$, $\ell \in \llbracket k+1, n-1 \rrbracket \setminus \{n-2\}$, thus reducing the complexity of this step.
Fact 37. Experimentally, $\dim(\mathcal{W}_{j_1, j_2}) = m$ and for odd $q$,

$$ \dim_{\mathbb{F}_q}(\cup_{\ell \in \llbracket k+1, n-1 \rrbracket, \ell \neq n-2} \mathcal{W}_{\ell, n-2}) = m(3m-2). $$

Step (4): eliminating $2m-1$ variables $Y_j$ using the linear polynomials from Step (1). We consider the case $q$ odd. Assuming Fact 37, the system $\oplus_{j_1 \in \llbracket k+1, n-3 \rrbracket} \mathcal{U}_{j_1, n-2}$ contains $m(3m-2)$ linearly independent polynomials, and they can all be expressed as linear combination of the monomials $Y_j, j \in \llbracket k+1, n-1 \rrbracket$ and $Y_j X_{j'}$ for $j \in \llbracket k+1, n-1 \rrbracket$ and $j' \in \llbracket k+1, n-3 \rrbracket$.

The system $\mathcal{V}_n$ contains $2m-1$ homogeneous linear polynomials in the $Y_j$'s, expressing the $Y_i$'s for $i \notin I$ in term of the $Y_\ell$'s for $\ell \in I$. If we use them to eliminate the $2m-1$ variables $Y_i$'s ($i \notin I$) from the polynomials in $\oplus_{j_1 \in \llbracket k+1, n-3 \rrbracket} \mathcal{U}_{j_1, n-2}$, we are left with polynomials that are linear combination of $m$ linear monomials $\{Y_\ell : \ell \in I\}$, and $m(3m-3)$ quadratic monomials $Y_j X_{j'}$ for $j \in I$ and $j' \in \llbracket k+1, n-3 \rrbracket$. This means that we have as many polynomials as monomials. However, the polynomials now have no reason to remain linearly independent, and in fact they are not.

Experimentally, after linearization, we get one polynomial expressing each quadratic term $Y_j X_{j'}$ in terms of the $m$ independent $\{Y_\ell : \ell \in I\}$ and $m$ reductions to zero, as we cannot get more than $2m-1$ linear polynomials relating the $Y_j$'s: a basis $\mathcal{U}$ of $\oplus_{j_1} \mathcal{W}_{j_1, n-2}$ modulo $\mathcal{V}_n$ has the shape

$$ \mathcal{U} \stackrel{\text{def}}{=} \{ Y_j X_{j'} + f_{j,j'} (Y_\ell : \ell \in I) \mid j \in I, j' \in \llbracket k+1, n-3 \rrbracket \}. $$

We can now use the polynomial $u_{n-2,n-1}$ from Proposition 34, that is a linear combination of the monomials $1, Y_{n-2}, Y_{n-1}$ and the $Y_j(2X_j-1)$ for $j \in \llbracket k+1, n-3 \rrbracket$. We eliminate the $Y_j$'s, $j \notin I$ using equations in $\mathcal{V}_n$ and the $Y_j X_{j'}$ for $j \in I, j' \in \llbracket k+1, n-3 \rrbracket$ using $\mathcal{U}$ and obtain a linear polynomials in the $Y_j$'s and 1. Note that, as we already have $2m-1$ homogeneous polynomials between the $Y_j$'s, we cannot have another homogeneous polynomials, hence the polynomials contains the constant 1.

To perform the elimination and the linearization, we can perform linear algebra on a matrix where the columns are the $Y_j X_{j'}$, hence $\mathcal{O}(m^2)$ columns, and the rows are the basis for $\mathcal{W}_{j,n-2}$ and the $X_i L_j$ with $L_j$ a linear polynomial in $\mathcal{V}_n$. This makes $\mathcal{O}(m^2)$ rows, and a complexity in $\mathcal{O}(m^{2\omega})$.

Step (5): computing linear polynomials for the $X$ variables.

**Proposition 38.** Assume that a basis of $\cup_{j \neq n-2} \mathcal{W}_{j, n-2}$ where the linear polynomials from $\mathcal{V}_n$ have been eliminated is given by

$$ (32) \qquad \left\{ \begin{array}{ll} Y_j X_{j'} + f_{j,j'} (Y_\ell : \ell \in I, 1), & \text{for all } j \in I, j' \in \llbracket k+1, n-3 \rrbracket, \\ \sum_{j \in I} a_j Y_j - 1, & \text{with } a_j \in \mathbb{F}_q. \end{array} \right. $$

Then the vector space generated by the polynomials (32) contains the polynomials

$$ (33) \qquad X_{j'} + \sum_{j \in I} a_j f_{j,j'} (Y_\ell : \ell \in I, 1), \qquad j' \in \llbracket k+1, n-3 \rrbracket. $$

*Proof.* We have

$$ \sum_{j \in I} a_j (Y_j X_{j'} + f_{j,j'} (Y_\ell : \ell \in I, 1)) = X_{j'} + X_{j'} \left( \sum_{j \in I} a_j Y_j - 1 \right) + \sum_{j \in I} a_j f_{j,j'} (Y_\ell : \ell \in I, 1) $$

so that we get in the ideal generated by (32) one affine linear polynomial expressing each $X_{j'}$ in terms of the $\{Y_\ell : \ell \in I, 1\}$. $\square$
**Step (6): the final Gröbner basis.** Now, if we use the polynomials in (33) to eliminate the $X_i$'s from the polynomials in (32), we get one linear polynomial for each term of degree 2 in $\mathbf{Y}$. Let $I_1$ be the set $I$ minus one element $i \in I$ such that $a_i \neq 0$. The final basis has the shape

$$ 
\left\{ 
\begin{array}{ll}
Y_j Y_{j'} + L'_{j,j'} (Y_\ell : \ell \in I_1, 1), & j, j' \in I_1, j < j', \\
X_{j'} + f_{j'} (Y_\ell : \ell \in I_1, 1), & j' \in [k+1, n-3], \\
Y_j + L_{j,n} (Y_\ell : \ell \in I_1), & j \notin I, \\
Y_i + L_{i,n} (Y_\ell : \ell \in I_1, 1), & i \in I \setminus I_1.
\end{array}
\right.
$$

This describes a variety of dimension 0 with $m$ solutions (as $\#I_1 = m - 1$, only $m$ monomials 1 and $Y_i : i \in I_1$ are not leading term of a polynomial in the ideal), that are exactly the $m$ solutions obtained by applying the Frobenius morphism. This proves that the basis is a Gröbner basis. It coincides with the basis that would have been computed from $S'$ plus the cubic polynomials from the $V_j$'s. However, our approach is more efficient, because it avoids unnecessary calculations.

**4.7. The particular $q = 2$ case.** Assumption 30 asserts that the rank of $S'$ is smaller than for all the other cases, namely $\text{Rank}(S') = \binom{3m}{2} - 3m$ instead of $\binom{3m}{2} - m$. This invalidates all the combinatorial arguments for the dimensions of $V_j$'s and for the number of degree falls. In this case we have

$$ \dim(V_j) = m - 1, $$

In particular for $j = n$, the number of independent linear polynomials in $\mathbf{Y}$ in a basis of $S'$ is $m - 1$. This result is rather technical, its essence not being merely combinatorial, and is beyond the scope of this article.

More interestingly, for the specific $q = 2$ case, we can use the following particular identity from [FGO$^{+}$13]:

$$ (34) \qquad (Y_i X_i^2)(Y_i)^2 = (Y_i)(Y_i X_i)^2. $$

Replacing as before the terms $Y_i X_i^a$ for $a \in [0, 2]$ by their values from (19) and, as $q = 2$, the terms $(Y_i X_i^a)^2$ for $a \in [0, 1]$ by $(Y_i X_i^a)^2 = - \sum_{j=k+1}^n p_{i,j} Y_j^2 X_j^{2a}$ leads, after expansion, to the system

$$ (35) \qquad \mathcal{B} \stackrel{\text{def}}{=} \left\{ \sum_{k+1 \le j < j' \le n} p_{i,j} p_{i,j'} Y_j Y_{j'} (Y_j + Y_{j'}) (X_j + X_{j'})^2 = 0 \mid i \in [1, k] \right\}. $$

**Proposition 39.** For $r = 3$, $q = 2$ with the specialization (22), the system $\mathcal{B}$ becomes

$$ (36) \qquad \mathcal{B}' = \left\{ \sum_{\substack{j,j'=k+1 \\ j < j'}}^{n-1} p_{i,j} p_{i,j'} Y_j Y_{j'} (Y_j + Y_{j'}) (X_j + X_{j'})^2 + \sum_{j=k+1}^{n-1} p_{i,j} p_{i,n} Y_j^2 \mid i \in [1, k] \right\}. $$

*Proof.* For $q = 2$ we use the identity $(Y_i X_i^2)(Y_i)^2 = (Y_i)(Y_i X_i)^2$, and the fact that for $q = 2$, $Y_i^2 = \sum_{k+1 \le j \le n-1} p_{i,j} Y_j^2$ and $(Y_i X_i)^2 = \sum_{k+1 \le j \le n-1} p_{i,j} Y_j^2 X_j^2$. After expansion, using the fact that $Y_j X_{j'}^2 + Y_{j'} X_{j'}^2 + Y_j X_j^2 + Y_j X_{j'}^2 = (Y_j + Y_{j'}) (X_j^2 + X_{j'}^2)$ for any $k+1 \le j < j' \le n-1$, we get $\mathcal{B}'$. $\square$
As before, we call $\mathcal{B}'$ the vector space generated by the polynomials $\mathcal{B}'$. The system $\mathcal{B}'$ contains $\binom{3m}{2} - 3m$ linearly independent polynomials of bidegree (2,3) in the two blocks of variables $\mathbf{X}$ and $\mathbf{Y}$ and its linearization involves $\binom{3m}{2}$ variables

$$ (37) \qquad \begin{cases} Y_j Y_{j'} (Y_j + Y_{j'})(X_j^2 + X_{j'}^2) & \text{for } k+1 \le j < j' \le n-1, \\ Y_j^2 & \text{for } j' = n \text{ and } X_n = \infty. \end{cases} $$

In a similar way to what done for $\mathcal{S}'$, we can define the vector spaces

$$ (38) \qquad \mathcal{E}_j = \frac{1}{Y_j} (\mathcal{B}' \cap Y_j \cdot \mathbb{F}_q[\mathbf{X}, \mathbf{Y}]_{\le 3}), \quad j \in [k+1, n-1] $$

and an associated basis $\mathcal{E}_j$ of $\mathcal{E}_j$. The following proposition is fundamental for the efficiency of the attack.

**Proposition 40.** For each $j \in [k+1, n-1]$, for each polynomial

$$ \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j)^2 \in \mathcal{V}_j $$

the polynomial

$$ \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j) $$

cancels on the solution $\mathbf{x}, \mathbf{y}$. This produces new bilinear polynomials that we can add to our algebraic system.

*Proof.* Comparing the polynomials in $\mathcal{S}'$ (23) and $\mathcal{B}'$ (36), it is clear that

$$ \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j)^2 \in \mathcal{V}_j \iff \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (Y_{j'} + Y_j)(X_{j'} + X_j)^2 \in \mathcal{E}_j, $$

hence $\dim(\mathcal{E}_j) = m-1$. We can split a polynomial in $\mathcal{E}_j$ in the following way:

$$ \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (Y_{j'} + Y_j)(X_{j'} + X_j)^2 \\ = \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'}^2 (X_{j'} + X_j)^2 + Y_j \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j)^2. $$

Since $Y_j \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j)^2$ is in the ideal generated by $\mathcal{V}_j$, we obtain

$$ \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'}^2 (X_{j'} + X_j)^2 \in \mathcal{E}_j + Y_j \mathcal{V}_j. $$

Since the coefficients $v_{j'} \in \mathbb{F}_2$, by applying the Frobenius map $m-1$ times, we get that the polynomial

$$ \left( \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'}^2 (X_{j'} + X_j)^2 = 0 \right)^{2^{m-1}} = \sum_{j' \in [k+1, n] \setminus \{j\}} v_{j'} Y_{j'} (X_{j'} + X_j) $$

belong to the ideal generated by $\mathcal{E}_j + \mathcal{V}_j$ and the polynomials $Y_{j'}^{2m} - Y_{j'}$, $X_{j'}^{2m} - X_{j'}$ for $j' \in [k+1, n]$, hence cancels on the solution. $\square$
As a consequence, from the computation of the $V_j$'s, and under the assumption that each $V_j$ contains $m-1$ linearly independent polynomials, we can produce $(m-1)(3m-1)$ bilinear polynomials that are independent, as well as $m-1$ linearly independent linear polynomials in $Y_j$'s (in $V_n$). A Gröbner basis of the system formed by those polynomials together with the polynomials in the systems $V_j$'s can then be computed, and experimentally all computations are done by staying at degree 3, which gives a global complexity that is polynomial in $m$. The last steps of the attack (computation of the solutions using the FGLM algorithm, and recovering of the entier support and multiplier) are identical to the $q > 2$ case.

4.8. Interlacing the algebraic recovering with the filtration. We now get back to distinguish between the full-length vectors $\mathbf{x}$ and $\mathbf{y}$ and their shortening due to the filtration attack. We can restore the information lost from the filtration shortening, by simply repeating the attack twice on different sets. This is possible because $2(r-3)$ is very small compared to $n$. Indeed, if we shorten the positions corresponding to $\mathcal{I}_1 \stackrel{\text{def}}{=} \llbracket 1, r-3 \rrbracket$ (the order is irrelevant) during the filtration attack, at the end of the algebraic recovering we have access to $m$ pairs of vectors

$$ \bar{x}_{\mathcal{I}_1} \quad \text{and} \quad \bar{y}_{\mathcal{I}_1} \left( \prod_{i \in \mathcal{I}_1} (\bar{x}_{\mathcal{I}_1} - x_i) \right). $$

Analogously, if we shorten the positions corresponding to $\mathcal{I}_2 \stackrel{\text{def}}{=} \llbracket (r-3)+1, 2(r-3) \rrbracket$ during the filtration attack, at the end of the algebraic recovering we have access to $m$ pairs of vectors

$$ \bar{x}_{\mathcal{I}_2} \quad \text{and} \quad \bar{y}_{\mathcal{I}_2} \left( \prod_{i \in \mathcal{I}_2} (\bar{x}_{\mathcal{I}_2} - x_i) \right). $$

In particular, if the same specialization has been chosen, we can couple $m$ pairs $(\bar{x}_{\mathcal{I}_1}, \bar{x}_{\mathcal{I}_2})$ such that the two vectors of each pair coincide on the last $n - (r-3)$ coordinates. We can easily detect them from the last $3m$ coordinates, so that we do not need to solve $2m$ linear systems but it is sufficient to choose one pair and solve only the 2 corresponding linear systems. In this way we obtain a full solution $\bar{x}$ for the original problem as

$$ \bar{x} = \left( \underbrace{\bar{x}_1, \dots, \bar{x}_{r-3}}_{\text{first } r-3} , \underbrace{\bar{x}_{r-2}, \dots, \bar{x}_{2(r-3)}}_{\text{first } r-3} , \underbrace{\bar{x}_{2(r-3)+1}, \dots, \bar{x}_{n-3}, 0, 1, \infty)}_{\text{last common coordinates of } \bar{x}_{\mathcal{I}_1} \text{ and } \bar{x}_{\mathcal{I}_2}} \right). $$

By replacing the found values in the corresponding $\bar{y}_{\mathcal{I}_1} (\prod_{i \in \mathcal{I}_1} (\bar{x}_{\mathcal{I}_1} - x_i))$ and $\bar{y}_{\mathcal{I}_2} (\prod_{i \in \mathcal{I}_2} (\bar{x}_{\mathcal{I}_2} - x_i))$, we retrieve $\bar{y}_{\mathcal{I}_1}$ and $\bar{y}_{\mathcal{I}_2}$. Similarly to what done for the support, we can put together the information of these two vectors and get

$$ \bar{y} = \left( \underbrace{\bar{y}_1, \dots, \bar{y}_{r-3}}_{\text{first } r-3} , \underbrace{\bar{y}_{r-2}, \dots, \bar{y}_{2(r-3)}}_{\text{first } r-3} , \underbrace{\bar{y}_{2(r-3)+1}, \dots, \bar{y}_{n-1}, 1)}_{\text{last common coordinates of } \bar{y}_{\mathcal{I}_1} \text{ and } \bar{y}_{\mathcal{I}_2}} \right). $$

So, a pair of valid support and multiplier has been recovered. However, $\bar{x} \notin \mathbb{F}_{q^m}^n$, because $\bar{x}_n = \infty$. The last question is therefore how to get a valid pair of support and multiplier such that both are defined over $\mathbb{F}_{q^m}$, i.e. how to get the alternant
representation. In other words, we need to determine some $f \in GL_2(\mathbb{F}_{q^m})$ and $\lambda \in \mathbb{F}_{q^m} \setminus \{0\}$ such that

$$ \bar{x}'_i = f(\bar{x}_i) \in \mathbb{F}_{q^m}, \quad \forall i \in [1, n] $$

and

$$ \bar{y}'_i = \lambda \theta(f, \bar{x}_i)^{r-1} \bar{y}_i \in \mathbb{F}_{q^m}, \quad \forall i \in [1, n]. $$

We observe that, since there are only $n-1$ coordinates of $\bar{x}$ in $\mathbb{F}_{q^m}$ and $n-1 < q^m$, there exists at least one element $\hat{x} \in \mathbb{F}_{q^m}$ that is different from all $\bar{x}$ coordinates. We also remark that $\hat{x} \neq 0$, since $\bar{x}_{n-2} = 0$, so the map $f$ on $\mathbb{F}_{q^m}$

$$ f \stackrel{\text{def}}{=} \frac{z}{z - \hat{x}} $$

is induced by an element of the linear group. We have $\theta(f, z) = z - \hat{x}$ if $z \in \mathbb{F}_{q^m}$ and $\theta(f, \infty) = 1$ and we choose $\lambda = 1$. Therefore

$$ \bar{x}'_i = \frac{\bar{x}_i}{\bar{x}_i - \hat{x}}, \quad i \in [1, n-1], $$

$$ \bar{x}'_n = 1, $$

$$ \bar{y}'_i = (\bar{x}_i - \hat{x})^{r-1} \bar{y}_i, \quad i \in [1, n-1], $$

$$ \bar{y}'_n = \bar{y}_n = 1. $$

We finally obtained a support and a multiplier with coordinates over $\mathbb{F}_{q^m}$ that define the public code. This concludes the key-recovery attack on high-rate random alternant codes.

## 5. CONCLUSION

**Breaking the $m = 2$ barrier.** Even if the first step of the attack, namely the derivation of an alternant code of degree 3 from the original alternant code is reminiscent of the filtration attack [COT14, COT17] which allowed to break a McEliece scheme based on wild Goppa codes of extension degree 2, it differs in a crucial way. In [COT14, COT17] building this decreasing sequence of codes is based on taking the shortening of the original Goppa code and square codes of it. This method is bound to fail when the extension degree $m$ is greater than 2 as explained in [COT17, §III]. This is a pity, because the square code is much easier to understand in this case since it retains the polynomial structure of the generalized Reed-Solomon super-code. To go beyond the $m = 2$ barrier, it is mandatory to look for the square of shortenings of the dual of the Goppa/alternant code instead. This square is much more difficult to understand, since taking the dual of a subfield subcode as is the case here yields by Delsarte's theorem the trace code of a generalized Reed-Solomon code which loses a lot of its polynomial description. This is certainly one of the main difficulties that had to be overcome to transform the distinguisher of [FGO+11] into an attack.

This is precisely what is achieved here. Besides shedding some light (together with [MT22]) on the structure of the square of such dual codes, this paper also introduces some other crucial ideas. The first one being the crucial role of being a generic alternant code rather than a Goppa code and then the condition $r \ge q+1$ which is crucial for our main result, Theorem 17 to hold. It is rather surprising that right now it seems that this filtration strategy needs apparently additional ideas to
make it work in the particular case of Goppa codes as observed in Subsection 3.2 despite the fact that Goppa codes are even more structured than plain alternant codes. However there are clearly reasons to believe that this paper has solved the main issue when it comes to transform the distinguisher of [FGO$^{+11}$] into an attack, which is to be able to work with the dual of the alternant/Goppa code, since this is the crux in breaking the $m = 2$ barrier. Therefore it is tempting to conjecture that Goppa codes, at least in the regime where they are distinguishable from random codes (which applies in particular to the CFS scheme [CFS01]) should eventually be attacked in polynomial by some variation the attack that has been given here.

**Understanding the Gröbner basis approach.** The filtration approach which amounts here to construct a nested sequence of dual of alternant codes of decreasing degree results in a final alternant code of degree 3 in the case at hand. From here, a dedicated Gröbner basis is used to recover the polynomial structure of the alternant code. This approach is able to take into account in a very efficient way the fact that the multiplier and the support that are sought should satisfy certain non standard constraints (all the entries of the multiplier are non zero whereas all the entries of the support are distinct). Taking these constraints into account as we do here results in speeding up significantly the algebraic system solving by adding many new low degree equations to the algebraic system. When the code is a Goppa code it turns out that solving the relevant system behaves a little bit differently (as for the case of the filtration). However, unlike the case of the filtration where right now this part of the attack does not work at all, it seems that here even if the Gröbner basis computation consists of more steps, solving the whole system should still be polynomial. Actually there are reasons to conjecture that the Gröbner basis approach stays of polynomial complexity for recovering the polynomial structure of an alternant code of degree 4 and even potentially beyond this, say up to a constant degree. More generally, this raises the issue of understanding how the complexity of recovering the polynomial structure of a degree $r$ alternant/Goppa code through this Gröbner basis approach scales with $r$. An obvious application of this study would be to be able to increase the values of the field size $q$ for which we still have a polynomial attack. This could also allow to get a better understanding how the McEliece resists to this algebraic approach in general and is desirable in the light of the ongoing NIST competition where the McEliece cryptosystem is an alternate fourth round candidate.

## REFERENCES

[ABC$^{+20}$] Martin Albrecht, Daniel J. Bernstein, Tung Chou, Carlos Cid, Jan Gilcher, Tanja Lange, Varun Maram, Ingo von Maurich, Rafael Mizoczki, Ruben Niederhagen, Edoardo Persichetti, Kenneth Paterson, Christiane Peters, Peter Schwabe, Nicolas Sendrier, Jakub Szefer, Cen Jung Tjhai, Martin Tomlinson, and Wang Wen. Classic McEliece (merger of Classic McEliece and NTS-KEM). https://classic.mceliece.org, October 2020. Third round finalist of the NIST post-quantum cryptography call.

[BBB$^{+17}$] Gustavo Banegas, Paulo S.L.M Barreto, Brice Odilon Boidje, Pierre-Louis Cayrel, Gilbert Ndollane Dione, Kris Gaj, Cheikh Thiécoumba Gueye, Richard Haeusler, Jean Belo Klamti, Ousmane N'diaye, Duc Tri Nguyen, Edoardo Persichetti, and Jefferson E. Ricardini. DAGS: Key encapsulation for dyadic GS codes. https://csrc.nist.gov/CSRC/media/Projects/Post-Quantum-Cryptography/documents/round-1/submissions/DAGS.zip, November 2017. First round submission to the NIST post-quantum cryptography call.
[BCGO09] Thierry P. Berger, Pierre-Louis Cayrel, Philippe Gaborit, and Ayoub Otmani. Reducing key length of the McEliece cryptosystem. In *Bart Preneel, editor, Progress in Cryptology - AFRICACRYPT 2009*, volume 5580 of LNCS, pages 77-97, Gammarth, Tunisia, June 21-25 2009.

[Ber00] Thierry P. Berger. On the cyclicity of Goppa codes, parity-check subcodes of Goppa codes and extended Goppa codes. *Finite Fields Appl.*, 6(3):255-281, 2000.

[BJMM12] Anja Becker, Antoine Joux, Alexander May, and Alexander Meurer. Decoding random binary linear codes in $2^{n/20}$: How $1+1=0$ improves information set decoding. In *Advances in Cryptology - EUROCRYPT 2012*, LNCS. Springer, 2012.

[BLM11] Paulo Barreto, Richard Lindner, and Rafael Misoczki. Monoidic codes in cryptography. In *Post-Quantum Cryptography 2011*, volume 7071 of LNCS, pages 179-199. Springer, 2011.

[BLP10] Daniel J. Bernstein, Tanja Lange, and Christiane Peters. Wild McEliece. In *Alex Biryukov, Guang Gong, and Douglas R. Stinson, editors, Selected Areas in Cryptography*, volume 6544 of LNCS, pages 143-158, 2010.

[BLP11] Daniel J. Bernstein, Tanja Lange, and Christiane Peters. Wild McEliece Incognito. In *Bo-Yin Yang, editor, Post-Quantum Cryptography 2011*, volume 7071 of LNCS, pages 244-254. Springer Berlin Heidelberg, 2011.

[BM17] Leif Both and Alexander May. Optimizing BJMM with Nearest Neighbors: Full Decoding in $2^{2/21^n}$ and McEliece Security. In *WCC Workshop on Coding and Cryptography*, September 2017.

[CBB+17] Alain Couvreur, Magali Bardet, Élise Barelli, Olivier Blazy, Rodolfo Canto Torres, Phillipe Gaborit, Ayoub Otmani, Nicolas Sendrier, and Jean-Pierre Tillich. BIG QUAKE. https://bigquake.inria.fr, November 2017. NIST Round 1 submission for Post-Quantum Cryptography.

[CC98] Anne Canteaut and Florent Chabaud. A new algorithm for finding minimum-weight words in a linear code: Application to McEliece's cryptosystem and to narrow-sense BCH codes of length 511. *IEEE Trans. Inform. Theory*, 44(1):367-378, 1998.

[CCMZ15] Ignacio Cascudo, Ronald Cramer, Diego Mirandola, and Gilles Zémor. Squares of random linear codes. *IEEE Trans. Inform. Theory*, 61(3):1159-1173, 3 2015.

[CFS01] Nicolas Courtois, Matthieu Finiasz, and Nicolas Sendrier. How to achieve a McEliece-based digital signature scheme. In *Advances in Cryptology - ASIACRYPT 2001*, volume 2248 of LNCS, pages 157-174, Gold Coast, Australia, 2001. Springer.

[CGG+14] Alain Couvreur, Philippe Gaborit, Valérie Gauthier-Umaña, Ayoub Otmani, and Jean-Pierre Tillich. Distinguisher-based attacks on public-key cryptosystems using Reed-Solomon codes. *Des. Codes Cryptogr.*, 73(2):641-666, 2014.

[CMCP17] Alain Couvreur, Irene Márquez-Corbella, and Ruud Pellikaan. Cryptanalysis of McEliece cryptosystem based on algebraic geometry codes and their subcodes. *IEEE Trans. Inform. Theory*, 63(8):5404-5418, 8 2017.

[COT14] Alain Couvreur, Ayoub Otmani, and Jean-Pierre Tillich. New identities relating wild Goppa codes. *Finite Fields Appl.*, 29:178-197, 2014.

[COT17] Alain Couvreur, Ayoub Otmani, and Jean-Pierre Tillich. Polynomial time attack on wild McEliece over quadratic extensions. *IEEE Trans. Inform. Theory*, 63(1):404-427, 1 2017.

[Del75] Philippe Delsarte. On subfield subcodes of modified Reed-Solomon codes. *IEEE Trans. Inform. Theory*, 21(5):575-576, 1975.

[Dür87] Arne Dür. The automorphism groups of Reed-Solomon codes. *Journal of Combinatorial Theory, Series A*, 44:69-82, 1987.

[FGLM93] Jean-Charles Faugère, Patrizia M. Gianni, Daniel Lazard, and Teo Mora. Efficient computation of zero-dimensional gröbner bases by change of ordering. *J. Symbolic Comput.*, 16(4):329-344, 1993.

[FGO+11] Jean-Charles Faugère, Valérie Gauthier, Ayoub Otmani, Ludovic Perret, and Jean-Pierre Tillich. A distinguisher for high rate McEliece cryptosystems. In *Proc. IEEE Inf. Theory Workshop- ITW 2011*, pages 282-286, Paraty, Brasil, October 2011.

[FGO+13] Jean-Charles Faugère, Valérie Gauthier, Ayoub Otmani, Ludovic Perret, and Jean-Pierre Tillich. A distinguisher for high rate McEliece cryptosystems. *IEEE Trans. Inform. Theory*, 59(10):6830-6844, October 2013.
[FOPT10] Jean-Charles Faugère, Ayoub Otmani, Ludovic Perret, and Jean-Pierre Tillich. Algebraic cryptanalysis of McEliece variants with compact keys. In *Advances in Cryptology - EUROCRYPT 2010*, volume 6110 of LNCS, pages 279–298, 2010.

[GUL09] Valérie Gauthier-Umaña and Gregor Leander. Practical key recovery attacks on two McEliece variants, 2009. IACR Cryptology ePrint Archive, Report2009/509.

[HP03] W. Cary Huffman and Vera Pless. *Fundamentals of error-correcting codes*. Cambridge University Press, Cambridge, 2003.

[KT17] Ghazal Kachigar and Jean-Pierre Tillich. Quantum information set decoding algorithms. In *Post-Quantum Cryptography 2017*, volume 10346 of LNCS, pages 69–89, Utrecht, The Netherlands, June 2017. Springer.

[LS01] Pierre Loidreau and Nicolas Sendrier. Weak keys in the McEliece public-key cryptosystem. *IEEE Trans. Inform. Theory*, 47(3):1207–1211, 2001.

[MB09] Rafael Misoczki and Paulo Barreto. Compact McEliece keys from Goppa codes. In *Selected Areas in Cryptography*, Calgary, Canada, August 13-14 2009.

[McE78] Robert J. McEliece. *A Public-Key System Based on Algebraic Coding Theory*, pages 114–116. Jet Propulsion Lab, 1978. DSN Progress Report 44.

[MMT11] Alexander May, Alexander Meurer, and Enrico Thomae. Decoding random linear codes in $O(2^{0.054n})$. In Dong Hoon Lee and Xiaoyun Wang, editors, *Advances in Cryptology - ASIACRYPT 2011*, volume 7073 of LNCS, pages 107–124. Springer, 2011.

[MO15] Alexander May and Ilya Ozerov. On computing nearest neighbors with applications to decoding of binary linear codes. In E. Oswald and M. Fischlin, editors, *Advances in Cryptology - EUROCRYPT 2015*, volume 9056 of LNCS, pages 203–228. Springer, 2015.

[MP12] Irene Márquez-Corbella and Ruud Pellikaan. Error-correcting pairs for a public-key cryptosystem. CBC 2012, Code-based Cryptography Workshop, 2012. Available on http://www.win.tue.nl/~ruudp/paper/59.pdf.

[MS86] Florence J. MacWilliams and Neil J. A. Sloane. *The Theory of Error-Correcting Codes*. North-Holland, Amsterdam, fifth edition, 1986.

[MT21] Rocco Mora and Jean-Pierre Tillich. On the dimension and structure of the square of the dual of a Goppa code. preprint, 2021.

[MT22] Rocco Mora and Jean-Pierre Tillich. On the dimension and structure of the square of the dual of a Goppa code. In *WCC 2022 - Workshop on Coding Theory and Cryptography*, 2022.

[Nie86] Harald Niederreiter. Knapsack-type cryptosystems and algebraic coding theory. *Problems of Control and Information Theory*, 15(2):159–166, 1986.

[Ran15] Hugues Randriambololona. On products and powers of linear codes under componentwise multiplication. In *Algorithmic arithmetic, geometry, and coding theory*, volume 637 of Contemp. Math., pages 3–78. Amer. Math. Soc., Providence, RI, 2015.

[RSA78] Ronald L. Rivest, Adi Shamir, and Leonard M. Adleman. A method for obtaining digital signatures and public-key cryptosystems. *Commun. ACM*, 21(2):120–126, 1978.

[Sen00] Nicolas Sendrier. Finding the permutation between equivalent linear codes: The support splitting algorithm. *IEEE Trans. Inform. Theory*, 46(4):1193–1203, 2000.

[Sho94] Peter W. Shor. Algorithms for quantum computation: Discrete logarithms and factoring. In S. Goldwasser, editor, *FOCS*, pages 124–134, 1994.

[SS92] Vladimir Michilovich Sidelnikov and S.O. Shestakov. On the insecurity of cryptosystems based on generalized Reed-Solomon codes. *Discrete Math. Appl.*, 1(4):439–444, 1992.

[Ste88] Jacques Stern. A method for finding codewords of small weight. In G. D. Cohen and J. Wolfmann, editors, *Coding Theory and Applications*, volume 388 of LNCS, pages 106–113. Springer, 1988.

## APPENDIX A. FURTHER RESULTS ABOUT TRACE CODES

We will state and prove here a simple result on the extension of trace codes which will be useful in our context.
**Proposition 41.** Let $\mathcal{C} \subseteq \mathbb{F}_{q^m}^n$ be an $\mathbb{F}_{q^m}$-linear code. Then

$$ \mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}} = \sum_{i=0}^{m-1} \mathcal{C}^{q^i}. $$

*Proof.* Take any $\mathbf{c} \in \mathcal{C}$. Then $\mathrm{Tr}(\mathbf{c}) = \mathbf{c} + \mathbf{c}^q + \cdots + \mathbf{c}^{q^{m-1}}$ also belongs to $\mathcal{C} + \mathcal{C}^q + \cdots + \mathcal{C}^{q^{m-1}}$. This proves that $\mathrm{Tr}(\mathcal{C}) \subseteq \sum_{i=0}^{m-1} \mathcal{C}^{q^i}$ and therefore $\mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}} \subseteq \sum_{i=0}^{m-1} \mathcal{C}^{q^i}$. On the other hand, let us prove that any $\mathcal{C}^{q^i}$ is a subspace of $\mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}}$ for any $i$. Consider an arbitrary $\mathbb{F}_q$-basis $\alpha \stackrel{\mathrm{def}}{=} \{\alpha_1, \cdots, \alpha_m\}$ of $\mathbb{F}_{q^m}$. Let $\mathbf{x}_i \stackrel{\mathrm{def}}{=} \mathrm{Tr}(\alpha_i \mathbf{c})$. Since

$$ \mathbf{x}_i = \alpha_i \mathbf{c} + \alpha_i^q \mathbf{c}^q + \cdots + \alpha_i^{q^{m-1}} \mathbf{c}^{q^{m-1}} $$

we have that

$$ (\mathbf{x}_1 \ \mathbf{x}_2 \ \cdots \ \mathbf{x}_m) = \left( \mathbf{c} \ \mathbf{c}^q \ \cdots \ \mathbf{c}^{q^{m-1}} \right) \underbrace{\begin{pmatrix} \alpha_1 & \alpha_2 & \cdots & \alpha_m \\ \alpha_1^q & \alpha_2^q & \cdots & \alpha_m^q \\ \vdots & \vdots & \ddots & \vdots \\ \alpha_1^{q^{m-1}} & \alpha_2^{q^{m-1}} & \cdots & \alpha_m^{q^{m-1}} \end{pmatrix}}_{\stackrel{\mathrm{def}}{=} \mathcal{M}(\alpha)} $$

$\mathcal{M}(\alpha)$ is the Moore matrix associated to $\{\alpha_1, \cdots, \alpha_m\}$ and is invertible because the $\alpha_i$'s are linearly independent over $\mathbb{F}_q$. Therefore

$$ (\mathbf{c} \ \mathbf{c}^q \ \cdots \ \mathbf{c}^{q^{m-1}}) = (\mathbf{x}_1 \ \mathbf{x}_2 \ \cdots \ \mathbf{x}_m) \mathcal{M}(\alpha)^{-1} $$

and therefore all the $\mathbf{c}^{q^i}$ are $\mathbb{F}_{q^m}$-linear combinations of the $\mathbf{x}_j$'s and belong therefore to $\mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}}$. This shows that $\mathcal{C}^{q^i} \subseteq \mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}}$ for any $i$ and shows therefore the reverse inclusion

$$ \mathcal{C} + \mathcal{C}^q + \cdots + \mathcal{C}^{q^{m-1}} \subseteq \mathrm{Tr}(\mathcal{C})_{\mathbb{F}_{q^m}}. $$

□

## APPENDIX B. PROOFS OF SUBSECTION 3.2

Let us recall the proposition we are going to prove.

**Proposition 24.** Let $\mathcal{G}(\mathbf{x}, \Gamma) \stackrel{\mathrm{def}}{=} \mathcal{A}_r(\mathbf{x}, \mathbf{y})$ be a Goppa code of degree $r$. We have for any code positions $i$ and $j$ with $i \neq j$:

$$ (16) \qquad \mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp} = \mathcal{G}(\mathbf{x}, \Gamma) + \langle \mathbf{1} \rangle_{\mathbb{F}_q} \quad (\text{from [Ber00, prop. 1]}) $$

$$ (17) \qquad \mathcal{A}_r(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i - \mathbf{x}_i))^{\perp} = \mathbf{S}h_i(\mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp}) $$

$$ (18) \mathbf{S}h_j(\mathcal{A}_r(\mathbf{x}_i, \mathbf{y}_i(\mathbf{x}_i - \mathbf{x}_i))^{\perp}) = \mathbf{S}h_i(\mathcal{A}_r(\mathbf{x}_j, \mathbf{y}_j(\mathbf{x}_j - \mathbf{x}_j))^{\perp}). $$

*Proof. Proof of* (16). (16) was proved in [Ber00, Prop. 1].

**Proof of** (17).

Choose an $\mathbb{F}_q$-basis $\{\alpha_1, \cdots, \alpha_m\}$ of $\mathbb{F}_{q^m}$. We are first going to prove that

$$ (39) \qquad \mathcal{G}(\mathbf{x}_i, \Gamma)^{\perp} + \langle \mathbf{1} \rangle_{\mathbb{F}_q} = \mathcal{A}_r(\mathbf{x}_i, i\tilde{y}(\mathbf{x}_i - \mathbf{x}_i))^{\perp} + \langle \mathbf{1} \rangle_{\mathbb{F}_q}. $$
$$
\begin{align*}
\mathcal{A}_r(\mathbf{x}_{\tilde{i}}, \mathbf{y}_i(\mathbf{x}_{\tilde{i}} - \mathbf{x}_i))^{\perp} &= \langle \operatorname{Tr}(\alpha_j \mathbf{x}_{\tilde{i}}^a(\mathbf{x}_{\tilde{i}} - \mathbf{x}_i)\mathbf{y}_{\tilde{i}}) | a \in [0, r-1], j \in [0, m-1] \rangle_{\mathbb{F}_q} \\
&= \langle \operatorname{Tr}(\alpha_j \mathbf{x}_{\tilde{i}}^{a+1}\mathbf{y}_{\tilde{i}}) - \operatorname{Tr}(\alpha_j x_i \mathbf{x}_{\tilde{i}}^a\mathbf{y}_{\tilde{i}}) | a \in [0, r-1], j \in [0, m-1] \rangle_{\mathbb{F}_q} \\
&\subseteq \langle \operatorname{Tr}(\alpha_j \mathbf{x}_{\tilde{i}}^b\mathbf{y}_{\tilde{i}}) | b \in [0, r], j \in [0, m-1] \rangle_{\mathbb{F}_q} \\
&= \mathcal{A}_{r+1}(\mathbf{x}_{\tilde{i}}, \mathbf{y}_{\tilde{i}})^{\perp} \\
&= \mathcal{G}(\mathbf{x}_{\tilde{i}}, \Gamma)^{\perp} + \langle \mathbf{1} \rangle_{\mathbb{F}_q}
\end{align*}
$$

On the other hand, since $\Gamma(x_i) \neq 0$, then $\Gamma(\tilde{x}_i) \notin \langle \tilde{x}_i^a(\tilde{x}_i - x_i) | a \in [0, r-1] \rangle_{\mathbb{F}_q}$.

Therefore $\langle \Gamma(\tilde{x}_i), \tilde{x}_i^a(\tilde{x}_i - x_i) | a \in [0, r-1] \rangle_{\mathbb{F}_q}$ is a vector space of dimension $r+1$ of evaluations of polynomials with degree at most $r$. Hence

$$
\langle \Gamma(\tilde{x}_i), \tilde{x}_i^a(\tilde{x}_i - x_i) | a \in [0, r-1] \rangle_{\mathbb{F}_q} = \langle \tilde{x}_i^b | b \in [0, r] \rangle_{\mathbb{F}_q}.
$$

Since Tr $(\alpha_j \Gamma(\tilde{x}_i) \tilde{y}_i)$ = Tr $(\alpha_j \cdot \mathbf{1}) \in \langle \mathbf{1} \rangle_{\mathbb{F}_q}$, we get

$$
\begin{align*}
\mathcal{G}(\tilde{x}_i, \Gamma)^{\perp} + \langle \mathbf{1} \rangle_{\mathbb{F}_q} &= \langle \operatorname{Tr}(\alpha_j \tilde{x}_i^b \tilde{y}_i) | b \in [0, r], j \in [0, m-1] \rangle_{\mathbb{F}_q} \\
&\subseteq \langle \operatorname{Tr}(\alpha_j \tilde{x}_i^a (\tilde{x}_i - x_i) \tilde{y}_i) | a \in [0, r-1], j \in [0, m-1] \rangle_{\mathbb{F}_q} + \langle \mathbf{1} \rangle_{\mathbb{F}_q} \\
&= \mathcal{A}_r(\tilde{x}_i, \tilde{y}_i(\tilde{x}_i - x_i))^{\perp} + \langle \mathbf{1} \rangle_{\mathbb{F}_q}.
\end{align*}
$$

Because of the last point, $\mathbf{Sh}_i(\mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp})$ is the set of codewords of $\mathcal{G}(\mathbf{x}, \Gamma) + \langle \mathbf{1} \rangle_{\mathbb{F}_q}$ which evaluate to 0 at position $i$. Clearly the elements of $\mathcal{A}_r(\tilde{\mathbf{x}}_i, \tilde{\mathbf{y}}_i(\tilde{\mathbf{x}}_i - \tilde{x}_i))^{\perp}$ viewed as polynomial evaluations and extended canonically at position $i$ as the linear space

$$
\{ \operatorname{Tr}(\mathbf{y}(\mathbf{x} - \mathbf{x}_i)P(\mathbf{x})) : P \in \mathbb{F}_{q^m}[X], \deg P < r \}
$$

belong to this set. Since the all one vector does not meet this property and because of (39) this implies the point (17).

**Proof of** (18). This is just a consequence that $\mathbf{Sh}_i(\mathbf{Sh}_j(\mathscr{C})) = \mathbf{Sh}_j(\mathbf{Sh}_i(\mathscr{C}))$
which holds for any code $\mathscr{C}$. Here we apply it to $\mathscr{C} = \mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp}$ and apply the
previous point:

$$
\begin{align*}
\mathbf{Sh}_i ( \mathcal{A}_r ( \tilde{\mathbf{x}}_j, \tilde{\mathbf{y}}_j ( \tilde{\mathbf{x}}_j - \tilde{x}_j ) )^{\perp} ) &= \mathbf{Sh}_i ( \mathbf{Sh}_j ( \mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp} ) ) \quad (\text{by (17)}) \\
&= \mathbf{Sh}_j ( \mathbf{Sh}_i ( \mathcal{A}_{r+1}(\mathbf{x}, \mathbf{y})^{\perp} ) ) \quad (\text{by the previous remark}) \\
&= \mathbf{Sh}_j ( \mathcal{A}_r ( \tilde{\mathbf{x}}_i, \tilde{\mathbf{y}}_i ( \tilde{\mathbf{x}}_i - \tilde{x}_i ) )^{\perp} ) \quad (\text{by (17)})
\end{align*}
$$

□

UNIV ROUEN NORMANDIE, NORMANDIE UNIV, LITIS UR 4108, 76000 ROUEN, FRANCE

Email address: magali.bardet@univ-rouen.fr

INRIA, 2 RUE SIMONE IFF, 75012 PARIS, FRANCE, SORBONNE UNIVERSITÉS, UPMC UNIV
PARIS 06

Email address: rocco.mora@inria.fr

INRIA, 2 RUE SIMONE IFF, 75012 PARIS, FRANCE

Email address: jean-pierre.tillich@inria.fr