ICML 2025

Making Hard Problems Easier with Custom Data Distributions
and Loss Regularization: A Case Study in Modular Arithmetic
Eshika Saxena * 1 Alberto Alfarano * 1 Emily Wenger † 1 2 Kristin Lauter † 1
Abstract
Recent work showed that ML-based attacks on
Learning with Errors (LWE), a hard problem used
in post-quantum cryptography, outperform classi-
cal algebraic attacks in certain settings. Although
promising, ML attacks struggle to scale to more
complex LWE settings. Prior work connected this
issue to the difficulty of training ML models to
do modular arithmetic, a core feature of the LWE
problem. To address this, we develop techniques
that significantly boost the performance of ML
models on modular arithmetic tasks—enabling
the models to sum up to N = 128 elements mod-
ulo q ≤974269. Our core innovation is the use
of custom training data distributions and a care-
fully designed loss function that better represents
the problem structure. We apply an initial proof
of concept of our techniques to LWE specifically
and find that they allow recovery of 2x harder se-
crets than prior work. Our techniques also help
ML models learn other well-studied problems bet-
ter, including copy, associative recall, and parity,
motivating further study.
1. Introduction
Modular arithmetic is a key component of many crypto-
graphic hard problems, including Learning with Errors
(LWE), which is the basis for several newly standardized
post-quantum cryptosystems (Chen et al., 2022). The LWE
problem involves determining a secret vector given two
pieces of information: A ∈Zm×n
q
, a matrix sampled uni-
formly at random mod q, and b = A·s+e ∈Zm
q , the noisy
dot product of A with the secret vector s mod q. When s
is binary, which often happens in practical applications of
LWE, computing b from A requires computing a subset
sum mod q, which is a modular addition problem.
*Equal contribution † Equal contribution 1Meta AI 2Duke Uni-
versity. Correspondence to: Eshika Saxena <eshika@meta.com>.
Proceed...