Reviewerly/exHarmony huggingface dataset:
Robust Hypothesis Testing with Kernel Uncertainty Sets

In this paper, the robust hypothesis testing problem is investigated, where under the null and the alternative hypotheses, the distributions are assumed to be in some uncertainty sets. The uncertainty sets are constructed in a data-driven manner, i.e., they are centered around empirical distributions. The distance between kernel mean embeddings of distributions in the reproducing kernel Hilbert space is used as the distance metric of uncertainty sets. The Bayesian setting is studied, where the goal is to minimize the worst-case error probability. An optimal test is firstly obtained for the case with a finite alphabet. For the case with an infinite alphabet, a tractable approximation is proposed to quantify the worst-case error probability, and a kernel smoothing method is further applied to design test that generalizes to unseen samples. A heuristic robust kernel test is also proposed and proved to be exponentially consistent. Numerical results are provided to demonstrate the performance of the proposed tests.
Private Information Retrieval from Colluding and Byzantine Servers with Binary Reed–Muller Codes

This paper is eligible for the Jack Keil Wolf ISIT Student Paper Award. In this work, a flexible and robust private information retrieval (PIR) scheme based on binary non-maximum distance separable (non-MDS) codes is considered. This combines previous works on PIR schemes based on transitive non-MDS codes on one hand, and PIR from MDS-coded Byzantine and nonresponsive servers on the other hand. More specifically, a PIR scheme employing binary Reed–Muller (RM) codes tolerant to colluding, Byzantine, and non-responsive servers is constructed, and bounds for the achievable rates are derived under certain conditions. The construction of such schemes turns out to be much more involved than for MDS codes. Namely, the binary query vectors have to be selected with great care to hit the desired information sets, which is technically challenging as will be shown.
Another Look at Universal Individual Learning

In recent papers we have proposed an individual setting for the batch learning problem and showed that it is solved by a known variant of the Normalized Maximum Likelihood (NML) which we termed pNML. In this paper we present a different possible definition for the batch learning problem in the individual setting and show that it is solved by another known variant of the normalized maximum likelihood, which we denote by pNML2. We further derive an exact expression of the pNML2 for the linear regression problem. We use this result, along with known results and new upper and lower bounds over the regret of the pNML2 learner, to compare between the two learners.
Differentially Private Distributed Matrix Multiplication: Fundamental Accuracy-Privacy Trade-Off Limits

The classic BGW algorithm of Ben Or, Goldwasser and Wigderson for secure multiparty computing demonstrates that secure distributed matrix multiplication over finite fields is possible over 2t+1 computation nodes, while keeping the input matrices private from every t colluding computation nodes. In this paper, we develop and study a novel coding formulation to explore the trade-offs between computation accuracy and privacy in secure multiparty computing for real-valued data, even with fewer than 2t+1 nodes, through a differential privacy perspective. For the case of t = 1, we develop achievable schemes and converse arguments that bound ϵ — the differential privacy parameter that measures the privacy loss — for a given accuracy level. Our achievable coding schemes are specializations of Shamir secret sharing applied to real-valued data, coupled with appropriate choice of evaluation points. We develop converse arguments that apply for general additive noise based schemes.
Multi-Party Quantum Purity Distillation with Bounded Classical Communication

We consider the task of distilling local purity from a noisy quantum state ρ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ABC</sup> , wherein we provide a protocol for three parties, Alice, Bob and Charlie, to distill local purity (at a rate P) from many independent copies of a given quantum state ρ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ABC</sup> . The three parties have access to their respective subsystems of ρ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ABC</sup> , and are only allowed to use local unitary operations. In addition, Alice and Bob can communicate with Charlie using a one-way multiple-access dephasing channel of link rates R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> and R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> , respectively. The objective of the protocol is to minimize the usage of the dephasing channel (in terms of rates R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> and R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> ) while maximizing the asymptotic purity that can be jointly distilled from ρ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ABC</sup> . To achieve this, we employ ideas from distributed measurement compression protocols, and in turn, characterize a set of sufficient conditions on (P, R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> ,R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> ) in terms of quantum information theoretic quantities such that P amount of purity can be distilled using rates R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> and R <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> .
Understanding Generalization via Leave-One-Out Conditional Mutual Information

We study the mutual information between the output of a learning algorithm and its n training data, conditional on a supersample of n+1 i.i.d. data from which the training data is chosen at random without replacement. We show that this variant of the conditional mutual information (CMI) of an algorithm (Steinke and Zakynthinou, 2020), which we dub leave-one-out CMI, determines the mean generalization error of any interpolating learning algorithm with a bounded loss function. For interpolating and non-interpolating classifiers, we demonstrate that bounded leave-one-out CMI implies generalization. Finally, as an application, we analyze the population risk of the One-Inclusion-Graph Algorithm, a transductive learning algorithm for VC classes in the realizable setting, and prove that our framework is the first information-theoretic framework that is able to achieve the optimal bound for learning VC classes in the realizable setting.
New Bounds on the Size of Binary Codes with Large Minimum Distance

Let A(n, d) denote the maximum number of code-words in a binary code of length n and minimum Hamming distance d. Deriving upper and lower bounds on A(n, d) has been a subject for extensive research in coding theory. In this paper, we examine upper and lower bounds on A(n, d) in the high-minimum distance regime, in particular, when $d = n/2 - \Theta (\sqrt n )$. We will first provide a lower bound based on a cyclic construction for codes of length n = 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</sup> − 1 and show that $A\left({n,d = n/2 - {2^{c - 1}}\sqrt n }\right) \geq {n^c}$, where c is an integer with 1 ⩽ c ⩽ m/2 − 1. With a Fourier-analytic view of Delsarte’s linear program, novel upper bounds on $A(n,n/2 - \sqrt n ){\text{ and }}A(n,n/2 - 2\sqrt n )$ are obtained, and, to the best of the authors’ knowledge, are the first upper bounds scaling polynomially in n for the regime with $d = n/2 - \Theta (\sqrt n )$.
A Novel Attack to the Permuted Kernel Problem

The Permuted Kernel Problem (PKP) asks to find a permutation of a given vector belonging to the kernel of a given matrix. The PKP is at the basis of PKP-DSS, a post-quantum signature scheme deriving from the identification scheme proposed by Shamir in 1989. The most efficient solver for PKP is due to a recent paper by Koussa et al. In this paper we propose an improvement of such an algorithm, which we achieve by considering an additional collision search step applied on kernel equations involving a small number of coordinates. We study the conditions for such equations to exist from a coding theory perspective, and we describe how to efficiently find them with methods borrowed from coding theory, such as information set decoding. We assess the complexity of the resulting algorithm and show that it outperforms previous approaches in several cases. We also show that, taking the new solver into account, the security level of some instances of PKP-DSS turns out to be slightly overestimated.
Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent System

Decentralized learning is an efficient emerging paradigm for boosting the computing capability of multiple bounded computing agents. In the big data era, performing inference within the distributed and federated learning (DL and FL) frameworks, the central server needs to process a large amount of data while relying on various agents to perform multiple distributed training tasks. Considering the decentralized computing topology, privacy has become a first-class concern. Moreover, assuming limited information processing capability for the agents calls for a sophisticated privacy-preserving decentralization that ensures efficient computation. Towards this end, we study the privacy-aware server to multi-agent assignment problem subject to information processing constraints associated with each agent, while maintaining the privacy and assuring learning informative messages received by agents about a global terminal through the distributed private federated learning (DPFL) approach. To find a decentralized scheme for a two-agent system, we formulate an optimization problem that balances privacy and accuracy, taking into account the quality of compression constraints associated with each agent. We propose an iterative converging algorithm by alternating over self-consistent equations. We also numerically evaluate the proposed solution to show the privacy-prediction trade-off and demonstrate the efficacy of the novel approach in ensuring privacy in DL and FL.
Distributed User-Level Private Mean Estimation

Traditionally, an item-level differential privacy framework has been studied for applications in distributed learning. However, when a client has multiple data samples, and might want to also hide its potential participation, a more appropriate notion is that of user-level privacy [1]. In this paper, we develop a distributed private optimization framework that studies the trade-off between user-level local differential privacy guarantees and performance. This is enabled by a novel distributed user-level private mean estimation algorithm using distributed private heavy-hitter estimation. We use this result to develop the privacy-performance trade-off for distributed optimization.
Distillation of Secret Key and GHZ States from Multipartite Mixed States

We consider two related problems of extracting correlation from a given multipartite mixed quantum state: the first is the distillation of a conference key when the state is shared between a number of legal players and an eavesdropper; the eavesdropper, apart from starting off with this quantum side information, also observes the public communication between the players. The second is the distillation of Greenberger-Horne-Zeilinger (GHZ) states by means of LOCC from the given mixed state. These problem settings extend our previous paper [FS & AW, IEEE Trans. Inf. Theory 68(2):976-988, 2022], and we generalise its results: using a quantum version of the task of communication for omniscience, we derive a novel lower bound on the distillable secret key from any multipartite quantum state by means of a so-called non-interacting communication protocol. Secondly, by making the secret key distillation protocol coherent, we derive novel lower bounds on the distillation rate of GHZ states. Full details in the long version [1].
An Integrated Method to Deal with Partial Stragglers and Sparse Matrices in Distributed Computations

The speed of distributed matrix computations over large clusters is often dominated by the stragglers (slow or failed worker nodes). Several techniques based on coding theory have been introduced to mitigate the straggler issue where every worker node is assigned smaller task(s) of multiplying encoded submatrices of the original matrices. However, many of these methods consider the stragglers as erasures, i.e., they discard the potentially useful partial computations done by the slower workers. Moreover, the "input" matrices can be sparse in many scenarios. In this case encoding schemes that combine a large number of input submatrices can adversely affect the worker computation time.In this work, we proposed an integrated approach which addresses both of the issues mentioned above. We allow limited amount of encoding for the submatrices of both A and B; this helps us to preserve the sparsity of the encoded matrices, so that the worker computation can be fast. Our approach provides a trade-off between straggler resilience and worker computation speed, while utilizing partial computations at the workers. Crucially, at one operating point we can ensure that the failure resilience of the system is optimal. Comprehensive numerical analysis done in Amazon Web Services (AWS) cluster confirms the superiority of our approach when compared with previous methods.
Neural Estimation of the Rate-Distortion Function for Massive Datasets

A fundamental question in designing lossy data compression schemes is how well one can do in comparison with the rate-distortion function, which describes the known theoretical limits of lossy compression. Motivated by the empirical success of deep neural network (DNN) compressors on large, real-world data, we investigate methods to estimate the rate-distortion function on such data, which would allow comparison of DNN compressors with optimality. While one could use the empirical distribution of the data and apply the Blahut-Arimoto algorithm, this approach presents several computational challenges when the datasets are large and high-dimensional, such as the case of modern image datasets. Instead, we reformulate the rate-distortion objective, and solve the resulting functional optimization problem using neural networks. We provide experimental results on popular image datasets, and provide theoretical evidence why our method can accurately estimate the rate-distortion function. Additionally, we show that the rate-distortion achievable by DNN compressors are within several bits of the rate-distortion function. Lastly, we connect the rate-distortion objective and entropic optimal transport, and describe a method to implement an operational lossy compression scheme with guarantees on the achievable rate and distortion.
Learning Maximum Margin Channel Decoders for Non-linear Gaussian Channels

The problem of learning a channel decoder for an unknown non-linear white Gaussian noise channel is considered. The learner is provided with a fixed codebook and a dataset comprised of n independent input-output samples of the channel, and is required to select a matrix for a nearest neighbor decoder with a linear kernel. The objective of maximizing the margin of the decoder is addressed. Accordingly, a regularized loss minimization problem with a codebook-related regularization term and a hinge-like loss function is developed, which is inspired by the support vector machine paradigm for classification problems. Expected generalization error bound for that hinge loss is provided for the solution of the regularized loss minimization, and shown to scale at a rate of O(1/(λn)), where λ is a regularization tradeoff parameter. In addition, a high probability uniform generalization error bound is provided for the hypothesis class, and shown to scale at a rate of $O(1/\sqrt n )$. A stochastic sub-gradient descent algorithm for solving the regularized loss minimization problem is proposed, and an optimization error bound is stated, which scales at a rate of $\tilde O(1/(\lambda T))$. The performance of the this algorithm is demonstrated by an example.
Asymptotic Nash Equilibrium for the Sequential Adversarial Hypothesis Testing Game

In this paper, we formulate the sequential binary hypothesis testing problem in which an adversary is active under both hypotheses. This problem is formulated as a sequential adversarial hypothesis testing game played between the decision maker and the adversary and it is a zero-sum and strategic one. The goal of the decision maker is to minimize the expectation of stopping time to make the test more efficient, while the adversary’s goal is to maximize it. We obtain the pair of strategies under which the asymptotic Nash equilibrium of the game is attained.
Differentially Private ℓ<sub>1</sub>-norm Linear Regression with Heavy-tailed Data

We study the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) with heavy-tailed data. Specifically, we focus on the ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> -norm linear regression in the ϵ-DP model. While most of the previous work focuses on the case where the loss function is Lipschitz, here we only need to assume the variates has bounded moments. Firstly, we study the case where the ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> norm of data has bounded second order moment. We propose an algorithm which is based on the exponential mechanism and show that it is possible to achieve an upper bound of $\tilde O\left( {\sqrt {\frac{d}{{n\varepsilon }}} } \right)$ (with high probability). Next, we relax the assumption to bounded θ-th order moment with some θ ∈ (1,2) and show that it is possible to achieve an upper bound of $\tilde O\left( {{{\left( {\sqrt {\frac{d}{{n\varepsilon }}} } \right)}^{\frac{{\theta - 1}}{\theta }}}} \right)$. Our algorithms can also be extended to more relaxed cases where only each coordinate of the data has bounded moments, and we can get an upper bound of $\tilde O\left( {\sqrt {\frac{d}{{n\varepsilon }}} } \right)$ and $\tilde O\left( {\frac{d}{{{{\left( {n\varepsilon } \right)}^{\frac{{\theta - 1}}{\theta }}}}}} \right)$ in the second and θ-th moment case respectively.
Multi-user Privacy Cooperation Game by Leveraging Users’ Service Flexibility

In location-based services (LBSs), it is promising for multiple users to cache and share their Point-of-Interest (PoI) information with each other to reduce overall query frequency and preserve location privacy. Yet most studies on multi-user privacy preservation overlook the opportunity of leveraging service flexibility, where many users are flexible and may add obfuscation to individual LBS query. This paper is the first to study how multiple users cooperate to query with obfuscation against the adversary’s optimal inference attack, by leveraging their mutual service flexibility. Unlike the literature, even if a user already finds the shared PoI information useful, we prove it beneficial for him to further query with obfuscated location to confuse the adversary. To save the computational complexity of the max-min adversarial game problem and derive the closed-form solution, we also propose a binary approximate solution, which is proved to guarantee good privacy performance for an average user. Perhaps surprisingly, the user with greater service flexibility should choose to query the LBS with less misreported location, to maximally confuse the adversary. Finally, we numerically compare our optimal and approximate solutions with the existing approaches to show our effective privacy improvement.
Density Estimation of Processes with Memory via Donsker Vardhan

Density estimation plays an important role in modeling random variables (RVs) with continuous alphabets. This work provides an algorithm that estimates the probability density function (PDF) of stationary and ergodic random processes using recurrent neural networks (RNNs). The main idea is to decompose the target PDF into a known auxiliary PDF and a likelihood ratio between the target and auxiliary PDFs. The algorithm focuses on estimating the likelihood ratio using the Donsker Vardhan (DV) variational formula of Kullback Leibler (KL) divergence. Together, the maximizer of the DV formula and the auxiliary PDF are used to construct the estimator of the target PDF in the form of a Gibbs density. The obtained estimator converges to the target PDF in total variation (TV) and in distribution. Also, we show that proposed estimator minimizes the cross entropy (CE) between the target and auxiliary distribution, and that with a proper choice of the auxiliary distribution, it defines a tight upper bound on the entropy rate. We demonstrate this approach by estimating the density of a Gaussian hidden Markov model.
The Gapped k-Deck Problem

The k-deck problem is concerned with finding the smallest positive integer S(k) such that there exist at least two strings of length S(k) that share the same k-deck, i.e., the multiset of subsequences of length k. We introduce the new problem of gapped k-deck reconstruction: For a given gap parameter s, we seek the smallest positive integer G <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</inf> (k) such that there exist at least two distinct strings of length G <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</inf> (k) that cannot be distinguished based on a "gapped" set of k-subsequences. The gap constraint requires the elements in the subsequences to be at least s positions apart within the original string. Our results are as follows. First, we show how to construct sequences sharing the same 2-gapped k-deck using a nontrivial modification of the recursive Morse-Thue string construction procedure. This establishes the first known constructive upper bound on G <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> (k). Second, we further improve this bound using the approach by Dudik and Schulman [6].
Quantifying the Cost of Privately Storing Data in Distributed Storage Systems

Consider a user who wishes to store a file in multiple servers such that at least t servers are needed to reconstruct the files, and z colluding servers cannot learn any information about the file. Unlike traditional models, where perfectly secure channels are assumed to be available at no cost between the user and each server, we assume that the user can only send data to the servers via public channels, and that the user and each server share an individual secret key with length n. For a given n, we determine the maximal length of the file that the user can store, and thus quantify the necessary cost to store a file with a certain length, in terms of the length of the secret that the user needs to share with the servers. Additionally, for this maximal file length, we determine (i) the optimal amount of local randomness needed at the user, (ii) the optimal amount of public communication from the user to the servers, and (iii) the optimal amount of storage requirement at the servers.