In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an information-theoretic setting, it is known that Fano's inequality typically does not give a sharp converse result (error lower bound) for channel coding problems. Moreover, recent work has shown that an argument based on binary hypothesis testing gives tighter results. We adapt this technique to the statistical setting, and argue that Fano's inequality can always be replaced by this approach to obtain tighter lower bounds that can be easily computed and are asymptotically sharp. We illustrate our technique in three applications: density estimation, active learning of a binary classifier, and compressed sensing, obtaining tighter risk lower bounds in each case.
An important historical remark is that Hayashi and Nagaoka @cite_5 first linked channel coding and binary hypothesis testing, with later work @cite_8 by Hayashi clarifying this approach and Nagaoka @cite_18 using similar ideas to derive strong converse results. The recent work by Vazquez- @cite_19 also provides results characterizing the average error probability of channel coding in terms of the Type I error of a binary hypothesis test. This link with channel coding has been used in other contexts to prove strong converse results, including @cite_12 , which derived strong converse results for the group testing problem.
@cite_18 @cite_8 @cite_19 @cite_5 @cite_12
1562048139 2035972614 2236569555 2167830490
Strong converse theorems for two different subjects in quantum information theory are presented. First, we show a lower bound on the probability of decoding error for a quantum communication channel, from which the strong converse to the quantum channel coding theorem is obtained. Second, we give the strong converse theorem for the quantum hypothesis testing as an application of a new inequality on the error probabilities. This inequality is also used to establish the quantum Stein's lemma. Two alternative exact characterizations of the minimum error probability of Bayesian @math -ary hypothesis testing are derived. The first expression corresponds to the error probability of an induced binary hypothesis test and implies the tightness of the meta-converse bound by ; the second expression is a function of an information-spectrum measure and implies the tightness of a generalized Verdu-Han lower bound. The formulas characterize the minimum error probability of several problems in information theory and help to identify the steps where existing converse bounds are loose. A formula for the capacity of arbitrary single-user channels without feedback (not necessarily information stable, stationary, etc.) is proved. Capacity is shown to equal the supremum, over all input processes, of the input-output inf-information rate defined as the liminf in probability of the normalized information density. The key to this result is a new converse approach based on a simple new lower bound on the error probability of m-ary hypothesis tests among equiprobable hypotheses. A necessary and sufficient condition for the validity of the strong converse is given, as well as general expressions for spl epsiv -capacity. > The capacity of a classical-quantum channel (or, in other words, the classical capacity of a quantum channel) is considered in the most general setting, where no structural assumptions such as the stationary memoryless property are made on a channel. A capacity formula as well as a characterization of the strong converse property is given just in parallel with the corresponding classical results of Verdu-Han (1994) which are based on the so-called information-spectrum method. The general results are applied to the stationary memoryless case with or without cost constraint on inputs, whereby a deep relation between the channel coding theory and the hypothesis testing for two quantum states is elucidated.