We consider the problem of representing Boolean functions exactly by "sparse" linear combinations (over R) of functions from some "simple" class C . In particular, given C we are interested in finding low-complexity functions lacking sparse representations. When C is the set of PARITY functions or the set of conjunctions, this sort of problem has a well-understood answer; the problem becomes interesting when C is "overcomplete" and the set of functions is not linearly independent. We focus on the cases where C is the set of linear threshold functions, the set of rectified linear units (ReLUs), and the set of low-degree polynomials over a finite field, all of which are well-studied in different contexts. Building on the new easy witness lemma of Cody Murray and the author, we provide generic tools for proving lower bounds on representations of this kind. Applying these, we give several new lower bounds for "semi-explicit" Boolean functions. Let α(n) be an unbounded function such that n α(n) is time constructible (e.g. α(n) = log ⋆ (n)). We show: • Functions in NTIME[n α(n) ] that require super-polynomially many linear threshold functions to represent (depth-two neural networks with sign activation function, a special case of depth-two threshold circuit lower bounds). • Functions in NTIME[n α(n) ] that require super-polynomially many ReLU gates to represent (depthtwo neural networks with ReLU activation function). • Functions in NTIME[n α(n) ] that require super-polynomially many O(1)-degree F p -polynomials to represent exactly, for every prime p (related to problems regarding Higher-Order Uncertainty Principles). We also obtain a function in E NP requiring 2 Ω(n) -size linear combinations. • Functions in NTIME[n poly(log n) ] that require super-polynomially many ACC • THR circuits of polynomial size to represent exactly (further generalizing the recent lower bounds of Murray and the author). We also obtain "fixed-polynomial" lower bounds for functions in NP, for the first three representation classes.
Most recently, Williams REF showed that there are functions in NQP can not be represented by a linear combination of polynomially many ACC • THR circuits.
Limits on representing Boolean functions by linear combinations of simple functions: thresholds, ReLUs, and low-degree polynomials
ArXiv
ArXiv
Mathematics Computer Science