Title: Convex Efficient Coding
Primary area: applications to neuroscience & cognitive science
Keywords: Neuroscience, Representation, Identifiability

Abstract: Why do neurons encode information the way they do? Normative answers to this question model neural activity as the solution to an optimisation problem; for example, the celebrated efficient coding hypothesis frames neural activity as the optimal encoding of information under efficiency constraints. Successful normative theories have varied dramatically in complexity, from simple sparse linear models (Olshausen & Field ,1996), to complex regularised deep neural networks (Lindsay 2021). What complex models gain in flexibility, they lose in tractability and often understandability. Here, we partially bridge this gap by constructing a set of tractable but flexible normative representational theories that develop efficient coding. Instead of optimising the neural activities directly, following Sengupta et al. 2018, we rewrite the optimisations as over the representational similarity, a matrix formed from the dot products of each pair of neural responses. Using this, we show that a large family of interesting optimisation problems are convex. This includes problems corresponding to linear and some non-linear neural networks, and problems from the literature not previously recognised as convex. We put these findings to work in two ways. First, we extend previous results on modularity and mixed selectivity in neural activity; in so doing we provide the first necessary and sufficient identifiability result for related matrix factorisations. Second, we seek to understand the meaningfulness of single neural tuning curves as compared to neural representations. In particular we derive an identifiability result stating that, for an optimal representational similarity matrix, if neural tunings are `different enough' then they are uniquely linked to the optimal representational similarity, partially justifying the use of single neuron tuning analysis in neuroscience. In sum, we identify an interesting space of convex problems, and use them to derive neural coding results.

TL;DR: We derive a family of convex representational optimisation problems and use them to derive identifiability results for matrix factorisation and single neuron tuning curves.