text:   information theory",5,"['How do artificial neural networks generalize? The answer may be in their causal structure. In this new paper we use information theory to track nodes’ causal relationships becoming more sensitive or degenerate. Training traces a path in this “causal plane” <LINK> <LINK>', 'Some really interesting results of this collaboration with Simon Mattsson and @ericjmichaud_ : the informativeness of a causal relationship between two nodes peaks at a certain characteristic edge weight, no matter what bin size you use (there is a manifold for multiple edges) https://t.co/dLN3gKAOVK', 'We can even measure a variant of the integrated information using these techniques. Normally the phi of a feedforward network is zero... but there are still integrated joint effects of one layer to another. Here we introduce a measure to capture those: ""phi feedfoward"" https://t.co/JkB253oeut', ""There's a great github package for those who want to try out these techniques. Ultimately we are hoping to offer a kind of alternative to the information bottleneck approach (although its not contradictory) that focuses more on causation\n\nhttps://t.co/zp3x3nd9L5"", '@BlaiseLucey00 @ericjmichaud_ I recently learned how to make gifs so get used to it']",20,10,1161