text:   Learning",6,"['New paper, short and sweet:\n""Limitations of Neural Collapse for Understanding Generalization in Deep Learning""\n<LINK>\nwith Like Hui, Misha Belkin.\n\nNeural collapse is often claimed to be, in some way, deeply relevant for generalization. But is it? 1/3 <LINK>', 'The literature is muddled because most work does not carefully distinguish between behavior on the train set (an *optimization* property) vs. behavior at test time (a *generation* property).\n\nWe clarify these issues by introducing more precise definitions of ""Neural Collapse"" 2/3 https://t.co/JZsr1bgMau', ""From these definitions, it's clear that Train Collapse may occur, but Test Collapse is often impossible. \n\nThus Neural Collapse is primarily an *optimization* phenomena, with unclear connections to generalization. See paper for more, incl cases where Collapse is undesirable.  3/3 https://t.co/dZqVjhIelz"", '@machinaut Good question, we argue in the paper that the right definition of ""collapse"" should be finite-sample, but infinite-time.\n(Because for infinite both, things just converge to bayes optimal, and are not interesting). https://t.co/xUqI5ySou3', ""@machinaut I don't know what happens to the representation-layer in grokking -- but now that you mention it, this sounds like a great experiment to try."", '@XYHan_ @GalantiTomer @weijie444 Thanks for the nice comments! And for the additional refs— I’ll take a look.']",22,02,1325