text: 416,78,1274066323920822273,841499391508779008,Zico Kolter,"Another new DEQ-related paper this week! @ezra_winston develops a framework for equilibrium models with unique fixed points convergent solvers, based upon monotone operator theory. Paper: <LINK> Code: <LINK> Talk: <LINK> At a high level, the work illustrates a strong connection between the infinite-depth limit of some ""simple"" deep networks, and monotone operator splitting methods. The result generally connects implicit-depth models to this well-studied paradigm. <LINK> Works better than the ""pure"" version of other implicit-layer models like (augmented) Neural ODEs, while still providing existence, uniqueness, and stability guarantees (unlike traditional DEQ models). Works well with convolutional networks using FFT-based techniques. <LINK>",https://arxiv.org/abs/2006.08591,"Implicit-depth models such as Deep Equilibrium Networks have recently been shown to match or exceed the performance of traditional deep networks while being much more memory efficient. However, these models suffer from unstable convergence to a solution and lack guarantees that a solution exists. On the other hand, Neural ODEs, another class of implicit-depth models, do guarantee existence of a unique solution but perform poorly compared with traditional networks. In this paper, we develop a new class of implicit-depth model based on the theory of monotone operators, the Monotone Operator Equilibrium Network (monDEQ). We show the close connection between finding the equilibrium point of an implicit network and solving a form of monotone operator splitting problem, which admits efficient solvers with guaranteed, stable convergence. We then develop a parameterization of the network which ensures that all operators remain monotone, which guarantees the existence of a unique equilibrium point. Finally, we show how to instantiate several versions of these models, and implement the resulting iterative solvers, for structured linear operators such as multi-scale convolutions. The resulting models vastly outperform the Neural ODE-based models while also being more computationally efficient. Code is available at this http URL ",Monotone operator equilibrium networks,3,"['Another new DEQ-related paper this week! @ezra_winston develops a framework for equilibrium models with unique fixed points convergent solvers, based upon monotone operator theory.\n\nPaper: <LINK>\nCode: <LINK>\nTalk: <LINK>', 'At a high level, the work illustrates a strong connection between the infinite-depth limit of some ""simple"" deep networks, and monotone operator splitting methods.  The result generally connects implicit-depth models to this well-studied paradigm. https://t.co/Tg3ijIOwvA', 'Works better than the ""pure"" version of other implicit-layer models like (augmented) Neural ODEs, while still providing existence, uniqueness, and stability guarantees (unlike traditional DEQ models).  Works well with convolutional networks using FFT-based techniques. https://t.co/uY8k5KwCjr']",20,06,749