Liquid Networks

Richard_Jenkins: <p>I just watched an MIT discussion on Liquid Neurons and Liquid Networks. They took 10,000 neurons in an FC layer and replaced with 19 liquid neurons connected to 2 more layers with fewer liquid neurons each. The math function for the liquid neurons is defined in their work. Would a new nn class type be implemented as a LiquidNeuralNetwork as opposed to the Linear type for example? Would it be fairly easy to override existing classes and substitute the formulas?<br>
Here is the video presentation on it <a href="https://youtu.be/0FNkrjVIcuk" class="inline-onebox" rel="noopener nofollow ugc">Liquid Neural Networks, A New Idea That Allows AI To Learn Even After Training - YouTube</a></p>

Richard_Jenkins: <p>Is anyone working on implementing this today in PyTorch ?</p>

J_Johnson: <p>You would just need to translate this LTCCell definition from Tensorflow to Pytorch.</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py">
  <header class="source">

      <a href="https://github.com/raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py" target="_blank" rel="noopener nofollow ugc">raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py</a></h4>


      <pre><code class="lang-py">import tensorflow as tf
import numpy as np
import time
import os
from enum import Enum

class MappingType(Enum):
    Identity = 0
    Linear = 1
    Affine = 2

class ODESolver(Enum):
    SemiImplicit = 0
    Explicit = 1
    RungeKutta = 2

class LTCCell(tf.nn.rnn_cell.RNNCell):

    def __init__(self, num_units):

</code></pre>



  This file has been truncated. <a href="https://github.com/raminmh/liquid_time_constant_networks/blob/master/experiments_with_ltcs/ltc_model.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>ChatGPT or some other code LLM could probably handle most of the heavy lifting for you, and then some minor debugging.</p>

banton: <p>It seems like a pytorch implementation of liquid NN layers already exists:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/mlech26l/ncps">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/mlech26l/ncps" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/f950dea164ae3e9e5b6d5e8ae54a2cf6131464d9e9c8041e9bc27abd664ecdb7/mlech26l/ncps" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/mlech26l/ncps" target="_blank" rel="noopener nofollow ugc">GitHub - mlech26l/ncps: PyTorch and TensorFlow implementation of NCP, LTC,...</a></h3>

  <p>PyTorch and TensorFlow implementation of NCP, LTC, and CfC wired neural models - GitHub - mlech26l/ncps: PyTorch and TensorFlow implementation of NCP, LTC, and CfC wired neural models</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>