Turn a sympy expression to a trainable equation

username: smu226
cooked: <p>Hello! I have some sympy expressions of the form: <code>c0*cos(c1*y) + c2 + c3*x**2</code> I want to  turn the parameters c0, c1, c2, c3 to trainable pytorch parameters and run gradient descent on them (as I would do with an actual NN), to fit them to some data of the form (x,y,z). Is there a way to do this? Thank you!</p>

username: ptrblck
cooked: <p>You can define <code>cX</code> as an <code>nn.Parameter</code>, which would make them trainable.<br>
Also, you can wrap them in an <code>nn.Module</code>, which could make handling them easier:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.c0 = nn.Parameter(torch.randn(1, 1))
        self.c1 = nn.Parameter(torch.randn(1, 1))
        self.c2 = nn.Parameter(torch.randn(1, 1))
        self.c3 = nn.Parameter(torch.randn(1, 1))
        
    def forward(self, x, y):
        out = self.c0 * torch.cos(self.c1 * y) + self.c2 + self.c3 * x**2
        return out

model = MyModel()
x, y = torch.randn(1, 1), torch.randn(1, 1)
out = model(x, y)
out.backward()
for name, param in model.named_parameters():
    print(name, param.grad)
</code></pre>

username: smu226
cooked: <p>Thank you for your reply! This is the final form that I want to get to. However, my question was if there is a way to go automatically from the sympy expression to this from (sorry if that was not clear). I have lots of sympy expressions, and I want to pass them automatically to a pytorch module for training, not write everything by hand. For example, for the line <code>out = self.c0 * torch.cos(self.c1 * y) + self.c2 + self.c3 * x**2</code> I would like some function that would turn my sympy expression to that automatically, something like: <code>out = sympy_to_pytorch(sympy_expression)</code>. And something similar for picking the parameters out of my sympy expression and doing the <code>self.c0 = nn.Parameter(torch.randn(1, 1))</code> part on its own.</p>

username: ptrblck
cooked: <p>Oh, in that case I misunderstood the use case. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>That’s an interesting idea. I’m unfortunately not deeply familiar with sympy, but do you know, if the “parameters” have some kind of flags?<br>
I assume we could use some parser to check for the “trainable flag” and create <code>nn.Parameters</code> out of them.</p>
<p>EDIT: I’m not sure, if the creation of the actual function would be easy. It seems we would need to have some mapping between sympy to PyTorch methods.</p>

username: Claytonius
cooked: <p>I’ve been working on doing exactly that by adding functionality to sympy that allows reasoning about pytorch tensors.</p>
<p>I’ve documented a few examples here:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/Zer0Credibility/symtorch" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars0.githubusercontent.com/u/22800528?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/Zer0Credibility/symtorch" target="_blank" rel="noopener nofollow ugc">Zer0Credibility/symtorch</a></h3>

<p>Initial Commit. Contribute to Zer0Credibility/symtorch development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: ptrblck
cooked: <p>That looks pretty cool! Thanks for sharing <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: strangemonad
cooked: <p>There’s also sympytorch which looks relatively complete <a href="https://github.com/patrick-kidger/sympytorch" class="inline-onebox" rel="noopener nofollow ugc">GitHub - patrick-kidger/sympytorch: Turning SymPy expressions into PyTorch modules.</a></p>

username: robinechuca
cooked: <p>There’s also cutcutcodec which is able to compile sympy expression to pytorch function with some optimizations. But not a lot of functions are yet implemented: <a href="https://framagit.org/robinechuca/cutcutcodec/-/blob/main/cutcutcodec/core/compilation/sympy_to_torch.py" class="inline-onebox" rel="noopener nofollow ugc">cutcutcodec/core/compilation/sympy_to_torch.py · main · robin richard / cutcutcodec · GitLab</a></p>

username: robinechuca
cooked: <p>Now, this module is very complete: <a href="https://framagit.org/robinechuca/cutcutcodec/-/tree/main/cutcutcodec/core/compilation/sympy_to_torch?ref_type=heads" class="inline-onebox" rel="noopener nofollow ugc">cutcutcodec/core/compilation/sympy_to_torch · main · robin richard / cutcutcodec · GitLab</a></p>
<ul>
<li>It supports a lot of functions.</li>
<li>Factorization of the common sub patterns for a complete suppression of the redundancy.</li>
<li>Dynamic or compiled evaluation for safety or speed.</li>
<li>Sort the function argument for better broadcasting efficiency.</li>
<li>Cache smart management for variant and invariant input tensors.</li>
<li>Reuse internal variable for limiting RAM and speed-up by limiting allocation.</li>
</ul>