From Scratch to PyTorch

username: 111398
cooked: <p>Hallo. I’m just starting with PyTorch.</p>
<p>I have built the following program in python and I want to translate it to PyTorch so I can compare the result but I still haven’t found how to do it. I’m currently attenting a course for PyTorch but it’s really slow and I need some help. Can anyone tell me how to write this code using PyTorch?<br>
It’s the code in the paper Deep Learning: An Introduction for Applied<br>
Mathematicians - Catherine F. Higham, Desmond J. Higham but I have translated it from MATLAB to Python</p>
<p>P.S. Thank you everyone in advance for your time and sorry if it’s against the rules of the Forum. Please delete it if it is.<br>
Also if you have any suggestion about Courses or YouTube videos/series that you found helpful while you were getting starting with PyTorch please let me know.</p>
<pre><code class="lang-auto">import numpy as np
from numpy.linalg import norm as norm
np.random.seed(5000)

def activate(x,W,b): #Evaluates sigmoid function. x is the input vector, y is the output vector W contains the weights, b contains the shifts. The i-th component of y is activate((Wx+b)_i) where activate(z) = 1/(1+exp(-z))
    z=W@x+breturn 1/(1+np.exp(-z))

def cost(W2,W3,W4,b2,b3,b4):
	costvec = np.zeros(10)
	for i in range(10):
		x=[x1[i],x2[i]]
		a2=activate(x,W2,b2)
		a3=activate(a2,W3,b3)
		a4=activate(a3,W4,b4)
		costvec[i] = norm(y[:,i]-a4,2)
		costvec[i]
	costval = norm(costvec,2)**2
	return costval

x1 = np.array([0.1,0.3,0.1,0.6,0.4,0.6,0.5,0.9,0.4,0.7])
x2 = np.array([0.1,0.4,0.5,0.9,0.2,0.3,0.6,0.2,0.4,0.6])
y=np.array([ [1,1,1,1,1,0,0,0,0,0],[0,0,0,0,0,1,1,1,1,1] ])

W2=np.random.random((2,2))
W3=np.random.random((3,2))
W4=np.random.random((2,3))
b2=np.random.random(2)
b3=np.random.random(3)
b4=np.random.random(2)
#forward and back propagate
eta = 0.05 #learning rate
Niter = 10**6 #number of SG iterations
savecost = np.zeros(Niter) #Value of cost at each iteration
for counter in range(Niter):
	k=np.random.randint(0,10) #choose a training point at random
	x=np.array([x1[k],x2[k]])
	#forward pass
	a2 = activate(x,W2,b2)
	a3 = activate(a2,W3,b3)
	a4 = activate(a3,W4,b4)
	#backward pass
	delta4 = a4*(1-a4)*(a4-y[:,k])
	delta3 = a3*(1-a3)*(W4.T@delta4)
	delta2 = a2*(1-a2)*(W3.T@delta3)
	#gradient step
	W2 = W2 - eta*delta2@x
	W3 = W3 - eta*delta3.reshape(3,1)@a2.reshape(1,2)
	W4 = W4 - eta*delta4.reshape(2,1)@a3.reshape(1,3)
	b2 = b2 - eta*delta2
	b3 = b3 - eta*delta3
	b4 = b4 - eta*delta4
	newcost = cost(W2,W3,W4,b2,b3,b4)
	savecost[counter]=newcost
	print(newcost , 'i=',counter)

import matplotlib.pyplot as plt
iterr=[i for i in range(counter+1)]
plt.ylim([10**(-4),10])
plt.xlim([0,Niter])
plt.semilogy(iterr,savecost)
plt.show()
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="111398" data-post="1" data-topic="100575">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/111398/48/30247_2.png" class="avatar"> 111398:</div>
<blockquote>
<p>I have built the following program in python and I want to translate it to PyTorch so I can compare the result but I still haven’t found how to do it. I’m currently attenting a course for PyTorch but it’s really slow and I need some help. Can anyone tell me how to write this code using PyTorch?</p>
</blockquote>
</aside>
<p>I think the best approach would be if you could explain what you have done so far, what is not working, and where you are stuck.<br>
Once we know this information we might be able to guide you through it. Especially if you are taking a course I would recommend to try to debug it yourself first and avoid looking at perfect solutions. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>