Mapping Trignometric Math functions as Neural Networks

username: siddu1998
cooked: <p>Hello All,</p>
<p>I am new to PyTorch and started with a simple project for learning purposes. Here is the data format I currently have current=(x1,y1) target=(x2,y2) and would like to predict the angle between the two points using the cosine formula = (x2-x1) / distance.euclidean(current,target). I have a simple neural network that takes the  2 pairs points in and has 2 outputs (sin, consine). The encoding for the angles is inspired from this post <a href="http://practicalcryptography.com/miscellaneous/machine-learning/encoding-variables-neural-networks/" rel="noopener nofollow ugc">Link.</a> My code with comments is as follows. The results i get are absurd loss values and totally wrong predictions clearly indicate I am going wrong. I am trying hard to figure out where but feeling lost. Any help on this would be appreciated. The activation function and loss function<br>
were inspired form this <a href="https://discuss.pytorch.org/t/custom-loss-function-for-discontinuous-angle-calculation/58579">Post</a></p>
<pre><code class="lang-auto">class Net(nn.Module):
  def __init__(self):

       super(Net, self).__init__()
       self.fc1 = nn.Linear(4,2)
       self.htanh = nn.Hardtanh(-1,1)

   def forward(self, x):
       return self.htanh(self.fc1(x))


df = pd.read_csv('data_robot_nav_edited.csv')

#Angle Encoding : http://practicalcryptography.com/miscellaneous/machine-learning/encoding-variables-neural-networks/
df['sin_desired'] = df['desired_heading'].apply(lambda x: math.sin(x*0.0174533))
df['cos_desired'] = df['desired_heading'].apply(lambda x: math.cos(x*0.0174533))

X = df[['x_t', 'y_t', 'x_c', 'y_c']]
y = df[['sin_desired','cos_desired']]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=RANDOM_SEED)
print(X.head())

X_train = torch.from_numpy(X_train.to_numpy()).float()
y_train = torch.squeeze(torch.from_numpy(y_train.to_numpy()).float())
X_test = torch.from_numpy(X_test.to_numpy()).float()
y_test = torch.squeeze(torch.from_numpy(y_test.to_numpy()).float())
print(X_train.shape, y_train.shape)
print(X_test.shape, y_test.shape)


net = Net()
optimizer = optim.Adam(net.parameters(), lr=0.001)
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
X_train = X_train.to(device)
y_train = y_train.to(device)

X_test = X_test.to(device)
y_test = y_test.to(device)

net = net.to(device)

criterion = nn.MSELoss()
criterion = criterion.to(device)

for epoch in range(1000):
   y_pred = net(X_train)
   y_pred = torch.squeeze(y_pred)
   train_loss = criterion(y_pred, y_train)
   print(f'''epoch {epoch} Train set - loss: {(train_loss)}, ''')
   optimizer.zero_grad()
   train_loss.backward()
   optimizer.step()

def get_desired_angle(x_t, y_t, x_c, y_c):
 t = torch.as_tensor([x_t, y_t, x_c, y_c]) \
     .float() \
     .to(device)
 output = net(t)
 return output


output = get_desired_angle(6,6,0,0)
print(output)
sin_theta = output[0]
cos_theta = output[1]
tan_theta = sin_theta/cos_theta
print(sin_theta,cos_theta,tan_theta) ```</code></pre>

username: ptrblck
cooked: <p>Could you check the shape of <code>y_pred</code> before using <code>torch.squeeze</code> as well as the shape of <code>y_train</code>?<br>
Since you are using <code>nn.MSELoss</code> both tensors should have the same shape as otherwise a broadcasting would be applied internally and will raise a warning in newer PyTorch versions.</p>