Problem migrating code from Torch to PyTorch

lakehanne: I have a code I wrote in Torch which I am migrating to PyTorch. Its a regression-based learning problem. In Torch, after defining my network, I define concatenate my inputs along the 2nd axis as follows <SCODE>    inputs = torch.cat({inputs[1], inputs[2], inputs[3], inputs[4], 
                        inputs[5], inputs[6], inputs[7], inputs[8],
                        inputs[9]}, 2)
<ECODE> And the network is as defined: <SCODE>  (lstm1): LSTM(9, 9)
  (lstm2): LSTM(9, 6)
  (lstm3): LSTM(6, 6)
  (drop): Dropout (p = 0.3)
  (fc1): Linear (6 -> 3)
  (fc2): Linear (3 -> 3)
<ECODE> In torch, I can forward call and backward call okay as the inputs tensor id of batch size [100, 9] torch.*Tensor. I construct the network the same way in PyTorch and concatenate my nine inputs along the 1st axis as <SCODE>	inputs = torch.cat((base_in, base_out, left_in, 
						left_out, right_in, right_out, 
						z, pitch, yaw), 1)
<ECODE> I define my model in PyTorch as <SCODE>class LSTMModel(nn.Module):
    def __init__(self, nFeatures, nCls, nHidden, nineq=12, neq=0, eps=1e-4, 
                noutputs=3,numLayers=1):
        super(LSTMModel, self).__init__()
        
        self.nFeatures = nFeatures
        self.nHidden = nHidden
        self.nCls = nCls
        self.nineq = nineq
        self.neq = neq
        self.eps = eps

        self.cost = nn.MSELoss(size_average=False)
        self.noutputs = noutputs
        # self.neunet = nn.Sequential()
        self.lstm1  = nn.LSTM(nHidden[0],nHidden[0],num_layers=numLayers)
        self.lstm2  = nn.LSTM(nHidden[0],nHidden[1],num_layers=numLayers)
        self.lstm3  = nn.LSTM(nHidden[1],nHidden[2],num_layers=numLayers)
        self.drop   = nn.Dropout(0.3)
        self.fc1 = nn.Linear(nHidden[2], noutputs)
        self.fc2 = nn.Linear(noutputs, noutputs)

        self.M = Variable(torch.tril(torch.ones(nCls, nCls)))
        self.L = Parameter(torch.tril(torch.rand(nCls, nCls)))
        self.G = Parameter(torch.Tensor(nineq/2, nCls).uniform_(-1,1))

        """
        define constraints, z_i, and slack variables, s_i,
        for six valves. z_i and c_i are learnable parameters
        """
        self.z0 = Parameter(torch.zeros(nCls))
        self.s0 = Parameter(torch.ones(nineq/2))
        self.z0p = Parameter(torch.zeros(nCls))
        self.s0p = Parameter(torch.ones(nineq/2))

    def forward(self, x):
        nBatch = x.size(0)
        # LSTM-dropout-LSTM-dropout-lstm-dropout-FC-QP-FC
        x = x.view(nBatch, -1)
        x = self.lstm1(x)
        x = self.drop(x)
        x = self.lstm2(x)
        x = self.drop(x)
        x = self.lstm3(x)
        x = self.drop(x)
        x = self.fc1(x)
<ECODE> But my calls to forward gives me runtime errors like so <SCODE>RuntimeError: matrices expected, got 1D, 2D tensors at /home/robotec/Documents/NNs/locuclab-pytorch/torch/lib/TH/generic/THTensorMath.c:1224
<ECODE> I wonder what I must be doing wrong.

jekbradbury: 

apaszke: 

lakehanne: <SCODE>        neunet = nn.Sequential()
       local rnn        
        rnn = nn.FastLSTM(9, 9, 5)
        neunet:add(rnn) 
        rnn = nn.FastLSTM(9, 6, 5)
        neunet:add(rnn) 
        rnn = nn.FastLSTM(6, 3, 5)
        neunet:add(rnn) 
        neunet:add(nn.Dropout(0.3))
        neunet:add(nn.Linear(3, 3, bias))      
       neunet = nn.Sequencer(neunet, 3)
<ECODE> <SCODE>class LSTMModel(nn.Module):
    def __init__(self, noutputs=3,numLayers=3):
        super(LSTMModel, self).__init__()
        
        self.nHidden = [9, 6, 6]
        self.cost = nn.MSELoss(size_average=False)
        self.noutputs = noutputs
        self.num_layers =  numLayers
        
        self.lstm0     = nn.LSTM(self.nHidden[0], self.nHidden[0], self.num_layers, 
                                batch_first= True, dropout=0.3)
        self.lstm1     = nn.LSTM(self.nHidden[1], self.nHidden[1], self.num_layers, 
                                batch_first= True, dropout=0.3)
        self.lstm2     = nn.LSTM(self.nHidden[2], self.nHidden[2], self.num_layers, 
                                batch_first= True, dropout=0.3)
        self.fc = nn.Linear(self.nHidden[2], noutputs)

    def forward(self, x):

        h0 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[0])) 
        c0 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[0]))
        
        #Now propagate the rnn 
        #rnn layer 1
        out0, _ = self.lstm0(x, (h0, c0))  
        
        #second rnn layer
        h1 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[1])) 
        c1 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[1]))
        out1, _ = self.lstm1(out0, (h1, c1)) 
        
        #third rnn layer
        h2 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[2])) 
        c2 = Variable(torch.zeros(self.num_layers, x.size(0), self.nHidden[2]))
        out2, _ = self.lstm2(out1, (h2, c2)) 
        
        #hidden layer of last time step
        out = self.fc(out2[:, -1, :])        
        return out
<ECODE> So I forward a DoubleTensor defined as <SCODE>RuntimeError: matrices expected, got 1D, 2D tensors at /home/robotec/Documents/NNs/locuclab-pytorch/torch/lib/TH/generic/THTensorMath.c:1224
<ECODE>

lakehanne: 

jekbradbury: 

lakehanne: Yes, it is a time series. Could you give a minimal working example of how I can correctly form a PyTorch model and a time series input tensor based off the snippet I wrote above? Sorry to be a pain.

jekbradbury: If x is 50000 by 9, then what’s your time dimension, what’s your batch dimension, and what’s your feature dimension?

lakehanne: mini-batch dimension is 100, feature dimension is 3, time dimension is 5.

jekbradbury: Then you need to pass in a tensor of dimension 5 x 100 x 3

lakehanne: 

sahar_khosravi: