foldl/torch-forum huggingface dataset:
Using previous output values as part of a loss function
Category: reinforcement-learning

yoonholee: I am trying to implement TRPO, and I need the gradient of the network parameters w.r.t. the KL divergence between the current action distribution and the action distribution after the parameters are changed. What is the best way to implement this? How do I make sure action_distribution0 doesn’t get backpropagated through?

apaszke: <SCODE>action_distribution0 = model(state).detach()
# detach() blocks the gradient
# also action_distribution0 doesn't require grad now, so it can be a loss target

# =========================================
# change the parameters of the network here
# =========================================

KL(Network(state), action_distribution0).backward()
<ECODE>

yoonholee: This worked great, thanks!

tomsercu: <SCODE>fake = netG(noise)
fake = fake.detach() # this is not an inplace op, so need to re-assign?
output = netD(fake)<ECODE>

smth:
Matrix-vector multiply (handling batched data)

emanjavacas: <SCODE># (batch x inp)
v = torch.randn(5, 15)
# (inp x output)
M = torch.randn(15, 20)
<ECODE> Compute: <SCODE># (batch x output)
out = torch.Tensor(5, 20)
for i, batch_v in enumerate(v):
    out[i] = (batch_v * M).t()
<ECODE>

apaszke: 

apaszke: 

emanjavacas: 

apaszke: I think these are the only Py3 versions that we support.

emanjavacas: I was wondering… Matrix batched-vector multiply is just matrix-matrix multiply, but what about the inverse situation: Batched-matrix vector multiply?

apaszke: No, there’s not. But you could do it by reshaping your vector to look like a matrix (no memory copy there, just stride tricks): <SCODE>M_batch = ... # batch x dim1 x dim2
v = ... # dim2
v_view = v.unsqueeze(0).expand(-1, len(v)).unsqueeze(2) # batch x dim2 x 1
output = M_batch.bmm(v_view) # batch x dim1 x 1
# optionally output.squeeze(2) to get batch x dim1 output
<ECODE>

emanjavacas: Alright, thanks for the info. Given efficient reshaping capabilities, all this operations can easily be implemented by the user.

emanjavacas: I got it to work passing the batch size instead of -1. <SCODE>v.unsqueeze(0).expand(M_batch.size(0), len(v)).unsqueeze(2)
<ECODE>

apaszke:
Titan X Maxwell (old) issues

Eugenio_Culurciello: I can confirm we can run Torch7 on those machines with no issues. Also on the new Pascal Titan X, we do not have this problem. Maybe an issues with driver? Anyone else has the same issue?

colesbury: We don’t support Python 3.4 Do you still have this issue with Python 3.5?

Eugenio_Culurciello:
Adding a scalar?

ririw: Dumb question, but how do I make a scalar Variable? I’d like to add a trainable parameter to a vector, but I keep getting size-mismatch problems. <SCODE># Works, but I can't make the 2 a 
# parameter, so I can't do gradient descent on it
Variable(torch.from_numpy(np.ones(5))) + 2 
<ECODE> Thanks!

fmassa: <SCODE>x = Variable(torch.from_numpy(np.ones(5)))
y = Variable(torch.Tensor([2]).double()) # numpy is double by default
z = x + y.expand(x.size())
<ECODE>

ririw: Great, thanks for that.
When will the conda package be updated?

junonia: I found some features like upsampling_nearest are in the github but not in the conda package.   Is there a timeline when the conda package will be updated?

smth: the conda package was updated yesterday evening with the 0.1.7 release which has the upsampling_nearest available

junonia: Thanks!  I need to use install instead update to get the new package.

Atcold: 

trypag: conda update pytorch torchvision -c soumith

Atcold: <SCODE>conda config --add channels soumith
<ECODE>

McLawrence:
Help clarifying repackage_hidden in word_language_model

WenchenLi: <SCODE>def repackage_hidden(h):
    """Wraps hidden states in new Variables, to detach them from their history."""
    if type(h) == Variable:
        return Variable(h.data)
    else:
        return tuple(repackage_hidden(v) for v in h)
<ECODE> I dont think I fully understand what the “history” includes, can somebody helps clarify this? Thanks!

apaszke: 

jekbradbury: 

apaszke: 

LaceyChen17: So we do not need to repackage hidden state when making predictions ,since we don’t do a BPTT ?

jdhao: 

ratishsp: <SCODE>def repackage_hidden(h):
    """Wraps hidden states in new Variables, to detach them from their history."""
    if type(h) == Variable:
        return Variable(h.data, requires_grad=True)
    else:
        return tuple(repackage_hidden(v) for v in h)
<ECODE> Thanks.

alphadl: it has already been updated to be compatible with the latest PyTorch version: <SCODE>def repackage_hidden(h):
    """Wraps hidden states in new Tensors, to detach them from their history."""
    if isinstance(h, torch.Tensor):
        return h.detach()
    else:
        return tuple(repackage_hidden(v) for v in h)
<ECODE>
CUDA out of memory error

Kalamaya: I ran my net with a large minibatch on the GPU with problem, and then ctrl-c out of it. However when I try to re-run my script I get: RuntimeError: cuda runtime error (2) : out of memory at /home/soumith/local/builder/wheel/pytorch-src/torch/lib/THC/generic/THCStorage.cu:66 Nothing changed and it just worked before. Is the memory on the GPU not being released? Is there a way to force the GPU memory to release its memory before a run? Thanks.

apaszke: 

Kalamaya: nvidia-smi shows me using 8GB out of the 12 GB… What puzzles me is that literally just stopped with a large minibatch size that training just fine, but then when I re-start the python script it craps out.

apaszke: 

Kalamaya: 

Kalamaya: 

mjchen611: Hi, Can you give me some suggestions? Thank you so much.

mjchen611: And when I set num_workers = 0,the (RAM, but not GPU) memory not increases largely with the increase of epoch… Can you give me some suggestions or instructions about the problem? Thank you so much.

Harikrishna.Vydana: i face the same error…is there a way to fin the variables which are getting cached on gpu
Inplace matrix modification

ypxie: 

apaszke: Yes it is. This should work: <SCODE>L = Variable(torch.Tensor(i_size, j_size))
# it's important to not specify requires_grad=True
# it makes sense - you don't need grad w.r.t. original L content,
# because it will be overwritten.
for i in range(i_size):
    for j in range(j_size):
        L[i, j] = # compute the value here
<ECODE> But beware, it might be very very slow! Not only because you’ll be looping over elements in Python, but also because it will involve a lot of autograd ops to compute this, and there’s a constant overhead associated with each one. It’s not a huge problem if you’re doing relatively expensive computation like matrix multiplication or convolution, but for simple ops it can be more expensive than the computation alone.

ypxie: <SCODE>loss = 0
L = Variable(torch.Tensor(i_size, j_size))
W = Parameter(torch.Tensor(20, 20)) #fake size
for t in range(time_step):
   for i in range(i_size):
       for j in range(j_size):
           L[i, j] = func(L[i,j]) # a function of wrt old L
   out = get_output(L, W)  #output computed from L and weights W
   loss  += loss_func(out, label[t])

<ECODE> In this case, can I still get gradient of loss wrt the weights W using autograd? It seems that L is overwritten at each timestep.

apaszke: 

ypxie: Tahnks！ That’s exactly what I want to know.
Repackage_hidden function in word_language_model example

erogol: Can you pls explain what this function do? <SCODE>def repackage_hidden(h):
    """Wraps hidden states in new Variables, to detach them from their history."""
    if type(h) == Variable:
        return Variable(h.data)
    else:
        return tuple(repackage_hidden(v) for v in h)<ECODE>

apaszke: 

erogol: I see thank you. Is this a general practice for BPTT in pytorch or just memory efficiency?

apaszke:
Torch.cat() throws error?

Xiaochen_Li: When running these code: I got an error message: But similar code like this: or this: works well. I’m totally confused. I really need to extract some elements from one tensor and cat them into another tensor. What should I do? Thanks!

apaszke: I’m not sure why does it work for regular tensors, I’ll need to look into that, but I think it’s an unintended behaviour.

Xiaochen_Li: But got this:

apaszke: <SCODE>>>> import torch
>>> x = torch.range(1, 2)
>>> y = torch.range(3, 4)
>>> torch.stack([x, y], 0)

 1  2
 3  4
[torch.FloatTensor of size 2x2]
<ECODE>
PyTorch only using one GPU?

Kalamaya: From nvidia-smi, I can see that during training, my pyTorch script is only using one GPU. Is there a way to make it use the others? (I have two). Thanks.

apaszke: 

Kalamaya: Now in my main script, I call: I took a look at the documentation for the DataParallel, and what I currently do is this: Now this seems to run, BUT, it complains that it doesnt know what “forward” is. (The forward member function that it). So I am confused as to what exactly I should be putting through this “DataParallel” exactly… every member function of my DNN?.. Thanks

apaszke:
Python 2.7, 3.5, and 3.6

sungtae: Dear All, Thanks!

smth: py3 versions (3.5 and 3.6) also have CUDA multiprocessing available. Apart from that feature difference, they are within epsilon of performance with 2.7.

sungtae: 

fromLittleAcorns: I changed from 2.7 to 3.6 but am having problems with the model zoo.  The Imagenet example will download pre-trained weights using a hickle file.  Unfortunately it seems that the hickle module is not compatible with Python V 3.* due to the way strings are handled, which makes it difficult to use the pre-trained examples. Does anybody know a way around this, I don’t  want to have to go back to 2.7 if I can avoid it John
PyTorch for older then 7.5 CUDA

Artsiom_Chapialiou: PS I have old laptop with GeForce 330M and the newest available driver for Ubuntu 16.04 is nvidia-340. So the max version of CUDA I can use is 6.5.

smth: The minimum CUDA version we want to support is 7.0. You can try building from source, but we dont plan to support 6.5

Chen-Wei_Xie:
Model produces random output when using loops to define hidden layers

Jordan_Campbell: Hi, I’m adapting the DCGAN models from the PyTorch examples and had a problem saving and loading the parameters. Specifically, when I saved the parameters and then read them back in at a later stage, I would get a different output for a consistent input. I debugged and found what I believed to be the problem, however I’m not sure of the cause. I am using a loop to define hidden layers of the MLP (I’m not using the conv nets): <SCODE>self.nhl = 4
self.nh = [2, 2, 2, 2]
self.fc = []
for i in range(self.nhl-1):
    self.fc.append(nn.Linear(self.nh[i], self.nh[i+1]))
<ECODE> which gave the following output on repeated executions of the program (with a model saved earlier): <SCODE> jordan [ src ] $ python main.py 

First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.558199584484
hidden layer --> 0.203323662281
output --> 0.305809795856 -0.139149516821 0.538217186928

 jordan [ src ] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.45568972826
hidden layer --> 0.0742047131062
output --> 0.425929844379 -0.366338938475 0.542093634605
 
jordan [ src ] $ python main.py 
First layer --> 0.0268119424582 0.0
hidden layer --> 0.0
hidden layer --> 0.0
hidden layer --> 0.460717827082
output --> 0.345742940903 -0.0932856351137 0.694804787636
<ECODE> Note that the outputs at each of the layers changes on each execution (I confirmed the parameters loaded are all consistent on each iteration). Forward is implemented as: <SCODE>    x = F.relu(self.input(_input))

    print 'First layer -->', x.data[0][0], x.data[0][1]
    for i in range(self.nhl-1):
        x = F.relu(self.fc[i]((x)))
        print 'hidden layer -->', x.data[0][0]

    output = self.output(x)
<ECODE> If I unroll the loops then it produces consistent output. Is there a bug in my loop or is there a reason you can’t use loops to define hidden layers?

NgPDat: 

Jordan_Campbell:
Cuda Numpy operations in custom Modules

Vijay_Rengarajan: Hi, Just converting the model and input to cuda results in “RuntimeError: numpy conversion for FloatTensor is not supported” error at “result = abs(rfft2(numpy_input)) line”. <SCODE>class BadFFTFunction(Function):
    
    def forward(self, input):
        numpy_input = input.numpy()
        result = abs(rfft2(numpy_input))
        return torch.FloatTensor(result)
    
    def backward(self, grad_output):
        numpy_go = grad_output.numpy()
        result = irfft2(numpy_go)
        return torch.FloatTensor(result)<ECODE>

apaszke: 

Vijay_Rengarajan: Okay, thanks! I will check.
Choose backends

junonia: It is not obvious to me how to select different backends.  In torch, you can mix layers from cudnn and cunn easily.  Is there an easy way to do the same in pytorch?

apaszke: Just out of curiosity, what’s the reason why you need to select a particular backend?

junonia: Thanks for the information.  Sometimes I extract extra information from the layers.  For example, for maxpooling layer,  I have cases that make use of the indices in the cunn implementation.  The cudnn implementation does not have the indices variable available. So in cases there is no corresponding cudnn implementation, cunn option will be selected for that layer?

apaszke: Yes, if you request the pooling function/module to return indices and cuDNN doesn’t support it, it will be automatically ignored. We’re always picking the fastest backend that supports all specified options.

junonia: Just to clarify,  if I have a network with some layers supported by cudnn, but if the net contains a layer that cudnn does not support (e.g.  FractionalMaxPooling),  will the entire net be run using cunn or just the FractionalMaxPooling layer?

apaszke: The decision is made on a per-operation basis. When we have a choice of multiple backends for a given op, we filter out these that don’t support the options you chose and pick the fastest one. For example cuDNN 5 doesn’t support dilated convolutions. If your model has both dilated and regular convolutions, THCUNN will be used for dilated and cuDNN for regular ones. This has the upside of always running at the max speed, across many different systems, that have only a subset of backends (e.g. no CUDA support), based on a single model definition.

supakjk: I’d like to use CUDNN libraries for deterministic modules but not for non-deterministic modules such as CNN libraries. If I can choose backends for each module, I would be able to get some degree of speedup while keeping determinic results. You don’t really have a plan to allow manually choosing backends?

apaszke: We want to add the ability to pick the backends in a more fine-grained manner, but we haven’t discussed any solutions yet.

Shubhankar: Any update on this? Is there a way to choose conv algorithms?
How to save your net in pytorch?

Kalamaya: I think it’s the torch.save(aNet, ‘myNet’), and this doesn’t complain, but I cant seem to load it… :-/

trypag: Code from the imagenet example : Loading: <SCODE>if args.resume:
        if os.path.isfile(args.resume):
            print("=> loading checkpoint '{}'".format(args.resume))
            checkpoint = torch.load(args.resume)
            args.start_epoch = checkpoint['epoch']
            best_prec1 = checkpoint['best_prec1']
            model.load_state_dict(checkpoint['state_dict'])
            print("=> loaded checkpoint '{}' (epoch {})"
                  .format(args.evaluate, checkpoint['epoch']))
<ECODE> Saving: <SCODE>torch.save({
            'epoch': epoch + 1,
            'arch': args.arch,
            'state_dict': model.state_dict(),
            'best_prec1': best_prec1,
        }, 'checkpoint.tar' )
<ECODE>

rajarsheem: Does this loading recover the last updated learning rate ?

jpeg729: Add the following to the state you save <SCODE>"optim_state": optim.state_dict(),
<ECODE> and load like this <SCODE>optimizer.load_state_dict(checkpoint['optim_state'])<ECODE>
Mask tensors from indexes (e.g from multinomial sampling)

ludc: Hi, I am looking for a ‘good’ way to transform matrix of indexes to a onehot matrix (line by line). For example,

apaszke: <SCODE>>>> x = torch.LongTensor([[1, 3], [2, 4]])
>>> torch.zeros(2, 5).scatter_(1, x, 1)

 0  1  0  1  0
 0  0  1  0  1
[torch.FloatTensor of size 2x5]
<ECODE>

1112: 

ptrblck: You could use this code snippet to get all indices of the value you are comparing to: <SCODE>x = torch.LongTensor([[1, 3], [2, 4]])
res = torch.zeros(2, 5).scatter_(1, x, 1)
(res == 1.).nonzero()
<ECODE>
Document for C API

longcw: 

apaszke: 

longcw: Thank you for your reply. I have found it in the source code.

alexis-jacq: float *int input_data = THFloatTensor_data(input)

ncullen93: 

Frank_Lee: <SCODE>long offset = input->storageOffset+i*input->stride[0]+j*input->stride[1];
float val = THFloatStorage_get(input->storage, offset);

<ECODE>

apaszke: 

fmassa: 

lupoglaz: 

alexis-jacq: 

apaszke: 

alexis-jacq: Ah ok ! thanks! Now I’m starting to understand better how you create a python library with C code. Also, this link is really helpfull: I don’t know if somewhere else you provide the list of all the “torch.x” available functions.

apaszke: 

alexis-jacq: Wow. How did I miss that

Andrei_Pokrovsky: I think the doc for the C api would be nice, there’s so much super cool functionality that gets a bit obscured by the macro concatenation in the C source and the source being scattered across files plus navigation tools getting confused by the macros (I really like the design of the API though and how the macros are used, it was clearly designed with maximum simplicity in mind which is awesome). Also incremental updates in release notes on what’s new in the C API would help keeping track of changes. I could also see how added overhead of something like this might be an opportunity cost. Just an idea.

jinserk:
Verifying proper usage of bach_norm and eval()

Kalamaya: I want to make sure that I am utilizing the batchnorm functionality properly: I understand how to code up the layers to perform the batchnorm, however one thing I want to make sure that I am doing right, is basically putting the net into “evaluation” mode, so that the parameters of the batch_norm do not keep getting updated, when I am say, periodically running through the validation set, or when I am done and running over new data etc. Is this correct? Thank  you.

Jordan_Campbell:
Build model from repeated template

Atcold: <SCODE>class Template(nn.Module):
    def __init__(self):
        super().__init__()
    def forward(self, *input):
        pass
<ECODE> <SCODE>class BuildModel(nn.Module):
    def __init__(self, number_of_layers):
        super().__init__()
        # self.layer_n = Template() for n in range(0, number_of_layers)
    def forward(self, *input):
        pass
<ECODE>

trypag: 

Atcold: <SCODE>b - a - A - B
<ECODE> <SCODE>c - b - a - A - B - C
<ECODE>

trypag: Just about the nn.Sequential point, I implemented a network with 3 branches in parallel, I am not using nn.Sequential, I just end up summing the variables of my branches, this is pure autograd. Hope it can help

ClementPinard: what about <SCODE>def template(num_layers):
    if num_layers == 1:
        return nn.Sequential([a,A])
    else:
        return nn.Sequential([f(num_layers), template(num_layers-1), F(num_layers)])
<ECODE>

Atcold: 

apaszke: 

apaszke: Also, we’ll be merging an official solution into the core today or tomorrow.

Atcold: Oh, that would be great. I would have been mucking around with string based attribute settings, otherwise … which should be just fine, no?

apaszke: 

Atcold:
Discussion about datasets and dataloaders
Category: vision

ClementPinard: However, there has been some issues that I had to solve in order to match my workflow. So I created this topic to either discuss about possible ameliorations in the dataset interface or ameliorations in my own workflow, which i like but may be far from perfect. So to my mind, dataset would be the class where you decide where to take samples, and data loader is the one deciding if we apply specific data augmentation routines or not. <SCODE>dataset = datasets.foo(data, split)
train_loader = torch.utils.data.DataLoader(
    dataset,batch_size,workers,
    transforms=[input_transform,target_transform,co_transform]
    )
test_loader = torch.utils.data.DataLoader(
    dataset,batch_size,workers,
    transforms=[input_transform_test,target_transform_test,co_transform_test]
    )
dataset.train()
enumerate(train_loader) #*batches from train set with data augmentation
dataset.eval()
enumerate(test_loader) #batches from test set without data augmentation
dataset.train()
enumerate(test_loader) #batches from train set without data augmentation
<ECODE> Last problem will be that graphic functions will be involved in a module that is not from vision (because data loaders are from pytorch/utils) but i think vision was separated from the rest because it involved PIL operations which was not necessary for some other problems such as text embedding. But I think if we work with tensors, these transform don’t have to be graphics and could be anything as long as tensors are given in output. sum up of ideas co_transforms splitted dataset dynamic samplers random sampling without replacement for epoch size < dataset size attach transformations to data loaders add tensor related image loading and transformations to avoid numpy HxWxC to CxHxW conversion

shaun: Do you have an update on this especially the data split with dataloader for train, validation, and test?

deepcode: same question, Do you have an update on this especially the data split with dataloader for train, validation, and test? Im about to do the same thing and dont want to write a new piece of code for this.

smth: 

chsasank: 

kevinzakka: The main logic of the code is as follows: Generate a train dataset and test dataset using the argument train in torchvision.datasets.XYZ where XYZ is your desired data (i.e. CIFAR10 or ImageNet). Figure out the length of your validation set num_valid. If say you want 10% of your training data to be used for validation, then you would multiply the total length of the training set by 0.1. Create a list of indices of size num_train, shuffle it, and then slice num_valid indices from it and call it valid_idx and store the rest in train_idx. Feed these indices to separate instances of SubsetRandomSampler. Finally feed these 2 samplers to torch.utils.data.DataLoader using the sampler argument. And voila!
Output of RNN is not contiguous

mrdrozdov: I would expect the output of RNN to be contiguous in memory. This doesn’t seem to be the case. For instance, the final output in this snippet has output.is_contiguous() == False. <SCODE>train = True
num_layers = 1
bidirectional = True
bi = 2 if bidirectional else 1

x = Variable(torch.from_numpy(_x), volatile=not train)
batch_size, seq_length, input_dim = x.size()

rnn = nn.LSTM(input_dim, model_dim / bi, num_layers,
    batch_first=True,
    bidirectional=bidirectional,
    )

h0 = Variable(torch.zeros(num_layers * bi, batch_size, model_dim / bi), volatile=not train)
c0 = Variable(torch.zeros(num_layers * bi, batch_size, model_dim / bi), volatile=not train)

print(x.is_contiguous())
# True

# Expects (input, h_0):
#   input => batch_size x seq_length x model_dim
#   h_0   => (num_layers x bi[1,2]) x batch_size x model_dim
#   c_0   => (num_layers x bi[1,2]) x batch_size x model_dim
output, (hn, cn) = self.encode(x, (h0, c0))

print(output.is_contiguous())
# False<ECODE>

apaszke: Yeah I think that’s expected. Depending on the chosen backend, a contig or non-contig result may be returned. Why is that a problem?

mrdrozdov: Okay. Noticed the same behavior for on cpu/gpu. I don’t have a specific problem, but assumed that if input is contiguous then output should/would be as well. Thanks for the response!

apaszke: No, I don’t think we’ve every guaranteed that. I’ll take a look at RNNs anyway, thanks for the notice!