Are there any recommended methods to clone a model?

mrdrozdov: I’m interested to clone a model for various reasons (makes it easy to stack untied versions of the same model for instance). Any recommended methods for doing so?

apaszke: 

mrdrozdov: Thanks! Seems like a reasonable approach.

mrdrozdov: Ended up going with something like: <SCODE>model_clone = model_cls(**kwargs)
model_clone.load_state_dict(copy.deepcopy(original_model.state_dict()))
<ECODE> Works well for me!

apaszke: No need to deepcopy the state dict. Its contents won’t be assigned, but copied into the clone.

rk.epfl: I have a situation where I am copying weights back and forth between two instances of the same model. Unfortunately, copy.deepcopy is not working for me. I am having to do: <SCODE>mp = list(model.parameters())
mcp = list(model_copy.parameters())
n = len(mp)
for i in range(0, n):
    mp[i].data[:] = mcp[i].data[:]
<ECODE> While this is fine, I wonder why deepcopy function is not working.

damaru: Did you figure it out? I am having the same problem

rk.epfl: I did not find a different way when I had this problem. But more recently, I used the “copy_” operation to copy weights layer by layer from one model to another: <SCODE>mydict = mymodel.state_dict()
layer_names = list(mydict)

# Now, to copy values for a particular layer using the name or index of it:

mydict[layer_names[index_of_layer]].copy_(some_data_with_matching_shape)
<ECODE> If there is a better way, I would be happy to learn.

Royi: What happens if I do this: <SCODE>hNetModel = Model()
    for trainBatch, trainLabels in hTrainLoader:
        <Train the Model by a Function>
        modelEvaluationMetric = hNetModel(Validation)
        if(modelEvaluationMetric < bestModelMetric):
            hBestModel = hNetModel
<ECODE>

albanD: 

Royi: <SCODE>hNet in NetList
hNet = TrainNet(hNetModel)
modelEvaluationMetric = hNetModel(Validation)
        if(modelEvaluationMetric < bestModelMetric):
            hBestModel = hNet
<ECODE> I thought at least when something gets back from a function it is a different copy of it (Yea, I’m not so experienced with Python).

Brando_Miranda: just to make your answer clear you mean: <SCODE>new_mdl = copy.deepcopy(model)
<ECODE> right?

Brando_Miranda: why is deep copy not working for you? in what way is it not working compared to what u expected?

Brando_Miranda: Does something inspired from: or not work for you?

X-czh: Hi, copy.deepcopy(model) works fine for me in previous PyTorch versions, but as I’m migrating to version 0.4.0, it seems to break. It seems to have something to do with torch.device. How should I do cloning properly in version 0.4.0? (torch.device device) (str type, int index)

kuzand: Deepcopy is not working for me. I have a function train(model) which returns the trained model, model_trained = train(model_untrained). However as result both are trained at the end, but I want the model_untrained to be unchanged. So I tried to deep-copy the model_untrained inside the function before the training loop, but It is not working – the model is not trained correctly. Any idea why is it happening?

ptrblck: 

kuzand: Yes I am training the copied model. You are right about the optimizer, I was passing the original model parameters to it. Thanks for spotting it!

shivamsaboo17: 

52153d3b7bd2d9066500: <SCODE>import pickle
copyed_model = pickle.loads(pickle.dumps(model))
<ECODE>