TypeError: cannot assign 'torch.FloatTensor' as parameter 'layer_weights' (torch.nn.Parameter or None expected)

username: reudrax
cooked: <p><span class="hashtag">#43835</span>  Issue description<br>
Traceback (most recent call last):<br>
File “run_techqa_layer.py”, line 605, in <br>
main()<br>
File “run_techqa_layer.py”, line 599, in main<br>
model = train(args, train_dataset, model, optimizer, tokenizer, model_evaluator)<br>
File “run_techqa_layer.py”, line 211, in train<br>
outputs = model(**inputs)<br>
File “C:\Users\Rudraksh\Anaconda3\envs\bert_pre\lib\site-packages\torch\nn\modules\module.py”, line 547, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “D:\techqa-layer\modeling.py”, line 228, in forward<br>
self.layer_weights = F.softmax(self.layer_weights,dim=0)<br>
File “C:\Users\Rudraksh\Anaconda3\envs\bert_pre\lib\site-packages\torch\nn\modules\module.py”, line 610, in <strong>setattr</strong><br>
.format(torch.typename(value), name))<br>
TypeError: cannot assign ‘torch.FloatTensor’ as parameter ‘layer_weights’ (torch.nn.Parameter or None expected)</p>
<h2>System info</h2>
<p>PyTorch version: 1.2.0<br>
Is debug build: False<br>
CUDA used to build PyTorch: 10.0</p>
<p>OS: Microsoft Windows 10 Education<br>
GCC version: (i686-posix-dwarf-rev0, Built by MinGW-W64 project) 8.1.0<br>
Clang version: Could not collect<br>
CMake version: Could not collect</p>
<p>Python version: 3.6 (64-bit runtime)<br>
Is CUDA available: False<br>
CUDA runtime version: No CUDA<br>
GPU models and configuration: No CUDA<br>
Nvidia driver version: No CUDA<br>
cuDNN version: No CUDA</p>
<pre><code>    def __init__(self, config):
	super(LayeredBertforQA, self).__init__(config)
	self.num_labels = config.num_labels
	self.bert = BertModel(config)
	self.n_weights = config.num_hidden_layers + 1
	self.layer_weights = nn.Parameter(torch.zeros(self.n_weights,dtype=torch.float,device="cpu"))
	self.qa_outputs = nn.Linear(config.hidden_size, config.num_labels)
	self.options = None
	self.init_weights()	

    def forward(self, input_ids=None, attention_mask=None, token_type_ids=None,start_positions=None,end_positions=None):		
	outputs = self.bert(
		input_ids,
		attention_mask=attention_mask,
		token_type_ids=token_type_ids,
	)

	hidden_states = outputs[2]
	stacked_hidden_states = torch.stack(hidden_states)

	self.layer_weights = F.softmax(self.layer_weights,dim=0)</code></pre>

username: suraj.pt
cooked: <p><code>self.layer_weights</code> is not a tensor but a <a href="https://pytorch.org/docs/stable/generated/torch.nn.parameter.Parameter.html" rel="nofollow noopener">Parameter</a> that is registered as an attribute of your module. That’s why when you assign it a tensor it throws an error. Instead, try<br>
<code>self.layer_weights = nn.Parameter(F.softmax(self.layer_weights,dim=0))</code></p>

username: reudrax
cooked: <p>If I make a self.layer_weights a Tensor, and then later on make it a Paramater, will it work? Just for information</p>

username: suraj.pt
cooked: <p>Yep, should work. <code>Parameter</code> is literally a <code>Tensor</code> with the convenience of automatically registering that tensor as a trainable parameter. If you leave it as a tensor, it won’t show up in your <code>module.parameters()</code>.</p>