I want to optimize a few output of my model's layers, how should i do?

username: Will97
cooked: <p>I have a ResNet18 network and have obtained a small set of specific parameters at each layer. Using a hook function, I have obtained the output of each layer. I want to optimize the model’s input to achieve specific values for their corresponding outputs. For example, in the ‘layer1.0.conv1’ layer (Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), I obtained the parameter [x][y][m][n] and the output of this layer. How should I optimize the input to achieve a specific value for the output corresponding to this parameter [x][y][m][n]? Please note that I want to optimize the input, not the parameter. Also, in a convolutional layer, one parameter may correspond to multiple outputs.</p>

username: soulitzer
cooked: <p>Does making your inputs requires_grad=True work? You may also need to update your update your optimizer to include the inputs e.g. something like optim.SGD(model.parameters() + [inputs])</p>

username: Will97
cooked: <p>YES,I already use requires_grad=True . THe problem is how to find the specfic target in output corresponding to the parameter [x][y][m][n]?</p>

username: soulitzer
cooked: <p>Could you do something like this?</p>
<pre><code class="lang-python">import torch
import torch.nn as nn
import torch.nn.functional as F

a = torch.rand(16, 10, requires_grad=True)
seq = nn.Sequential(nn.Linear(10, 10), nn.Linear(10, 10))

intermediate = [None]

def fn(m, input, output):
    intermediate[0] = output
seq[0].register_forward_hook(fn)
seq[0].weight.requires_grad_(False)
seq[0].bias.requires_grad_(False)

out = seq(a)

target = torch.rand_like(intermediate[0])
loss = F.l1_loss(intermediate[0], target)
loss.backward()

print(a.grad)

</code></pre>

username: Will97
cooked: <p>it seems you dont understand my question.</p>
<blockquote>
<p>import torch<br>
import torch.nn as nn<br>
import torch.nn.functional as F</p>
<p>a = torch.rand(3,16, 10, requires_grad=True)<br>
net = nn.Sequential(<br>
nn.Conv2d(3, 16, 3, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(16, 32, 3, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(32, 10, 3, padding=1),<br>
nn.ReLU()<br>
)<br>
intermediate = [None]<br>
weights=[None]<br>
def fn(m, input, output):<br>
intermediate[0] = output<br>
weights[0]=m.weight</p>
<pre><code># print('weight',m.weight.shape)
</code></pre>
<p>net[2].register_forward_hook(fn)<br>
out = net(a)<br>
print(intermediate[0].shape)</p>
<p>weight=weights[0][0][0][0][0]</p>
<h1>What is the index of the output corresponding to this parameter?</h1>
<h1>If I cannot obtain the index, I cannot optimize my input using the output of this layer.</h1>
<p>target_output=intermediate[0].[?][?][?]</p>
<p>optimizer=torch.optim.SGD(a,lr=0.1)</p>
<p>loss=(target_output-10)**2<br>
loss.backward()<br>
optimizer.step()</p>
</blockquote>
<p>I am grateful for you help.</p>

username: soulitzer
cooked: <p>If I understand correctly, by “correspond” you mean “if one perturbs the parameter but some amount, which indices of the output would change”. If that is the case, there are at least two ways:</p>
<p>You could derive it potentially by looking at what convolution does, e.g. given weight[c_out][c_in][i][j]. I perhaps the following view of output is affected: output[:, c_out, i, h - k_h - i + 1, w - k_w - i + 1] where h, w are the height and width of your input, k_h, k_w are the dimensions of the kernel, etc.</p>
<p>A super dumb way to do this is just to set the particular index of your parameter to inf or nan and observe which outputs become inf or nan. You could also use this dumb method as a test for the programmatic approach.</p>