How to interpret layerwise relevance propogation for RNN classification?

swap: <p>I am deriving Layerwise Relevance Propogation for an RNN classification model.</p>
<pre><code class="lang-auto">model configuration:

dnn(
  (encoder): Encoder(
    (embedding): Embedding(44, 32, padding_idx=0)
    (gru): GRU(32, 64)
  )
  (fc): ModuleList(
    (0): Linear(in_features=19584, out_features=128, bias=True)
    (1): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): Linear(in_features=128, out_features=128, bias=True)
    (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (4): Linear(in_features=128, out_features=2, bias=True)
  )
)

Max sequence length = 306

Dictionary size = 44

Dictionary:
{'_': 0, '?': 1, '\t': 2, '\n': 3, '#': 4, '(': 5, ')': 6, '+': 7, '-': 8, '1': 9, '12': 10, '2': 11, '21': 12, '23': 13, '24': 14, '3': 15, '32': 16, '34': 17, '35': 18, '4': 19, '43': 20, '5': 21, '6': 22, '64': 23, '7': 24, '73': 25, '=': 26, 'B': 27, 'Br': 28, 'C': 29, 'Cl': 30, 'F': 31, 'I': 32, 'N': 33, 'O': 34, 'P': 35, 'S': 36, '[': 37, ']': 38, 'c': 39, 'i': 40, 'n': 41, 'o': 42, 's': 43}
</code></pre>
<p>For an original input of ‘c1ccccc1C(C(NC)C)O’, tokenized input is as follows:</p>
<pre><code class="lang-auto">tensor([[ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  2, 39,
          9, 39, 39, 39, 39, 39,  9, 29,  5, 29,  5, 33, 29,  6, 29,  6, 34,  3]])
</code></pre>
<p>This input has been predicted as “positive” by above classification model.</p>
<p>My layerwise relevance propogation output is</p>
<pre><code class="lang-auto">R[0].shape
torch.Size([44, 306])
</code></pre>
<p>Lets look at position 305 in relevance tensor</p>
<pre><code class="lang-auto">R[0][:,304]
tensor([     0.0000,  -2519.1868,  -6721.7515,   1773.0427,   2274.4043,
      1315.9221,    689.7231,  -3970.1589,   -290.6332,   1902.7957,
     -3552.2190,   2137.6050,   7068.9321,    824.2842,  -2354.6023,
      -888.0222,    183.6073,  -1015.4807,  -1693.0706,   5424.1680,
     -1585.7656,    456.7188,   -424.3035,   -849.8544,  -3942.9111,
     -2586.9678,     38.4876,   2865.9019,  -4129.2144,  -6311.8076,
      3502.2493,   -495.0044,  -1296.6990,   5003.5410,   4107.5718,
      1664.5162, -10036.3223,   1197.8634,  -1420.5979,  -2937.0493,
     -5572.0791,   4293.1611,   5572.4443,  -3519.6023],
   grad_fn=&lt;SelectBackward0&gt;)
</code></pre>
<p>I want to find which of the 18 characters contributed most in classifying given input to positive class.<br>
How can I intepret R[0] tensor? For shown R[0][:,304], how will I know which of these 44 relevance score are of my 305th token (i.e. 34 in tokenized input)? any help will be deeply appreciated!</p>