Understanding this diagram

username: vainaijr
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/2/62a9b213e5ae90887cac725439367cc52cf05772.png" data-download-href="https://discuss.pytorch.org/uploads/default/62a9b213e5ae90887cac725439367cc52cf05772" title="Screenshot%20(36)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62a9b213e5ae90887cac725439367cc52cf05772_2_669x500.png" alt="Screenshot%20(36)" data-base62-sha1="e4OmwsVrFctuWLzaDoa6SyhdgdA" width="669" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62a9b213e5ae90887cac725439367cc52cf05772_2_669x500.png, https://discuss.pytorch.org/uploads/default/original/3X/6/2/62a9b213e5ae90887cac725439367cc52cf05772.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/2/62a9b213e5ae90887cac725439367cc52cf05772.png 2x" data-dominant-color="A9A9A9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20(36)</span><span class="informations">952×711 41.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>from <a href="https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">NLP From Scratch: Translation with a Sequence to Sequence Network and Attention — PyTorch Tutorials 2.2.0+cu121 documentation</a><br>
how do I interpret this diagram, does this mean that the word ‘five’ was translated because of ‘ans’, the words ‘to’, ‘scared’ were translated because of ‘mourir’, how does one read such diagrams?</p>
<p>thanks</p>

username: albanD
cooked: <p>Hi,</p>
<p>If you want to get a helpful answer, you should explain what you’re trying to do, how and why did you plot these diagrams and what you expected to see in them.</p>

username: alex.veuthey
cooked: <p>From a purely linguistic perspective, it looks like the diagram was shifted “one block” to the right.</p>
<p>Also, what direction are you translating into? Because there are nonsensical sentences in both english and french (“elle est trop petit” has wrong grammatical agreement, “he’s a bit young man” doesn’t make sense…).</p>
<p>It’s surprising as well that you get mostly 1.0 values throughout the diagrams, there doesn’t seem to be a lot of variation in the values, except for the third sentence pair…</p>

username: albanD
cooked: <p>Ho it’s the attention maps from the NLP tutorial ! ok.</p>
<p>I am not a specialist of this, <a class="mention" href="/u/spro">@spro</a> correct me if I’m wrong here !</p>
<ul>
<li>I think the model not not trained to see the difference between “petit” and “petite” in french and “petit” is considered correct for both. This would be to reduce the size of the dictionary.</li>
<li>There maps show, when using the attention model, which part of the input was used to produce each output. What it shows is that in most cases, only the previous word of the input is enough to produce the next word in the output. But there are some cases where a given word is used to output multiple outputs or a combination of two input words is used for one output word.</li>
<li>This is run on a small dataset and the model may be overfitting to the dataset: If every time you have a “cinq” you should output “five”, then there is no reason to look at other part of the sentence.</li>
</ul>

username: alex.veuthey
cooked: <p>Thanks for the clarifications, I didn’t see it was from the tutorial either <img src="https://discuss.pytorch.org/images/emoji/apple/grimacing.png?v=9" title=":grimacing:" class="emoji" alt=":grimacing:"></p>