Repository: OpenNMT/OpenNMT-tf

Issue:
can we output alignment of ParallelInputter?
I was using the ParallelInputter and transformer, and I found that we can't output the alignment,
  

> (optional) For compatible models, also output the alignments (can be: null, hard, soft,default: null).
>   with_alignments: null

according the turtorial, want to know which part is not supported, and how to get the alignment?

my model is simliar as https://github.com/OpenNMT/OpenNMT-tf/blob/master/config/models/multi_source_transformer.py 





Discussion:
The alignment information is not returned in this case because it's unclear what to do: return alignment for first source? for second? for both?

What do you suggest?
for my specific case, i want to use alignment for first source.

for generic cases, maybe can add parameter to control output what alignments.
As a rule of thumbs, let's always return the attention of the first source. It seems better than returning nothing at the moment.

Resolving patch:
diff --git a/opennmt/decoders/self_attention_decoder.py b/opennmt/decoders/self_attention_decoder.py
--- a/opennmt/decoders/self_attention_decoder.py
+++ b/opennmt/decoders/self_attention_decoder.py
@@ -74,7 +74,7 @@ def maximum_sources(self):
 
   @property
   def support_alignment_history(self):
-    return self.num_sources == 1
+    return True
 
   def map_v1_weights(self, weights):
     m = []
diff --git a/opennmt/layers/transformer.py b/opennmt/layers/transformer.py
--- a/opennmt/layers/transformer.py
+++ b/opennmt/layers/transformer.py
@@ -356,12 +356,12 @@ def __init__(self,
     self.self_attention = TransformerLayerWrapper(
         self.self_attention, dropout)
     self.attention = []
-    for _ in range(num_sources):
+    for i in range(num_sources):
       attention = MultiHeadAttention(
           num_heads,
           num_units,
           dropout=attention_dropout,
-          return_attention=num_sources == 1)
+          return_attention=i == 0)
       attention = TransformerLayerWrapper(
           attention, dropout)
       self.attention.append(attention)


Test patch:
diff --git a/opennmt/tests/decoder_test.py b/opennmt/tests/decoder_test.py
--- a/opennmt/tests/decoder_test.py
+++ b/opennmt/tests/decoder_test.py
@@ -93,14 +93,15 @@ def _testDecoder(self,
         training=True)
     self.assertEqual(outputs.dtype, dtype)
     output_time_dim = tf.shape(outputs)[1]
-    if decoder.support_alignment_history and num_sources == 1:
+    if decoder.support_alignment_history:
       self.assertIsNotNone(attention)
     else:
       self.assertIsNone(attention)
     output_time_dim_val = self.evaluate(output_time_dim)
     self.assertEqual(time_dim, output_time_dim_val)
-    if decoder.support_alignment_history and num_sources == 1:
-      attention_val, memory_time = self.evaluate([attention, tf.shape(memory)[1]])
+    if decoder.support_alignment_history:
+      first_memory = memory[0] if isinstance(memory, list) else memory
+      attention_val, memory_time = self.evaluate([attention, tf.shape(first_memory)[1]])
       self.assertAllEqual([batch_size, time_dim, memory_time], attention_val.shape)
 
     # Test 2D inputs.
@@ -111,7 +112,7 @@ def _testDecoder(self,
         step,
         state=initial_state)
     self.assertEqual(outputs.dtype, dtype)
-    if decoder.support_alignment_history and num_sources == 1:
+    if decoder.support_alignment_history:
       self.assertIsNotNone(attention)
     else:
       self.assertIsNone(attention)