In this paper we show that a simple beam approximation of the joint distribution between attention and output is an easy , accurate , and efficient attention mechanism for sequence to sequence learning . The method combines the advantage of sharp focus in hard attention and the implementation ease of soft attention . On five translation and two morphological inflection tasks we show effortless and consistent gains in BLEU compared to existing attention mechanisms . In structured input-output models as used in tasks like translation and image captioning , the attention variable decides which part of the input aligns to the current output . Many attention mechanisms have been proposed ( Xu et al . , 2015 ; Bahdanau et al . , 2014 ; Luong et al . , 2015 ; Martins and Astudillo , 2016 ) but the de facto standard is a soft attention mechanism that first assigns attention weights to input encoder states , then computes an attention weighted ' soft ' aligned input state , which finally derives the output distribution . This method is end to end differentiable and easy to implement . Another less popular variant is hard attention that aligns each output to exactly one input state but requires intricate training to teach the network to choose that state . When successfully trained , hard attention is often found to be more accurate ( Xu et al . , 2015 ; Zaremba and Sutskever , 2015 ) . In NLP , a recent success has been in a monotonic hard attention setting in morphological inflection tasks ( Yu et al . , 2016 ; Aharoni and Goldberg , 2017 ) . For general seq2seq learning , methods like Sparse-Max ( Martins and Astudillo , 2016 ) and local attention ( Luong et al . , 2015 ) were proposed to bridge the gap between soft and hard attention . * Both authors contributed equally to this work In this paper we propose a surprisingly simpler alternative based on the original joint distribution between output and attention , of which existing soft and hard attention mechanisms are approximations . The joint model couples input states individually to the output like in hard attention , but it combines the advantage of end-to-end trainability of soft attention . When the number of input states is large , we propose to use a simple approximation of the full joint distribution called Beam-joint . This approximation is also easily trainable and does not suffer from the high variance of Monte-Carlo sampling gradients of hard attention . We evaluated our model on five translation tasks and increased BLEU by 0.8 to 1.7 over soft attention , which in turn was better than hard and the recent Sparsemax ( Martins and Astudillo , 2016 ) attention . More importantly , the training process was as easy as soft attention . For further support , we also evaluate on two morphological inflection tasks and got gains over soft and hard attention . In this paper we showed a simple yet effective approximation of the joint attention-output distribution in sequence to sequence learning . Our joint model consistently provides higher accuracy without significant running time overheads in five translation and two morphological inflection tasks . An interesting direction for future work is to extend beam-joint to multi-head attention architectures as in ( Vaswani et al . , 2017 ; Xu Chen , 2018 ) .

Challenge: Softmax attention models are popular because of their differentiable and easy to implement nature while hard attention models outperform them when successfully trained.
Approach: They propose a method to approximate the joint attention-output distribution which provides sharp attention as hard attention and easy implementation as soft attention.
Outcome: The proposed approach outperforms soft attention models and recent hard attention and Sparsemax models on five translation tasks and also on morphological inflection tasks.