# Research on Applying the \"Shift\" Concept to Deep Attention Matching
## 3 Improved DAM
### Shifted Granularity Matching  
Explaining the meaning of each level of granularity in DAM is difficult, but we can make an analogy in another way, for a sentence, its granularity from shallow to deep can be represented as a whole sentence, small clause, phrase, and word. What is achieved in DAM is similarity matching at the same level of granularity, e.g., sentence-to-sentence, phrase-to-phrase. However, we believe that similarity matching is also possible between different levels, where sentences and words are related, for example, \"colourful flowers bloom in the park\" and \"flowers\". In this view, DAM ignores the association between the different granularities, and the space of association can be further explored. Therefore, we propose a shifted granularity matching approach, as shown in Figure 1. In the matching phase of the model, the granularity of the response is shifted and matched with the granularity of the utterance. For example, the 1 st layer granularity of utterance is matched to the 2 nd layer granularity of response, the 2 nd layer granularity of utterance is matched to the 3 rd layer granularity of response, and so on. Figure 1 assumes that there are 5 layers of the stacked self-attention, i.e., there are 5 layers of granularity representations. \(U_{i}^{l}\) denotes the \(i\)th utterance of the \(l\)th layer of granularity, and \(R^{l}\) denotes the \(l\)th layer of granularity of the responses. The shifted distance can not only be 1, the matching module with a shifted distance of 2 is illustrated in Figure 1b. When the shifted distance is \(d\),\(\ldots\ U^{5}\) and \(R^{1}\ldots\ R^{d}\) are not required to be matched, the following equation represents the definition of the shifted matrix,  
\[M^{u_{i}\sigma,l}_{self}=\{U^{l}_{i}[k]^{T}.R^{l+d}[t]\}_{n_{u_{i}}\times n_{r}} \tag{15}\]  
where \(U^{l}_{i}[k]^{T}\) denotes \(k^{th}\) embedding vector in \(U^{l}_{i}\), \(R^{l+d}[t]\) denotes \(t^{th}\) vector in \(R^{l+d}\), \(i\) denotes \(i^{th}\) utterance, \(l\) denotes an \(l^{th}\) layer of granularity, and \(d\) denotes the shifted distance. Considering the full use of the information, we can make use of the unused granularities, we can match the tail of the utterance which is not matched with the head of the response. Figure 2a shows the model for a complete match at a shifted distance of 1, i.e., matching the representation at the 5 th layer of the utterance with the representation at the 1 st layer of the response, and Figure 2b shows the model for a complete match at a shifted distance of 2, which matched the representation at 4 th and 5 th layers of the utterance with the representation at 1 st and 2 nd layer of the response respectively. In the aggregation phase, the matrix obtained from the shifted granularity matching module will be added to the matching part of DAM and together with the original self-attention matching matrix and the cross-attention matching matrix, it will be integrated into a new 3D matching matrix, each pixel value in the new matrix \(Q_{i,k,l}\) is specified by  
\[Q_{i,k,t}=[M^{u_{i}\sigma,l}_{self}[k,t]]^{L}_{l=0}\oplus[M^{u_{i}\sigma,l}_{ cross}[k,t]]^{L}_{l=0}\oplus[M^{u_{i}\sigma,l}_{shift}[k,t]]^{L-d}_{l=0} \tag{16}\]  
Figure 1: Shifted granularity matching module. (**a**) In the matching phase of the model, the granularity representation of response is mismatched with the granularity representation of utterance for the multi-layer granularity representation generated by the stacked self attention modules. (**b**) a matching module with a misalignment distance of 2. In Figure 1, the yellow blocks represent the granularity of utterance, orange blocks represent the granularity of response. While the orange boxes are used for one-to-one matching. Finally, the red-shifted granularity matching matrix is generated. Figure 2: Shifted granularity matching module in full matching.