## 2 Methods
### Object Manipulation T-DRL
#### 2.2.1 Transformer Network with Attention Mechanism  
The transformer network illustrated in Figure 2a is composed of two modules to derive an optimal hand pose for natural grasping: the 6-DoF object affordance encoder and the 21-DoF hand pose encoder-decoder. The object affordance encoder first uses a self-attention mechanism to discover the local relationship between every element in the source input of 6-DoF object affordance. The attention layer in the encoder uses three independent feedforward networks to transform the input (i.e., the hand pose of 21-DoF and object affordance of 6-DoF) into query (\(Q\)), key (\(K\)), and value (\(V\)) tensors of dimension \(d_{q}\), \(d_{v}\) and \(d_{v}\), respectively. In the self-attention layer, \(Q\), \(K\), and \(V\) are calculated from each encoder input. Then, the self-attention layer finds the correlation between encoder inputs. Then, calculating the attention value, the query and key tensors are dot-produced and scaled. A SoftMax function then obtains the attention probabilities and the resulting tensor takes a linear projection using the value tensor. The attention is then computed with the following equation [38]:  
\[Attention(Q,K,V)=softmax\bigg{(}\frac{QK^{T}}{\sqrt{d_{k}}}\bigg{)}V \tag{1}\]  
The computed attention weights represent the relative importance of the input in the sequence (keys) for each particular output (query) and multiply it by a dynamic weight (value). To improve the range of input features that could be learned via attention, the  
Figure 2: The architecture of the transformer Network. (**a**) In Training, it takes both 6-DoF object affordance and 21-DoF hand pose as input to estimate an optimal 21-DoF hand pose for grasping the object. (**b**) At test time, 6-DoF object affordance is generated through GraspNet from an RGB-D image taken in the environment. Then, the trained transformer network estimates the optimal hand pose for generated object affordance. multi-head attention runs multiple self-attention layers in parallel and learns different projections of the input data. These are expressed as:  
\[ \begin{cases}MultiHead(Q,K,V)=Concat(h_{1},\dots,h_{h})W^{O}\\ h_{i}=Attention\Big{(}QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V}\Big{)}\end{cases}  \tag{2}\]  
where linear transformations \(W_{i}^{Q}\in\mathbb{R}^{d_{model}\times d_{k}}\), \(W_{i}^{K}\in\mathbb{R}^{d_{model}\times d_{k}}\), \(W_{i}^{V}\in\mathbb{R}^{d_{model}\times d_{k}}\), and \(W_{i}^{O}\in\mathbb{R}^{hd_{p}\times d_{model}}\) are parameter matrices. The parameter \(h\) represents the number of subspaces we compute with \(d_{k}=d_{v}=d_{model}/h=4\). Therefore, the two self-attention layers shown in Figure 2 compute the attention matrix to find similarity within the 6-DoF object affordance and 21-DoF hand pose individually. The self-attention output is an embedding of dimension \(d\) that carries the mutual information of each element in the input. Then, a residual connection expands this embedding, concatenating with the original object affordance. Later, a normalization layer and two feedforward networks, both with a non-linear activation function, output the encoded representation (attention value) of the object affordance from the encoder. The hand pose encoder does the same job as the object affordance encoder, with the target input of the hand pose being 21-DoF. The output of the hand pose encoder is an embedding vector of dimension \(d\), which computed the mutual information of the input elements through a self-attention mechanism, using a residual connection and two feedforward layers. The outputs of both encoders are used as input for the hand pose decoder. The cross-attention layer learns to map between the encoded values of the hand pose of 21-DoF and the object affordance of 6-DoF. Through this mapping the network can estimate the optimal hand pose for the object affordance \(\phi^{M}\). The decoder uses the cross-attention mechanism.