## 2 Methods
### Imputing a Muscle Activation to Joint Kinematics Relationship and Classification via CNN  
Figure 3 illustrates the examples of \(\mathcal{I}_{\text{easy}}\), \(\mathcal{I}_{\text{fair}}\), and \(\mathcal{I}_{\text{chal}}\), which were produced by our proposed approach introduced below. The subscript of \(\mathcal{I}\)--easy, fair, and chal (=challenging)--represents the level of manipulation for the recognizability of input image features from human eyes' point of view. For example, from Figure 3a, we could easily distinguish dumbbell curl and dumbbell kickback by the spectral color of upper- or lower-stripes. In Figure 3b, in contrast, the bottom stripe seemed to be related to the upper stripes but separated; especially, in case of dumbbell curl. Indeed, Figure 3a was produced by interleaving the interchannel normalized elbow joint angle data in-between biceps and triceps channels, whereas the elbow joint angle data were simply added at the bottom of the 8-channel sEMG data. From Figure 3c, we can see diagonal patterns but the patterns themselves are rather challenging to be described. The underlying rationale of the above manipulation was that we wanted to know whether producing an input image data set with considering human eyes' point of view had an effect on classification via CNN or not. Especially, we want to investigate how it affected feature extraction for imputing a muscle activation to joint kinematics relationship, which could be implied by the training loss and the test accuracy. In this sense, we summarized the feature characteristics of \(\mathcal{I}_{\text{easy}}\), \(\mathcal{I}_{\text{fair}}\), \(\mathcal{I}_{\text{chal}}\) as follows:  
* \(\mathcal{I}_{\text{easy}}\): an image contained features that were easy to be recognized by human eyes, e.g., geometry, color, etc. * \(\mathcal{I}_{\text{fair}}\): an image contained features that were fair to be recognized by human eyes, e.g., simple rules, simple pattern, local differences, etc. * \(\mathcal{I}_{\text{chal}}\): an image contained features that were challenging to be recognized by human eyes, e.g., mathematically defined patterns such as correlation, attraction, bifurcation, fractal, etc. \(\mathcal{I}_{\text{easy}}\), \(\mathcal{I}_{\text{fair}}\), \(\mathcal{I}_{\text{chal}}\) can be produced according to manipulations introduced below. Let \(s_{m}(n)\) and \(a(n)\) be the \(m\)th channel sEMG data and the elbow joint angle data after signal processing, respectively. Recall that both data were of \(\mathbb{R}^{1\times 16}\). First, \(\mathcal{I}_{\text{easy}}\) was defined to be  
\[\mathcal{I}_{\text{easy}}(m,n)=\left[ \begin{array}{c}s_{1}(n)\\ \vdots\\ s_{4}(n)\\ a(n)\\ s_{5}(n)\\ \vdots\\ s_{8}(n)\end{array} \right]=\left[ \begin{array}{cccc}s_{1}(1)&s_{1}(2)& \cdots&s_{1}(16)\\ \vdots&\vdots&\ddots&\vdots\\ s_{4}(1)&s_{4}(2)&\cdots&s_{4}(16)\\ a(1)&a(2)&\cdots&a(16)\\ s_{5}(1)&s_{5}(2)&\cdots&s_{5}(16)\\ \vdots&\vdots&\ddots&\vdots\\ s_{8}(1)&s_{8}(2)&\cdots&s_{8}(16)\end{array} \right]\in\mathbb{R}^{9\times 16}. \tag{1}\]  
Namely, \(a(n)\) was interleaved in-between the biceps and triceps channels of the sEMG. Second, we defined \(\mathcal{I}_{\text{fair}}\) as  
Figure 3: The examples of \(\mathcal{I}_{\text{easy}}\), \(\mathcal{I}_{\text{fair}}\), and \(\mathcal{I}_{\text{chal}}\): For all (**a**–**c**), a pair of figures represents dumbbell curl (**left**) versus dumbbell kickback (**right**). \[\mathcal{I}_{\text{fair}}(m,n)=\left[ \begin{array}{c}s_{1}(n)\\ s_{2}(n)\\ \vdots\\ s_{8}(n)\\ a(n)\end{array} \right]=\left[ \begin{array}{cccc}s_{1}(1)&s_{1}(2)&\cdots&s_{ 1}(16)\\ s_{2}(1)&s_{2}(2)&\cdots&s_{2}(16)\\ \vdots&\vdots&\ddots&\vdots\\ s_{8}(1)&s_{8}(2)&\cdots&s_{8}(16)\\ a(1)&a(2)&\cdots&a(16)\end{array} \right]\in\mathbb{R}^{9\times 16}. \tag{2}\]  
Thus, \(a(n)\) was merged at the bottom of the sEMG channels. Third, to define \(\mathcal{I}_{\text{chal}}\), we first defined matrices \(S_{m}\) and \(A\) which could be generated by replicating \(s_{m}(n)\) and \(a(n)\) to row- and column-wise, respectively:  
\[S_{m}=\left[