Matrix factorization under the constraint of connectivity between observed and source data ~ Muscle synergy analysis based on connectivity between muscle and brain activities ~
Authors: Takashi Isezaki
Michiaki Suzuki
Yukio Koike
Ryosuke Aoki
Yukio Nishimura
Venue: Submitted to ICLR 2023 (2023)
Keywords: Matrix factorization
Muscle synergy

Abstract: Matrix factorization is a popular method to investigate the hidden elements in observed data for tasks such as speech separation and muscle synergy analysis. The hidden elements may be closely related to the source phenomenon that cause the observed phenomenon. 
 However, conventional methods do not always factorize the observed phenomenon elements with the connectivity between the observed and source phenomena because they only use the observed phenomenon. This paper proposes a matrix decomposition method that constrains the connectivity between observed and source data by using the representations from a decoding model from source data to observed data. We applied our method to the corticomuscular system, which is made up of corticospinal pathways between the primary motor cortex and muscles in the body and creates muscle synergies that enable efficient connections between the brain and muscles. In this context, muscle activities are the observed phenomenon and brain activities are the source. Many previous studies have analyzed muscle synergies using only observed muscle activity, but there may be unrevealed muscle synergies under the constraint of the connectivity between brain and muscle activity. We therefore simultaneously recorded the brain activity from multiple regions of an extensive cortical area and the activity of multiple muscles of a monkey's forelimb while it performed a reach and grasp task throughout the course of recovery from a partial spinal cord injury (SCI). Analysis from a dataset of the monkey before SCI showed that some of the muscle synergies calculated from the proposed method using brain and muscle activities, did not exhibit a high degree of similarity to synergies obtained from the conventional method. The proposed method results obtained from the monkey after SCI showed an adaptive change in the number of muscle synergies associated with the degree of functional recovery. Specifically, the numbers of muscle synergies obtained by the proposed method initially increased immediately after SCI and then gradually decreased, while those obtained by a conventional method maintained the same number before and after SCI. These results suggest that our method is able to capture the unrevealed connectivity in the corticomuscular system that contributes to functional recovery: in other words, that it can factorize the observed data under the constraint of the connectivity between the observed and source data. Our work thus demonstrates the importance of using not only observed data but also source data to reveal unknown hidden elements.

Under review as a conference paper at ICLR 2023
MATRIX FACTORIZATION UNDER THE CONSTRAINT OF CONNECTIVITY BETWEEN OBSERVED AND SOURCE DATA
MUSCLE SYNERGY ANALYSIS BASED ON CONNECTIVITY BETWEEN MUSCLE
AND BRAIN ACTIVITIES
Anonymous authors Paper under double-blind review
ABSTRACT
Matrix factorization is popular method to investigate the hidden elements in ob- served data for tasks such as speech separation and muscle synergy analysis. The hidden elements may be closely related the source phenomenon that cause the observed phenomenon. However, conventional methods do not always factorize the observed phenomenon elements with the connectivity between the observed and source phenomena because they only use the observed phenomenon. This paper proposes matrix decomposition method that constrains the connectivity between observed and source data by using the representations from decoding model from source data to observed data. The proposed method factorizes matri- ces by extracting and combining representations and weights from the regression model in the regression process. We applied our method to the corticomuscular system, which is made up of corticospinal pathways between the primary motor cortex and muscles in the body and creates muscle synergies that enable efficient connections between the brain and muscles. In this context, musele activities are the observed phenomenon and brain activities are the source. Many previous stud- ies have analyzed muscle synergies using only observed muscle activity, but there may be unrevealed muscle synergies under the constraint of the connectivity be- tween brain and muscle activity. We therefore simultaneously recorded the brain activity from multiple regions of an extensive cortical area and the activity of mul- tiple muscles
monkey's 's forelimb while it performed a reach and grasp task throughout the course of recovery from partial spinal cord injury (SCI). Analysis from a dataset of the monkey before SCI showed that some of the muscle syner- gies calculated from the proposed method using brain and muscle activities, did not exhibit a high degree of similarity to synergies obtained from the conventional method. The proposed method results obtained from the monkey after SCI showed an adaptive change in the number of muscle synergies associated with the degree of functional recovery. Specifically, the numbers of muscle synergies obtained by the proposed method initially increased immediately after SCI and then gradually decreased, while those obtained by a conventional method maintained the same number before and after SCI. These results suggest that our method is able to cap- ture the unrevealed connectivity the corticomuseular system that contributes functional recovery: in other words, that it can factorize the observed data under the constraint of the connectivity between the observed and source data. Our work thus demonstrates the importance of using not only observed data but also source data to reveal unknown hidden elements.
INTRODUCTION
As human beings, we observe complex mixture of real-world events as phenomena, and each observed phenomenon contains elements of the events in the real-world. For example, the sounds hear may contain the voice listening environmental sounds. Separating the elements in the observed phenomena helps us to understand the events individually. Matrix factorization is used investigate the elements in observed phe-
the voices of other speakers, and various

---

Under review as a conference paper at ICLR 2023
nomena. For example, specific speech sound can be extracted from sound containing mixture of multiple sounds (Smaragdis, 2007), the appropriate recommendation can be calculated from a customer's ' purchase history (Li al., 2006), and muscle synergies can be extracted from multiple muscle activities (Shourijeh et al., 2016).
In some cases, the observed phenomenon is caused by functional connection with the source phe- nomenon. This connectivity a functional connection that causes an observed phenomenon from a source phenomenon, and the factors of connectivity include such as human connections and anatom- ical connections. In the case of muscle synergy analysis, the corticospinal pathways between the pri- mary motor cortex and muscles function events normally propagate to the peripheral muscles. Studies on humans and monkeys have reported that brain and muscle activity are closely associated with cortico- mot-onuroooonn connections( Lemon, 2008; Baldissera & Cavallari, 1993; Lemon & Griffiths, 2005). In this case, muscle activities are the observed phenomenon and brain activities are the source. The corticomuscular system enables us to perform complex body movements by neurologically combining sets of simpler movements. These sets are observed as the basic pattern of muscle activity, namely, muscle synergies. Muscle activity can be observed by using electromyography (EMG), technique that reveals bioelectric potential signals. Since the framework of conventional matrix factorization uses only observed data to factor- ize observed data, it might not factorize the observed phenomenon elements with the connectivity between the observed and source phenomena.
corticomuscular system (Liu al., 2019), and cortical
We have therefore developed method to capture the elements in the observed data by considering the connectivity between both the observed data and the source data. Our basic idea is to use the representations in deep neural network (DNN) model to predict the observed data from the source data. In this paper, we assume both source and observed data can be measured. Specifically, the factorized matrices can function as an activation scalar value and vector of weights for the observed data at each sample. We can then obtain the factorized matrix under the connectivity constraints between the observed and source data by extracting these values.
In this paper, we report the results of applying the proposed method to muscle synergy analysis. As stated earlier, the corticomuseular system composed of corticospinal pathways between the primary motor cortex and muscles (Liu et al., 2019), and we can consider the muscle activities as observed data and the brain activities as source data. To obtain brain activities closely related to mus- cle activity, we simultaneously measured a monkey's 's brain signals using both electrocorticooraphy (ECoG) and electromyography (EMG). We measured the muscle and brain activities monkey before and after partial spinal cord injury to investigate the potential of the proposed method under the conditions of a stable and a dynamically changing nervous system.
Our main contributions are follows.
We show a novel matric factorization framework under the constraint of connectivity be- tween both observed data and source data, in contrast to the conventional approach that uses only observed data.
We propose method that utilizes model representation to predict the observed data from the source data as a factorization matrix.
We demonstrate the potential of the proposed method capture unrevealed matrix factors by applying it to muscle synergy analysis with comparison to the non-negative matrix approach.
2 RELATED WORKS
Matrix Factorization in Statistics: One approach taken in statistics is to factorize the data by ex- tracting the components that best represent the variation in the data. Principal component analysis (PCA) is typically used for this, where the dimensionality-reduee ized matrix (Lee & Seung, 1999). Another approach to separate independent components by assuming that the data contains multiple data originating from independent sources. Independent component analysis (ICA) separates signals by independence on the basis of higher-order statistics temporal correlations (Comon, 1994). The fast fixed-poin algorithm for independent component analysis (FastCCA) was proposed as a method to improve the convergence of ICA (Hyvärinen & Oja, 2000). Non-negative matrix factorization (NMF) factorizes the matrix and restricts it to be positive
data representation is a factor-
2

---

Under review as a conference paper at ICLR 2023
(Lee & Seung, 2000; Wang & Zhang, 2013). Since the real-world events observe often com- posed linear sums of non-negative values, they have a high affinity with NMF analysis, so has accordingly been used in variety of applications (Rajapakse & Wyse, 2003; Smaragdis. 2007; Shourijeh et al., 2016). One study on muscle synergy analysis showed that NMF is the most popular method for analysis (Rabbi al., 2020). However, these methods factorize the observed data using only the observed data.
The deep matrix factorization approach has shown promise in studies that focus on interpretability. Several methods to improve the interpretability by using representa- tions in an autoencoder process as multi-factorizer matrices have been proposed (Ye et al., 2018; Song et al., 2015; Wang & Zhang, 2021). To ensure meaning in the multilayered factor matrix, Trigeorgis et al. (2014) proposed method that pre-trains each layered factor matrix to represent elements such as facial pose, expressions, and identity. Sparseness is also essential to capture the essence of the data. Chang et al. (2021) proposed making the factorized matrix sparse in not only decoding but also encoding. One concept similar to our own is the use of implicit feedback (e.g., purchase history and unobserved ratings) in recommendation systems (Xue et al., 2017). While we feel that such feedback can contribute to improving accuracy, it is not the same source phe- nomenon. And technically, there are few approaches for matrix factorization to use both parameters in the regression process and the weights inside the model.
Deep Matrix Factorization:
Applications: There are a variety of general applications of the matrix factorization approach. In the biological field, metagenes and molecular pattern discovery have been studied by applying NMF (Brunet et al., 2004; Devarajan, 2008). In the engineering field, image processing tasks such as face recognition (Rajapakse & Wyse, 2003), acoustic processing (e.g.. speech-music separation) (Smita et al., 2007), and speech separation (Chien & Chen, 2006) have been studied. As for medical applications. fetal heart rate determination has been explored (Szalai Mozes, 2014). In the kine- matics and kinetics field, muscle synergy analysis is significant application of matrix factorization Shourijeh et (2016). Matrix factorization has also been applied to community structure detection (e.g., social networks, collaboration networks, and citation networks) to study interaction (Ye et al., 2018).
3 MATRIX FACTORIZATION UNDER CONSTRAINTS BETWEEN OBSERVED AND SOURCE DATA
Suppose that the observed data are arranged as matrix Y .y....., yrr RTXT, which is n channels sampled T times in time series. The observed vector at time t represents [yl 32 ........ y²) .........."... The goal of the proposed method is approximately factorize the observed data as
Y VU.
(1)
where V = [ul..... J, Raxr and U ju...., ur] E R'xT are the factor matrices to reconstruct the matrix Y. and r is the number of factorized components. Suppose the source data is the m channels sampled T times in time series, and the source data at time t is represented as the matrix X, = [= (mm,t)] RmxX which m channels sampled times time series. Considering the time difference (r) between observed and source data, the vector of source data at time t consists of data from time to - T. The i-th source data vector at time is (i.1) = (P(,.))) (((.c(()) ........... ((.tt)))) ".,,,t))...,,,,..-.).))..
The basic idea underlying the factorization of matrices under constraints between observed and source data to factor matrices from a model that includes connectivity constraints between ob- served and source data. We obtain this model by constructing a DNN prediction model from the source data to the observed data, where the factorized vectors e and u can assigned as the weight for each channel of observed data and the activation for the weights, respectively.
3.1 FRAMEWORK AND ARCHITECTURE
We designed regression model M that computes the observed data y, from the source data X,- As shown in the Fig. 1, the representation in the model designed to contain activation (scalar) and weights (vector).
3

---

Under review as a conference paper at ICLR 2023
Source data X, Regression model .M
Observed data V/
TCNet (c.
Mean
Scalarize
Sigmold (1)
Linear
Factors: th
Figure 1: Network architecture of proposed method. Activation (scalar) We is obtained from the Sigmoid layer. Vector is obtained from the weights in the Linear layer.
It is not only the source data that contributes to the factorization at time i; data over certain time range up to is also considered be involved in some cases. Temporal convolutional networks (TCN) have outperformed recurrent networks such as LSTM and GRU (Bai et al., 2018) 8) for handling multi-channel time-series data and have been applied to gesture recognition using muscle activity (Rahimian et al., 2021; 2022). Inspired by such research, we utilize TCN to incorporate the time- series features in our method.
TCNet: We implement TCNet the TCN in the regression model to capture the features from sequence data. As in (Bai et al.. 2018: Rahimian al., 2021; 2022), the TCNet layer consists of sequence of TBlock layers, as shown in Fig. 2. The TBlock is the dilated causal convolution, with the dilation factor exponentially increased (I, 2, 4, ...) and the number of filters e and the kernel size k. Each TBlock consists of two peated sequences consisting of extended causal convolutions ("DCconv")) weight norm, ReLU and Dropout, and finally residual connection.
TCNet (c, r)
TBlock 1.
TBlock (
TBlock Is
TBlock ei.2
The parameters for TCNer are the number fil- ters c, the kernel size k, and the number of di- lations d. When d set to three, the dilation of TBlock is 1, 2, and 4 sequentially. This layer outputs matrix E (exrr to Mean for each in- put data X,.
DCcomw
Waighthomm
ReLU
Dropout
DCconv
Weightoom
ReLU
Dropout
Sum
Mean, Scalarization, and Sigmoid layers: We designed these layers so that the captured fea- tures from TCNer would have a positive scalar value (activation u.). The Mean layer calcu- lates vector (R*) by averaging each filter's 's re- sult. The Scalarization layer is a linear layer to calculate scalar value from vector. The Sigmoid layer is set keep the computed sealar value w within a positive range. This scalar value u, is calculated at each time by sliding the window of the source data.
Figure 2: Architecture of temporal convolutional network.
Linear layer: The Linear layer inputs positive scalar value and calculates the reconstructed ob- served vector y. [5] ..,, 07') as
V: u,
where [w 2 ..... w"] is the weight for each element of observed vector. When acquiring vector of non-negative weights, the weights are clipped so that they are between 0 and 1 during training.
3.2 MATRIX FACTORIZATION THROUGH REGRESSION CALCULATIONS
First, the regression model M is trained using X, and y: (I, Similar to the previous approach, (Shourijeh et al., 2016; Rajapakse & Wyse, 2003), the proposed matrix factorization is verified by the restoration degree of the observed data, so there is no need to separate source and observed data into training and test data. The regression model M predicts the observed data yı from the source data X, as M(X,). An activation u, is obtained from the representation (scalar) from the layer of Sigmoid, and weights vector obtained by extracting the weights at the Linear

---

Under review as a conference paper at ICLR 2023
layer in model M. u, is the activation of the component time t. X, is sample extracted from the time series source data with a r-width time window. When the window locates at t = 1, X1 is extracted from the time series source data. Regression model M imputs X1 and outputs yı. and W1 is obtained from this calculation process. By sliding the window, regression model M inputs Xt, i E (1,...,7) T) and outputs ye: i E (1, (1,...: 7). We; f (1,...: : T) is obtained from each time. By concatenating úe from each time, we get the activation vector u = [u1, U2, UT]. In this way, we can obtain an activation and weight vector of component.
Algo. 1 explains the factorization procedure. The reconstructed observed data Et can be more accurately reconstructed by ensembling M as
Algorithm Factorization procedure
Input: X, and ye {1...., .T). and r.
y M,(X,).
repeat
Update parameters in M,; r),...,
where is the number of factorized components and M, is the i-th model. In computing ma- trix factorization, the number of components
Clip u in Linear: < w <
until objective function is converged
u; = [us].t E [l.....T7 T} from Sigmoid in M,
r needs to be given. The appropriate number
0; < w from Linear in M:
of factorized components r' is determined ac- cording to evaluation indices such as the degree of restoration. M,, E (1,...:7) r) are trained in parallel to minimize the objective function using the same dataset across models. From each M,, we obtain the activation vector u, and the weight vector v, The factorized matrix V is obtained by combining the weight vectors as [U,....,yy] Uy]. The factorized matrix U is also obtained by combining the activation vectors as [ut....,u, u,]
V < [u,], E fl.....r"
U = [u], E (I....,r)
Output: Y = VU
The objective function is
min Clye, Σ (x(X.)) R(w)
(4)
where L(y,, E M,(X,)) is the reconstruction loss and R(w) is the regularization term for the weight. The parameters of the models are updated minimize the objective function are denoted as M, At each iteration of learning model, each element of the weights is clipped from O to to obtain the positive factors of the matrix. The weight vector w M, used as U, V and U are obtained by concatenating of vectors v; and u,, {1..... r). respectively. Reconstructed observed data Y [yc,, E {1...., T} is reconstructed by V and U.
4 EXPERIMENT
We applied the proposed method to muscle synergy analysis. As stated in the introduction, our bod- ies contain corticospinal pathways between the primary motor cortex and muscles that function as corticomuscular system (Liu et al., 2019). Normally, cortical events propagate to the peripheral muscles, and the corticomuscular system enables us to perform complex body movements by neuro- logically combining sets of simpler movements. These simpler movements are observed as the basic pattern of muscle activity, namely, muscle synergies. Muscle synergy analysis is designed to capture this efficient neural connection between the brain and muscle. However, the conventional approach uses only muscle activities (observed phenomena) to capture the muscle synergies, and there may still be unexplored muscle synergies that remain hidden. applying the proposed method muscle synergy analysis in this experiment, we consider the muscle activities as the observed phenomenon and the brain activities as the source phenomenon. To test the proposed method, we measured the muscle and brain activities simultaneously in a monkey.
4.1 DATA MEASUREMENT
We measured the biopotential signals derived from the muscles and brain activitics. The invasively measured electrocorticographic (FCoG) signal selected as the brain activity data because
5

---

Under review as a conference paper at ICLR 2023
contains more detailed brain activity than non-invasively measured signal. Thus, to obtain both the brain activity and the muscle activity, we recorded ECoG and electromyographie (EMG) signals simultaneously from the contralateral hemisphere and the ipsilateral forelimb muscles of a monkey, respectively.
The monkey used in this experiments was an adult Japanese macaque monkey (Macaca fuscata; Monkey D: female, 6.0 kg. 7 years old). The Ethical description is in A.1.
ECoG signals were down- to 500 samples per second, similar to previous works (Shin et al., 2012; Hidenori et al., 2012). Fourth-order Butterworth bandpass filters were applied to each ECoG signal, dividing them into multiple specifie bands (Ø(1.5 4Hz), 0(4 8H2), a1 1, (14 - 2). (2((20- 20H2). 2 7300 - 50H z). 22(00 - 20H2), 73(90 120H2) and 74 =)) The nine bandpass filters split each of the 15-channel ECoG signals into nine signals to produce 135 channels of handpass-filtered signals. We performed normalization and smoothing the same as in (Shin al., 2012: Hidenori al., 2012). A bandpass filter that passes 20-500-Hz frequency was applied to the EMG signals for removing motion artifacts. Bandpassed signals were then rectified and passed through a fourth-order low-pass filter with a cut-off frequency of 4 Hz. Finally, the signals were down-sampled to 500 Hz. To account for the nerve signal transmis- sion delay between brain and muscles, delayed signal values is also used. The value discrete-tine time step-size At is set to 20 ms in the previous study (Shin et al., 2012). The muscle activity at time t is predicted using 10 time points starting 200 ms hefore the target time t. The i-th ECoG data vector at time t is (i,t) (z((,))) (V. 20); (i, -.. -fii and the ECoG data matrix time t is X, (1,t) ................................ (135,t))
Processing:
Baseline: We utilized non non-negative matrix factorization (NMF) as a baseline and used the Python library scikit-lear. All parameters were set to the same those for the pre-set.
Metric: Variance accounted for (VAF) is the main metric of the degree of reconstruction and used to determine the number of synergies (Steele et al., 2013; Delis et al., 2013), where the dimension of factors is determined when VAF exceeds 0.9. The dimension of factors is used as the synergy number in some cases (Hug et al., 2010; Frère & Hug. 2012: Chvatal & Ting, 2013). The specific calculation is decribed in A.4. In our analysis, the similarity of the muscle synergy components resulting from the two methods (proposed and NMF) is calculated by the cosine similarity of the weights of the synergies.
Model selection: We performed three times optimization and adopted the model with the largest VAF because the VAF of the proposed method varies slightly each time.
4.2 MUSCLE SYNERGY ACQUISITION POTENTIAL USING ONLY MUSCLE ACTIVITY (CONVENTIONAL FRAMEWORK)
First, we examined the proposed method's availability of muscle synergies by using only muscle activities (conventional framework). The muscle synergies were determined based on the degree of reconstruction (VAF) and the number of synergies. Theoretically, as the num- ber of synergies increases, the VAF becomes higher because the reconstruction accuracy im- proves. Researchers have shown that the VAF of NMF increases with the increase of the num- ber of muscle synergies in the case of us- ing only muscle activities (Rabbi et al., 2020; Gui & Zhang, 2016). If the proposed method can calculate muscle synergies by factorizing the input matrix, the VAF should increase with the number of muscle synergies. We tested the VAF of the proposed method to see it would increase and exceed 0.9, as the number of syn- ergies increases only when using muscle activ- ity. The proposed method inputs EMG vec-
0.95
0.90
0.85
0.75
0.70
0.65
[M
[4.1
[1611]]
[54.4]
MMF
[128. 128)
5
6
Synergy number
Figure 3: VAF of proposed and NMF methods for each number of synergies. In the proposed method, we experimentally explored the number of filters from [1, 1] to [128, 128).

---

Under review as a conference paper at ICLR 2023
and the output is also supposed to be y without changing the architecture as tor y: = 19 ................................ 41 ........,,.,,, following
D M.(y)).
(5)
We used data from one-day experiment that included 90 trials in the intact state (no neurological disorders). In the proposed method, the number of filters e of the temporal convolutional networks in the activation layer is a key parameter to determine the potential of synergy acquisition. Thus, we experimentally tested eight types of filters here: [1. 1]. [2. 21, [4, 4J, [8, 8), [16. 16], [32, 32], [64, 64), and [128, 128). The vector length of the number of filters the number of dilations in the temporal convolutional network, which is two in all eight types. For example, in the case of [8, 8), the first 88 is the number of filters at the first dilation, and the second "8 the number of filters for the second dilation.
Figure 3 shows the VAF results the proposed method (eight different numbers of filters) and NMF by increasing the synergy number for the same data. Most of the VAF results calculated by the proposed method increased with the synergy number, as did those of NMF. The dashed horizontal line indicates VAF of 0.9, which determines the muscle synergy number. For the same number of muscle synergies, VAF tended to be higher for a larger number of filters. By confirming that VAF increases with increasing the number of synergies, verified the possibility that the proposed method can analyze muscle synergies.
4.3 NUMBER OF FILTERS
We also tested whether the proposed method can acquire muscle synergy under connectivity constraints between brain and muscle activity. Fig. 4 shows the VAF per muscle synergy num- ber for the proposed method and NMF. Most of the proposed method's VAF values increased with the number of muscle synergies not de- pending on the number of filters. The higher number of filters tended to make VAF higher.
1.00
[1.1]
[4/11
[15.16]
[84,04]
NMI
0.96
(22,5]]
(128 120)
0.85
0.80
We can see that the determined number of mus- cle synergies, where VAF exceeds 0.9 at first, depends on the number of filters. At the num- bers of filters [32. 32] and [64, 64], the deter- mined number of muscle synergies eight. At the number of filters [128, 128), the determined number of muscle synergies is six. Defining the number of filters depends how the muscle synergies are used. To compare the components of muscle synergies from the proposed method and NMF, we set the number of filters to [32, 32], which is the minimum number of filters when the number of muscle synergies eight.
0.65-
0.60
4
5
6
7
Synergy number
Figure 4: Muscle synergy analysis by factoriza- tion using brain and muscle activities in the pro- posed method.
4.4 COMPARISON OF MUSCLE SYNERGY COMPONENTS
We examined whether the proposed method can capture the unrevealed muscle synergy components that are difficult for the conventional approach to extract. As a muscle synergy component, a vector of weights and activations is obtained from a muscle synergy. A, B, and C in Fig. 5 respectively indicate the eight components from the proposed method using only EMG, NMF, and the proposed method using ECoG and EMG. The left panels in the figure show the weight the muscles of each synergy, while the right panels show the activation of each synergy, which is calculated by averaging all trials. To compare the similarity of each component, we calculate the similarity matrix. D in Fig. 5 shows the similarity matrix using the weights from these three approaches. The upper panel in shows the similarity matrix of the weights between the proposed method (only using EMG) and the NMF method, while the lower panel in D shows that between the proposed method
7

---

Under review as a conference paper at ICLR 2023
A: Proposed EEMG-EMG)
Sy13
Syn2
Syna
Synd Syst Rynd Syn7
Syn8
D: Similarity
9xx1
1:21
3.21
1:35
1.23
%,40
1:30
1:22
8:35
103
5,n6
(EMG EMGI
3:22
3030
2
5,19
3.30
20
9.22
33
0:37
3.5
3D
25
B: NMF
kyrk
3.59
37
2
9:21
Synt Syd
8.20
3.00
8:22
2.0
33
5,10
3.09
322
9:22
8.32
2016
5ma
Senk
MMF
Syd
Senk
Seril
5,15
1043
5,17
5.
1.2
1:55
115
1:30
M.
321
00
2.2
C: Proposed (ECoG-EMG)
EMG)
Syd1
Syn2
Syna Synd Syst
Smnd syn7 Syst
3:37
3.35
5
25
C
21
8:37
9-11
5,10
2:0
2
5V
3:3
8:29
331
9010
3.10
1yd
3:22
3.31
9:06
8.55
1:50
8:0
8.37
MMF
Figure 5: Weights and activations of muscle synergy components from the proposed method (using
only EMG), NMF (using only EMG), and the proposed method (using EMG and ECoG).
using ECoG and the EMG and NMF methods. Using only EMG data, the similarity of almost all components of muscle synergy was close to 1, meaning that they were all identical for the proposed and NMF methods. The averaged computation times of the proposed method (basically using GPU: NVIDIA RTX3090) and NMF (CPU: AMD Ryzen Threadripper 3970X) are 6209.3 sec and 12.0 sec, respectively. The computational complexity of the proposed method is higher than that of the NMF. The low similarity of synergies 6 in both the proposed method (using only EMG) and NMF may have been due to the high degree of freedom of activation in the proposed method. As result, synergies 4 and 7 of the proposed method represent the corresponding synergies in NMF, while also representing synergy 6 in NMF by the cooperative activation of the two synergies. It may be possible to solve this problem by setting a constraint term such that the activation acts independently between the synergies.
Some of the similarities between the proposed components (using ECoG and EMG) and NMF did exhibit a high degree of similarity. For example, the similarities of the proposed method's synergy and 2 to synergy 1 in NMF were 0.76 and 0.82, respectively. The results indicate that the proposed method separates synergy 1 of NMF into two synergies. These separated components may be the result of capturing the connectivity between brain and muscle activities. The same can be said for synergy and of the proposed method.
4.5 ADAPTIVE CHANGE ANALYSIS
We next investigated whether the proposed method can capture the adaptive change of muscle syn- ergy components during nervous recovery. First, brain and muscle activity was measured from an intact (no neurological disorders) monkey. Then, performed partial SCI surgery the monkey and measured its activity for 30 days (not every day) after the surgery. We explored the number of components of each of the experiment days in the proposed and NMF methods.
8

---

Under review as a conference paper at ICLR 2023
14.
Intact
SCI
Proposed
NM
11-
8
10-
82-
Prestpion [N]
80
5
1894
685.8
2686
Dates from spinal cord injuyy surgery
Figure 6: Number of components (upper panel) and task precision (lower panel) in each day. The numbers of components of the proposed and NMF methods are plotted in red and blue lines, respec- tively, in the upper panel. The days depicted by the red dashed lines are those for which no muscle synergy was computed below 14. The vertical dashed green line indicates the SCI surgery day.
Figure 6 shows the number of components (upper panel) and task precision (lower panel) in each day. We can see the precisions during the intact period were almost 100 %, then moved to nearly % for month after the surgery, and then improved around 80 % after three months. These results suggest that the monkey performed the task before the surgery based on the pathways between the brain and muscles acquired so far. Immediately after the partial SCI surgery, monkey to use the pathways because many of the previously acquired pathways were disconnected and it was more difficult to move the upper limb. The monkey regained the function to perform the task by adaptively changing the pathways using the residual nervous connections, and improved the precision of the task. The numbers of components of the proposed and NMF methods are plotted in red and blue lines, respectively. The days dashed in red (proposed) are days where muscle synergy was not calculated below (the number of measured muscles).
difficult for the
The numbers of components of NMF's muscle synergy were stable both before and after SCI (Fig. 6). While those of the proposed method were similar during the intact period, they exceeded 14 about month after the surgery and then gradually decreased and converged around three months after the surgery. We presume that the number of components increased immediately after the surgery because the monkey could not use the pathways obtained so far, and had to the residual pathways (not sophisticated). The number of components decreases and converges in the adaptive change of the nervous system, the transition of the proposed method's results might be explainable from the neurological point of view. This result indicates that the proposed method can potentially be used analyze muscle synergy with adaptive nervous change, which difficult extract in the conventional approach.
CONCLUSION
In this paper, we proposed matrix factorization method under the constraint of connectivity be- tween observed and source data. The core idea of our method is to use the representation in a model that predicts the observed data from the source data as a factored matrix. Applying the proposed method to muscle synergy analysis using a monkey's brain and muscle activities demonstrated its potential to reveal unknown muscle synergy components, which are difficult for the conventional approach to capture. Specifically, the proposed method's transitions of the number of components after partial SCI surgery indicated the adaptive changes in the corticomuscular system. Through two analysis experiments, source data to extract the elements hidden in observed data, and showed that the proposed method's potential to capture these hidden elements.
demonstrated the importance of using not only observed data but also

---

Under review as a conference paper at ICLR 2023
REFERENCES
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, 2018. URL ftttps://arii...o/ / 1803. 1803.0127 01271.
Baldissera and P Cavallari. Short-latency subliminal effects of transcranial magnetic stimulation on forearm motoneurones. Experimental brain research, 96(3):513-518, 1993.
Jean-Philippe Brunet, Pablo Tamayo, Todd R. Golub, and Jill P. Mesirov. Metagenes and molecular pattern discovery using matrix factorization. Proceedings of the National Academy of Sciences, 101(12)441444116, 2004. doi: 10. htt7://paa..0053110000000 URL https: /www. pnas org/ doi/abs/10: 1073 /ppaa.. 0308531101.
Joshua C Chang, Patrick Fletcher, Jungmin Han, Ted L Chang, Shashaank Vattikuti, Bart Desmet, Ayah Zirikly, and Carson C Chow. Sparse encoding for more-interpretable feature-selecting rep- resentations in probabilistic matrix factorization. In International Conference on Learning Rep- resentations, 2021. URL https: httpp/pppww......eeopeee
/eveeppee//p.........e...... net www.:nruntnotoooooeoeo
forum parum?id-DD Pid-D __
Jen-Tzung Chien and Bo-Cheng Chen. A new independent component analysis for speech recog- nition and separation. IEEE Transactions Audio, Speech, and Language Processing, 14(4): 1245-1254, 2006. doi: 10.11099TSA2200..558666
Stacie Chvatal and Lena Ting. Common muscle synergies for balance and walking. Frontiers in Computational Neuroscience, 7, 2013. ISSN 1662-5188, doi: 10.3389/fncom.200 201.200013.. URL 3389/
https:/ wwwp///wwww//oo/////////////t http://wwfffotttt www.oorrntsinnnmoo org wttpe/otigrtttleeeeeeeeeeeeaaeee 3389/fncom. fncomm 2013. 00048.
Pierre Comon. Independent wwmpppmmnnammlllylsseee...ocoooo Signal Processin,,, 36(3):287-3)) 1994. ISSN 0165- 1684. doi: http:://ob.ooo////.0000006661112 URL https: // www. sci encedi rect. com http:///..ccaaaeeerreee///////// Order Statistics.
Higher
Joannis Delis, Bastien Berret, Thierry Pozzo, and Stefano Panzeri. Quantitative evaluation of mus- cle synergy models: a single-trial task decoding approach. Frontiers in Computational Neuro- science, 7, 2013. ISSN 1662-5188. doi: (0.3389/fncom.2013.. 2013.00008. URL https: //wp:://www.. 3389/ fncom. 000000
Karthik Devarajan. Nonnegative matrix factorization: analytical and interpretive tool compu- tational biology. PLOS Computational Biology, 4(7):1-12, 07 2008. doi: 10.137 131.13////oaalpccc..... 1000029. URL hetptttps//... / wttp//.....////orraaal.....mmm.. www.ornnn....b...... 1000029.
Julien Frère and François Hug. Between-subject variability of muscle synergies during plex motor skill. Frontiers in Computational Neuroscience, 6, 2012. ISSN 1662-5188, doi:
10.3389/fncom.2212..00999 URL https://www. frontiersin wrg/articles//1..
3389/ wnw.m..coc.mmm/ommm//m/oo.../... 2012. 00099.
Kai Gui and Dingguo Zhang. Influence of locomotion speed on biomechanical subtask and muscle synergy. Journal of Electromyography and Kinesiology, 30:209-215, 2016. ISSN 1050-6411.
doi: http:///cneeerrrrceeee//////o110 com/ com/scienceeeee
URL https: /wwww/wwwwk. www.sccienceiirect.
Watanabe Hidenori, Sato Masa-aki, Suzuki Takafumi. Nambu Atsushi, Nishimura Yukio, Kawato Mitsuo, and Isa Tadashi. Reconstruction of movement-related intracortical activity from micro- clectrocorticogram array signals in monkey primary motor cortex. Journal of 9()):036006, 05 2012. ISSN 1741 2560. cir. miiniiiii ac.
Neural Engineering, 10.188//7442220009333666606
doi: 10. 1088/1741 2560/9/3/0360066 URL https://
François Hug. Nicolas A. Turpin, Arnaud Guével, and Sylvain Dorel. Is interindividual variability of emg patterns in trained cyclists related different muscle synergies? Journal of Applied Physiology, 10876) 2010. doi: 10. 10.1152/japplphysio..00 doi doi.org/10. 1152, happ wwwpppphyylllloom 01305. 2009. PMID: 20299611.
URL https://
10

---

Under review as a conference paper at ICLR 2023
A. Hyvärinen and E. Oja. Independent component analysis: algorithms and applica-
tions. Neural Networks, 13(4):41--430 2000. ISSN 0893-6080. doi: https://oi.org//
URL
Daniel Lee and H. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788-91, 11 1999. doi: 10. 10.1038/4456.
Daniel Lee and H. Sebastian Seung, Algorithms for non-negative matrix factorization. In T. Leen, T. Dietterich, and V. Tresp (eds.), Advances in Neural Information Processing Systems, vol- neurips. wwtppper///...CC/C////// .ddf.
ume 13. MIT Press, 2000. URL File/f9d11 9d115254 00s1112547ccbbeeee 8306 B000
Roger N. Lemon. Descending pathways in motor control. Annual Review of Neuroscience, 31(1); 195-218, 2008, doi: 10. 10.11/4///mmree.euuo......00 www.:/muurrr....0000777222555541 URL https:
10. 1..1146/ 2..4///annree...me neuro. 31 31. 060407.....000 125547.
PMID: 18558853.
Roger N Lemon and James Griffith.. Comparing the function of the corticospinal system in different species: organizational differences for motor specialization? Muscle & Nerve: Official Journal of the American Association of Electrodiagnostic Medicine, 32(3):261-27,, 2005.
Tao Li, Jiandong Wang, Huiping Chen, Xinyu Feng, and Feiyue Ye. A nmf-based collaborative filtering recommendation algorithm. In 2006 6th World Congress on Intelligent Control and Automation, volume 2. pp. 6082-6086, 2006. doi: 10.1109/WCCC..2000 1714249,
Jinbiao Liu, Yixuan Sheng, and Honghai Liu. Corticomuscular coherence and its applications: A review. Frontiers in Human Neuroscience, 13, 2019. ISSN 1662-5161. doi: 10.3389/fnhum. 2019.00100. URL httpt::/p///.....t. //w:://www..w. frontiersin. 00100.
www.en//erssi..ogrrrtiil.ee..eee wwtp/ooicleeoee/eoooooooooooceat 3389/ fnhum.
Mohammad Rabbi, Claudio Pizzolato, David Lloyd, Christopher Carty, Daniel Devaprakash, and Laura Diamond. Non-negative matrix factorisation is the most appropriate method for extraction of muscle synergies in walking and running. Scientific Reports, 10, 05 2020. doi: 10.1038/ 1398-020-65257-w.
Elahe Rahimian, Soheil Zabihi, Amir Asif, S. Farokh Atashzar, and Arash Mohammadi. Few-shot learning for decoding surface electromyography for hand gesture recognition. ICASSP 2021 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1300-1304, 2021. doi: 10.1109/CASSP33972... 20....022.922 9411558....
Elahe Rahimian, Soheil Zabihi, Amir Asif, Dario Farina, S. Farokh Atashzar, and Arash Mo- hammadi. Hand gesture recognition using temporal convolutions and attention mechanism. In ICASSP 2022 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1196-1200, 2022. doi: N010099CAASPP44322222229977
M. Rajapakse and L. Wyse. Nmf vs ica for face recognition. In 3rd International Symposium Image and Signal Processing and Analysis. 2003. ISPA 2003. Proceedings of the, volume 2, pp- 605-610 Vol.2, 2003. doi: 10.1110// 2003. 1296348.
Duk Shin, Hidenori Watanabe, Hiroyuki Kambara, Atsushi Nambu, Tadashi Isa, Yukio Nishimura, and Yasuharu Koike. Prediction of muscle activities from electrocorticograns in primary motor cortex of primates. PLOS ONE, 7-10):1-10, 10 2012. doi: 10. Ljouumal. 0047999.. URL
https: doi. /tc.//..... 1371/ harp///oooraalla pone 0047992.
Mohammad S. Shourijeh, Teresa E. Flaxman, and Daniel L. Benoit. An approach for improv- ing repeatability and reliability of non-negative matrix factorization for muscle synergy analy- sis. Journal of Electromyographh and Kinesiology, 26:36-43, 2016. ISSN 1050-6411. doi:
URL https: //w.tt/s:///...../. sciencedirect. ott.eaaeotee///eooeeeeeeeeeeeeee com/
science//ariiilee/iii///
Paris Smaragdis. Convolutive speech bases and their application supervised speech separation.
IEEE Transactions on Audio, Speech, and Language Processing. 15(1):1-12, 2007. doi: 10.1 109/ AASI.2006.876777
11

---

Under review as a conference paper at ICLR 2023
Silk Smita, Sharmila Biswas, and Sandeep Solanki. Audio signal separation and classification: A review paper. 3297, 12 2007.
Hyun Ah Song, Bo-Kyeong Kim, Thanh Luong Xuan, and Soo- Young Lee. Hierarchical fea- ture extraction by multi-layer non-negative matrix factorization network for classification task. Neurocomputting, 165:63-74, 2015. ISSN 0925-2312. doi: 1011 2014.08.095. URL https: /ww. www.::sieecedireeeooooe cww/ http://swt...oeeeeeeeeeeeareeaaa 50925231215004588
Katherine Steele, Matthew Tresch, and Eric Perreault. The number and choice of muscles impact the results of musele synergy analyses. Frontiers in Computational Neuroscience, 7, 2013. ISSN
doi: 0..3389fnccom2213300000 articles es 3389/ffcom.....mmo.mmm fncom. 2013. 3. 00105.
1662-5188.
URL https: //w::///wwwwww// wttp://wwwwotiiiiincoooo
János Szalai and Ferenc Emil Mozes. Determining fetal heart rate using independent component analysis. In 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 11-16, 2014. doi: 10.1 2...2..01..9.93973..
George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Björn W. Schuller. A deep semi-nmf model for learning hidden representations. In Proceedings of the 31st International Conference International Conference on Machine Learning Volume 32, ICML'14, pp- 11-1692-11-1700. JoLR.org, 2014.
Jianyu Wang and Xiao-Lei Zhang. Deep NMF topic modeling. CoRR, abs/2102.12998, 2021. URL https:/ //axxivv.... wwwriii.rrg.....eo htrg/bb/// abs/ 2212..2292........
Yu-Xiong Wang and Yu-Jin Zhang. Nonnegative matrix factorization: A comprehensive review. IEEE Transactions Knowledge and Data Engineering, 25(6):1336-133 1353, 2013. doi: 10.1109/
TKDE.2012.51.
Hong-Jian Xue, Yu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. Deep matrix fac- torization models for recommender systems. Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI'17 pp. 3203-3209. AAAI Press, 2017. ISBN 9780999241103.
Fanghua Ye, Chuan Chen, and Zibin Zheng. Deep autoencoder-like nonnegative matrix factoriza- tion for community detection. In Proceedings of the 27th ACM International Conference In- formation and Knowledge Management, CIKM '8, pp. 1393-1402, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 978 47784450660444 42. doi: 10.1 10.1145/3269206332116777 URL httph::: / www.ooioorgg/111 11455.......1 32662226622662222 3271697 697.
A APPENDIX
A.1 ETHICS
All animal experimental procedures performed in accordance with the guidelines of the Na- tional Institute Health and the Ministry of Education, Culture, Sports, Science, and Technology of Japan, and approved by the Institutional Animal Care and Use Committee of the Tokyo Metropolitan Institute of Medical Science (Approval No. 18034). The monkey was monitored closely, and animal welfare was assessed daily or, if necessary, several times day.
A.2 ECoG AND EMG SIGNALS
We implanted a 15-channel grid electrode array in which the diameter of each electrode was mm and the inter-electrode distance was 5 mm (Unique Medical, Japan) beneath the dura mater. To im- plant the grid electrode array the cortical surface over the sensorimotor cortex, the cortices around the arcuate sulcus and the central sulcus on the right side were exposed by craniotomy. As to the muscle activity, electrodes were implanted in 14 muscles of the left forelimb: three elbow muscles [triceps brachii (TR), biceps brachii (BB), brachialis (BR)), five wrist muscles [extensor carpi ra- dialis (ECR), extensor carpi ulnaris (ECU), flexor carpi radialis (FCR), flexor carpi ulnaris (FCU),
12

---

Under review as a conference paper at ICLR 2023
palmaris longus (PL)), five digit muscles [extensor digitorum communis (EDC), extensor digito- rum 4,5 (ED45), flexor digitorum superficialis (FDS), flexor digitorum profundus (FDP), abductor pollicis longus (APL)), and one intrinsic hand musele [adductor pollicis (ADP)). ECoG and EMG signals were recorded simultaneously using a Cerebus data acquisition system (BLACKROCK MICROSYSTEMS, Utah, USA) sampling rate of 2,000 Hz.
A.3 BEHAVIOR TASK
The monkey performed the task of reaching and grasping for piece of food multiple times in one experimental day. On each day, the decision to perform the measurement experiment and the number of trials were determined on the basis of the monkey's 's condition. In total, conducted the measurement experiment for 41 days. Each recording session consisted "100 trials.
A.4 VAF CALCULATION
Variance accounted for (VAF) is metric of the degree of reconstruction. This calculation uses true value vectors y = [ ..7T, ] m E (L...., 14) and reconstructed vectors ý = [DTT ..... ...,1T1 m (I,..., 14), where the number of measure muscles. The VAF is calcu- lated as following
VAF 1 Em Var(y" TT)
(6)
Em=1 Var(y")
where Var(y" - )) is the variance of y D" the square of the standard deviation. Var(y") the variance of n
13


Reviews:
[{"review_id": "z_qQ7wHy4u", "reviewer": "Reviewer_89Ck", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Alternatively to the common way of finding latent factors in the data via a matrix factorization, the paper proposes to interpret a deep regression model with a linear layer as a matrix factorization of the output data. The main intention of this re-interpretation is to be able to interpret the soft-maxed input to the final linear layer as the weights, and the linear layer weights as the features.\n", "summary_of_the_review": "An interesting application paper with a misleading narrative that inflates the novelty of the proposed modification of a deep convolutional regression model. \n", "strength_and_weaknesses": "# Strengths\n- The paper is mostly clearly written apart some undefined notions and at times lack of explanations.\n- The application is interesting and potentially extensible to other fields and situations. In a way, the paper proposes an autoencoder with deep convolutional encoder and linear decoder, only this architecture is not auto-restoring the input, but predicts the output via reconstruction loss, i.e. what in deep learning is called a regression task.\n# Weaknesses\n- My main concern is the lack of technical novelty. It is a solid application paper, where a regression model is used to interpret latent factors of one of the signal modalities.\n- The writing of the paper is sometimes confusing.\n  - For example, the introduction talks about \"connectivity\", but it is unclear what kind of connectivity is implied, anatomical, functional, hypothetical in some implicit data manifold, or something else. The meaning of the \"connectivity\" does not become clear further.\n  - The Introduction could clarify that both the source and the measured data are actually measured. This is to avoid the confusion with matrix factorization models, where sources commonly denote latent factors (e.g. in ICA).\n  - Section 3 around expression 1 could be clearer if the fact that the model operates on time-windows would have been made explicit.\n  - Algorithm 1 needs to specify $\\mathbf{Y}_t$ as an input as well to be trained as a regression model.\n- It would be informative to compare computational complexity of the proposed approach and NMF. The results correlate highly (Figure 5). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RSP6ROyhY9", "reviewer": "Reviewer_jF4J", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a matrix factorization technique that incorporates also information from a, so called, source data in addition to the data from the direct observations. The paper’s authors motivate the challenge by focusing the discussion on the application of the method to corticomuscular system with brain activity associated to several muscles of a monkey’s forelimb (reach and grasp tasks) during recovery of spinal cord injury. The model in itself if constructed with a DNN prediction model that links the $T$-dimensional source data with $m$ channels with the observed $T$-dimensional observational data with $n$ channels. The architecture of the DNN include: a TCNet, mean layer, scalarization layer, a sigmoid layer, and finally a linear layer. The source data is fed to the TCNet, the factorization DNN reconstructs the observed data where the sigmoid and linear layers provide the factors of the procedure.  The evaluation on the data from the corticomuscular system aimed to reconstruct the “synergies” between the brain and the muscle. This evaluation first considers only muscle activity data, then uses brain and muscle activity, and analyses the synergies identified and the changes identified before and after the SCI surgery. ", "summary_of_the_review": "The paper has its pros, such as the relevance and the potential applicability. However, there are several reservations I have with respect to both the clarity of the technical contribution, the modeling, and the the choice of performance measure. More work is needed to polish the description of the contribution and the evaluations in order to have a realistic assessment of the quality of this work. I detailed my comments on what could be improved in the Strength and Weaknesses section. ", "strength_and_weaknesses": "With respect to its strengths, the paper is well organized, the problem is relevant to the ML community and the neuroscience community. The paper provides a very didactic description of the proble, although the description of the contribution is rather limited. Another strength of the paper is that the evaluations consider various perspectives of the problem. However, the measure used for evaluating performance provides a limited perspective about the performance of the models. \nThat being said, there are some items that need some work. The description of the factorization is not very rigorous. The factorization of $Y$ is rigorous but the factorization that incorporates $X$ is not formally introduced. The factorization itself is an application of ideas of Bai et al. (2018) Rahimian et al. (2021; 2022) to the reconstruction of observed data $Y$ and other ideas. The main novel component of this work is the application to modeling corticomuscular synergies. The evaluation is interesting but there is still work that could be done. In particular VAF is defined but it is not clear what the ground truth or the reference for the variance comes from.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "N2sownt2-Z", "reviewer": "Reviewer_VHFL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a deep non-negative matrix factorization approach for time series that offers the ability to constrain the factorization to common factors between a source and a receiver. They apply this approach to the detection of muscle synergies detected by EMG at the forearm of a monkey performing a reach and grasp task, with and without the constraint of being driven by ecog activity from the sensorimotor cortex. The new model is compared to traditional non-negative matrix factorization, a gold standard for synergy detection in the field. The proposed model works by processing an input timeseries with a variable number of parallel stacked TCN networks and scalarizing and normalizing each networks output. Compared to traditional NMF analysis the proposed approach demonstrates improved variance accounted for (VAF) for low numbers of synergies, and recovers comparable number of synergies from EMG alone. When using the ecog as the input to the TCN model, there are more differences in the recovered synergies. Finally the later approach using the TCN input is compared to NMF for synergy detection after inducing spinal cord injury, which shows large differences in detection. ", "summary_of_the_review": "Overall I found the TCN model presented somewhat interesting and potentially novel and I believe the problem of synergy detection itself is important. However I found the framing and goal of the paper somewhat misguided from a neuroscience bent, and a lack of alternative models discussed. It is possible that a revision could alleviate some of these concerns, but this would require a fairly thorough rewrite of problematic neuroscientific framing and a much deeper consideration of alternative architectures or modeling approaches to prove the importance of the one chosen.\n", "strength_and_weaknesses": "Strengths:\n(1) Muscle synergy analysis is widely used, but is based on a fairly ad hoc statistical approach and so new approaches could be very impactful. \n(2) The model appears technically sound, and the comparisons provided with NMF and across filter numbers are worthwhile.\n\nWeaknesses\n(1)  The framing of synergies and their neuroscientific context is somewhat lacking. The premise of the paper is that muscle synergies can be predicted from the cortical inputs, e.g. We applied our method to the corticomuscular system, which is made up of corticospinal pathways between the primary motor cortex and muscles in the body and creates muscle synergies that enable efficient connections between the brain and muscles.”, however synergies are thought to be generated in the spinal cord and some of their first characterization was in frogs, a species without a motor cortex. \nOne reason this is problematic is that the paper is framed as revealing *new* synergies, e.g. “However, the conventional approach uses only muscle activities (observed phenomena) to capture the muscle synergies, and there may still be unexplored muscle synergies that remain hidden” However, based on the model design it seems like it should be detecting a subset of the muscle-only synergies. Moreover, synergies is largely defined in a muscle-centric way. It is certainly the case that discovering cortico-muscular shared synergies is interesting, but the framing is very different. \n(2) There are no details provided on the TCN training, and importantly how data was split up for train and test splits. This is especially important for the SCI experiments. \n(3) There are multiple alternative models that could be considered, for instance performing NNMF on a linear or nonlinear prediction of EMG from ecog activity. The specific motivation for the increased number of parameters and model structure of the TCN is not provided. One of the appeals of NNMF is its simplicity, allowing it to be used across paradigms and contexts, and this TCN introduces a lot of added complexity, with only minimal gains in VAF at high numbers of syneries. \n(4) For the SCI experiments – there is no ground truth present and so it is impossible to evaluate which technique is ‘correct’. As noted above, without knowledge of how much data is required for model training, it is hard to know if the increase in number of synergies observed is a result of \n\nNits:\n-\t‘connectivity’ is misleading, as it isn’t using the structural connections between the brain and body. \n-\tFigure 6: would help to have an estimate of variance for the number of synergies, e.g. from using different subsets of data to train/test. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject