Original: ReCell: replicating recurrent cell for auto-regressive pose generation ANONYMOUS AUTHOR(S)
Revised: ReCell: replicating recurrent cell for auto-regressive pose generation ABSTRACT

Original: This paper describes Anonymous gesture generating system entry for the GENEA Challenge 2022.
Revised: This paper describes FineMotion’s gesture generating system entry for the GENEA Challenge 2022.

Original: Our system is based on autoregressive approach imitating recurrent cell.
Revised: Our system is based on autoregressive approach imitating recurrent cell.

Original: Combined with  special windowed auto-encoder and training approach this system generates plausible gestures appropriate to input speech.
Revised: Combined with a special windowed auto-encoder and training approach this system generates plausible gestures appropriate to input speech

Original:  ACM Reference Format: Anonymous Author(s).
Revised: CCS CONCEPTS ACM Reference Format: Vladislav Korzun, Anna Beloborodova, and Arkady Ilin.

Original: ReCell: replicating recurrent cell for auto-regressive pose generation .
Revised: ReCell: replicating recurrent cell for auto-regressive pose generation .

Original: In Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY.
Revised: In INTERNA- TIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’22 Companion), November 7–11, 2022, Bengaluru, India.

Original: ACM, New York, NY, USA, 6 pages.
Revised: ACM, New York, NY, USA, 4 pages.

Original: https://doi.org/XXXXXXX.XXXXXXX
Revised: https://doi.org/10.1145/3536220.

Original: 1  INTRODUCTION
Revised:  INTRODUCTION

Original: Automatic animation generation for humanoid characters is becoming increasingly popular.
Revised: Automatic animation generation for humanoid characters is becoming increasingly popular.

Original: Most 3D characters are still animated using expensive motion capture or a small set of predefined animations.
Revised: Most 3D characters are still animated using expensive motion capture or a small set of predefined animations.

Original: Recent advances in facial animation generation[1, 4] provide a realistic facial expression from audio.
Revised: Recent advances in facial animation generation [1, 4] provide a realistic facial expression from audio.

Original: Moreover, 
Revised: Moreover, JALI Viseme

Original: JALI Viseme Model[2] was already used in recent projects like Cyberpunk2077.
Revised: Model [2] was already used in recent projects like Cyberpunk2077.

Original: Automatic gesture generation, on the other hand, has several existing approaches, including [3, 6, 7, 10], but no standard, ready-for-production solution.
Revised: Automatic gesture generation, on the other hand, has several existing approaches, including [3, 6, 7, 10], but no standard, ready-forproduction solution.

Original: One possible explanation for that is the lack of a unified method for reliable comparison.
Revised: One possible explanation for that is the lack of a unified method for reliable comparison.

Original: Multiple standards for humanoid rigs and skeletons are also present making it even more challenging to use open-source solutions. 
Revised: Multiple standards for humanoid rigs and skeletons are also present making it even more challenging to use open-source solutions. Furthermore, some Permission to make or

Original: Furthermore, some existing approaches generate motions in 3D, while others generate motions in 2D. As a result, comparing them is difficult.
Revised: ACM ISBN 978-1-4503-9389- 11...$15.00 https://doi.org/10.1145/3536220. existing approaches generate motions in 3D, while others generate motions in 2D. As a result, comparing them is difficult.

Original: The GENEA Challenge 2020 [8] was held to address this issue.
Revised: The GENEA Challenge 2020 [8] was held to address this issue.

Original: The main goal of this challenge was to determine which models for automatic gesture generation based on speech recording performed better on the same data.
Revised: The main goal of this challenge was to determine which models for automatic gesture generation based on speech recording performed better on the same data.

Original: The main result of this challenge was that some models outperformed the previous year’s best methods results while remaining far from real motion.
Revised: The main result of this challenge was that some models outperformed the previous year’s best methods results while remaining far from real motion.

Original: Even though the target domain was relatively simple: the dataset only contains recordings for one person, and the generated motion was examined for the upper body without fingers.
Revised: Even though the target domain was relatively simple: the dataset only contains recordings for one person, and the generated motion was examined for the upper body without fingers.

Original: Because there are numerous ways to improve the challenge target, the organizers decided to hold another challenge.
Revised: Because there are numerous ways to improve the challenge target, the organizers decided to hold another challenge.

Original: In comparison to the previous one, the task of GENEA Challenge 2022 is more difficult.
Revised: In comparison to the previous one, the task of GENEA Challenge 2022 is more difficult.

Original: The proposed models should generate motions for more than one person during the conversation.
Revised: The proposed models should generate motions for more than one person during the conversation.

Original: A person’s behavior during a dialogue differs greatly from that of a single performance.
Revised: A person’s behavior during a dialogue differs greatly from that of a single performance.

Original: The training and testing datasets included examples of the target person gesticulating while listening to the companion.
Revised: The training and testing datasets included examples of the target person gesticulating while listening to the companion.

Original: However, these gestures are independent of speech.
Revised: However, these gestures are independent of speech.

Original: This behavior could have a significant impact on motion generation systems based on input speech.
Revised: This behavior could have a significant impact on motion generation systems based on input speech.

Original: Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.

Original: Copyrights for components of this work owned by others than ACM must be honored.

Original: Abstracting with credit is permitted.

Original: To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

Original: Request permissions from permissions@acm.org.

Original: June 03– NY Anon. We provide a solution that is based on both the speech input and the history of previously generated motions.
Revised: We provide a solution that is based on both the speech input and the history of previously generated motions.

Original: This system generates plausible gestures from speech using a special windowed auto-encoder and training approach.
Revised: This system generates plausible gestures from speech using a special windowed auto-encoder and training approach.

Original: We also tried to explain the difference between challenges and resulting models by testing one of the previous approaches that performed well on the previous challenge.
Revised: We also tried to explain the difference between challenges and resulting models by testing one of the previous approaches that performed well on the previous challenge.

Revised: Our code and some video examples are publicly availableto help other researchers reproduce our systems.

Original: Our paper is organized as follows: Section 2 describes data processing, which is shared by all experiments; Sectiondescribes our models; Section 4 discusses our results; and Section 5 is for the conclusion.
Revised: Our paper is organized as follows: Section 2 describes data processing, which is shared by all experiments; Section 3 describes our models; Section 4 discusses our results; and Section 5 is for the conclusion.

Original: 2  DATA PROCESSING
Revised:  DATA PROCESSING

Original: This time, the organizers of the challenge provided a large dataset based on the "Talking With Hands 16.2M"[9] dataset.
Revised: This time, the organizers of the challenge provided a large dataset based on the «Talking With Hands 16.2M» [9] dataset.

Original: It includes motion capture recordings as well as audio and textual transcripts for several persons having a conversation.
Revised: It includes motion capture recordings as well as audio and textual transcripts of several people having a conversation.

Original: Mocap data includes motion data for the entire body, includingthe fingers.
Revised: Mocap data includes motion data for the entire body, including fingers.

Original: Exploring the data we found several peculiarities which could affect the data-processing pipeline:
Revised: Exploring the data we found several peculiarities which could affect the data-processing pipeline:

Original: • Mocap skeleton differs from common industry formats (for example, Blender’s Meta-Rig): the spine has a discontinuity, shoulders have extra bones and hands have a different wrist structure.
Revised: • Mocap skeleton differs from common industry formats (for example, Blender’s Meta-Rig): the spine has a discontinuity, shoulders have extra bones and hands have a different wrist structure.

Original: • Hand motion is inconsistent: some recordings lack data for finger motion, while others contain glitches and jerks.
Revised: • Hand motion is inconsistent: some recordings lack data for finger motion, while others contain glitches and jerks.

Original: • Audio recordings contain speech for both speakers.
Revised: • Audio recordings contain speech for both speakers.

Original: Some audio cuts are applied to the track in order to conceal confidential information, which greatly complicates speech processing.
Revised: Some audio cuts are applied to the track in order to conceal confidential information, which greatly complicates speech processing.

Original: To overcome the aforementioned difficulties, we attempted to construct the following data processing pipeline.
Revised: To overcome the aforementioned difficulties, we attempted to construct the following data processing pipeline.

Original: The pose is represented by 3-dimensional axis-angle rotation vectors for all bone joints and also the position of the root bone.
Revised: The pose is represented by 3-dimensional axis-angle rotation vectors for all bone joints and also the position of the root bone.

Original: We also leave only non-constant values and normalize them over the training dataset
Revised: We also leave only non-constant values and normalize them over the training dataset

Original: ’s mean and maximum values.
Revised: ’s mean and maximum values.

Original: As the result, pose representations with 164 real value features are produced.
Revised: As the result, pose representations with 164 real value features are produced.

Original: We tried different representations for audio (MFCC, mel-spectrogram, log mel-spectrogram) with different framerate but found no significant difference during training.
Revised: We tried different representations for audio (MFCC, Mel-spectrogram, log Mel-spectrogram) with different framerate but found no significant difference during training.

Original: We chose MFCC extracted at the same framerate as motion (30fps).
Revised: We chose MFCC extracted at the same framerate as motion (30fps).

Revised: Asmentioned before, audio recordings contain speech for both speakers in dialogue.

Revised: However, organizers provide transcripts only for the target person’s speech.

Revised: We assumed, that using audio while the target person is silent could help to predict visual response to the companion’s speech.

Original: To distinguish the target person’s speech from the other, we added the textual information as follows.
Revised: But to help model distinguishing the target person’s speech from the other, we added the textual information as follows.

Original: First, we built a vocabulary of symbols on the training dataset.
Revised: First, we built a vocabulary of symbols on the training dataset.

Original: Second, we use one-hot vectors to represent symbols.
Revised: Second, we use one-hot vectors to represent symbols.

Original: Finally, we concatenate audio features with obtained vectors that are uniformly distributed within word spans.
Revised: Finally, we concatenate audio features with obtained vectors that are uniformly distributed within word spans.

Original: Figure 1 shows an example of such alignment.
Revised: Figure 1 shows an example of such alignment.

Original: 3  MODELS 3.1 Seq2seq
Revised:  MODELS 3. Seq2seq

Original: We started to design our architecture taking inspiration from a solution that performed well in the previous competition.
Revised: We started to design our architecture taking inspiration from a solution that performed well in the previous competition.

Original: The seq2seq model by [5] based on both audio and text features achieved the highest median score on human-likeness and the second highest on appropriateness.
Revised: The seq2seq model by [5] based on both audio and text features achieved the highest median score on human-likeness and the second highest on appropriateness.

Original: Unlike the previous work, we propose eliminating separate encoders for audio and text features.
Revised: Unlike the previous work, we propose eliminating separate encoders for audio and text features.

Original: We also decided to use a simple seq2seq instead of a context encoder.
Revised: We also decided to use a simple seq2seq instead of a context encoder.

Original: To keep the context encoder’s main feature of using more context, we decided to use a wider input features window than the length of the output predicting sequence.
Revised: To keep the context encoder’s main feature of using more context, we decided to use a wider input features window than the length of the output predicting sequence.

Original: Other components, such as the loss function and decoder initialization, remained unchanged.
Revised: Other components, such as the loss function and decoder initialization, remained unchanged.

Original: Figure 2. depicts the final seq2seq model scheme.
Revised: Figure 2 depicts the final seq2seq model scheme.

Original: Here 𝑎 𝑖 − 𝑝 , . .
Revised: Here 𝑎 𝑖 − 𝑝 , . .

Original: .,𝑎 𝑖
Revised: .,𝑎 𝑖

Original: + 𝑓 is an input features window for output motionwindow 𝑚 𝑖 , . .
Revised: + 𝑓 is an input features window for output motion 𝑚 𝑖 , . .

Original: .,𝑚 𝑖 + 𝑘 ,  𝑓 > = 𝑘 , 𝑚 𝑖 − 𝑝 , . . .,𝑚
Revised: .,𝑚 𝑖 + 𝑘 , where 𝑓 > = 𝑘 . 𝑚 𝑖 − 𝑝 , . . .,𝑚

Original: 𝑖 −motions from the previous step.
Revised: 𝑖 − 1 are the output poses from the previous step.

Original: Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Original: During our tests, this model produced good loss function values, but the resulting motions converged to minor movements around the rest pose.
Revised: This model produced good loss function values during our tests, but the resulting motions converged to minor movements around the rest pose.

Original: We also tried some training strategies, such as feeding zeros instead of previous poses and adding additional dropouts inside the motion decoder to force the model to focus on input features.
Revised: We also tried some training strategies, such as feeding zeros instead of previous poses and adding additional dropouts inside the motion decoder to force the model to focus on input features.

Original: However, this shows only minor improvements.
Revised: However, this shows only minor improvements.

Original: We believe this defect in the method manifested itself due to the significant differences in the datasets used in this and previous challenges.
Revised: We believe this defect in the method manifested itself due to the significant differences in the datasets used in this and previous challenges.

Original: The last year’s challenge dataset only included one actor’s performance.
Revised: The last year’s challenge dataset only included one actor’s performance.

Original: There were no long pauses in monologue, in contrast to the dialogue speech, and no audio artifacts, allowing the model to create a consistent mapping between speech and gestures.
Revised: There were no long pauses in monologue, in contrast to the dialogue speech, and no audio artifacts, allowing the model to create a consistent mapping between speech and gestures.

Original: As a result, we require a new model to address the following issues:
Revised: As a result, we require a new model to address the following issues:

Original: (1) Predict expressive motions from actual speech (2) Realistically continue motion in the absence of speech
Revised: (1) Predict expressive motions from actual speech (2) Realistically continue motion in the absence of speech

Original: We tried to solve the aforementioned problems with the following architecture.
Revised: We tried to solve the aforementioned problems with the following architecture.

Original: 3.2 ReCell
Revised: 3. ReCell

Original: We discovered that the model provided in [6] generates a variety of poses but they are unstable.
Revised: We discovered that the model provided in [6] generates a variety of poses but they are unstable.

Original: This model creates a singleframe of animation  from a sliding window of audio features.
Revised: This model creates a single animation frame from a sliding window of audio features.

Original: We used auto-regressive input to stabilize predicted motions.
Revised: We used autoregressive input to stabilize predicted motions.

Original: We also tweaked the feature window encoder.
Revised: We also tweaked the feature window encoder.

Original: The original model for frame 𝑖 takes input a i-k/2,...
Revised: The original model for frame 𝑖 takes input 𝑎 𝑖 − 𝑘 / 2 , . .

Original: ,a 𝑖 + 𝑘 / 2, passes it through a single-direction GRU, and takes the last output.
Revised: .,𝑎 𝑖 + 𝑘 / 2 , passes it through a single-direction GRU, and takes the last output.

Original: We decided to take the 𝑘 / 2-th output from bidirectional GRU, which corresponds to the current frame.
Revised: We decided to take the 𝑘 / 2-th output from bidirectional GRU, which corresponds to the current frame.

Original: During our tests, this modification produced slightly better results.
Revised: We found that this modification produced slightly more expressive motion, which could be seen in our supplementary materials.

Original: We called this model "ReCell" because it replicates recurrent cell behavior.
Revised: We called this model "ReCell" because it replicates recurrent cell behavior.

Original: Figure 3 depicts the model’s architecture.
Revised: Figure 3 depicts the model’s architecture.

Original: To avoid the seq2seq model’s convergence to a rest pose, we should strike a balance between auto-regressive input and speech features.
Revised: To avoid the seq2seq model’s convergence to a rest pose, we should strike a balance between auto-regressive input and speech features.

Original: The authors of [7] faced the same problem.
Revised: The authors of [7] faced the same problem.

Original: To overcome this they forced the model to extract useful features from the speech input by pretraining the model without auto-regression.
Revised: To overcome this, they forced the model to extract useful features from the speech input by pretraining the model without auto-regression.

Original: Then they also used the teacher forcing technique to let the model properly integrate auto-regressive input.
Revised: Then they also used the teacher forcing technique to let the model properly integrate auto-regressive input.

Original: As the result, we attempted to implement these ideas in the following manner.
Revised: As the result, we attempted to implement these ideas in the following manner.

Original: Our validation step uses a short sequence of  successive frames as well as the previous step’s output frame.
Revised: In order to optimize the training procedure and allow teacherforcing we first slice input data sequences into small sub-sequences of 10 successive poses and speech features each. On each training step, the model receives a batch of sub-sequences along with a pose corresponding to ( 𝑖 − 1)-th element, where 𝑖 indicates the start of the sub-sequence.

Original: Then we iterate through this sequence, passing the output from the previous iteration to our model speech features.
Revised: Then we iterate through this sub-sequence starting from ( 𝑖 − 1)-th pose in an auto-regressive manner, passing the output of the previous iteration as an input in the current iteration.

Original: In the training step we only change auto-regressive inputs:
Revised: We additionally apply small changes in training input depending on epoch number:

Original: • During the first 10 epochs, we zero the last frame from the previous training step as well as some frames between iterations.
Revised: • During the first 10 epochs, we zero the starting pose as well as some poses between iterations.

Revised: A number of epochs were chosen empirically, as the loss stopped decreasing afterepochs without zeroing.

Original: • After the first 10 epochs, we do not zero the last frame from the previous training step, but instead, replace some intermediate frames with real ones.
Revised: • After the first 10 epochs, we use starting pose from data, but instead, replace some intermediate predicted poses with real ones imitating the teacher-forcing technique.

Original: The precise strategy for replacing framesbetween iterations along with  model parameters will be provided later.
Revised: The precise strategy for replacing frames along with a description of model hyperparameters will be provided in 3.4.

Original: On prediction, we go through the entire recording frame by frame, beginning with zero frame as auto-regressive input.
Revised: On prediction, we go through the entire recording frame by frame, beginning with zero pose as auto regressive input

Original: 3.3 Windowed auto-encoder
Revised: 3. Windowed auto-encoder

Original: To overcome motion instability we trained a simple linear auto-encoder to reduce the dimension of the output vectors.
Revised: To overcome motion instability we trained a simple linear autoencoder to reduce the dimension of the output vectors.

Original: It has two major components, as usual: encoder and decoder.
Revised: It has two major components, as usual: encoder and decoder.

Original: Input poses are projected into lower-dimensional space by the encoder.
Revised: Input poses are projected into lower-dimensional space by the encoder.

Original: In turn, the decoder must reconstruct the input pose from its lower-dimensional representation.
Revised: In turn, the decoder must reconstruct the input pose from its lower-dimensional representation.

Original: Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Original: The encoder has one hidden layer, that converts the input vector to a size of 512.
Revised: Theencoderhas one hidden layer, that converts the input vector to a size of 512.

Original: After that, we apply ReLU as an activation function.
Revised: After that, we apply ReLU as an activation function.

Original: Then we apply another linear layer and Tanh function to normalize.
Revised: Then we apply another linear layer and Tanh function to normalize.

Original: The final size of the output vector  is 60.
Revised: We set the final size of the output vector equal to 60 as it is the smallest possible size that allows retaining enough information while also providing compact representation. When we decreased the size of the output vector we started to notice visible artifacts on the reconstructed motion.

Original: We use one additional hidden linear layer to restore the vector from the encoded state, which translates the  vector into dimension  512.
Revised: We use one additional hidden linear layer to restore the vector from the encoded state, which translates the encoder output vector of dimension 60 into 512.

Original: Next, a ReLU  and an output linear layer were used, which translates the vector  into dimension 164.
Revised: Next, a ReLU function and an output linear layer were used, which translates the vector of 512 into dimension 164.

Original: Since normalized features from the value range  [0, 1] were fed to the auto-encoder input, the decoder output was also normalized by  Tanh  and shifted to [0, 1].
Revised: Since normalized features with values that range between [0, 1] were fed to the auto-encoder input, the decoder output is then normalized by the Tanh function and shifted to [0, 1], which showed better results than Sigmoid during our experiments.

Original: We discovered that when such an auto-encoder is trained, the reconstruction is mostly similar to the input pose, but it occasionally provides unrealistic poses for the left shoulder.
Revised: We discovered that when such an auto-encoder is trained, the reconstructed pose is mostly similar to the input pose, but it occasionally provides unrealistic poses for the left shoulder.

Original: We believe this behavior is related to rare jerks in training data.
Revised: We believe this behavior is related to rare jerks in training data.

Original: To address this issue, we decided to improve our auto-encoder by reconstructing a short sliding window of poses rather than the single one.
Revised: To address this issue, we decided to improve our auto-encoder by reconstructing a short sliding window of poses rather than the single pose.

Original: As a result, a configured vector of three  frames was fed  to the auto-encoder, implying that the initial and output states were of dimension 492.
Revised: As a result, we stack three sequential frames into a combined vector and fed it to the auto-encoder, implying that the initial and output vectors were of dimension 164 × 3.

Original: Otherwise, the auto-encoder architecture was similar to that described above.
Revised: In everything else, the auto-encoder architecture is similar to that described above.

Original: We average the outputs in which the current frame participates during inference.
Revised: In order to get a pose for the current frame we average its corresponding poses from all sequential windows it appears in (e.g for 5th pose we average pose vectors 𝑝 5 from sequences [ 𝑝 3 ,𝑝 4 ,𝑝 5 ] , [ 𝑝 4 ,𝑝 5 ,𝑝 6 ] , [ 𝑝 5 ,𝑝 6 ,𝑝 7 ] ).

Original: 3.4 Final pipeline
Revised: 3. Final pipeline

Original: Our final approach is as follows.
Revised: Our final approach is as follows.

Original: As the main model, we used the ReCell model with the following parameters.
Revised: As the main model, we used the ReCell model with the following parameters.

Original: First of all, our Bi-directional encoder consists of  linear highway and GRU.
Revised: First of all, our Bi-directional encoder consists of a linear highway and GRU.

Original: Linear highway, in turn, consists of three sequential blocks with linear layer, batch normalization, and dropout with ReLU as the activation function to encode audio features.
Revised: Linear highway, in turn, consists of three sequential blocks with linear layer, batch normalization, and dropout with ReLU as the activation function to encode audio features.

Original: As mentioned before, we use GRU to extract contextual features from the window of audio features.
Revised: As mentioned before, we use GRU to extract contextual features from the window of audio features.

Original: The size of this window is  61.
Revised: The size of this window is equal to 61.

Original: All hidden sizes are  150.
Revised: All hidden sizes are equal to 150.

Original: At the same time, we put the pose from  previous step throughthe another highway with  bottleneck of size 40.
Revised: At the same time, we put the pose from the previous step through another highway with a bottleneck of size 40.

Original: It contains only two linear layers with ReLU between them and dropout to prepare model output for auto-regression.
Revised: It contains only two linear layers with ReLU between them and dropout to prepare model output for auto-regression.

Original: It enables us to obtain more complex and informative features, as well as translate vectors to the required size.
Revised: It enables us to obtain more complex and informative features, as well as translate vectors to the required size.

Original: This operation we call Hidden Highway.
Revised: This operation we call Hidden Highway.

Original: Finally, outputs of  Bi-directional encoder and Hidden Highway are concatenated together to create  final feature vector.
Revised: Finally, outputs of the Bi-directional encoder and Hidden Highway are concatenated together to create the final feature vector.

Original: Then, we apply another batch normalization, ReLU and Dropout to the resulting feature vector and push it through the last linear layer to produce the next pose.
Revised: Then, we apply another batch normalization, ReLU, and Dropout to the resulting feature vector and push it through the last linear layer to produce the next pose.

Original: During the teacher-forcing, we zeroed frames between iterations with a probability of 0.5 in the first ten epochs, and we also replaced some frames with real ones with a probability of 0.5 in the following epochs.
Revised: During the teacher-forcing, we zeroed frames between iterations with a probability of 0.5 in the first ten epochs, and we also replaced some frames with real ones with a probability of 0.5 in the following epochs.

Original: To train the network, we use MSELoss and the Adam optimizer with default parameters from PyTorch.
Revised: To train the network, we use MSELoss and the Adam optimizer with default parameters from PyTorch.

Original: We also train the main model to predict audio-encoder representations rather than poses.
Revised: We also train the main model to predict audio-encoder representations rather than poses.

Original: Then, using the Savitsky Golay filter, we reconstruct the poses and smooth the predictions.
Revised: Then, using the Savitsky-Golay filter, we reconstruct the poses and smooth the predictions.

Original: We use26 MFCC coefficients concatenated with one-hot encodings of text transcripts as input speech representation.
Revised: We use MFCC coefficients concatenated with one-hot encodings of text transcripts as input speech representation.

Revised: It’s also worth noting that our auto-encoder was trained using the entire train dataset.

Revised: However, in order to train the ReCell model we selected samples from the most frequently encountered speaker.

Revised: We also removed audio cuts-related fragments.

Original: 4  RESULTS AND DISCUSSION
Revised:  RESULTS AND DISCUSSION

Original: As in the previous challenge, the organizers provide [11] human-evaluation results for comparing the system.
Revised: As in the previous challenge, the organizers provide [11] humanevaluation results for comparing the systems.

Original: As before there are  two main metrics: human-likeness and appropriateness.
Revised: As before there are [Table] two main metrics: human likeness and appropriateness.

Original: The first evaluates how predicted motions are realistic, and the second estimates the correspondence between speech and predicted motions from it.
Revised: The first evaluates how predicted motions are realistic, and the second estimates the correspondence between speech and predicted motions from it.

Original: While human-likeness evaluation follows the previous challenge, the approach to estimating appropriateness is different.
Revised: While human-likeness evaluation follows the previous challenge, the approach to estimating appropriateness is different.

Original: Investigating the results of the previous challenge organizers found that calculated appropriateness was highly correlate with the quality of generation motion.
Revised: Investigating the results of the previous challenge organizers found that calculated appropriateness was highly correlated with the quality of generation motion.

Original: Therefore, this time they decided to calculate appropriateness inAnon. a different way.
Revised: Therefore, this time they decided to calculate appropriateness in a different way.

Original: They ask evaluation study participants not to estimate the result videos alone but compare the corresponding audio-video pair with the mismatched one.
Revised: They ask evaluation study participants not to estimate the result videos alone but compare the corresponding audio-video pair with the mismatched one.

Original: It could help to find out if generated motion really appropriate to speech or just random.
Revised: It could help to find out if generated motion really appropriate to speech or just random.

Revised: Table 1 presented by organizers shows the results of human-evaluation.

Original: Our system  achieved slightly poor results for both metrics for the full-body motion: 34  median scores for humanlikeness and 51.5 of matched percent for appropriateness.
Revised: Our system (FSD) achieved slightly poor results on both metrics for the full-body motion: 34 on the human-likeness median score and 51.5% on Appropriateness percent matched.

Original: Nevertheless, it outperforms the provided baseline by human-likeness. Although, there are models that showed significantly better results.
Revised: Our model outperforms the provided baseline by human-likeness, although, there are models that showed significantly better results.

Original: Moreover, there is a model that outperforms even real motion by human-likeness.
Revised: Moreover, there is a model that outperforms even real motion by human-likeness.

Original: This leads us to the fact that there isa huge room to improvement.
Revised: This leads us to assume that there is huge room for improvement in our approach.

Original: The low score in human-likeness could be connected with insufficient fluidity of movement and periodic jerks.
Revised: The low score in human-likeness could be connected with insufficient fluidity of movement and periodic jerks.

Original: It could be smoothed with an even stronger filter but it could make movements «like in jelly».
Revised: It could be smoothed with an even stronger filter but that could result in inexpressive, overly smoothed «jelly-like» movements.

Original: Although the lowest score of appropriateness means that generated motions do not have any semantics.
Revised: Also the lowest score of appropriateness means that generated motions do not have any semantics.

Original: It is only hand movements supporting the speech rhythm.
Revised: It is only hand movements that follow the speech rhythm.

Original: To add semantics the different features could be used, like Glove or BERT embeddings for  textual transcripts.
Revised: Additional features, like Glove or BERT embeddings for a textual transcript, could be used to add semantics.

Original: It is also could be improved with extensive data cleanup.
Revised: We suppose that results can further be improved by extensive data cleanup.

Original: For example, only motions corresponding to actual speech could be used for training.
Revised: For example, only motions corresponding to actual speech can be used for training.

Original: Thus, even the seq2seq approach could be improved and maybe shows better results.
Revised: Thus, even the seq2seq approach has room for improvement and presumably is able to show better results.

Revised: During the previous 2020 GENEA Challenge organizers also found that objective metrics such as average jerk and Hellinger distance may not correlate with visual quality.

Revised: Therefore, we decided not to rely on objective metrics during the development of our system and to use only visual assessment.

Revised: However, organizers also provided [11] some objective metrics for the final evaluation of the submitted approaches.

Original: 5  CONCLUSION
Revised:  CONCLUSION

Original: The proposed challenge raised issues that are difficult to solve directly with a tried-and-true method.
Revised: The proposed challenge raised issues that are difficult to solve directly with a tried-and-true method.

Original: However, with additional data processing and training techniques, it may be possible to solve the problem relatively successfully.
Revised: However, with additional data processing and training techniques, it may be possible to solve the problem relatively successfully.

Original: Although our approach is far from ideal, it has the potential to be improved.
Revised: Although the proposed approach is far from ideal it has the potential to be improved.

Original: Wealso believe that, with some modifications, the  sequence-to-sequence approach could  produce better results in the future.
Revised: We believe that, with some modifications, the alternative sequence-to-sequence approach could also produce better results.