Pose Sequence Generation
with a GCN and an initial Pose Generator
The University of Electro-Communications Kento Terauchi, Keiji Yanai

Experiments

MethodBackground

Examples of generated results on Run action

Make two changes to Action2Motion

Method: change(1) Add module for initial frame consideration

Cai et al[CVPR, 2017]

Related Works
Action2Motion[ACM Multi Media, 2020]

We encode initial frame encoder to encode initialframe and initial condition generator to the model ofAction2Motion.
We encode the first pose with the initial frame
encoder and decode it to condition of decoder withthe initial condition decoder.
After the second frame, Learn to bring the outputs ofthe previous frame encoder and the next frame
decoder closer to each other.

At the time of generation, the first frame is acondition generated from the noise of the normal
distribution using the initial frame condition decoder.
After the second frame, the next frame issequentially generated by encoding the previous
frame with the previous frame encoder.

Ours

Some two-step video generation consisting of generation of posesequences and video generation from pose sequences.
↓By making pose generation natural, it is possible to generate human motion
videos that perform natural motions.

Action2Motion

DatasetResearch Objective: natural pose sequence generation

Method: Change(2)introduce GCN to the decoder

Qualitative Evaluation
The proposed method, the results of which are represented as ``Ours(FULL)'' in thetable, outperforms the existing methods in FID while maintaining the same accuracy.

They introduces Prior Loss which brings the distributions of the outputs ofthe encoder of the previous frame and the encoder of the next frame
closer together.
During testing, the encoder of the previous frame produces an output
similar to the encoder of the next frame.
From the pose of the previous frame, the pose of the next frame can begenerated sequentially.

It learn the structural representation by using three layers of GCN to get a complexrepresentation.
The output is obtained by passing through a fully connected layer with different weights foreach joint.

Ours (IPG only) outperformed the baseline regarding FID.
Its accuracy was inferior to Ours(FULL), and it could not generate a clear posesequence.
Ours (GCN only) failed to generate a clear pose sequence because both Accuracy
and FID are by far less than others.

Pose Sequence Generation
with a GCN and an initial Pose Generator

Kento Terauchi, Keiji Yanai
The University of Electro-Communications

Research Objective: natural pose sequence generation

学習時

Cai et al[CVPR, 2017]

Some two-step video generation consisting of generation of posesequences and video generation from pose sequences.
↓By making pose generation natural, it is possible to generate human motion
videos that perform natural motions.

Related Works: Action2Motion[ACM Multi Media, 2020]↓Encoder or the next frame
They introduces Prior Loss whichbrings the distributions of the outputsof the encoder of the previous frame
and the encoder of the next frame
closer together.

⇅Decoder⇅Encoder or the previous frame

During testing, the encoder of the previous
frame produces an output similar to theencoder of the next frame.
From the pose of the previous frame, thepose of the next frame can be generated
sequentially.

Method: change(1) Add module for initial frame consideration

We encode initial frame encoder to encode initial frame andinitial condition generator to the model of Action2Motion.
We encode the first pose with the initial frame encoder anddecode it to condition of decoder with the initial condition
decoder.
After the second frame, Learn to bring the outputs of theprevious frame encoder and the next frame decoder closer toeach other.

At the time of generation, the first frame is acondition generated from the noise of the normal
distribution using the initial frame condition decoder.
After the second frame, the next frame issequentially generated by encoding the previousframe with the previous frame encoder.

Method: Change(2)introduce GCN to the decoder

It learn the structural representation by using three layers of GCN to get acomplex representation.
The output is obtained by passing through a fully connected layer withdifferent weights for each joint.

Ours

Action2Motion

Qualitative Evaluation: Examples of generated results on Run Action

Interpolation of latent space
Interclass interpolation between jump and sit action
Jump↑

middle↓SitQualitative Evaluation
The proposed method, the results of which are represented as ``Ours(FULL)'' in the table, outperforms theexisting methods in FID while maintaining the same accuracy.

Ours (IPG only) outperformed the baseline regarding FID.
Its accuracy was inferior to Ours(FULL), and it could not generate a clear pose sequence.
Ours (GCN only) failed to generate a clear pose sequence because both Accuracy and FID
are by far less than others.

Conclusion• In this study, we proposed a model that explicitly captures the structural
information by considering the initial frame with the intial pose generator
and incorporating GCN.
• The qualitative results showed that the proposed method was capable ofgenerating complex and diverse motions.
• Quantitatively, the proposed method outperformed the existing methods
and showed more natural generation.