3D Human Pose Estimation from Multiple Dynamic Views via Single-view Pretraining with Procrustes Alignment

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-04 15:15:35

Abstract: 3D Human pose estimation from multiple cameras with unknown calibration has received less attention than it should. The few existing data-driven solutions do not fully exploit 3D training data that are available on the market, and typically train from scratch for every novel multi-view scene, which impedes both accuracy and efficiency. We show how to exploit 3D training data to the fullest and associate multiple dynamic views efficiently to achieve high precision on novel scenes using a simple yet effective framework, dubbed \textit{Multiple Dynamic View Pose estimation} (MDVPose). MDVPose utilizes novel scenarios data to finetune a single-view pretrained motion encoder in multi-view setting, aligns arbitrary number of views in a unified coordinate via Procruste alignment, and imposes multi-view consistency. The proposed method achieves 22.1 mm P-MPJPE or 34.2 mm MPJPE on the challenging in-the-wild Ski-Pose PTZ dataset, which outperforms the state-of-the-art method by 24.8% P-MPJPE (-7.3 mm) and 19.0% MPJPE (-8.0 mm). It also outperforms the state-of-the-art methods by a large margin (-18.2mm P-MPJPE and -28.3mm MPJPE) on the EgoBody dataset. In addition, MDVPose achieves robust performance on the Human3.6M datasets featuring multiple static cameras. Code will be released upon acceptance.

Main paper: 
 related work: 
 Multi-view 3D HPE. Supervised multi-view 3D HPE methods [4, 10, 36] , including multi-view single person [15] and multi-person [2, 4, 18] methods, can predict highly accurate poses, but typically require precise camera calibration during both training and inference. Most methods [9, 14, 17, 33] obtain the 2D pose by running a CNN over 2D poses given in multiple views. To exploit temporal information, Chen et al. [4] couple cross-view tracking and multihuman 3D pose estimation in a unified framework. TesseTrack [34] handles multi-person 3D body joint reconstruction and association in space and time in a single end-to-end learnable framework.VoxelPose [36] aggregates features of all camera views in the 3D voxel space and directly operates in the 3D space to avoids making incorrect decisions in each camera view. However, supervised methods has a few issues. First, obtaining camera calibration is in convenient or impractical during inference. Second, due to the high cost of motion capture, the scarcity of 3D training data becomes a bottleneck. Third, and more importantly, these methods cannot be applied to dynamic (possibly moving) cameras without camera calibration. Some works turn to weakly or self-supervised solutions [5, 16, 22, 40] . EpipolarPose uses 2D poses from multi-view images and then utilizes epipolar geometry to obtain a 3D pose and camera geometry, which are subsequently used to train a 3D pose estimator. Iqbal et al. [16] propose a weakly-supervised baseline to predict pixel coordinates of joints and their depth in each view and penalize the discrepancy between rigidly aligned predictions for different views during training. RepNet [39] and Chen et al. [3] use more realistic data to train 2D-to-3D lifting networks. CanonPose [40] presents a self-supervised approach that learns a single image 3D pose estimator from unlabeled multi-view data. These solutions, however, do not allow pose inference from multiple images.3D HPE from Multiple Dynamic Views. Human pose estimation from multiple cameras with unknown calibration has received less attention than it should. MetaPose [37] is the first multi-view 3D HPE method that requires no camera parameters at both training and inference time, and thus can be applied to dynamic (possibly moving) cameras. MetaPose aggregates pose predictions and uncertainty estimates across multiple views and trains the network without 3D Supervision. The advantage of MetaPose is that it requires only 2D keypoint data for training, which is suitable for cases where 3D data is unavailable. Yet, the downside of MetaPose is it does not utilize the 3D training data when available. FLEX [13] presents an extrinsic parameter-free multi-view model, in the sense that it does not require extrinsic camera parameters. It learns fused deep features through a multi-view fusion layer. HMVformer [46] proposes a hierarchical multi-view fusion transformer framework for 3D HPE, incorporating cross-view feature fusion methods into the spatial and temporal feature extraction process in a coarse-tofine manner. In contrast, we show that without designs in deep feature fusion, by exploiting a single-view pretrained motion model and transforming multi-view 3D poses into a unified coordinate system with Procrustes alignment which is robust to the relative positions of multiple views, we can obtain highly accurate 3D estimates in the wild.3D HPE with Transformer backbones. 3DHPE with Transfomer backbones, such as PoseFormer [45] , MHFormer [24] , MixSTE [42] , and MotionBERT [48] , have gained great success due to its competence in handling sequences. MotionBERT [48] proposes a pretraining stage in which a motion encoder is trained to recover the underlying 3D motion from noisy partial 2D observations, and implements the motion encoder with a Dual-stream Spatio-temporal Transformer (DSTformer) neural network. Recently, PoseFormerV2 [44] exploits a compact representation of lengthy skeleton sequences in the frequency domain to efficiently scale up the receptive field and boost robustness to noisy 2D joint detection. HSTFormer [32] focuses on capturing multi-level joints' spatial-temporal correlations from local to global gradually for accurate 3D HPE, and [8] focuses on parameter reduction for 3D HPE by leveraging dynamic multi-headed convolutional attention. Mo-tionAGFormer [29] presents an Attention-GCNFormer (AGFormer) block that divides the number of channels by using two parallel Transformer and GCNFormer streams, and reportedly achieves impressive results.Orthogonal to these efforts in improving the Transformer backbones, we simply take advantage of the Transformer backbone as our motion encoder, and focus on improving the performance and flexibility of 3D HPE from multiple dynamic views. We adopt the DSTFormer proposed by motionBERT [48] . 
method 3.1 overview: 
 The overview of the proposed framework is demonstrated in Fig. Given the input 2D keypoint sequences from 𝑁 dynamic views x ∈ R 𝑁 ×𝑇 ×𝐽 ×𝐶 𝑖𝑛 , and 3D joints sequence from multiple dynamic views X ∈ R 𝑁 ×𝑇 ×𝐽 ×𝐶 𝑜𝑢𝑡 , we project the 2D keypoints to a highdimensional feature F 0 ∈ R 𝑁 ×1×𝐽 ×𝐶 𝑓 following motionBERT [48] .𝑁 is the number of views,𝑇 is the pose sequence length, 𝐽 is the number of joints in the human skeleton, and 𝐶 𝑖𝑛 is the number of input channels. Learnable spatial positional encoding 𝑃 𝑇 𝑝𝑜𝑠 ∈ R 𝑁 ×1×𝐽 ×𝐶 𝑓 and temporal positional encoding 𝑃 𝑇 𝑝𝑜𝑠 ∈ R 𝑁 ×𝑇 ×1×𝐶 𝑓 are then added to the extracted feature. The sequence-to-sequence model DSTformer is then used to calculate F 𝑑 ∈ R 𝑁 ×𝑇 ×𝐽 ×𝐶 𝑓 (𝑑 = 1, ..., 𝐷), where 𝐷 is the network depth. Same as motionBERT, a linear layer is applied to F D to compute the motion feature E ∈ R N×T×J×C e . 𝐶 𝑖𝑛 , 𝐶 𝑓 , 𝐶 𝑒 , and 𝐶 𝑜𝑢𝑡 are the channel numbers of input, feature, embedding, and output respectively. 𝐶 𝑜𝑢𝑡 equals 3, i.e. the three dimensions 𝑋, 𝑌, 𝑍 . We let batch size equal the number of viewpoints 𝑁 during training.To extract the motion features, the Dual-stream Spatio-temporal Transformer which stacks the spatial and temporal MHSA blocks alternately in two branches [48] and the output features of the two branches are adaptively fused by an attention regressor. The dual-stream-fusion module is repeated for D times:EQUATIONwhere 𝑑 ∈ 1, ..., 𝐷 is the network depth and F 𝑑 denotes the feature embedding at depth 𝑑. 𝑆 and 𝑇 are spatial and temporal blocks. Adaptive fusion weights 𝛼 𝑑 𝑆𝑇 , 𝛼 𝑑 𝑇 𝑆 ∈ R 𝐷 ×𝑁 ×𝑇 ×𝐽 are given by a shallow layer, following the same architecture as in motionBERT [48] .The DSTFormer architecture is illustrated in Fig. 3 . The building blocks of DSTFormer consists of spatial and temporal Multi-Head Self Attention (MHSA) blocks stacked in different orders, forming two parallel computation streams which are expected to specialize in different spatial-temporal aspects. DSTFormer takes multi-view 2D keypoint sequences x as input and passes it through a fully connected (FC) later before feeding it into the DSTFomer.Spatial MHSA.The Spatial MHSA (S-MHSA) models the relationship among the joints from a view within the same time step. Note that each view is computed separately. It is defined as:EQUATIONwhere W 𝑃 𝑆 is a projection parameter matrix. Query 𝑄 ℎ 𝑆 , key 𝐾 ℎ 𝑆 , and value 𝑉 ℎ 𝑆 are per frame spatial feature from input 𝐹 𝑆 from each attention head ℎ. 𝑑 𝐾 is the feature dimension of 𝐾𝑆. Residual connection and layer normalization (LayerNorm) are used to the S-MHSA block following classical Transformers [38] .Temporal MHSA. The Temporal MHSA (T-MHSA), on the other hand, models the relationship across the time steps for a body joint and applies to a feature 𝐹 𝑇 ∈ R 𝑇 ×𝐶 𝑒 . The notation is similar to the spatial MHSA.T -MHSA(Q 𝑇 , K 𝑇 , V 𝑇 ) = [ℎ 1 , ...ℎ 𝐻 ]W 𝑃 𝑇 ℎ ℎ = Softmax Q ℎ 𝑇 (𝐾 ℎ 𝑇 )⊤ √︁ 𝑑 𝐾 V ℎ 𝑇 .(3)We use the pretrained motion encoder given by motionBERT [48] ,where the learned feature embedding E serves as a 3D-aware and temporal-aware human motion representation. Since the motion encoder adopts masking strategy during pretraining, it can realize pose track inpainting as well. Our work focuses on the 3D human pose estimation task; therefore, we fine-tune the network to achieve 2D-to-3D lifting and inpainting, i.e. 3D human pose estimation.The loss for multi-view 3D HPE finetuning is computed as:EQUATION)where 𝑁 is the number of viewpoints. L 3𝐷 is the pose regression loss, L 𝑂 is the velocity loss. Predicted velocity is calculated byO 𝑡 = X 𝑡 -X 𝑡 -1, and likewise for ground truth velocity Ô𝑡 .EQUATIONwhere 𝐽 is the number of joints, and 𝑇 the sequence length.We design the framework such that the model can take a arbitrary number of viewpoints as input. This is done by gating the data of any viewpoint with a certain probability during training, only keeping some of the viewpoints. Let E 𝑖 ∈ R 𝑇 ×𝐽 ×𝐶 𝑜𝑢𝑡 be the motion feature and X 𝑖 ∈ R 𝑇 ×𝐽 ×𝐶 𝑜𝑢𝑡 be the output from View 𝑖, we add a control gate 𝐺 𝑖 ∈ 0, 1 at each view to mask the feature and 3D pose output sequence.EQUATIONWhen 𝐺 𝑖 = 1, View 𝑖 is valid, otherwise it is invalid. 𝑃 𝑖 is the probability of having 𝑖 valid views.EQUATIONWhen there are multiple viewpoints in the input, we associate multiple viewpoints by 3D human joints themselves and apply multi-view consistency constraint. This process can be done almost for free. The multi-view consistency loss. L MV is computed as follows.(1) We predict the 3D human pose of each viewpoint 𝑖 at time 𝑡 as X 𝑡 𝑖 ∈ R 𝐽 ×𝐶 𝑜𝑢𝑡 . 𝐶 𝑜𝑢𝑡 = 3. (2) Using the first viewpoint as the reference view, we procrustesalign the 3D pose estimated from the other viewpoints to the first viewpoint if there should be other viewpoints, i.e. 𝑁 ≥ 2. PROC(x, y) denotes the procrustes alignment of 3D joints.PROC(x, y) = 𝑠 (Rx + t)EQUATIONThe procrustes alignment calculates the scale 𝑠, rotation R, and translation t between two sets of 3D point correspondences, which is the same as calculating P-MPJPE from MPJPE [41] .EQUATION(3) Using the procruste-aligned X𝑡 𝑖 , compute the multi-view consistency loss:EQUATIONThus, the overall loss of our framework becomes:EQUATIONwhere 𝑁 is the number of viewpoints. I is a binary indicator function to decide whether to include the multi-view consistency loss. 
experiments 4.1 experimental settings: 
 Data. We evaluate the performance of our proposed method on three 3D human pose estimation benchmarks, i.e., one challenging in-the-wild dataset SkiPose [12] , one recent dataset EgoBody [43] , and one well-established dataset Human3.6M [15] . Ski-Pose PTZ (SkiPose) is a challenging in-the-wild dataset with six moving pan-tilt-zoom cameras that is perfect for evaluating our proposed method. This multi-view pant-tilt-zoom-camera (PTZ) dataset features competitive alpine skiers performing giant slalom runs. It provides labels for the skiers' 3D poses in each frame, their projected 2D pose in all 20k images, and accurate per-frame calibration of the PTZ cameras.The model is trained and tested using an official training and testing set. EgoBody is another large-scale dataset for 3D human motions during social interactions in complex 3D scenes.For each sequence, multiple Azure Kinects capture the two-subject interactions from different views with RGBD streams, and a synchronized HoloLens2 worn by one subject captures the first-person view image. We only use the RGB data from Kinect cameras for training in our experiments. Human3.6M is a large-scale 3D dataset of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, providing a diversity of human activities.We keep the same training and test split as in [28, 48] .Metrics. We report (1) Mean Per Joint Position Error (MPJPE), and (2) Procrustes aligned Mean Per Joint Position Error (P-MPJPE) that measure the L2-error of 3D joint estimates after applying the optimal rigid alignment (including scale) [7, 41] . Some other methods report (3) normal mean per joint position error (NMPJPE) that normalizes MPJPE to exclude the effect of scale.Implementation details. We use 2 RTX3090 to train our model. DSTformer is trained with depth 𝐷 = 5, number of heads ℎ = 8, feature size 𝐶 𝑓 = 512, embedding size 𝐶 𝑒 = 512, following motionBERT. The probability of having 1 valid viewpoint during training is set to 𝑃 1 = 0.5. 𝜆 𝑀𝑉 = 1, 𝜆 3𝐷 = 1, 𝜆 𝑂 = 1. We set batch size to be the number of views 𝑁 , which is 6 for SkiPose and 4 for Human3.6M. For Human3.6M, the 2D skeletons are provided by 2D pose estimator trained on MPII [1] and Human3.6M following the common practice of [48] .Table 1 : Comparisons with SOTA methods on SkiPose. The results of AniPose, Rhodin et al. [35] , CannonPose [40] , MetaPose [37] were provided by [37] . IR means iterative refinement, the details of which can be found in motionBERT. ↓ means the lower the better. Best in bold.AniPose w/ GT [20] [23] , METRO [25] , PARE [21] and EFT [19] were provided by the official repository of EgoBody [43] Quantitative Evaluation on SkiPose. We compare our method with the state-of-the-art methods for 3D HPE using multiple dynamic views on the SkiPose dataset, and the results are listed in Table 1 . Our method achieves an unprecedented 22mm P-MPJPE (after procrustes alignment) as well as 34mm MPJPE. The P-MPJPE(↓) is 7.4mm (25.2%) lower than the second-best model HMVPose [46] (22mm vs. 29.4mm). Note that the MetaPose performance was achieved with a series of iterative refinement steps, which we do not perform. Even if we use a weak baseline without heavy efforts in feature fusion, we were able to easily achieve highly accurate pose predictions, showing the efficacy of the proposed method on challenging outdoor data Quantitative Evaluation on EgoBody. Table 2 presents the quantitative evaluations on the EgoBody dataset, where two subjects interact in natural scenes. We first evaluate the results when an off-the-shelf 2D keypoint detector [11] is used during reference. The proposed method significantly outperforms the state-of-the-art methods on EgoBody, surpassing the EgoBody Challenge champion MEEV [30] by 18.2mm P-MPJPE and 28.3mm MPJPE. Subsequently, we show the upper bound of our method when ground truth 2D keypoints are fed into the network during inference, listed as "MDV-Pose(gt)". The experimental results show that our method produces highly accurate 3D pose estimates for novel natural videos without bells and whistles.Quantitative Evaluation on Human3.6M. Table 3 presents the quantitative results on Human3.6M. The proposed MDVPose achieves state-of-the-art accuracy on the competitive benchmark, proving that the MDVPose is robust and generalizes well. Note that our method cannot be compared with HMVPose or FLEX because of two reasons. First, HMVPose and FLEX are not intrinsic-parameter free on Human3.6M, whereas our MDVPose uses neither intrinsic parameters nor extrinsic parameters. Second, the input 2D keypoints of HMVPose were given by CPN while ours were inherited from motionBERT [48] . It is also worth mentioning that 3D HPE from static cameras are not the focus of this paper; yet, MDVPose Fig. 4 illustrates several examples from the SkiPose (left) and the EgoBody (right) dataset. Result 3D pose sequences of synchronized views at different time steps are plotted. For SkiPose, the 2D keypoints are provided by the dataset, following previous works FLEX [13] and HMVFormer [46] . For EgoBody, the input 2D keypoints are extracted by an off-the-shelf 2D keypoint detector [11] . As shown in Fig. 4 , the predicted 3D poses (green skeletons) all align well with the ground truth (red skeletons). Fig. 5 compares MDVPose output with the original motionBERT output. Both using the DSTFormer as backbone, our MDVFormer performs significantly better than motionBERT.Effect of multi-view consistency loss. We conduct ablation studies on the SkiPose dataset with multiple dynamic views. The Performance with the multi-view consistency loss L MV versus without L MV are listed in Table 1 . The model with multi-view consistency loss outputs -3 mm lower P-MPJPE, which is more important in analyzing the behavior of the individual human. This is reasonable because the multi-view consistency loss (Eq. 10) basically penalizes P-MPJPE. Training from scratch versus fine-tuning. The results trained from scratch using SkiPose training data is reported in Row 1, Table 1 . As shown, the fine-tuning strategy outperforms the train-fromscratch strategy by a large margin, proving that mining 3D training data to the fullest can yield the best results. In addition, we plot the error curves of MPJPE and P-MPJPE in Fig. 6 . The fine-tuned model converges significantly faster than that of the model trained from scratch, making it more efficient and practical for real-world applications.Multi-view versus single view. We compare the qualitative results using the original motionBERT model and our MDVPose model in Fig. ??. Using the same backbone, MDVPose shows superior performance than motionBERT on the SKiPose dataset. In summary, the benefit of the proposed method are three-folds: first, With a relatively small amount of novel scene training data, MDVPose can make highly accurate predictions on the novel scenes with multiple dynamic views. Second, the proposed simple yet effective framework does not demand camera calibration, making it extremely practical for scenarios where camera calibration is hard, such as skiing. Third, MDVPose adapts to an arbitrary number of dynamic views to be practical and flexible in complicated real-world scenarios.Limitations and future work. MDVPose only uses a weak baseline without heavy efforts on multi-view feature fusion. We believe that with stronger backbones and more designs in cross-view relations the accuracy could be further improved. Currently, MDV-Pose can only handle single person 3D HPE without considering multi-person tracking. The framework can be extended to multiperson 3D HPE using multiple dynamic views in future, which is a further step towards analyzing human behaviors in the wild.  
: 
 that limit their application in practice. First, obtaining camera calibration is inconvenient or impractical, especially during inference. Second, due to the high cost of motion capture, the scarcity of 3D training data becomes a bottleneck. Since motion capture data is usually recorded in a constrained space, there is a lack of in-thewild data [47] . Third, and more importantly, these methods cannot be applied to dynamic (possibly moving) cameras without camera calibration.Human pose estimation from multiple cameras with unknown calibration has received less attention than it should. For sports capture where close-ups of players are captured in front of moving cameras, camera calibration cannot easily be estimated [37] , as shown in Fig. 1(a) . Another scenario is in a dynamic camera network for autonomous driving, a pedestrian's pose can be estimated collaboratively using several potentially moving cameras in a traffic Internet of Things (IoT) network, such as vehicle-installed cameras and surveillance cameras (See Fig. 1 (b) ).MetaPose [37] is the first multi-view 3D HPE method that requires no camera parameters at both training and inference time, and thus can be applied to dynamic cameras. MetaPose aggregates pose predictions and uncertainty estimates across multiple views, and outperforms both classical bundle adjustment and weaklysupervised monocular 3D baselines on the Human3.6M dataset, as well as the more challenging in-the-wild Ski-Pose PTZ dataset. Nevertheless, MetaPose exploits only 2D keypoint data for training and chooses not to make use of 3D joint annotations, whereas we exploit 3D data when available. FLEX [13] and HMVformer [46] design cross-view deep feature fusion to integrate the relationship between views, to avoid camera calibration. Nonetheless, FLEX [13] and HMVformer [46] train from scratch on every novel scene, overlooking existing pretrained motion model empowered by largescale and heterogeneous data resources, thus leading to deficiency in both accuracy and efficiency on the novel scene. In contrast, we show that by exploiting a single-view pretrained motion model and transforming multi-view 3D poses into a unified coordinate system with Procrustes alignment, we can also bypass camera calibration and obtain highly accurate 3D estimates in the wild without bells and whistles, even if with a weak baseline without heavy efforts in feature fusion.We propose the Multiple Dynamic View Pose Estimation (MDV-Pose) framework, to exploit available 3D joint annotations to the fullest and associate different views by human 3D joints themselves. With a relatively small amount of novel scene training data, MDV-Pose can make highly accurate predictions on novel scenes. Specifically, 2D keypoint sequences from multiple views are fed into a single-view pretrained motion encoder, which can be adapted to the downstream 3D pose estimation task (essentially 2D-to-3D motion lifting and inpainting). Then, we fine-tune the single-view motion encoder using limited 3D annotations of novel scenarios if they are available, and predict the 3D pose in each view. Subsequently, we associate and procrustes-align all views by the predicted 3D human joints themselves in a unified coordinate, and apply multiview consistency. By exploiting the powerful pretrained motion encoder combined with transfer learning, and further imposing constraints of multi-view consistency, high accuracy is achieved for novel scenarios such as skiing. Our proposed MDVPose shows a superior performance of 22.1 P-MPJPE or 34.2 MPJPE, which exceeds the state-of-the-art method by 24.8% P-MPJPE (-7.3 mm) and 19.0% MPJPE (-8.0 mm) on the challenging in-the-wild Ski-Pose PTZ dataset [12] (See Fig. 2 ). It also outperforms the state-of-the-art methods by a large margin (-18.2mm P-MPJPE and -28.3mm MPJPE) on natural scenes in the EgoBody dataset. In addition, MDVPose achieves robust performance on the Human3.6M datasets featuring multiple static cameras even if using a weak baseline without heavy efforts on multi-view feature fusion.Moreover, MDVPose designs a flexible strategy to train and test on an arbitrary number of dynamic views. In a complicated environment in the wild, the number of views can vary over time. In a skiing motion capture scenario, if the subject skies in long tracks, it is likely that he/she would be out of sight from some of the camera views. In a traffic IoT network, as the car-mounted cameras move, the pedestrian is probably only available in some of the views. Therefore, we train arbitrary number of dynamic views, enabling the algorithm to adapt to real-world scenarios.Our contributions are summarized as follows:• We propose a simple yet effective framework that estimates 3D human pose from multiple dynamic (possibly moving) views which does not require camera calibration, and outperforms state-of-the-art methods by a large margin. • We utilize a shared pretrained single-view motion encoder to extract multi-view features, followed by exploiting 3D training data of the novel scenario when it is available to fine-tune the novel scene poses. Further, we procrustes-align the multiple dynamic views almost for free in a unified coordinate, and impose multi-view consistency to achieve high accuracy. Experiments on multiple challenging datasets show the effectiveness of our framework in natural scenes. • We propose a flexible strategy to train and test on arbitrary number of dynamic views, enabling the algorithm to be practical and flexible in real-world scenarios. 
 
 Appendix:  


Decision: This paper addresses the problem of 3D Human pose estimation from multiple cameras with unknown calibration.
The proposed approach is novel and technically sound, it achieves competitive results, it generalizes to arbitrary views, it is well presented. 
The only shortcome is the computationally complexity that is much higher with respect to competitive approaches. However, the computational time is reasonable (also considering the limited computational resources used by the authors) and wouldn't hampers its use real applications, where it is very common to have uncalibrated cameras. 
I recommend acceptance.

Reviews: review: paper_summary: paper_summary: This paper proposes a new muti-view 3d pose estimation method that utilizes a pre-trained backbone with a new control gate design for extending to arbitrary views without camera calibrations.
strength_weakness: strength_weakness: 1.	The proposed method got SOTA performance on three benchmark datasets.
2.	The proposed method can be applied to arbitrary views.
limitations: limitations: 1. What's the relation between Pi and Gi according to Eq. 7.
2. According to Eq. 7, as N increases, Pi gradually decreases. What is the logic of this design? Later views have less influence?
3. In L752: Fig.?? L854 Capital O should be o.
4. In Table 4, when multi-view consistency loss is added, MPJPE increases in fine-tuning setting. Does this happen on other datasets? 
5. Any visualization analysis on views that not be selected? Why are they dropped by the network?
paper_summary: paper_summary: The paper proposes MDVPose, which is a framework for 3D human pose estimation from multiple dynamic cameras with unknown calibration. While the baseline models either exploits only 2D keypoint data or does not utilize pre-trained motion models, MDVPose successfully avoid these. It leverages Procrustes alignment to transform multi-view 3D poses into a unified coordinate system and exploits single-view pre-trained motion model. By doing so, MDVPose not only obtains highly accurate 3D estimation in the wild but also bypasses the camera calibration which is impractical. It outperforms all stated baseline models both qualitatively and quantitatively in 3 different datasets.
strength_weakness: strength_weakness: - Capable of leveraging existing 3D annotations and pre-trained motion model encoder of motionBERT.
- Achieves highly accurate predictions on novel scenes even with relatively smaller novel scene data.
- Capable of handling arbitrary number of viewpoints
- Extensive experimental results with SkiPose, EgoBody, and Human3.6M, comprehensively show MDVPose exceeds state-of-the-art baselines in both in-the-wild and natural settings.
- Supplementary videos present temporally consistent prediction quality.
limitations: limitations: - What is the reason for adding both spatial positional encoding and temporal positional encoding? Does it perform the best compared to other operations like concatenation?
- Since I am not an expert on this task, I will refer to other reviewers' comments for my final decision.

Minor comment:
- L810: Incorrect reference to a Figure.
paper_summary: paper_summary: The paper proposes a simple yet effective framework that estimates 3D human pose from multiple dynamic (possibly moving) views which does not require camera calibration, and outperforms state-of-the-art methods by a large margin. A shared pretrained single-view motion encoder is designed to extract multi-view features, followed by exploiting 3D training data of the novel scenario when it is available to fine-tune the novel scene poses. Further, the paper procrustes-aligns the multiple dynamic views almost for free in a unified coordinate, and imposes multi-view consistency to achieve high accuracy. Experiments on multiple challenging datasets show the effectiveness of our framework in natural scenes.
strength_weakness: strength_weakness: 1. This paper is novelty, and the DSTformer effectively enhance the model's performance by spatial-temporal details.
2. Comprehensive comparisons to competing methods.
3. Although uncalibrated cameras, intrinsic and extrinsic camera parameters are not given, the performance is still excellent. This demonstrates the effectiveness and applicability of the method.
limitations: limitations: 1. The visualization experiments are insufficient, and the existing multi-view methods are lacking in the comparative analysis.
2. There are errors in the figure citations in this paper; please check carefully.
3. The design advantages of the dual-branch network lack visual experimental validation.
4. There is a lack of analysis regarding the computational complexity of the dual-branch network.