Proactive Multi-Camera Collaboration for 3D Human Pose Estimation
Authors: Hai Ci
Mickel Liu
Xuehai Pan
fangwei zhong
Yizhou Wang
Venue: ICLR 2023 poster (2023)
Keywords: Multi-Cameras Collaboration
Multi-Agent Credit Assignment
Active Vision
Human Pose Estimation

Abstract: This paper presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration in 3D Human Pose Estimation in dynamic human crowds. Traditional fixed-viewpoint multi-camera solutions for human motion capture (MoCap) are limited in capture space and susceptible to dynamic occlusions. Active camera approaches proactively control camera poses to find optimal viewpoints for 3D reconstruction. However, current methods still face challenges with credit assignment and environment dynamics. To address these issues, our proposed method introduces a novel Collaborative Triangulation Contribution Reward (CTCR) that improves convergence and alleviates multi-agent credit assignment issues resulting from using 3D reconstruction accuracy as the shared reward. Additionally, we jointly train our model with multiple world dynamics learning tasks to better capture environment dynamics and encourage anticipatory behaviors for occlusion avoidance. We evaluate our proposed method in four photo-realistic UE4 environments to ensure validity and generalizability. Empirical results show that our method outperforms fixed and active baselines in various scenarios with different numbers of cameras and humans.

Published as a conference paper at ICLR 2023
PROACTIVE MULTI-CAMERA COLLABORATION FOR 3D HUMAN POSE ESTIMATION
Hai Ciris, Mickel Liu Xuehai Pan Fangwei Zhong Yizhou Wang
i School of Computer Science, Peking University
$ Nat'l Key Lab. of GAI & Beijing Institute for GAI (BIGAI)
School of Intelligence Science and Technology. Peking University
Inst. for AI, Peking University Nat'l Eng. Research Center of Visual Technology
(cihai, XuehaiFan, złw, yizhou. wang 8pku. www..edu.cc.mm.. en, Liickeliusstu.pke
ABSTRACT
This paper presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration 3D Human Pose Estimation dynamic human crowds. Traditional fixed-viewpoint multi-camera solutions for human motion capture (MoCap) are limited in capture space and susceptible to dynamic occlusions. Active camera approaches proactively control camera poses find optimal viewpoints for 3D reconstruction. However, current methods still face challenges with credit assignment and environment dynamics. To address these issues, our proposed method introduces a novel Collaborative Triangulation Contri- bution Reward (CTCR) that improves convergence and alleviates multi-agent credit assignment issues resulting from using 3D reconstruction accuracy as the shared reward. Additionally, we jointly train our model with multiple world dynamics learning tasks to better capture environment dynamics and encourage anticipatory behaviors for occlusion avoidance. We evaluate our proposed method in four photo-realistic UE4 environments ensure validity and generalizability. Empirical results show that our method outperforms fixed and active baselines in various scenarios with different numbers of cameras and humans.
(a) Dynamic occlusions lead failed reconstruction
(b) Constrained MoCap area
Active MoCap in the wild
Figure 1: Left:
Two critical challenges in fixed camera approaches. Right: Three active cameras collaborate to best reconstruct the 3D pose of the target (marked in V).
INTRODUCTION
Marker-less motion capture (MoCap) has broad applications in many areas such as cinematography, medical research, virtual reality (VR), sports, and etc. Their successes be partly attributed to recent developments in 3D Human pose estimation (HPE) techniques (Tu et al., 2020; Iskakov al., 2019; Jafarian et al., 2019; Pavlakos al., 2017b; Lin & Lee, 2021b). A straightforward implementation to solve multi-views 3D HPE is to use fixed cameras. Although being a convenient solution, less effective against dynamic occlusions. Moreover, fixed camera solutions confine tracking targets within a constrained space, therefore less applicable to outdoor MoCap. On the contrary, active cameras (Luo et al., 2018; 2019; Zhong et al., 2018a; 2019) such as ones mounted on drones can maneuver proactively against incoming occlusions. Owing to its remarkable flexibility, the active approach has thus attracted overwhelming interest (Tallamraju et al., 2020; Ho et al., 2021; Xu et al., 2017; Kiciroglu et al.. 2019; Saini et al., 2022: Cheng et al., 2018; Zhang et al., 2021).
"Equal Contribution. "CCoresponding author. Project Website: httpt:://////. //s googl e com/vi ew/ httpacctveeddeee. ictivedeesse

---

Published as a conference paper at ICLR 2023
Previous works have demonstrated the effectiveness of using active cameras for 3D HPE on single target in indoor (Kiciroglu et al., 2019; Cheng et al.. 2018), clean landscapes (Tallamraju et al., 2020; Nägeli et al., 2018; Zhou et al., 2018: 8: Saini et al., 2022) or landscapes with scattered static obstacles (Ho et al., 2021). However, to the best of our knowledge, we have not seen any existing work that experimented with multiple (n > 3) active cameras conduct 3D HPE in human crowd. There are two key challenges: First, frequent human-to-human interactions lead to random dynamic occlusions. Unlike previous works that only consider clean landscapes or static obstacles, dynamic scenes require frequent adjustments of cameras' viewpoints for occlusion avoidance while keeping a good overall team formation to ensure accurate multi-view reconstruction. Therefore, achieving hand-crafted control addition, the complex behavioural pattern of human crowd makes the occlusion patterns less comprehensible and predictable, further increasing the difficulty in control. Second, as the team size grows larger, the multi-agent credit assignment issue becomes prominent which hinders policy learning of the camera agents. Concretely, multi-view 3D HPE as team effort requires inputs from multiple cameras to generate an accurate reconstruction. Having more camera agents participate in reconstruction certainly introduces more redundancy, which reduces the susceptibility to reconstruction failure caused by dynamic occlusions. However, it consequently weakens the association between individual performance and the reconstruction accuracy of the team, which leads to the "lazy agent" problem (Sunehag et al., 2017).
optimality in dynamic scenes by implementing fixed camera formation policy challenging.
In this work, we introduce a proactive multi-camera collaboration framework based on multi-agent reinforcement learning (MARL) for real-time distributive adjustments of multi-camera formation for 3D HPE in a human crowd. In our approach, multiple camera agents perform seamless collaboration for successful reconstructions of 3D human poses. Additionally, it is a decentralized framework that offers flexibility over the formation size and climinates dependency on a control hicrarchy or a centralized entity. Regarding the first challenge, we argue that the model's ability to predict human movements and environmental changes is crucial. Thus, we incorporate World Dynamics Learning (WDL) to train a state representation with these properties, i.e., learning with five auxiliary tasks to predict the target's 's position, pedestrians' positions, self state, teammates' states, and team reward.
To tackle the second challenge, we further introduce the Collaborative Triangulation Contribution Reward (CTCR), which incentivizes each agent according to its characteristic contribution 3D reconstruction. Inspired by the Shapley Value (Rapoport, 1970), CTCR computes the average weighted marginal contribution to the 3D reconstruction for any given agent over all possible coalitions that contain it. This reward aims to directly associate agents' levels of participation with their adjusted return, guiding their policy learning when the team reward alone is insufficient to produce such direct association. Moreover, CTCR penalizes occluded camera agents more efficiently than the shared reward, encouraging emergent occlusion avoidance behaviors. Empirical results show that CTCR can accelerate convergence and increase reconstruction accuracy. Furthermore, CTCR is a general approach that can benefit policy learning in active 3D HPE and serve as new assessment metric for view selection in other multi-view reconstruction tasks.
For the evaluations of the learned policies, we build photo-realistic environments (UnrealPose) using Unreal Engine 4 (UB4) and UnrealCV (Qiu et al., 2017). These environments can simulate realistic- realistic-beaavngg crowds with assurances of high fidelity and customizability. We train the agents on Blank environment and validate their policies three unseen scenarios with different landscapes, levels of illumination, human appearances, and various quantities of cameras and humans. The empirical results show that our method can achieve more accurate and stable 3D pose estimates than off-the-shelf passive- and active-camera baselines. To help facilitate more fruitful research on this topic, we release our environments with OpenAI Gym-API (Brockman et al., 2016) integration and together with a dedicated visualization tool.
Here we summarize the key contributions of our work:
Formulating the active multi-camera 3D human pose estimation problem as a Dec-POMDP and proposing a novel multi-camera collaboration framework based on MARL (with n 2 3). Introducing five auxiliary tasks to enhance the model's ability to learn the dynamics of highly dynamic scenes.
Proposing CTCR to address the credit assignment problem in MARL and demonstrating notable improvements in reconstruction accuracy compared to both passive and active baselines. Contributing high-fidelity environments for simulating realistic-looking human crowds with au- thentic behaviors, along with visualization software for frame-by by-fame video analysis.
2

---

Published as a conference paper at ICLR 2023
2 RELATED WORK
3D Human Pose Estimation (HPE) Recent research on 3D human pose estimation has shown significant progress in recovering poses from single monocular images (Ma et al., 2021; Pavlakos et al., 2017a; Martinez et al., 2017: Kanazawa et al., 2018; Pavlakos et al., 2018: Sun et al., 2018; Ci et al., 2019; Zeng et al., 2020; Ci et al., 2020) or monocular video (Mehta et al., 2017; Hossain & Little, 2018; Pavllo et al., 2019; Kocabas et al., 2020). Other approaches utilize multi-camera systems for triangulation to improve visibiliiyy and eliminate ambiguity (Qiu et al., 2019; Jafarian ct al., 2019; Pavlakos ct al., 2017b; Dong et al.. 2019; Lin & Lee, 2021b; Tu et al., 2020; Iskakov et al., 2019). However, these methods are often limited to indoor laboratory environments with fixed cameras. In contrast, our work proposes an active camera system with multiple mobile cameras for outdoor scenes, providing greater flexibility and adaptability.
Proactive Motion Capture Few previous works have studied proactive motion capture with a single mobile camera (Zhou et al., 2018; Cheng et al., 2018; Kiciroglu et al., 2019). In comparison, more works have studied the control of multi-camera team. Among them, many are based on optimization with various system designs, including marker-based (Nägeli al., 2018), RGBDD-based (Xu et al., 2017), two-stage system (Saini et al., 2019; Tallamraju et al., 2019), hierarchical system (Ho et al., 2021), etc. It is important note that all the above methods deal with static occlusion sources or clean landscapes. Additionally, the majority of these works adopt hand-crafie optimization objectives and some forms of fixed camera formations. These factors result in poor adaptability to dynamic scenes that are saturated with uncertainties. Recently, RL-based methods have received more attention due to their potential for dynamic formation adjustments. These works have studied active 3D HPE the Gazebo simulation (Tallamraju et al., 2020) or Panoptic dome (Joo et al., 2015; Pirinen et al., 2019; Gärtner et al., 2020) for active view selection. Among them, AirCapRL (Tallamraju et al., 2020) shares similarities with our work. However, it is restricted to coordinating between two cameras in clean landscapes without occlusions. We study collaborations between multiple cameras (n 2 3) and resolve the credit assignment issue with novel reward design (CTCR). Meanwhile, study more challenging scenario with multiple distracting humans serving as sources of dynamic occlusions, which requires more sophisticated algorithms to handle.
Multi-Camera Collaboration Many works in computer vision have studied multi-camera collabora-
tion and designed active camera systems accordingly. Earlier works (Collins al., 2003; Qureshi & Terzopoulos, 2007; Matsuyama et al., 2012) focused on developing network of pan-tile-zoom (PTZ) cameras. Owing to recent advances in MARL algorithms (Lowe et al., 2017: Sunehag al., 2017; Rashid et al., 2018; Wang et al., 2020; Yu et al., 2021; Jin et al., 2022), many works have formulated multi-camera collaboration as a multi-agent learning problem and solved it using MARL algorithms accordingly (Li et al., 2020; Xu et al., 2020; Wenhong et al., 2022; Fang et al., 2022: Sharma et al., 2022; Pan et al., 2022). However, most works focus on the target tracking problem. whereas this work attempts to solve the task of 3D HPE. Compared with the tracking task, 3D HPE has stricter criteria for optimal view selections due to correlations across multiple views, which necessitates intelligent collaboration between cameras. To our best knowledge, this work is the first to experiment with various camera agents (n 2 3) to learn multi-camera collaboration strategies for active 3D) HPE.
PROACTIVE
COLLABORATION
This section will explain the formulation of multi-camera collaboration in 3D HPE as a Dec-POMDP. Then, we will describe our proposed solutions for modelling the virtual environment's complex dynamics and strengthening credit assignment in the multi-camera collaboration task.
3.1 PROBLEM FORMULATION
We formulate the multi-camers 3D HPE problem as a Decentralized Partially-Observaa Markov Decision Process (Dec-POMDP), where each camera is considered as an agent which decentralized- controlled and has partial observability over the environment. Formally, a Dec-POMDP is defined as (8, O, A" n, P,r,7), where S denotes the global state space of the environment, including all human states and camera states in our problem. O; O denotes the agent i's local observation, i.e., the RGB image observed by camera i. AA denotes the action space of an agent and AA" represents the joint action space of all agents. P. S x A" + S is the transition probability function P(s++1|s ss a'), ), in which a' E A is a joint action by all n agents. At each timestep t, every agent obtains local view of from the environment s and then preprocess of to form i-th agent's 's local
3

---

Published as a conference paper at ICLR 2023
Next Environment Step
C Cameea Agent An
Agent
Camera Agent #1
Target Prodiction
UE4 Environment
a
S tate Encoder
a2
Observaton Process
MLP
Actor
aj
Envrrmment Consentss
images
r~Agents Decentralized Execution
Figure 2: Simplified Pipeline Architecture. For i agent at t step, the environment returns local camera view of and environment constants that are pre-processed to generate local observation o (described here). Target Prediction and Pedestrian Prediction modules generate predictions Pgg and jf respectively. The parameters of the Mixture Density (MIW)rk(MIDN) within the Target Prediction Module concatenated to the state encoder's output to improve the quality of the representation.
observation ot. The agent performs action of ~ (-1ot) and receives its reward r(s", a²). (0, is the discount factor used to calculate the cumulative discounted reward G(t) Σ 6' 7 r(t').
In a cooperative team, the objective is to learn a group of decentralized policies { (a) that denote as the agent index, [r|]
maximizes (=, [G(t]]. the set of all n agents, and -i [<] {i) are all agents except agent i.
[G([)] For convenience,
n is
Observation Camera agents have partial observability over the environment. The pre- rre-processed observation o; = (pi, E- of the camera agent i consists of: (1) pi, set of states of visible humans to agent i, containing information, including the detected human bounding-boo in the 2D local view, the 3D positions and orientations of all visible humans measured both local coordinate frame of camera i and world coordinates; (2) own camera pose & showing the camera's 's position and orientation in world coordinates; (3) peer cameras poses Ç-i showing their positions and orientations in world coordinates and are obtained via multi-agent communication.
Action Space The action space of each camera agent consists of the velocity of 3D egocentric
translation (xr, 2) and the velocity of 2D pitch-yaw rotation (0, v)) To reduce the exploration space for state-action mapping, the agent's action space is discretized into three levels across all five dimensions. At each timestep, the camera agent can move its position by [+8, 0, -5] in (xr, y, directions and rotate about pitch-yaw axes by [+7, O, -nj degrees. In our experiments, the camera's 's pitch-yaw angles are controlled by rule-based system.
3.2 FRAMEWORK
This section will describe the technical framework that constitutes our camera agents, which contains a Perception Module and a Controller Module. The Perception Module maps the original RGB images taken by the camera to numerical observations. The Controller Module takes these numerical observations and produces corresponding control signals. Fig. 2 illustrates this framework.
Perception Module The perception module executes a procedure consisting of four sequential stages: (1) 2D HPE. The agent performs 2D human detection and pose estimation the observed RGB image with the YOLOv3 (Redmon & Farhadi, 2018) detector and the HRNet-w32 (Sun et al., 2019) pose estimator, respectively. Both models are pre-trained on the COCO dataset, (Lin al., 2014) and their parameters are kept frozen during policy learning of camera agents to ensure cross- scene generalization. (2) Person RelD. RelD model (Zhong et al., 201 is used to distinguish people in a scene. For simplicity, an appearance dictionary of all to webbeapppereed people is built in advance following (Gärtner al., 2020). test time, the ReID network computes features for all detected people and identifies different people by comparing features to the pre-built appearance dictionary. (3) Multi-agent Communication. Detected 2D human pose, IDs, and own camera pose are broadcasted other agents. (4) 3D HPE. 3D human pose is reconstructed via local triangulation after receiving communications from other agents. The estimated position and orientation of a person can then be extracted from the corresponding reconstructed human pose. The communication process is illustrated in Appendix Fig. 9.
4

---

Published as a conference paper at ICLR 2023
2:0!
111
1111
9x(1) 31
(Tag -Tps) +
30 Roid Rasg
(123)
210
1111
1111
9x(2)
(1223) -P(1)) Tria
5 (121
210
1111
1111
9+G)
Iovalid subses for trinngalation:
5- [23)
Figure 3: An instance of computing Collaborative Triangulation Contribution Reward (CTCR) for every camera. Left: Three cameras tasked to perform HPE on the target person avoiding occlusions from two pedestrians. Middle: Three cameras can constitute four valid coalitions, each generating different 3D HPE. Right: Compute CTCR (without re-scaling by n) for each camera.
Controller Module The controller module consists of a state encoder E' and an actor network A. The state encoder E' takes of input, encoding the temporal dynamics of the environment via LSTM. The future states of the target, pedestrians, and cameras are modelled using Mixture Density Network (MDN) (Bishop, 1994) account for uncertainty. During model inference, it computes target position prediction, and then the (é, μ4, o) parameters of target prediction MDN are used as part of the inputs to the actor network to enhance feature encoding. Please refer to Section 3.4 for more details regarding training the MDN.
Feature Embedding
2f MLP(O
(1)
Temporal Modeling
h LSTM( b/
(2)
Human Trajectory Prediction
s++1 = MDN(22) h Pigupa):
(3)
Final Embedding e E (ot, (,,,, Concat(=), h4 {(o, n,, o) JMDN
(4)
where Pigr and Ppd refer to the state of the target and the pedestrian, respectively, The actor network A consists of 2 fully-connected layers that output the action, aj = A(E h, h(-1)).
3.3 REWARD STRUCTURE
To alleviate the credit assignment issue that arises in multi-camera collaboration, propose the Collaborative Triangulation Contribution Reward (CTCR). We start by defining a base reward that reflects the reconstruction accuracy of the triangulated pose generated by the camera team. Then explain how our CTCR is computed based on this base team reward.
Reconstruction Accuracy as a Team Reward To directly reflect the reconstruction accuracy, the reward function negatively correlates with the pose estimation error (Mean Per Joint Position Error, MPJPE) of the multi-camera triangulation. Formally,
0,
X < 1, X 2 2.
r (X)
Gemen (MPJPE(X)),
Where the set X represents the participating cameras in triangulation, and employs the Geman- McClure smoothing function, Gemen(-) = (7-)=41- to stabilize policy updates, where c = 50mm in our experiments. However, the shared team reward structure in our MAPPO baseline, where each camera in the entire camera team receives a common reward r(X), presents a credit assignment challenge, especially when camera is occluded, resulting in reduced reward for all cameras. To address this issue, we propose new approach called Collaborative Triangulation Contribution Reward (CTCR).
2(
Collaborative Triangulation Contribution Reward (CTCR) CTCR computes each agent's individual reward based on its marginal contribution to the collaborative multi-view triangulation. Refer to Fig. 3 for rundown of computing CTCR for a 3-cameras team. The contribution of agent i can be measured by:
[S |S| 1)!
CTCR(i) n (i),
pr(i)=
Σ
(r)) -r(S), (6)
SCI
5

---

Published as a conference paper at ICLR 2023
Where denotes the total number of agents. S denotes all the subsets of [n]] not containing agent | the normalization term. [r(S U {i}) r(S) means the marginal contribution of agent i. Note that [ig (Px(1) r[[=]). We additionally multiply a constant n to rescale the CTCR to have the same scale as the team reward. Especially in the 2-cameras case, the individual
CTCR should be equivalent to the team reward, i.e., CTCR(i = 1) = CTCR(i = 2) = r((1,2))
For more explanations CTCR, please refer to Appendix Section G.
3.4 LEARNING MULTI-CAMERA COLLABORATION VIA MARL
We employ the multi-agent learning variant of PPO (Schulman al., 2017) called Multi-Agent PPO (MAPPO) (Yu et al., 2021) to learn the collaboration strategy. Alongside the RI loss, we jointly train the model with five auxiliary tasks that encourage comprehension of the world dynamics and the stochasticity in human behaviours. The pseudocode can be found in Appendix A.
World Dynamics Learning (WDL) We use the encoder's hidden states (zf, ht) as the basis to model the world. Three WDL objectives correspond to modelling agent dynamics (1) learning the forward dynamics of the camera Pi (5, (c!+2 |2t, hi, af). (2) prediction of team reward P2(r1 h4, hi,af, af). (3) prediction of future position of peer agents Pa |z], hf. a²). Two WDL objectives correspond to modelling human dynamics: (4) prediction of future position of target person (p |2 h4 Pig.).
(5) prediction of future position pedestrians, (p above are approximated using Mixture Density Networks (MDNs) (Bishop, 1994).
( |2f, M3 P(a). All the probability functions
Total Training Objectives Crain = LRL + AWDL LWDL- The LRL is the reinforcement learning loss consisting of Cpppppe loss and centralized-critic network loss similar MAPPO (Yu et al., 2021). LWDL Σ, de E, Ellog (- h4, a[]] is the world dynamics learning loss that consists of MDN supervised losses on the five prediction tasks mentioned above.
4 EXPERIMENT
In this section, first introduce novel environment, UNREALPOSE, used for training and testing the learned policies. Then we compare our method with multi-passive-caeet baselines and perform ablation study the effectiveness of the proposed CTCR and WDL objectives. Additionally, evaluate the effectiveness of the learned policies by comparing them against other active multi-camera methods. Lastly,
our method in four different scenarios showcase its robustness.
4.1 UNREALPOSE: A VIRTUAL ENVIRONMENT FOR PROACTIVE HUMAN POSE ESTIMATION
We built four virtual environments for simulating active HPE the wild using Unreal Engine (UE4), which is a powerful 3D game engine that can provide real-time and photo-realistic renderings for making visually-stunning video games. The environments handle the interactions between realistic- behaving human crowds and camera agents. Here is a list of characteristics of UNREALPOSS that we would like to highlight: Realistic: Diverse generations of human trajectories, built-in collision avoidance, and several scenarios with different human appearance, terrain, and level of illumination. Flexibility: extensive configuration in numbers of humans, cameras, or their physical properties, more than 100 MoCap action sequences incorporated. RL-Ready: integrated with OpenAI Gym API, overhaul the communication module in the UnrealCV (Qiu et al., 2017) 7) plugin with inter-process communication (IPC) mechanism. For more detailed deseriptions, please refer to Appendix Section B.
(a) Blank
(b) School Gym
(c) Urban Street
(d) Wildemesss
Figure 4: UNREALPPSE: Our virtual environments for active 3D HPE in human crowd

---

Published as a conference paper at ICLR 2023
MPJPE vs. Number of Cameras
MPJPE vs. Number of Humans
Nixed Cameras
120
Fized Camens (RMEAAC)
Fieed Cameras PMasdS
MAPPO CTCR
NAPE immo
WOL
popps (mm)
150
60
40
Hummns
Success Rate vs. Number of Cameras
Success Rate vs. Number Humann
1.00
1.00
0.95
Success Rr (oweer=20000 mm0
Ruze m300mm)
0.30
0.95
0.85
0.94
0.50
0.92
0.75
Fieed Cemeras
Fixed Cameras (RANSAC)
8.00
0.70
Nitd Camerax (Paandsoeptaae)
MAPPO
0.88
0.85
MAPPO CTCR
HAPP CTCR VOL
0.86
0.60
Number Camoras
Number Humons
Figure 5: Comparative evaluation of mean per-joint position error (MPJPE) and success rate (T = 200mm) between the baseline methods and the proposed approaches. Left column: av- erage performance w.r.t. the number of camera agents employed in environments containing one to six humans. Right column: average performance w.r.t. the number of humans in the environment, where three to five cameras are utilized.
4.2 EVALUATION METRICS
We use Mean Per Joint Position Error (MPJPE) as our primary evaluation metric, which measures the difference between the ground truth and the reconstructed 3D pose on a per-frame basis. However, using MPJPE alone may not provide a complete understanding of the robustness of a multi calli-c collaboration policy for two reasons: Firstly, cameras adjusting their perception quality may take multiple frames to complete, and secondly, high peaks in MPJPE may be missed by the mean aggregation. To address this, introduce the "success rate" metric, which evaluates the smooth execution and robustness of the learned policies. Success rate is calculated as the ratio of frames in an episode with MPJPE lower than т. Formally, RateessRate(?) is a temporal measure that reflects the integrity of multi-view coordination. Poor coordination may cause partial occlusions or too many overlapping perceptions, leading to a significant increase in MPJPE and a subsequent decrease in the success rate.
P(MPJPE r). This metric
4.3 RESULTS AND ANALYSIS
The learning-bast based control policies were trained in a total of 28 instances of the BlankEnv, where each instance uniformly contained I to humans. Each training run consisted of 700,000 steps, which corresponds to 1,000 training iterations. To ensure a fair evaluation, we report the mean metrics based the data from the latest 100 episodes, each comprising 500 steps. The experiments were conducted in a 10m x 10m area, where the cameras and humans interacted with each other.
Active vs. Passive To show the necessity of proactive camera control, we compare the active camera control methods with three passive methods, i.e., Fixed Camera, Fixed Camera (RANSAC), and Fixed Camera (PlaneSweepPPose.. "Fixed Cameras" denotes that the poses of the cameras are fixed, hanging 3m above ground and -35" camera pitch angles. The placements of these fixed cameras are carefully determined with strong priors, e.g., right-angle, triangle, square, and pentagon formations for 2, 3, 4, and 5 cameras, respectively. "RANSAC" denotes the method that uses RANSAC (Fischler & Bolles, 1981) for enhanced triangulation. "PlaneSweepPoss represents the off shel learning- based method (Lin & Lee, 2021b) for multi-view 3D HPE. Please refer to Appendix F.2 for more implementation details. We show the MPJPE and Success Rate versus
7

---

Published as a conference paper at ICLR 2023
Carreran
Cameraa
Cameras
275
250
225
125
100
SchoolCym
Schooltym
Bym
Figure 6: Evaluation results for testing our multi-camera collaboration policy (three to five cameras) in four different environments that have six walking humans (including the target human).
different number of cameras in Fig. 5. We observe that all passive baselines are being outperformed by the active approaches due to their inability to adjust camera views against dynamic occlusions. The improvement of active approaches is especially significant when the number of cameras is less, iec when the camera system has little or no redundancy against occlusions. Notably, the MPJPE attained by our 3-camera policy is even lower than the MPJPE from 5 Fixed cameras. This suggests that proactive strategies can help reduce the number of cameras necessary for deployment.
The Effectiveness CTCR and WDL We also perform ablation studies on the two proposed modules (CTCR and WDL) to analyze their benefits performance. We take "MAPPO" as the active-camera baseline for comparison, which is our method but trained instead by a shared global reconstruction reward and without world dynamics modelling. Fig. 5 shows consistent performance gap between the "MAPPO" baseline and our methods (MAPPO + CTCR + WDL). The proposed CTCR mitigates the credit assignment issue by computing the weighted marginal contribution of each camera. Also, CTCR promotes faster convergence. Training curves are shown in Appendix Fig. 13. Training with WDL objectives further improves the MPJPE metric for our 2-Camera model. However, its supporting effect gradually weakens with the increasing number of cameras. We argue that is caused by the more complex dynamics involved with more cameras simultaneously interacting the same environment. Notably, we observe that the agents trained with WDL are of better generalization unseen scenes, as shown Fig.
Versus Other Active Methods To show the effectiveness of the learned policies, we further com- pare our method with other active multi-camera formation control methods 3-cameras BlankEnv. "MAPPO" (Yu et al., 2021) and AirCapRL (Tallamraju et al., 2020) are two learning-based based meth- ods based on PPO (Schulman et al., 2017). The main difference between these two methods is the reward shaping technique, i.e., AirCapRL additionally employs multiple carefully-designee rewards (Tallamraju et al., 2020) for learning. We also programmed a rule-based fixed formation control method (keeping equilateral triangle, spread by 120") to track the target person. Results are shown in Table 1. Interestingly, these three baselines achieve comparable performance. Our method outperforms them, indicating a more effective multi-camera collaboration strategy for 3D HPE. For example, our method learns a spatially spread-out formation while automatically adjusting avoid impending occlusion.
Generalize to Various Scenarios We train the control policies in BlankEnv while testing them three other realistic environments (SchoolGym, UrbanStreet, and Wilderness) to evaluate their generalizability to unseen scenarios. Fig. 6 shows that our method consistently outperforms baseline
Table 1: A comparison with other active control methods on the control of three cameras. We also report fixed camera results as reference. MPJPE is averaged over six environment instances, each with different number of humans (1 to 6, respectively).
Fixed
AirCapRL (Reproduced)
Rule-based Formation
MAPPO
Ours
Cameras
MPJPE (mm) J.
122.4
71.6
70.2
70.0
56.7
8

---

Published as a conference paper at ICLR 2023
(a) Cameras
(b) 4 Cameras
(c) 5 Cameras
Figure 7: Visualizations for the emergent formations in controlling three to five cameras. The target person is colored in blue and others are colored in pink. The camera agents learn spread out and avoid occlusions while maintaining good reconstruction accuracy without a centralized controller.
methods with lower variance in MPJPE during the evaluations in three test environments. We report the results in the BlankEnv reference.
Qualitative Analysis In Figure 7, we show six examples of the emergent formations of cameras under the trained policies using the proposed methods (CTCR + WDL). The camera agents learn spread and ascent above humans avoid occlusions and collision. Their placements in emergent formation are not assigned by other entities but rather determined by the decentralized control policies themselves based on local observations and agent-to-agent communication. For more vivid examples of emergent formations, please refer to the project website for the demo videos. For more analysis on the behaviour mode our 3-, and 5-camera models, please refer Appendix Section H.
5 CONCLUSION AND DISCUSSION
To our knowledge, this paper presents the first proactive multi-camera system targeting the 3D reconstruction a dynamic crowd. It also the first study regarding proactive 3D HPE that promptly experimented with multi-camera collaborations at different scales and scenarios. We propose CTCR to alleviate the multi-agent credit assignment issue when the camera team scales up. We identified multiple auxiliary tasks that improve the representation learning of complex dynamics. As final note, we release our virtual environments and the visualization tool to facilitate future research.
Limitations and Future Directions Admittedly, couple of aspects of this work have room for improvement. Firstly, the camera agents received the positions for their teammates via non-disrupting broadcasts of information, which may be prone packet losses during deployment. One idea is to incorporate a specialized protocol for multi-agent communication into our pipeline, such as ToM2C(Wang et al., 2022). Secondly, intended initially reduce the communication bandwidth (for example, to eliminate the need for image transmission between cameras), the current pipeline comprises Human Re-Identification module requiring a pre-scan memory on all co-be-appeared human subjects. For some out-of disttriibbuiiinn humans' appearances, the current RelD module may not be able to recognize. Though ACTIVE3DPOSE can accommodate a more sophisticated RelD module (Deng et al., 2018) to resolve this shortcoming. Thirdly, the camera control policy requires accurate camera pose, which may need to develop a robust SLAM system (Schmuck & Chli, 2017: Zhong et al., 2018b) to work in dynamic environments with multiple cameras. Fourthly, the motion patterns of targets in virtual environments are based on manually designed animations, which will lead to the poor generalization of the agents unseen motion patterns. In the future, multi-agent game (Zhong et al., 2021) 1) in
enrich
the diversity by incorporating a pnoperative-competiinn training. Lastly, assume near-perfect calibrations for group of mobile cameras, which might be complicated to sustain in practice. Fortunately, we are seeing rising interest in parameter-free pose estimation (Gordon et al., 2022: Ci et al., 2022), which does not require online camera calibration and may help to resolve this limitation.
Project Website for demo videos: https: / tes. .oogl com/vr v ew/ act veddpose

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
Our research into active 3D HPE technologies has the potential to bring many benefits, such as biomechanical analysis in sports and automated video-assisted coaching (AVAC). However, we recognize that these technologies can also be misused for repressive surveillance, leading to privacy infringements and human rights violations. We firmly condemn such malicious acts and advocate for the fair and responsible use of our virtual environment, UNREALPOSE, and all other 3D HPE technologies.
ACKNOWLEDGEMENT
The authors would like to thank Yuanfei Wang for discussions on world models; Tingyun Yan for his technical support on the first protolype of UNREALPOSE. This research was supported by MOST-2022ZD01 14900, NSFC-6206 136001, China National Post-doctoral Program for Innovative Talents (Grant No. BX2021008) and Qualcomm University Research Grant.
REFERENCES
Epic games incorporated. unreal engine 4. fttEps://www.neee http://wwwwwreaaeniiiiiaa com.
Christopher M. Bishop. Mixture density networks. Workingpaper, Aston University, 1994.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint arXiv: 01540, 2016.
Géry Casiez, Nicolas Roussel, and Daniel Vogel. I filter: a simple speed- based low-pass filter for noisy input in interactive systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2527-2530. 2012.
Wei Cheng, Lan Xu, Lei Han, Yuanfang Guo, and Lu Fang. ihuman3d: Intelligent human body 3d reconstruction using single flying camera. Proceedings of the 26th ACM International Conference on Multimedia, pp. 1733-1741, 2018.
Hai Ci, Chunyu Wang, Xiaoxuan Ma, and Yizhou Wang. Optimizing network structure for 3d human pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2271, 2019.
Hai Ci, Xiaoxuan Ma, Chunyu Wang, and Yizhou Wang. Locally connected network for monocular 3d human pose estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 2020.
Hai Ci, Mingdong Wu, Wentao Zhu, Xiaoxuan Ma, Hao Dong, Fangwei Zhong, and Yizhou Wang. Gfpose: Learning 3d human pose prior with gradient fields. arXiv preprint arXiv:2212.086/4 2022.
Robert T Collins, Omead Amidi, and Takeo Kanade. Acquiring multi-view video with an active camera system. In Multisensor Surveillance Systems, pp. 135-147. 2003.
Weijian Deng, Liang Zheng, Qixiang Ye, Guoliang Kang, Yang, and Jianbin Jiao. Image- image domain adaptation with preserved self self-aiiil and domain-dissimilargy for person re- identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 994-1003, 2018
Junting Dong, Wen Jiang, Qixing Huang, Hujun Bao, and Xiaowei Zhou. Fast and robust multi person 3d pose estimation from multiple views. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 7792-7801, 2019.
Zeyu Fang, Jian Zhao, Mingyu Yang, Wengang Zhou. Zhenbo Lu, and Houqiang Li. Coordinate-aligned multi-camera collaboration for active multi multi-object tracking. arXiv preprint arXiv:2202. 10881, 2022.
10

---

Published as a conference paper at ICLR 2023
Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24 (6):381-395, 1981.
Erik Gärtner, Aleksis Pirinen, and Cristian Sminchisescu. Deep reinforcement learning for active hu- man pose estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 10835-10844, 2020.
Brian Gordon, Sigal Raab, Guy Azov, Raja Giryes, and Daniel Cohen-Or. Flex: Extrinsic parameters- free multi-view 3d human motion reconstruction. In European Conference on Computer Vision (ECCV), pp. 176-196. Springer, 2022.
Cherie Ho, Andrew Jong, Harry Freeman, Rohan Rao, Rogerio Bonatti, and Sebastian Scherer. 3d human reconstruction in the wild with collaborative aerial cameras. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5263-5269, 2021.
Mir Rayat Imtiaz Hossain and James J Little. Exploiting temporal information for 3d human pose estimation. Proceedings of the European Conference on Computer Vision (ECCV), pp. 68-84, 2018.
Karim Iskakov, Egor Burkov, Victor Lempitsky, and Yury Malkov, Learnable triangulation of human pose. In Proceedings of the TEEF/CVF International Conference on Computer Vision, pp. 7718-7727,2011
Yasamin Jafarian, Yuan Yao, and Hyun Soo Park. Monet: Multiview semi-supervised keypoint via epipolar divergence. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 753-762, 2019.
Kefan Jin, Jian Wang. Hongdong Wang. Xiaofeng Liang, Yongjin Guo, Mianjin Wang, and Hong Yi. Soft formation control for unmanned surface vehicles under environmental disturbance using multi-task reinforcement learning. Ocean Engineering, 260:1 12035, 09 2022. doi: 10.1016/j. oceaneng.2022 12035.
Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, lain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: massively multiview system for social motion capture. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1334-3342, 2015.
Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7122-7131, 2018.
Sena Kiciroglu, Helge Rhodin, Sudipta N Sinha, Mathieu Salzmann, and Pascal Fua. Activemo- cap: optimized drone flight for active human motion capture. Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 16-20, 2019.
Muhammed Kocabas, Nikos Athanasiou, and Michael J Black. Vibe: Video inference for human body pose and shape estimation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp.
2020.
Jing Li, Jing Xu, Fangwei Zhong, Xiangyu Kong, Yu Qiao, and Yizhou Wang. Pose-assisted multi- camera collaboration for active object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 759-766. 2020.
Jiahao Lin and Gim Hee Lee. Multi-view multi-person 3d pose estimation with plane sweep stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11886-18895 June 2021a.
Jiahao Lin and Gim Hee Lee. Multi-view multi-person 3d pose estimation with plane sweep stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1 11886-1 11 1895, 2021b.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference Computer Vision, pp. 740-755, 2014.

---

Published as a conference paper at ICLR 2023
Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor- critic for mixed peperrative-oopettiiii environments. Neural Information Processing Systems, 30, 2017.
Wenhan Luo, Peng Sun, Fangwei Zhong, Wei Liu, Tong Zhang, and Yizhou Wang. End-to-end active object tracking via reinforcement learning. In International Conference on Machine Learning, pp. 3286-3295, 2018.
Wenhan Luo, Peng Sun, Fangwei Zhong, Wei Liu, Tong Zhang, and Yizhou Wang. End-to-end active object tracking and its real-world deployment via reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(6):1317-1332, 2019.
Xiaoxuan Ma, Jiajun Su, Chunyu Wang, Hai Ci, and Yizhou Wang. Context modeling in 3d human pose estimation: unified perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 6238-6247, 2021.
Julieta Martinez, Rayat Hossain, Javier Romero, and James Little. A simple yet effective baseline for 3d human pose estimation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2640-2649, 2017.
Takashi Matsuyama, Shohei Nobuhara, Takeshi Takai, and Tony Tung. Active camera system for object tracking and multi-view observation. In 3D Video and Its Applications, pp. 45-85. Springer, 2012.
Dushyant Mchta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans- Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. Vnect: Real-time 3d human pose estimation with a single rgb camera. ACM Transactions on Graphics (TOG), 36(4):1-14 2017.
Tobias Nägeli, Samuel Oberholzer, Silvan Plüss, Javier Alonso-Mora, and Otmar Hilliges. Flycon: real-time wwvironment-independeee multi-view human pose estimation with aerial vehicles. ACM Transactions on Graphics (TOG), 37(6):1-14, 2018.
Xuehai Pan, Mickel Liu, Fangwei Zhong, Yaodong Yang, Song- Zhu, and Yizhou Wang. MATE: Benchmarking multi-agent reinforcement learning in distributed target coverage control. In Thirty- sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL https:tt///ccc..... /pe.revenneeeeeeeeeeeeeeeeeeeeee n Forum? www.mmmisssyyyyyyooooooo
Georgios Pavlakos, Xiaowei Zhou, Konstantinos G Derpanis, and Kostas Danilidis. Coarse-to-fine volumetric prediction for single-image 3d human pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7025-7034, 2017a
Georgios Pavlakos, Xiaowei Zhou, Konstantinos G Derpanis, and Kostas Daniilidis. Harvesting multiple views for marker-less 3d human pose annotations. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 6988-6997 2017b.
Georgios Pavlakos, Luyang Zhu, Xiaowei Zhou, and Kostas Daniilidis. Learning to estimate 3d human pose and shape from a single color image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 459,468, 2018.
Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estimation in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7753-7762. 2019.
Alcksis Pirinen, Erik Gärtner, and Cristian Sminchisesou. Domes to drones: Self-supervised active triangulation for 3d human pose reconstruction. Advances in Neural Information Processing Systems, 32, 2019.
Haibo Qiu, Chunyu Wang, Jingdong Wang, Naiyan Wang. and Wenjun Zeng, Cross view fusion for 3d human pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4342 4351, 2019.
Weichao Qiu, Fangwei Zhong, Yi Zhang, Siyuan Qiao, Zihao Xiao, Tae Soo Kim, and Yizhou Wang. Unrealev: Virtual worlds for computer vision. In Proceedings of the 25th ACM International Conference Multimedia, pp. 1221-1224, 2017.
12

---

Published as a conference paper at ICLR 2023
Faisal Qureshi and Demetri Terzopoulos. Smart camera networks in virtual reality. In 2007 First ACM/IEEE International Conference on Distributed Smart Cameras, pp. 87-94, 2007. doi: M.L1109/CDDSC2220.43555111
Anatol Rapoport. N-person game theory concepts and applications Anatol Rapoport. University of Michigan Press Ann Arbor, 1970. ISBN 0472050176.
Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shi- mon Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference Machine Learning, pp. 4295-4304, 2018.
Joseph Redmon and Ali Farhadi. Yolov3: arXiv:1804.02767, 2018.
An incremental improvement. arXiv preprint
Nitin Saini, Eric Price, Rahul Tallamraju, Rafli Enficiaud. Roman Ludwig, Igor Martinovic, Aamir Ahmad, and Michael Black. Markerless outdoor human motion capture using multiple au- tonomous micro aerial vehicles. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 923-832, 2019.
Nitin Saini, Elia Bonetto, Eric Price, Aamir Ahmad, and Michael Black. Airpose: Multi-view fusion network for aerial 3d human pose and shape estimation. arXiv preprint arXiv:2201.08093 2022.
Patrik Schmuck and Margarita Chli. Multi-uav collaborative monocular slam. In 2017 IEEE International Conference Robotics and Automation (ICRA), pp. 3863-3870. IEEE, 2017.
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint 10Xiv:550.0044 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 17X.:07777.03477 2017.
Anil Sharma, Saket Anand, and Sanjit K Kaul. Intelligent camera selection decisions for target tracking in camera network. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3388-3391 January 2022.
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693-5703, 2019.
Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. Integral human pose regression. In Proceedings of the European Conference on Computer Vision, pp. 529-545, 2018
Peter Sunehag, Guy Lever, Audrunas Gruslys. Wojeiech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Mare Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls. et al. valop-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv: 770605526. 2017.
Rahul Tallamraju, Eric Price, Roman Ludwig, Kamalakar Karlapalem, Heinrich H Bülthoff, Michael Black, and Aamir Ahmad. Active perception based formation control for multiple aerial vehicles. IEFE Robotics and Automation Letters, 111144491149999 2019.
Rahul Tallamraju, Nitin Saini, Elia Bonetto, Michael Pabst, Yu Tang Liu, Michael Black, and Aamir Ahmad, Aircaprl: autonomous aerial human motion capture using deep reinforcement learning. IEEE Robotics and Automation Letters, 5(4))6678-666 6685, 2020.
Hanyue Tu, Chunyu Wang, and Wenjun Zeng. Voxelpose: Towards multi-camera 3d human pose estimation in wild environment. In Proceedings of the European Conference on Computer Vision. pp. 197-212, 2020.
Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, and Yunjic Gu. Shapley q-value: A local reward approach to solve global reward games. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 7285-7292, 2020.
13

---

Published as a conference paper at ICLR 2023
Yuanfei Wang, Fangwei Zhong, Jing Xu, and Yizhou Wang. Tom2e: Target-orienter multi- agent communication and cooperation with theory of mind. In International Conference on Learning Representations, 2022. URL https: openreview net//orumidddMS333 8MH1BK.
Zhou Wenhong, LI Jie, LIU Zhihong, and SHEN Lincheng. Improving multi-target cooperative tracking guidance for uav swarms using multi-agent reinforcement learning. Chinese Journal of Aeronautics, 35(7):100-112, 2022.
Jing Xu, Fangwei Zhong, and Yizhou Wang. Learning multi-agent coordination for enhancing target coverage in directional sensor networks. In Advances in Neural Information Processing Systems, volume 33, pp. 10053-10064, 2020.
Lan Xu, Yebin Liu, Wei Cheng, Kaiwen Guo, Guyue Zhou, Qionghai Dai, and Lu Fang. Flycap: Markerless motion capture using multiple autonomous flying cameras. IEEE Transactions Visualization and Computer Graphics, 24(8):2284-2297, 2017.
Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre M Bayen, and Yi Wu. The surprising effectiveness of mappo in cooperative, aggent games. arXiv preprint arXiv:2103.01955, 2021.
Ailing Zeng, Xiao Sun, Fuyang Huang, Minhao Liu, Qiang Xu, and Stephen Lin. Smeet: Improving generalization in 3d human pose estimation with aplit-and-recombbe approach. In European Conference on Computer Vision, pp. 507-523, 2020.
Kaiqing Zhang, Zhuoran Yang. and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pp- 321-384, 2021.
Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. Ad-vat: An asymmetric dueling mechanism for learning visual active tracking. International Conference on Learning Representations, 2018a.
Fangwei Zhong,
Sheng Wang, Ziqi Zhang, Chen Zhou, and Yizhou Wang. Detect-slam: Making object detection and slam mutually beneficial. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1001-1010. IEFE, 2018b.
Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. Ad-vat+: An asymmetric dueling mechanism for learning and understanding visual active tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1467-1444 2019.
Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. Towards distraction-robust active visual tracking. In International Conference Machine Learning, pp. 12782-12792.
PMLR, 2021.
Zhun Zhong, Liang Zheng, Zhedong Zheng, Shaozi Li, and Yi Yang. Camera style adaptation for person re-identification. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 51.51 5157-5166, 2018c.
Xiaowei Zhou, Sikang Liu, Georgios Pavlakos, Vijay Kumar, and Kostas Danillidis. Human motion capture using a drone. 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 2027-2033, 2018.
14

---

Published as a conference paper at ICLR 2023
A TRAINING ALGORITHM PSUEDOCODE
Algorithm I Learning Multi-Cames Collaboration (CTCR + WDL)
I: Initialize: agents with tied-weights MAPPO policy and mixture density models (MDN) for WDL
prediction models (Paer, Prewad, Paec, Pig; P(a) +> & parallel environment rollouts
for Iteration 1, 2,
3:
In each & environment rollouts, each agent E [n]] collects trajectory with length T:
T [[d], 0+ 0, a,,, r4. ,8" aa-! ,r'")>"]_
4: Substitute individual reward of each agent r' with CTCR: r 4- CTCR(i) (equation 6) For each step T, compute advantage estimates ......, A? with GAE (Schulman et al., 2015) each agent i E [[r]]]
6:
Yield training batch D of the size & x T for Mini-batch SGD Epoch 1,2, ... K do
7:
8:
Sample a stochastic mini-batch of size form D, which B IDI/K Compute 2 and bf using the encoder model Ea
9:
10:
Compute PPO-CLIP objective loss Levo, global critic value loss Lvatue adaptive KL loss LKL
11:
Compute the objectives that constitutes LWDE:
Self-State Prediction Loss
Lielt = - E, [lo P( |2f, 14, a])]
Reward Prediction Loss
Creward = - Er (lgg P(r 14f, hi, a!)
Peer-State Prediction Loss
Spont E- flog P N4. a.)
Target Prediction Loss
Lug E. flog P( 1+1 | h1s p(p)l
Pedestrians Prediction Loss
Lps R., (log P(r
4+1 |e|, M4 Pf0)
12:
Lrain Arro Avalue + 3KI. Cxt. + Awnr LWDL.
Optimize Ciraia w.r.l current policy parameter 0x
15

---

Published as a conference paper at ICLR 2023
B UNREALPOSE: ACCESSORIES AND MISCELLANEOUS ITEMS
Our UnrealPose virtual environment supports different active vision tasks, such as active human pose estimation and active tracking. This environment also supports various settings ranging from single-target single-camera settings to multi-target multi-camera settings. Here we provide a more detailed deseription of the three key characteristics of UnrealPose:
Realistic The built-in navigation system governs the collision avoidance movements of virtual humans against dynamic obstacles. In the meantime, also ensures diverse generations of walking trajectories. These features enable the users to simulate realistic-looking crowd exhibiting socially acceptable behaviors. We have also provided several pre-set scenes, e.g., school gym, wilderness, urban crossing, and etc. These scenes have notable differences in illuminations, terrains, and crowd appearances to reflect the dramatically different-looking scenarios real-life.
Extensive Configuration The environment can be configured with different numbers of humans and cameras and swapped across other scenarios with ease, which we demonstrated in Fig 4(a-d). Besides simulating walking human crowds, the environment incorporates over 100 Mocap action sequenees with smooth animation interpolations to enrich the data variety for other MoCap tasks.
RL-Ready We the UnrealCV (Qiu et al., 2017) plugin the medium acquire images and annotations from the environment. The original UnrealCV plugin suffers from unstable data transfer and unexpected disconnections under high CPU workloads. To ensure fast and reliable data acquisition for large-scale MARL experiments, we overhauled the communication module in the UnrealCV plugin with inter-process communication (IPC) mechanism, which eliminates the aforementioned instabilities.
B. VISUALIZATION TOOL
provide a visualization tool to facilitate per-frame analysis of the learned policy and reconstruction
results (shown in Fig. 8). The main interface consists of four parts: (1) Live 2D views from all cameras. (2) 3D spatial view of camera positions and reconstructions. (3) Plot of statistics. (4) Frame control bar. This visualization tool supports different numbers of humans and cameras. Meanwhile, written Python to support easy customization.
3D Spatial View
Reconstruction Radio Plat
Local Camera View
Figure 8: Visualization Tool. RGB videos at the two sides show the local camera view for each individual camera. The center part shows the live 3D reconstructions and corresponding MPJPE.
B.2 LICENSE
All assets used in the environment commercially-availae and obtained from the UE4 Marketplace. The environment and tools developed in this work are licensed under Apache License 2.0.
16

---

Published as a conference paper at ICLR 2023
C OBSERVATION PROCESSING
Fig.9 shows the pipeline of the observation processing. Each camera observes an RGB image and detects the 2D human poses and IDs via the Perception Module described in the main paper. The camera pose, 2D human poses and IDs are then broadcast to other cameras for multi-view 3D triangulation. The human position calculated as the median of the reconstructed joints. Human orientation is calculated from the cross product of the reconstructed shoulder and spine.
Camera Posp
Local Camera
Detection, RelD, 2D HPE
Human Pose, ID
3D Online Triangulation
Feature Parsing
PigT
Observation
Orientation Human Position
RGB
Human Pone, ID
P
0
Human Pare
Camer Pose
Camera
Receiver
Camera Agent i
Sender
Broadcast Channel
Houran Poss, ID...
Human Poin,
Comrerta Fose
Camera
Camera Agent j
Sender
Receiver
ID
Human Camora PosC
Posp...
Human Porce
Local Camera Observation
Detection, RelD, HPE
Triangulation
Feature Parsing
Pigs
RGB
3D HPE
Orientation Human Posltion
Pd
Figure 9: Decentralized Observation Processing
17

---

Published as a conference paper at ICLR 2023
D IMPLEMENTATION DETAILS
D.1 TRAINING DETAILS
All control policies are trained in the BlackEnv scene. At the testing stage, we apply zero- shot transfer for the learned policies to three realistic scenes: SchoolGym, UrbanStreet, and Wilderness.
To simulate dynamic human crowd that performs high random behaviors, sample arbitrary goals for each human and employ the built-in navigation system to generate collision-free trajectories. Fach human walks random speed. To ensure the generalization across different number of humans, we train our RL policy with a mixture of environments of to 6 humans. The learning rate is set to x 10-4 with scheduled decay during the training phase. The annealing schedule for the learning rate is detailed in Table. 2. The maximum episode length is 500 steps, discounted factor y is 0.99 and the GAE horizon is 25 steps. Each sampling iteration produces training batch with the size of 700 steps, then we perform 16 iterations every training batch with 2 SGD mini-batch updates for each iteration (i.e. SGD batch size 350).
Table 2 shows the common training ppper-parameters shared between the baseline models (MAPPO) and all of our methods. Table. 4 shows the hyperparameters for the WDL module.
Table 2: Common hyperparameters for models in baselines and our methods
Common Hyperparameter
Baseline, Our Methods
Max Episode Length
500
Rollout Fragment Length
25
Number Rollouts
28
Train Batch Size
700
SGD Mini-batch Size
350
Number SGD Iterations (Sample Reuse)
16
gamma (7)
0.99
GAE lambda (A)
1.0
Initial Coefficient
0.2
Tagge
0.01
Entropy Regulation Coefficient
0.0
Value Function Loss Coeflicient
0.1
Value Function Clipping
1000
Gradient Clipping
50
Initial Learning Rate
0.0005
Learning Rate Schedule (Ir, end step)
[(0, SE-4), (200E3, 5E-4), (200E3, 1E-4), (400E3,
IE-4), (600E3, SE-5). (600E3, SE-5)]
LSTM Cell Size
128
Encoder Hidden Layers
[128, 128, 128]
Value Network Hidden Layers
[128]
Actor Network Hidden Layers
[128]
D.2 DIMENSIONS OF FEATURE TENSORS IN THE CONTROLLER MODULE
Table 3 serves as a complementary description for Eqn. 1, 2, 3, 4. The table shows the dimensions of the feature tensors used in the controller module." "B" denotes the batch size. In the current model design, the dimension of local observation is adjusted based on the maximum number of camera agents (N cammax) and the maximum number of observable humans (N human in an environment. In our experiments, N human has been set to 7. The observation pre-processe processor will zero-pad each observation to length equal N Juumanmax X 18 if the current environment
18

---

Published as a conference paper at ICLR 2023
instance has less than N human humans. "9" and "18" correspond to the feature length for a camera and human, respectively. The MDN the Target Prediction module has 16 Gaussian components, in which each component outputs (Ø, Ma. xx My: σy) and is the weight parameter of component. The current implementation of MDN only predicts the x-y location of human, which simplification since the coordinate of simulated human barely changes across an episode compared to the x and y coordinates. The dimension of an MDN output has a length of 80 and the exact prediction produced by e-weighted averaging. In Eqn. { μ4, o MDN.,. an encoded feature produced by passing the MDN output to 2-layer MLP that has an output dimension of 128.
Table 3: Dimensions of the feature tensors used in the controller module
Feature
Shape
0z
(B, N cammmax x9+N Jeu x 18)
(B, 128)
h4
(B, 128)
Pigt, Ppd
(B, 128)
[(0, 11,0) 0) M
(B, 128)
(B, 384)
D.3 COMPUTATIONAL RESOURCES
We used 15 Ray workers for each experiment to ensure consistency in the training procedure. Each worker carries Gym vectorized environment consisting of 4 actual environment instances. Each worker demands approximately 3.7 GB of VRAM. We run each experiment with 8 NVIDIA RTX 2080 Ti GPUs. Depending on the number of camera agents, the total training hours required for an experiment to run 500k steps will vary between 4 to 12 hours.
Table 4: Hyperparamters for WDL
WDL Hyperparameter
Our Methods
Prediction Loss Coefficient (AwaL)
1.0
Camera Prediction Coefficient
1.0
Other Camera Prediction Coefficient
1.0
Reward Prediction Coefficient
1.0
Human Prediction Coefficient
1.0
Pedestrian Prediction Coefficient
0.1
MDN: Hidden Layers
[128, 128]
MDN: Number of Gaussian Distributions
D.4 TRAINING CURVE
Fig. 13 shows the training curves of method and the baseline method. We can find that our methods converge faster and improve reconstruction accuracy than the bascline method.
D.5 TOTAL INFERENCE TIME
Our solution can run real-time. Table reports inference time for each module of the proposed Active3DPose pipeline.
19

---

Published as a conference paper at ICLR 2023
[3 Cameras]: Baseline Vs. Our Method
30
MPIPE (cm)
25
MAPPO CTCR
WDL
(Our Nathod)
HAPPO
(Baseline)
[4 Cameras]: Baseline Method
10
Timesteps
100k
200k
300k
400k
500k
NAPPO
CTCR WDL
(Our Nethod)
HAPPO (Baseline)
[5 Cameras]:
Baseline Vs. Our Method
esteps
MAPPO CTCR
WDL
(Our Nathod)
NAPPO
(Baseline)
Figure 10: Comparing the training curves (MPJPE (cm) vs. Environment Steps) for 3, 5 cameras policies trained to 500k timesteps for the baseline and our method. Intervals for one standard deviation are shown, and each interval is computed based on samples from 3 different runs on different seeds.
Table 5: The total inference speed of current implementation is around FPS. The average time data are generated on a computing node with one NVIDIA RTX 2080TI GPU and one Intel Xeon E5-26699 CPU. These data from testing our 5-camera model over 2500 environment steps in the Blank Environment.
Average Time (ms)
Human Detection
20.05
ReID Module
28.88
2D Pose Estimation
31.16
3D Triangulation
1.43
Observation Processing
3.97
RL Model Inference
40.76
Total Inference Time
126.25
20

---

Published as a conference paper at ICLR 2023
E ADDITIONAL EXPERIMENT RESULTS
E ABLATION STUDY ON WDL OBJECTIVES
We perform a detailed ablation study regarding the effect of each WDL sub-task on the model performance. As shown in Fig. 11, we can observe that the MPJPE metric gradually decreases as we incorporate more WDL losses. This aligns with our assumptions that training the model with world dynamics learning objectives will promote the model'ss ability to capture a better representation of future states, which in turn increases performance. Our method additionally demonstrates the importance of incorporating information regarding the target's future state into the encoder's s output features. Predicting the target's s future states should not only be used as an auxiliary task but should also directly influence the inference process of the actor model.
NPJE 3D vs. Number of Humans
Succoss Rate Number of Humars
Duts Muran reed AL Prod
0.50
Ours Human fied RL hrod Cur Numan Fred
Ours Tiuman Frdd
-8--
Ours
Ours
2000mee
10y2 30 (0m1
0.20
130
0.75
100
0.30
Figure 11: Detailed ablation study WDL modules in the 2-cameras environment. Here commence the study by running three separate experiments differed by the number of auxiliary tasks involved. A minus sign (-) followed by the name of a module indicates the absence of this module in this trial. RLPred: learn agent's 's forward dynamics, Human Pred: predict future target position and predict future pedestrian position. Ours is our fully implemented method (MAPPO+WDL) for controlling 2 camera agents.
E.2 BASELINE - FIXED- CREDE-SAMERAS
In addition to the triangulation and RANSAC baselines introduced in the main text, we compare and elaborate on two more baselines that use fixed cameras: (1) fixed cameras with RANSAC-based triangulation and temporal smoothing (TS) (2) an off-shelf 3D pose estimator PlaneSweepPoss (Lin & Lee, 202
In the temporal smoothing baseline, we applied low-pass filter (Casiez et al., 2012) and temporal fusion where the algorithm will fill in any missing key points in the current frame with the detected key points from the last frame.
In the PlaneSweep baseline, as per the official instructions, we train three separate models (3 cams to 5 cams) with the same camera setup as in our testing scenarios. We have tested the trained models in different scenarios and reported the MPJPE results in Table 6, 7, 8 and 9. Note that, this off-shelf pose estimator performs better than the Fixed-Camera Baseline (Triangulation) but still underperforms compared to active method. Fig. 12 illustrates the formations of the fixed camera baselines.
Table 6: Four fixed-camera baselines compared our method in the 6-human Blank environment. Results for all baselines are evaluated over 20 episodes of 500 steps. RANSAC: RANSAC-based Triangulation TS: Temporal Smoothing with temporal fusion PlaneSweep: PlaneSweepPose (Liin Lee, 2021a).
Fixed Cameras (Triangulation)
Fixed Cameras (RANSAC)
Fixed Cameras (RANSAC+TS)
Fixed Cameras (PlaneSweep)
BlankEnv
Ours
Cameras
142.0
129.0
123.5
123.3
73.3
Cameras
122.3
116.6
106.0
116.4
54.7
Cameras
87.0
72.3
77.7
86.5
51.6
21

---

Published as a conference paper at ICLR 2023
Table Four fixed-camera baselines compared our method in the 6-human SchoolGym environ- ment. Results for all baselines are evaluated over 20 episodes of 500 steps. RANSAC: RANSAC- based Triangulation TS: Temporal Smoothing with temporal fusion PlaneSweep: PlaneSweep.
Pose (Lin & Lee, 2021a).
Fixed Cameras (Triangulation)
Fixed Cameras (RANSAC)
Fixed Cameras (RANSAC+TS)
Fixed Cameras (PlaneSweep)
SchoolGym
Ours
Cameras
149.1
149.8
144.1
132.6
99,4
Cameras
133.3
124.5
124.7
120.3
67.7
Cameras
100.2
76.8
83.0
92.8
64.6
Table Four fixed-camera baselines compared our method the 6-human UrbanStreet environ- ment. Results for all baselines are evaluated over 5 episodes of 500 steps. RANSAC: RANSAC-based Triangulation TS: Temporal Smoothing with temporal fusion PlaneSweep: PlaneSweepPose (Lin Lee, 2021a).
Fixed Cameras (Triangulation)
Fixed Cameras (RANSAC)
Fixed Cameras (RANSAC+TS)
Fixed Cameras (PlaneSweep)
UrbanStreet
Ours
3 Cameras
513.4
210.1
194.2
160.3
103.6
4 Cameras
279.4
190.8
174.5
154.5
66.4
5 Cameras
136.7
94.1
94.4
106.5
61.9
Table 9: Four fixed-camera baselines compared our method in the 6-human Wilderness environment. Results for all baselines are evaluated over 20 episodes of 500 steps. RANSAC: RANSAC-based Triangulation TS: Temporal Smoothing with temporal fusion PlaneSweep: PlaneSweepPose (Lin Lee, 2021a).
Fixed Cameras (Triangulation)
Fixed Cameras (RANSAC)
Fixed Cameras (RANSAC+TS)
Fixed Cameras (PlaneSweep)
Wilderness
Ours
3 Cameras
234.0
159.6
160.5
139.2
72.6
4 Cameras
165.6
143.9
146.6
129.5
71.3
Cameras
108.1
74.9
86.5
95.1
54.3
22

---

Published as a conference paper at ICLR 2023
2-cam
3-cam
4-cam
5-cam
Figure 12: Illustration of fixed camera placements
(a) 6 Cams
Hexagon
(h) Cams Octagon
(c) 16 Cams Double Octagon
(d) 16 Cams Hexadecagon
(e) 16 Cams Double Octagon (OH)
(0) 24 Cams
"Xmas Tree"
Figure 13: In search of topology fixed-camera system that could outperform our 5-camera model, we have designed six more 3D topologies that contain 8 to 24 cameras.
Camera placements for all fixed-camera baselines are shown in Fig. 12. These formations are carefully designed not to disadvantage the fixed-camera baselines on purpose. Especially for the 5-cameras pentagon formation, which helps the fixed-camera baseline obtain satisfactory performance on the 5-camera setting as shown Tables 6, 7, 8 and 9.
Table 10: Comparing six more fixed-camera network topologies with our 5-camera method. Although these topologies have significantly more cameras than our 5-active-camera model, the fixed-camera approach cannot close the performance gap between fxxe-camera systems and the learned policy. We also observe inversions in MPJPE when increasing the number of cameras from to 16 to 24. These results suggest that the fixed-camera system has saturated the reconstruction accuracy beyond cameras, and there are no further upsides to increasing the scale. We reason this is due to the errors and biases in the 2D pose model that subsequently affect the triangulation algorithm's 's effectiveness..
MPJPE (mm)
Cams Hexagon
88.0
8 Cams Octagon
78.5
16 Cams Double Octagon
86.7
16 Cams Hexadecagon
81.7
16 Cams
Double Octagon (OH)
94.9
24 Cams "'Xmas Tree"
87.0
Ours, 5 Cams, active cameras
51.6
23

---

Published as a conference paper at ICLR 2023
E.3 OUR METHOD ENHANCED WITH BANSAC-BASED TRIANGULATION
RANSAC is generic technique that can be used to improve triangulation performance. In this experiment, we also train and test our model with RANSAC. The final result (Table 11) shows further improvement our original triangulation version.
Table 11: Evaluate the effect of adding RANSAC- based Triangulation to our method. Both models are trained from scratch mix of Blank environment instances with to 6 humans. MPJPE(mm) averaged over training results in six different configurations Blank environment instances (1-6 humans).
Ours (5 Cams)
Ours (5 Cams RANSAC)
MPJPE (mm))
46.7
35.0
F GENERATING SAFE AND SMOOTH TRAJECTORIES
F.1 COLLISION AVOIDANCE
In order to generate safe trajectories, in this section, introduce and evaluate two different ways to enforce collision avoidance between cameras and humans.
Obstaele Collision Avoidance (OCA) OCA resembles feed-forward PID controller on the final control outputs before execution. Concretely, OCA adds constant reverse quantity to the control output if it detects any surrounding objects within its safety range. This "detouring" mechanism safeguards the cameras from possible collisions and prevents them from making dangerous maneuvers.
Action Action-Masking (AM) AM also resembles a feewaforward controller but is instead embedded into the forward step of the deep-leane seep-learning model. At each step. AM module first identifies the dangerous actions among all possible actions, then modifies the probabilities (output by the policy model) of choosing the hazardous actions to be zero so that the learning- -ased control policy will never pick them. Note that AM must be trained with the MARL policy model.
We proposed the minimum Camera-Human distance (in which the scope includes the target and the pedestrians) the safety metric. It measures the distance between the closest human and the camera at timestep. Fig. 14 shows the histograms of Min Camera-Humai distance sampled over five episodes of 500 steps.
Histogram of Min Camera-Human Distance (5 Cameras Humans)
Vanilla
Obstacle Callision Avaidance
Masking
2.0
1.5
Probability
1.0
0.50
0.75
1.00
1.25
1.50
1.75
2.00
2.25
Min Cameer-Hummn Distance (m)
Figure 14: Histograms of minimum Camera-to- Human distances to demonstrate the safety integrity level for three different 5-Cameras methods. Vanilla: original deep MARL policy without any safety feature. Obstacle Collision Avoidance: deep MARL policy added with OCA module Action Masking: deep MARL policy trained with AM module.
24

---

Published as a conference paper at ICLR 2023
F.2 TRAJECTORY SMOOTHING
Exponential Moving Average (EMA) To generate smooth trajectories, we introduce EMA to smooth the outputs of our learned policy model. EMA is common technique used in smoothing time-series data. In our case, the smoothing operator is defined as
â u + 1j (a+ 0c-1)
Where a, is the action (or control signal) output by the model the current step, â is the smoothed action from the last step and is the smoothing factor. A smaller 7) results in greater smoothness. âe the smoothed action that the camera will execute.
F.3 ROBUSTNESS OF THE LEARNED POLICY
In this section, evaluate the robustness of our model different external perturbations to the control signal. In conclusion, our model shows resilience against the delay effect and random noise.
Delay EMA also brings delay effect while smoothing the generated trajectory. The level of smoothing positively correlates with larger delay factor. Here, we evaluate our model's 's robustness the EMA simulated delay.
Random Action Noise Control devices in real-life are inevitably affected by emror.. For example, the control signal of a controller may be over-damped or may overshoot. We simulated this type of random error by multiplying the output action by uniformly-samplee noise.
In Table 12, we intend to observe the effects EMA delay and random action noise on the recon- struction accuracy of our model, marked "Vanilla".
Table 12: Adding delay (EMA) and action noise proposed method. In this experiment, 0.50 and noise factor between [0.80, 1.20). Results are reported on 5 episodes of 500 steps.
Add action noise and (ElAy(EMA)
Ours(Vanill))
Add delay(EMA)
Add action noise
MPJPE (mm)
(51.6)
+0.8
+2.1
+3.9
25

---

Published as a conference paper at ICLR 2023
G MORE EXPLANATIONS ON CTCR
Figure 3 is an example of using Eq. 6 to compute CTCR for each of the three cameras. The CTCR is incentivized by the Shapley Value. The main idea is that the overall optimality needs also account for the optimality of every possible sub-formation. For camera agent to receive the highest CTCR possible, its current position and view must be optimal both in terms of its current formation and any sub-formation possible.
Note: group of collaborating players is often referred to as a "coalition" in other literatures. Here we apply the same concept but to group of cameras, so we used the more initiative term "formation" instead.
Eq. 6 can be further breakdown as follows:
[S]!(n - [S] 1)!
(r(1)) = Σ
(t))
n!
ScIDM M0
[S|:(n 1- |S|)!
E
(i(SU - r(()
n MC
(n (n-1))
Σ MO
n |S| (r(S
where [n] 1,2,... 1, denotes the set of all cameras and the binomial coefficient (
ml k[in-A]1 0 < k S n. S denotes a formation (a subset) without camera i. ("is)) is the number normalization term.
of combinations of subset S (i.e., the binomial coefficient), which serves r(S) computes the reconstruction accuracy of the formation S. And [r(5 {t)) r()) computes the marginal improvement after adding the camera to sub- hobooormation S. So this equation means iterate over all possible S and compute the marginal contribution of camera and average over all possible combinations of (S,
Suppose we have a 3-cameras formation, similarly shown in Figure 3. So n = 3, the number of cameras. Let's name these cameras (1,2,3), and let's say only care about Camera 1 for now. Since we are computing the average marginal contribution for Camera 1, we are looking for those formations that do not have Camera because want to see how much of an increase in performance resulted from the addition of Camera 1 to those formations. For all possible formations denoted by the set [r.]. four formations satisfy this condition, S [n] {i} + S (0, {2), (3), 3)).
The binomial coefficient (is) for 2-camera nub-formation in a 3-cameras case is (2) which makes sense because there exists only one unique combination that does not contain Camera 1, which is sub-formation {2, 3). r((2,3)) computes the reconstruction accuracy of the formation {2, and r((2,3) (1))) computes the reconstruction accuracy after adding Camera to sub-formation {2, 3}- Their difference gives the marginal contribution of Camera 1, As we sum over all subsets S of [n]] not containing Camera 1, and then divide by ((ss)) and the number of cameras we have the average marginal contribution of Camera (v)((1))) the collaborative triangulated reconstruction.
Further multiplying this term by n, you have the CTCR, as shown in Eq. 6.
26

---

Published as a conference paper at ICLR 2023
H ANALYSIS ON MODES OF BEHAVIORS OF THE TRAINED AGENTS
Here Figure 15 we provide statistics and analysis on the behaviour mode the agents controlled by our 3-, 4- and 5-camera policies, respectively. We are interested in understanding the characteristics of the emergent formations learned by our model. Hence we proposed three quantitative measures to understand the topology of the emergent formations: (1) the min-angle between the cameras' orientation and the per- per-frame-maan of the min-camer angle, (2) the camera's 's pitch angle, and (3) the Gamera-human distance. Hence we provide rigorous definitions as follows:
min-camera anglo(i) = min camera i, axis camera 3944
1
per-frame-men meann of min-camera angle
min-camera angle(i)
72
In simpler terms, min-camera angle(i) finds the minimum angle between camera i and any other camera j. "per-frame- mean of min-camera angle" is the mean of min-camera angle(i) for all camera i in one frame. A positive camera's 's pitch angle means looking upward, and negative camera's pitch angle means looking downward. The camera-human distance measures the distance between the target human and the given camera.
Regarding the distance between cameras and humans, the camera agents actively adjust their position relative to the target human. The cameras learn to keep camera-human distance between 2m and 3m. is neither too distant from the target human nor too close, violating the safety constraint. The camera agents surround the target at appropriate distance have better resolution on the 2D views. In the meantime, as the safe-distance constraint enforced during training, the camera agents prohibited from aggressively minimizing the camera-human distance.
The histograms for the camera's pitch angle suggest that the cameras mostly maintain negative pitch angles. Their preferred strategy is to hover over the humans and capture images a higher altitude. This is likely because of emergent occlusion avoidance. The cameras also emerge to fly at an even level with the humans (where the pitch angle approximately equals 0) capture more accurate 2D poses of the target human. This propensity is apparent from the peaks at x = O angles in the histograms. relatively wide distribution of the pitch angle histogram suggests that the camera formation is spread out in the 3D space and dynamic adjustments of flying heights and pitch angles by the camera agents.
For the average angle between the cameras' orientations, this statistic shows that the cameras in various formations will maintain reasonable non-zero spatial angles between each other. Therefore, their camera views are less likely to more reliable 3D triangulation.
coincide and provide more diverse perspectives to generate
27

---

Published as a conference paper at ICLR 2023
0.05
0.04
0.00
0.000
120
00.
300
(a) 3 camera agents, humans
amain
con milvceeera
Phoced 1//0ee00
0.35
0.00
0.35
0.34
0.02
dittacc
Megrel
(b) 4 camera agents, 6 humans
3.07
senfrrme
0.12
0.008
0.59
0000
distaic 1un
50
-39
-20
10
20
182
139
camera agents, humans
Figure 15: Histograms of the three metrics for 3-, 4- and 5-camera methods. Left Column: pdf of camera-to-targee distance. Middle Column: pdf of camera pitch angle. Right Column: pdf of per-fran per-frame-mar mean of min-camera angle. Detailed descriptions of each metric can be found in Section
28


Reviews:
[{"review_id": "TsOkynvB476", "reviewer": "Reviewer_eqPb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a multi-agent reinforcement learning approach with a Collaborative Triangulation contribution the credit assignment by using their weighted average marginal contribution for 3D human pose estimation in dynamic human crowds. The model is trained with multiple world dynamics learning tasks. The method is evaluated in four photo-realistic UE4 environments. The results show the proposed method outperforms the fixed and active baselines in different scenarios with various numbers of cameras and humans. ", "summary_of_the_review": "The paper introduces a novel dynamic multi-camera collaboration framework via multi-agent reinforcement learning to solve the 3D Human pose estimation problem. The paper is clearly written and well-structured with strong empirical evaluation in various simulated scenarios and conditions. One potential drawback of the proposed method is the practicality of the dynamic multi-camera deployment in real-world environments. ", "strength_and_weaknesses": "# Strength:\n- The proposed method novel in using multiple (>3) active cameras to perform 3D HPE in human crowd. \n- The proposed method introduces a decentralized framework via multi-agent reinforcement learning for multi-camera collaboration at different scales and scenarios (in simulated enviromnment). \n# Weakness:\n- The proposed approach relies on human recognition module to distinguish people in a scene. For some out-of-distribution human appearances, the current ReID module may not work, which will then hinder HPE task. \n- The practicality of the algorithm seems limited. To perform 3D human pose estimation, dynamic cameras seem quite difficult to deploy in real-time. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "QUXFsymBcHD", "reviewer": "Reviewer_XxGE", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper develops an active vision scheme where multiple non-stationary cameras reconfigure themselves to achieve high-quality 3D human pose estimation.  The camera control problem is cast within the multi-agent reinforcement learning framework.  The paper also presents a new reward formulation that incentivizes the cameras according to their weighted marginal contribution to the 3D human pose reconstruction quality.  This reward seems to address the multi-agent credit assignment issue.  The proposed model is trained within a 3D environment populated with lifelike pedestrians.  The model is jointly trained with world-dynamics-learning task; therefore, it exhibits anticipatory occlusion avoidance, improving the reconstruction accuracy.    ", "summary_of_the_review": "This is an application paper.  It makes a number of technological contributions---the 3D environment, model, etc.  The paper also makes some scientific contributions---model setup and credit assignment strategy.  The paper will be of interest to the machine learning community.  It is particularly useful for those who are interested in multi-agent systems coordination problems where the individual agents rely upon vision as their primary perception modality.\n", "strength_and_weaknesses": "This is a well-written paper that studies an interesting problem.  The paper claim four \"key\" contributions; however, not all of these contributions have the same scientific or technological impact.  I suggest to divide this list into primary and secondary lists.  The key conributions, I feel, are: 1) problem setup and 2) the new reward formulation.  The engineering effort in setting up the 3D world that others can use for their own work seems to me a secondary contribution.\n\nWhen speaking of active camera collaboration schemes for scene analysis and pedestrian tracking and the use of 3D worlds to study such multicamera systems, the following work comes to mind \n\nSmart Camera Networks in Virtual Reality. Qureshi, F.Z.; and Terzopoulos, D. Proceedings of the IEEE (Special Issue on \"Smart Cameras\"), 96(10): 1640–1656. October 2008.\n\nThis is highly relevant to the work presented in this paper, and it should be cited in Multicamera Collaboration Section.  This work studies multi-camera control and uses a 3D world with autonomous pedestrians to develop and evaluate the camera coordination on the task of pedestrian tracking.\n\nIt may be useful to provide a more detailed caption for Figure 2.  Specifically, please define the variables not defined anywhere else.    \n\nIt will be beneficial to provide the number of dimensions for each variable used in equations 1 to 4.  I feel that this will increase the clarity of the proposed approach.\n\nIn Section 3.4 what is the difference between the future position of target person and the future position of pedestrians?\n\nI tried but failed to wrap my head around Eq. 6.  Perhaps it is possible to revise this discussion.  One possibility is to construct Eq. 6 term by term.  This equation deals with the reward for each camera, which, alongwith problem setup, I feel is one of the primary scientific contribution of this work. \n\nI was pleased to see that the paper includes a discussion about the limitations of the proposed method.  Perhaps this discussion should be expanded.  For example, it seems that this work assumes near-perfect camera calibration, wich is not always possible to achieve in practice.  It is especially hard to maintain the calibration of a group of cameras that move around.\n\nI was also pleased see the paragraph about the ethical consideration of using such systems.  I ask that this dicussion should be moved to the main body of the apaper.   It is too important to be relegated to the appendix.\n\nPlease use a different \"bibstyle.\"  As it stands it is tedious to connect a citation to its reference.\n\nThe paper asserts that the group of cameras learn occlusion-avoiding anticipatory behavior.  Actually this was one of the reasons why the model was also trained on world-dynamics-learning-tasks.  Perhaps there is a way to provide some results that support this claim. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "HJk5tJmTxe7", "reviewer": "Reviewer_W1S5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a proactive multi-camera collaboration method for 3D multi-person human pose estimation with unmanned aerial vehicles (UAVs). The method is based on multi-agent reinforcement learning that treats cameras as agents and utilizes a world dynamics model to improve the performance of the model. Specifically, by additionally predicting the dynamics of camera agents and humans, the paper is able to extract more useful features for the control of the camera agents. Inspired by prior work, the paper also proposes a collaborative triangulation contribution reward (CTCR) that uses the shapley value to improve the credit assignment of reconstruction quality to agents. The paper builds its own RL environment based on Unreal Engine 4 (UE4) where human crowds are simulated with collision avoidance in various scenes. Experiments in this environment demonstrate the superiority of the proposed approach over the baselines. Extensive ablation studies are also carried out to validate the design of the method.", "summary_of_the_review": "Overall, the paper presents an interesting approach to a challenging and useful problem. However, its evaluation is lacking, especially the real-world evaluation, which is very important given the safety implication and concerns in practicality.\n\n== After rebuttal\n\nThe authors' response addressed most of my concerns, so I updated my rating.", "strength_and_weaknesses": "**Strength:**\n\n- The problem the paper aims to solve is interesting and challenging, i.e. reconstructing dynamic human motion in the wild where occlusions often happen and static camera formations are not sufficient to address them.\n- Extensive experiments and ablation studies are done in the simulated environment, which shows the method outperforms the baselines and also validates the usefulness of the proposed techniques.\n- The paper also plans to open-source the UE4-based RL environment used in the method, which would be beneficial to future research on proactive multi-camera human pose estimation.\n\n**Weakness:**\n\n- The method is not evaluated with safety metrics such as collision or interference with humans, which is rather important given the trajectory produced by the method could potentially harm the subjects.\n- The simulation does not consider the dynamics of real-world UAVs and many of the jittery trajectories produced by the method might not be realizable by UAVs.\n- A major concern of mine is that all the experiments are performed in simulation without real-world validation of the proposed approach. Given the method is potentially unsafe and the trajectories might be unrealizable, it is important to demonstrate its real-world applicability as prior work (e.g., AirCapRL) does.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0Mdf4a4QC_w", "reviewer": "Reviewer_FtmL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper describes a method for active (cameras can move proactively) multi-camera motion capture. The problem is formulated under multi-agent reinforcement learning framework, where the authors propose a new reward (Collaborative Triangulation Contribution Reward) to  incentivize agents according to their weighted average marginal contribution to the 3D reconstruction. The proposed method is evaluated on UE4 environments, where the method has demonstrated notable improvements over baseline, and competing methods.\n", "summary_of_the_review": "\n\nOverall it's an interesting work. The methodology is clearly motivated, and I found it quite novel to use multi-agent reinforcement learning for multi-camera motion capture. Also, the method is extensively evaluated with plenty of experiments and analysis. \n\nI have some concerns regarding generalizability of the method, and that the understanding of the learned policy seems limited. But none of my concerns are deal-breakers and I hope to discuss those in the rebuttal. \n\nOverall I believe it's a decent contribution, and I would recommend it for acceptance.\n\n", "strength_and_weaknesses": "\n\n# Strength\n- The methodology is clearly motivated, and I found it quite novel to use multi-agent reinforcement learning for multi-camera motion capture.\n- The method is extensively evaluated with plenty of experiments and analysis.\n- The proposed virtual environment also seems a very useful tool, and can facilitate future research in the direction if this can be open-sourced.\n\n# Weakness\nThough it's a very interesting work, I am mainly concerned with two aspects: generalizability, and the understanding of the learned policy seems limited.\n- **Generalizability**: as the model training seems only possible with virtual environments, and the real world can exhibit significant gaps with such environments, e.g. different body motion, different type of occlusion etc, it's not clear whether the learned policy can be generalizable to real environments. The experiments in Fig7 are helpful, I think it would be better to also add an upperbound model (\"ours\" trained/finetuned in the same environment) to get a sense of how the model is impacted by the domain gap. Nevertheless, I think the experiment only provides a partial view, still more evidence (e.g. experiments on real datasets) may be needed to validate the generalizability.\n- **Understanding of the learned policy is limited**: I appreciate the amount of experiments provided. But still it is not fully clear what is actually learned by the policy and why it is better than the baseline. It would be helpful if authors can provide more statistics/analysis on the behavior mode of the agents and visualizations. Besides, it would also be useful to provide experiments with more cameras until the performance gap between fixed cam/learned policy is closed. (I would expect with more cameras, it will spread out to cover most views, thus the improvement from a learner polocy can become marginal)\n\n# Other questions\nI have also several minor questions, would be good if the authors can discuss them:\n- wrt 3.1 Action Space, why rotation is limited to 2D and there is no roll?\n- wrt fig6, there is a sudden increase in error for MAPPO at 5-6 human, it would be interesting to understand why this happens, and to provide some illustration to demonstrate the error mode. Also, would be useful to provide the same visualization for MAPPO + RLPred to understand why RLPred is so helpful in such cases.\n- wrt 3.2 Perception Module, it can be helpful to provide an ablation study on used 2D pose models, and see how the errors in 2D pose would impact the learned policy. Currently it's not very clear whether the policy learned to encourage collaboration among cameras, or it's more encouraging camera to be at a view more optimal for 2D pose model. Would be interesting to provide some results with GT 2D pose (from virtual environment), and some cross results to test the generalization ability (e.g. learned from GT 2D pose, test with yolo v3 etc.)\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster