Real-Time Cinematic Tracking of Targets in Dynamic Environments
A BSTRACT
Tracking in a cinematic way a moving target inside a 3D dynamic environment remains a challenging problem.
This requires to simultaneously ensure a low computational cost, a good degree of reactivity and a high cinematic quality despite sudden changes.
In this paper, we draw on the idea of Motion-Predictive Control to propose an efﬁcient real-time camera tracking technique which ensures these properties.
Our approach relies on the predicted motion of a target to create and evaluate a very large number of camera motions using hardware ray casting.
Our evaluation of camera mo- tions includes a range of cinematic properties such as distance to target visibility, collision, smoothness and jitter.
Experiments are conducted to display the beneﬁts of the approach with relation to prior work.
1 I NTRODUCTION
The automated generation of qualitative camera motions in 3D vir- tual environments is a key problem for a number of computer graphics applications such as computer games, automated generation of virtual tours or virtual storytelling.
The ﬁrst and foremost problem is to identify what are the intrinsic characteristics of good camera motions.
While ﬁlm literature provides a thorough and in-depth analysis of what makes a qualitative viewpoint in terms of framing, angle to target, aesthetic composition, depth-of-ﬁeld or lighting, the characterisation of camera motions has been far less addressed.
This pertains to speciﬁcs of real camera rigs (dollies, cranes) that physically limit the range of motions, and also the limited use of long camera sequences in movies with the exception of Steadicam sequence shots.
In addition, characteristics of camera motions in movies are strongly guided by the narrative intentions which need to be conveyed ( e.g. rhythm, excitation, or soothing atmosphere) that are difﬁcult to formalize.
In an attempt to transpose this knowledge to the tracking of targets in virtual environments, one can however derive a number of desirable cinematic characteristics such as visibility (avoiding occlusion of the tracked target and collisions with the environment), smoothness (avoiding jerkiness in trajectories) and continuity (avoiding large changes in viewing angles and distances to target).
In practice, however, these characteristics are often contradictory: avoiding a sudden occlusion requires a strong acceleration, or an abrupt change in angle.
Furthermore, the computational cost of evaluating visibility, continuity and smoothness along trajectories limits the possibility of evaluating many alternative camera motions.
Existing work have either addressed the problem using global motion planning techniques typically based on precomputed roadmaps [5, 10, 11], or local planning techniques using ray casting [12] and shadow maps for efﬁcient visibility computations [1, 2, 4].
While global motion planning techniques excel at ensuring visibility given their full prior knowledge of the scene, local planning techniques excel in handling strong dynamic changes in the environment.
The main bottleneck of both approaches remains the limited capacity in evaluating at run time the cinematic properties along a camera motion or in the local neighborhood of a camera position.
Ourapproach builds on the idea of performing a mixed local + global approach by exploiting a ﬁnite-time horizon that is large enough to perform a global planning, yet efﬁcient enough to react in real-time to sudden changes.
This sliding window exploits recent hardware raycasting techniques to enable the real-time evalu- ation of thousands of camera motions.
As such, our approach draws inspiration from Motion-Predictive Control techniques [13] by opti- mizing a ﬁnite time-horizon, only implementing the current timeslot and then repeating the process on the following time slots.
To implement this approach, we make the hypothesis that the target object is controlled by the user through interactive inputs.
Its motions and actions can therefore be predicted within a short time horizon h .
Our system comprises 2 main stages, illustrated in Figure 1.
In the ﬁrst stage, we predict the motion of the target over our given time horizon h by using the target’s current position (at time t i ) and the user inputs.
We then select an ideal camera position at time t
i + h and propose to deﬁne a camera animation space as a collection of smooth camera animations that link the current camera position (at time t i ), to the ideal camera location (at time t i + h ).
In the second stage, we perform an evaluation of the quality of the camera animations in this animation space by relying on hardware raycasting techniques and select the best camera animation.
In a way similar to Motion-Predictive Control [13], we then apply part of the camera animation and re-start the process at a low frequency (4 Hz) or when a change in the user inputs is detected.
Finally, to better adapt the camera animation space to the scene topology ( e.g. cluttered environments vs. open environments), we dynamically update a scaling factor on the animation space.
As a whole this process allows generating a continuous and smooth camera animation which enables the real-time tracking of a target object in fully dynamic and complex environments.
Our contributions are:
• the design of a camera animation space as a ﬁnite time horizon space in which to express a range of camera trajectories; • an efﬁcient evaluation technique using hardware ray casting; • a motion predictive control approach that exploits the cam- era animation space to generate real-time cinematic camera motions.
2 R ELATED W ORK
We narrow the scope of related work to real-time camera planning techniques.
For a broader view of camera control techniques in computer graphics, we refer the reader to [3].
Global camera planning
Global camera path-planning techniques build on well-known results from robotics such as probabilistic roadmaps, regular cell decomposition or Delaunay triangulation.
All have in common the computa- tion of a roadmap as a graph where nodes represent regions of the conﬁguration-free space (points, regular cells or other primitives), and edges represent collision-free links between the nodes.
Niewenhuisen and Overmars exploited probabilistic roadmaps (PRM) to automatically perform camera path queries within the graph struc- ture, to link given starting and ending camera conﬁgurations [10].
Different heuristics were used to smooth the camera trajectories and avoid sudden changes in position and camera angles.
Later, Oskam et al.
[11] proposed a visibility-aware roadmap by using a sphere- sampling of the conﬁguration-free space, and by precomputing the sphere-to-sphere visibility using stochastic ray-casting.
Lino et al.
[7] exploited spatial partitioning techniques to com- pute dynamically evolving volumes around targets.
The adjacency between the volumes was then exploited to dynamically create a roadmap through which camera paths were computed while accounting for visibility and viewpoint semantics along the path.
More recently, Jovane et al.
[5] exploited the topological representations of 3D environments to create topology-aware camera roadmaps that lower the complexity (compared to probablistic roadmaps) and enable the generation of different cinematic styles.
Yet, the cost of precomputing the roadmap together with the difﬁculty in dynamically updating it to account for changes limits the practical applicability of such techniques in strongly dynamic environments such as those met in computer games or storytelling applications.
Local camera planning
Local camera planning techniques rely on a limited knowledge of the environment.
By sampling the local neighborhood around the current camera location, such systems are able to take decisions as to were to move the camera next.
The decision is guided by the thorough evaluation of cinematic properties such as visibility, smoothness and continuity on the camera samples.
To reduce the computational cost in the evaluation of target’s visilibity, Halper et al.
[4] exploit hardware rendered shadow maps that compute potential visible sets.
Results are coupled with a hierarchical solver to handle other cinematic properties.
Later, Normand and Christie exploited slanted rendering frustums to compose spatial and tem- poral visibility for two targets over a small temporal window ( frames) [2].
Additional criteria were added in order to select the best move to perform at each frame and to balance between camera smoothness and camera reactivity.
Litteneker et al.
[8] proposed a local planning technique based on an active contour algorithm.
Hardware rendering was also exploited by Burg et al.
[1] who performed shadow map projections from the targets to the surface of the Toric manifold (a speciﬁc manifold space dedicated to camera control [6]).
The visibility information provided by the shadow maps was then exploited to move the camera on the surface of the Toric manifold while ensuring secondary visual properties.
Recently, for the speciﬁc case of drone cinematography, Nageli et al.
[9] built a non-linear model predictive contouring controller to jointly optimize 3D motion paths, the associated velocities and control inputs for an autonomous drone.
Our approach partly builds on the work of Nageli et al. , by borrowing the idea of a receding horizon process in which motion planning is performed for a large enough time horizon (few sec- onds).
The processed is repeated at a higher frequency to account for dynamic changes in the environment.
Rather than addressing the problem using a non-linear solver, we propose in our paper to exploit the hardware raycasting capacities of recent graphics cards to efﬁciently detect collisions and occlusion and evaluate visual properties of thousands of camera trajectories for each time horizon.
3 O VERVIEW
Our system aims at tracking in real-time a target object traveling through a dynamic 3d environment by generating series of smooth cinematic camera motions.
In the following, we will present the construction of our camera animation space (Section 4) and then detail the evaluation of camera animations using hardware ray casting (Section 5). We will then show how the camera animation space
H i
Time horizon for iteration i (between times t i and t
i + h )
B i ( t )
Target behavior (predicted position) at time t ∈ H
i
V i
Set of preferred viewpoints at time t
i
+ h
Q i
Camera animation space for horizon H
i
M i
Transform matrix of the camera animation space, for H
i q ij ( t ) 3D position in camera animation q ij ∈ Q i , at time t ∈ H
i q istart
Starting camera position.
q ij ( t i ) =
q istart , ∀ ( i , j ) q igoal
Goal camera position.
q ij ( t i + h ) =
q igoal ∈ V i , ∀ ( i , j ) ˙
[Equation]
t )
Tangent vector of a camera track at time t
D ij [Equation] t )
The camera view vector at time t ( x , y ) angle between two vectors x and y
G ( x , σ )
Gaussian decay, equals to e − x 2 / ( 2 σ 2 )
E ( x , λ )
Exponential decay, equals to e − x / λ can be dynamically recomputed to adapt to the characteristics of the scene topology (cluttered vs. open environments) and how this adaptation improves our results (Section 6).
4 C AMERA ANIMATION SPACE
We propose the design of a Camera Animation Space as a relative local frame deﬁned by an initial camera conﬁguration q start and ﬁnal camera conﬁguration q goal (see Figure 2).
This local space deﬁnes all the possible camera animations that link q start at time to q goal at time .
Our goal is to compute the optimal camera motion within this space considering a number of desired features on the trajectory .
We propose to follow a 3-step process: (i) anticipate the target’s behavior ( i.e. its next positions) within a given time horizon, (ii) choose a goal camera viewpoint from which to view the target at the end of the time horizon, and (iii) given this goal viewpoint, and the current one, build and evaluate the space of possible camera animations between them using our camera animation space.
4.1 Anticipating the target behavior
We here make a strong assumption that we can anticipate the next positions of the tracked target within a time horizon H i .
This is classical in character animation engines in order to decide which animations to trigger ( e.g. motion matching).
We consider H i begins at time t i and has a constant user-deﬁned duration of h seconds.
Moreover, we consider that the target’s behavior will be consistent over the whole horizon
H i (a behaviour being a motion among walking, running, turning). In our implementation, we consider the target as a rigid body ( e.g. a capsule) with a current speed, acceleration, and behavior. We then simulate the target’s motion over time horizon H i (avoiding collisions with obstacles) and store all simulated positions over time. .
With this anticipation, we account for the scene geometry which might inﬂuence future user inputs.
We then refer to the anticipated positions as the target simulated behavior, expressed in the form of a 3d animation curve B
i ( t ) with t ∈ H i (see Figure 3).
Note that one may use another technique to anticipate the target behavior.
a 3d animation curve over time, it will not change the overall workﬂow of our camera system.
4.2 Selecting a goal viewpoint
We now make a second assumption that the user deﬁnes a set of viewpoints to portray the target object.
By default, one might use a list of stereotypical viewpoints in movies such as 3-quarter front
Considering all viewpoints are in V , we pop viewpoints by order of priority.
We propose to stop as soon as a viewpoint is promising enough, i e. at time t
i + h
neither the target will be occluded from this viewpoint nor the camera will be in collision with the scene geometry.
We then refer to this selected viewpoint as the goal viewpoint q goal .
Knowing the current camera viewpoint q start (at time t i ) and this goal viewpoint q goal (at time t i + h ) we can deﬁne our camera
[Figure]
4.3 Sampling camera animations
Given the target behavior to track represented as a curve B i ( t ) and the two key viewpoints q start and q goal we propose to sample a large set of camera animations between the key viewpoints.
We will hereafter note this stochastic set of camera animations as Q i , and a sampled camera animation as q ij , where j is the sample index.
Two requirements should be considered on this sampled space: (i) sampled camera animations should be as-smooth-as-possible, i.e. with low jerk, and (ii) the sampled animation space should en- force continuity between successive horizons.
To do so, we propose to encode each sampled camera animation as a cubic spline curve on all 3 camera position parameters, as they offer C 3 continuity between key-viewpoints.
In practice, we make use of Hermite curves which eases the sampling by randomly selecting tangent vectors to the spline curve at start and end positions.
C 1 continuity between successive Hermite curve portions is enforced by aligning both positions and tangents at connecting positions.
In practice, we propose for each camera animation to complement the starting and the goal camera positions q start and q goal by two tangents, i.e. the camera velocities ˙ q start and ˙ q goal (ﬁgure 2).
To offer a good coverage of the whole animation space , we use a uniform sampling of these tangents in a sphere of radius r (in our tests, we used r = 5).
The number of sampled animations is left as a user-deﬁned parameter.
An evaluation of results for different values is provided in Section 7.2. [Figure] [Figure]
The frequent recomputation of the tangent sampling and camera path construction has two drawbacks: its computational expense and the lack of stability over time.
To avoid the recomputation, we propose to precompute a graph of uniformly sampled camera animations, in an orthonormal coordinate system (as illustrated in ﬁgure 2).
In this system, q start and q goal have coordinates ( 0 , 0 , 0 ) and ( 0 , 0 , 1 ) respectively.
Then, for any horizon H i , we apply a proper 4 × 4 transform matrix M i to align the graph onto the computed viewpoints q istart and q igoal .
It is worth noting that in M i the 3d translation, 3d rotation and the scaling on the z axis will lead this axis to match the vector ( q igoal − q istart ) .
Two parameters remain free: the scaling for the other two axes ( x and y ).
As a ﬁrst assumption we could use the same scaling as for z .
However, we will further explain how to choose a better scaling in section 6, in order to adapt the sampled space to the scene geometry.
5 E VALUATING CAMERA ANIMATIONS
In this ﬁrst stage, we have computed a set of camera animations Q i , that can portray the target objects’ behavior within time horizon H
i .
We now need to select one of these animations as the one to apply to the camera.
5.1 Evaluating camera animation quality
Our second stage is devoted to evaluating of the quality of all ani- mations and selecting the most promising one in an efﬁcient way.
In the following, we will ﬁrst detail our evaluation criteria, before focusing on how we perform this evaluation.
A camera animation that portrays the motions of a target object should follow a number of requirements, among which the most important are: avoid collisions with the scene and enforce visibility on the target object, while
Occluder
[Figure]
Fail Success
{+Success Fail2  3 0 0 013 6 
 [Figure]+} offering a smooth series of intermediate viewpoints to the viewer.
To evaluate how well these requirements are satisﬁed along a camera animation q ij , we propose to rely on a set of costs C k ( t ) ∈ [ 0 , 1 ] :
Occlusions and Collisions To evaluate how much the target object is occluded from a camera position q ij ( t ) , we rely on ray casting as recommended by [12].
We ﬁrst approximate the target’s geometry with a simple abstraction .
Second we sample a set of points s ∈ [ 0 , N ] on this abstraction, which we position at the object ’s anticipated position B i ( t ) .
Third, we launch a ray from the camera to each point s (see ﬁgure 4).
We note R s ( t ) the result of this ray cast.
We use the same ray to also evaluate if the camera is in collision ( i.e. inside another object of the scene), by setting its value as:
[Equation]
We distinguish a collision from a simple occlusion as follows.
By looking at the normal at the hit geometry, we know if the ray has hit a back face or a front face.
When the ray hits a back face, q ij ( t ) must be inside a geometry, hence we consider it as a camera collision.
Conversely, when the ray hits a front face, q ij ( t ) must be outside a geometry.
If the ray does not reach s , we consider s as occluded, otherwise we consider it as visible.
Knowing R s ( t ) , we deﬁne our collision and occlusion costs as:
[Equation]
and
[Equation]
c
[Equation]
t
[Equation]
In our tests, we used N = 20.
Minimizing visual changes A smooth camera motion is a motion that avoids sudden changes in visual properties (distance to target and angle to target).
We therefore propose an addition metric to evaluate how much the viewpoint changes over time.
We split this evaluation into two distinct costs: one on the camera view angle, and one on the distance to the target object.
Costs are evaluated for each time step δ t .
Let us denote D ij ( t ) the view vector connecting the target object to the camera computed as:
[Equation]
From this view vector, we deﬁne the view angle change as:
[Equation]
In a way similar, we propose to rely on a squared distance varia- tion, deﬁned as:
[Equation]
We then deﬁne a cost on this distance change which we further normalize as:
[Equation]
where E is an exponential decay function, for which we set parameter λ to 10 − 4 .
Preferred range of distances One side effect of the above costs is that for large distances, changes on the view angle and distance will be less penalized.
In turn, this will favor large camera animations.
It is worth noting that, in the same way, placing the camera too close to the target object is also not desired in general.
We should then penalize both behaviors.
To do so, we propose to introduce a last cost, aimed at favoring camera animations where the camera remains within a prescribed distance range [ d min , d max ] .
We formulate this cost as:
[Equation]
5.2 Selecting a camera animation
In a ﬁrst step, we deﬁne the total cost of a camera animation as a weighted sum of its single-criteria costs integrated over time:
[Equation]
where w k ∈ [ 0 , 1 ] is the weight of criterion k .
G is a Gaussian decay function, where we set standard deviation σ to the value of h / 4.
We also slightly tune the decay to converge towards 0 .
25 (instead of 0).
This way, we give a higher importance to the costs of the beginning of the animation, yet still considering the end.
Indeed, , our assumption is that the camera will only play the ﬁrst part of it (10% in our tests), while the remaining part still brings a long term information on what could be a good camera path.
We compute the total cost for any camera animation q ij ∈ Q i .
We hereafter refer to this total cost as C ij .
In a second step, we propose to choose the most promising cam- era animation for time horizon H
i , denoted as q i , as the one with minimum total cost, i.e. :
[Equation]
5.3 GPU-based evaluation
We have presented our evaluation metric on camera animations.
However, some costs are expensive to compute.
In particular, occlusion and collision testing requires to trace a large number of rays ( i.e. N rays, for many time steps, for hundreds of camera animations).
It is worth noting that many of our computations are independent and can therefore be performed in parallel.
In a similar way, the evaluation of a cost at time steps along a given animation are also independent.
Hence, we propose to cast our evaluation of single costs into a massively-parallel computation on GPU.
We design our system in a way the we only need to send the animation space (in orthonormal coordinate system) once to the GPU.
Then, when we need to reposition the camera animation space, for horizon H
i , we simply update the 4 × 4 transform matrix M i .
And, from this data, one can straightforwardly compute any camera position q ij ( t ) for any time t .
Second, for occlusion and collision computations, we propose to rely on the recent RTX technology allowing to perform real-time ray casting requests on GPU.
we run kernels per track .
Each kernel launches N rays, one per sample s picked onto the target object. .
The result of these computations are stored into a 2D texture (as shown in ﬁgure 5), where the texture coordinates u and v map to one time step t and one animation of index j , respectively.
Occlusion and collision costs are stored into two different channels.
Third, we rely on a compute shader to compute all other costs, and combine them with occlusion and collision costs.
This shader uses one kernel per camera animation.
It stores the total cost of all animations into a GPU buffer, ﬁnally sent back to CPU where we perform the selection step
[Figure]
Until now, we have considered a nominal situation where we evaluate the animation space and select one camera animation for one given time horizon H i .
We now need to consider two other requirements.
First, the camera should be animated to track the target object for an unknown duration, larger than h .
Changes in the target behavior may also occur, due to interactive user inputs.
Second, for any horizon H i , some camera animations could be in collision with the scene, or the target could be occluded.
This would prevent ﬁnding a proper animation to apply.
In other words, the space of potential camera animations should be inﬂuenced by the surrounding scene geometry.
Hereafter, we explain how we account for these requirements.
6.1 User inputs and interactive update
We here assume the camera is currently animated along a curve q i .
We then need to compute a new camera animation for a time horizon H i + 1 in two cases.
First, when the target’s behavior is changed . This event indeed of the currently played animation for future time steps.
Second, it appears reasonable to , collision and occlusion information The duration of these ﬁrst steps is speciﬁed as a user- deﬁned ratio of progress along animation q i .
In our tests we used a horizon length h = 5 seconds and a of 10%.
In turn, the new horizon generally starts at t i +
= t i + 0 .
1 h , while we set q i + 1 start =
q i ( t i + 1 ) .
When an update is required (behavior change, or ratio of progress reached), we recompute a new camera animation for the next horizon
H i + 1 : we select a new goal viewpoint ( i.e. q i + 1 goal ) and update the transform matrix ( i.e. M i + 1 ) to position the camera animation space
Q i + 1 . We then evaluate all camera animations in Q i + 1 .
Animation transitions .
To enforce continuity between ani- mation q i and the animation q i + 1 that is to be selected we rely on an additional cost designed to favor a smooth transition between consecutive animations.
This cost penalizes abrupt changes when transitioning between two camera animation curves.
Our idea is to penalize a wide angle between the tangent vector to camera animation
q i and the tangent vector to animation q
i + 1 j ∈ Q i + 1 , at connection time t i + 1 .
We write this cost as:
[Equation]
We then rewrite the selection of camera animation q i + 1 as:
[Equation]
where w i , i + 1 is the relative weight of the transition cost with regards to other costs .
6.2 Adapt to scene geometry
Different scene geometries obviously impose different constraints on the camera animations and a single sampled camera animation space may not enough to tackle all situations. For example, clut- tered environment such as corridors would ideally require dedicated samples.
In fact, our camera animation space should exhibit as few collisions and occlusions as possible, while still covering as much as possible the free space between the target behavior and the scene geometry. To address this problem, rather than recomputing a new sampling of camera trajectories dedicated to these speciﬁc situations, we propose a technique to dynamically adapt our camera animation space to the scene geometry.
To do so, while we evaluate the quality of camera animations for an horizon H i , we analyse how much collisions and occlusions occur.
This informs us if the free space is well covered or not.
We then propose to dynamically rescale the camera animation space to make it grow or shrink in the next time horizon H
i + 1 .
This rescaling applies when we update the transform matrix M i + 1 , and on the x and y axes only.
It is worth noting that the free space might not be symmetrical around the target behavior (as illustrated ﬁgurewhere the free space is larger on the left than on the right of the target).
The same applies to the free space above or below the target.
Consequently our idea is to compute four scale values, on all four directions {− x , + x , − y , + y } along the axis of the camera animation space.
For any camera position along a camera animation, we then apply either two of them, depending on the sign of the position’s x and y coordinates in the non-transformed animation space.
To compute this scaling we ﬁrst leverage the occlusions and collisions evaluation to store additional information: we count fails and successes along each axis.
We consider a launched ray along a camera animation ( i.e. from the camera position at a given time step) as a fail if it is marked as occluded or collided, and as a success if not.
Second, we store this information in height arrays: for each half-axis ( e.g. + x or − x ), we count successes in one array, and fails in another array.
We further discretize this half-axis by using a given resolution R and output two histograms of fails and successes (as illustrated in ﬁgure 7).
Note that R here deﬁnes the scale precision on each axis.
At last we use both histograms to compute the new scale to apply.
We compute the indices
i f
and i s of the medians of both arrays (fails and successes, respectively).
By comparing them, we deﬁne how much we should rescale animations along this half-axis.
If i s < i f , we consider that there are too many fails, and multiply the current scale by i f / R to shrink animations.
Otherwise, we consider the free space is not covered enough, and apply a passive inﬂation to the current scale.
The aim of this inﬂation is to help return to a maximum scale value, when the surrounding geometry allows for large camera animations.
7 I MPLEMENTATION AND R ESULTS 7.1 Implementation
We implemented our camera system within the Unity3D 2019 game engine.
We compute our visibility and occlusion textures through raytracing shaders provided with Unity’s integrated pipeline and perform our scores for all sampled animations and timesteps through
Unity Compute Shaders.
All our experimentations (detailed in sec- tion 7.2) have been performed on a laptop computer with a Intel
Core i9-9880H CPU @ 2.30GHz and a NVIDIA Quadro RTX 4000.
7.2 Results
We split our evaluation into three parts.
We ﬁrst validate our adaptive scale mechanism.
We then evaluate the robustness of our system, by comparing its performances when using a different number or set of reference camera animations.
We ﬁnally validate the ability of our system mixing local and global planning approaches to outperform a purely local camera planning system.
To do so, we compare results obtained with our system and the one of Burg et al.
[1], on the same test scenes.
Tovalidate our adaptive scale, we study its impact on the quality of the animation space.
For the other evaluations, we compare camera systems with regards to two main criteria: how much the camera maintains the visibility on the target object and how smooth camera motions are.
We compute visibility by launching rays onto the target object, and calculate the ratio of rays reaching the target.
A ratio of 1 (respectively 0) means that the target is fully visible (respectively fully occluded).
When relevant, we additionally provide statistics on the duration of partial occlusions.
We then compare the quality of camera motions through their time derivatives (speed, acceleration and jerk), which provide a good indication of motion smoothness.
Our comparisons have been performed within 4 different scenes (illustrated in the accompanying video).
We validated our system by using (i) a Toy example scene where the target is travelling through a maze containing several tight corridors with sharp turns, an open area inside a building, and a ramp.
We then performed [Figure] the comparisons with the technique of Burg et al.
[1], by using two static scenes and a dynamic scene, which the target goes through: (ii) a scene with a set of columns and a gate ( Columns+Gate ), (iii) a scene with set of small and large spheres ( Spheres ) and (iv) a fully dynamic scene with a set of randomly falling and rolling boxes, and a randomly sliding wall ( Dynamic ).
To provide fair comparisons, in the dynamic scene, the random motions of boxes and of the wall are the same for both scenes.
In addition, for all tests in a scene, we play a pre-recorded trajectory of the target avatar, but let the camera system run as if the avatar was interactively controlled by a user, to ensure motions are the same in all tests.
7.2.1 Impact of adaptive scale
We validate our adaptive scale (section 6.2) by comparing results obtained (i) when we compute and apply the adaptive scale on allhalf-axes ( − x , + x , − y , + y ), and (ii) when we simply apply the same scale as for the z axis (which we will call the naive scale technique).
We ran our tests by using the toy example scene.
For each technique, each time we evaluate a new set of camera animations, we output the new scale values and the ratio of fails on each half-axis.
In addition, we output and plot the mean cost of the 5 best animations in this set.
Results are presented in ﬁgure 8.
Figure 8a shows how much our mechanism tightens the animation space (compared to the naive scaling technique) when the avatar is entering corridors, and grows back to the same scale when the avatar reaches less cluttered areas ( e.g. in the open interior room, or the outdoor area).
As expected, our mechanism allows to adapt the scale on half-spaces in a non-symmetrical way.
As shown by ﬁgure 8b, with our adaptive mechanism, the scaled animation space also exhibits less fails than using the naive scale technique.
As shown by ﬁgure 8c, it allows ﬁnding animations with lower costs most of the time.
One exception is between 40s and 50s, where the camera conﬁguration isn’t the same because the scale is different.
In the naive case the camera is high above the character while in the adaptive case, the camera is closer to the ground, thus the scores is not relevant in this case because the two conﬁgurations are too different to be compared.
In the next evaluations, we consider that the adaptive scale mech- anism is always activated.
7.2.2 Robustness
We study the robustness of our system regarding our randomly generated camera animation space.
In a ﬁrst step, we evaluate how performances vary if we run our real-time system multiple times on the toy example scene.
We also consider two cases: (i) using the same seed for every run ( i.e. the same animation space is used), and (ii) using a new seed for every run ( i.e. a new animation space is randomly sampled for each run).
For each case, we sample a set of 2400 animations.
Results are presented in ﬁgure 9.
As illustrated, with as many sampled animations, all runs lead to very similar results both on the visibility enforcement and on the camera motion smoothness.
Differences are mainly due to variations in the actual framerate of the game engine, hence the rate at which the system takes new decisions.
In a second step, we evaluate how the size of the animation space ( i.e. the number of sampled animations) impacts performances.
We ran our system with 4 different sizes: 2400, 1600, 800 or 100 animations.
For each size, we performed 5 runs with random seed, and combined the results in ﬁgures 10, 11 and 12.
Figures show that lowering the size (at least until 800 animations) still delivers [Figure] good performances.
Our camera system is able to ﬁnd a series of camera animations maintaining enough visibility on the target object through smooth camera motions.
As we expected, for 100 animations, our system’s performances are poor: it becomes harder to ﬁnd animations with sufﬁcient visibility and ensuring smooth camera motions.
Obviously as we lower the number of camera animations, the distribution of tangents becomes very sparse, hence breaking our assumption of a uniform sampling.
If the sampled animation space does not cover enough the free space, it prevents the ﬁnding of qualitative animations.
7.2.3 Comparison to related work
We also compare our system mixing local and global planning approaches to a purely local camera planning system.
We have run our proposed camera system and the local camera plan- ning system of Burg et al.
[1] in 3 different scenes: two static scenes ( Columns+Gate and Spheres ) and a fully dynamic scene ( Dynamic ).
The Columns+Gate is the same as in [1] where the avatar is moving between some columns and go through a doorway.
In the Spheres scene, the avatar is travelling a scene ﬁlled with a large set of spheres, which makes it moderately challenging for the camera systems.
In the Dynamic scene, the avatar must go through a ﬂat area, where a set of boxes are randomly ﬂying, falling, rolling all over the place, and a wall is randomly sliding.
This makes it challenging for camera systems to anticipate the scene dynamics and ﬁnd occlusion-free and collision-free camera paths.
In our camera system, we used 2400 animations, the recomputa- tion rate is set to 0.25s and the adaptive scaling is on.
We present results of our tests in ﬁgures 13, 14, 15, 16, and 17.
We ﬁrst compare camera systems along their ability to enforce visibility on the target object (ﬁgure 13).
Our tests show that for [Figure] moderately challenging scenes, both lead to relatively good results.
Few occlusions occur.
However, for a more challenging scene ( Dynamic ), our system outperforms Burg et al. ’s system.
Even if occlusions may occur more often, the degree of occlusion is lower (ﬁgure 13b).
Moreover, for all 3 scenes, when partial occlusions occur, they are shorter when using our system (ﬁgure 13c).
This is explained by the fact that when no local solution exist, our system can still ﬁnd a locally occluded path respecting other constraints, and leading to a less occluded area.
This demonstrates our system’s ability to better anticipate occlusions especially in dynamic scenes.
Second, we compare the smoothness of camera motions in both camera systems.
Figure 14 presents the side-by-side distributions of speed, acceleration and jerk for each system.
We also provide the speed, acceleration and jerk along time in ﬁgures 15, 16, and 17.
One observation we make is that Burg et al.
’s system leads to lower camera speeds, as it restricts itself to simply following the avatar.
In our camera system, the camera is allowed to move faster, to bypass the avatar when visibility or another constraint may be poorly satisﬁed.
Yet, our system provides smoother motions ( i.e. less jerk).
One explanation is that local systems often need to steer the camera from local minima ( e.g. low visibility areas).
A side effect is that it may lead, for successive iterations, to an indecision on which direction the camera should take to reach better visibility.
In turn, this leads to frequent changes in camera acceleration (hence higher jerk).
Conversely, our system has a more global knowledge on the scene, allowing to more easily ﬁnd a better path, which avoids sacriﬁcing the smoothness of camera motions.
8 D ISCUSSION AND C ONCLUSION
Our system presents a number of limitations.
Despite the ability to evaluate thousands of trajectories, strongly cluttered environments 45  (a)         45  ( [Figure] [Figure]
[Figure] [Figure] remain challenging. As smoothness is enforced, visibility may be lost in speciﬁc cases and designing a techniques that could properly balance between the properties to handle speciﬁc cases need to be addressed.
Also while the dynamic scale adaptation does improve results by compressing the trajectories in different half spaces, low values in scales prevent the camera from larger motions where necessary.
A future work could consist in biasing the sampling in the animation space in order to adapt the space to typical local topologies of the 3D environment.
Despite the limitations, the proposed work improves over existing contributions by proving an efﬁcient camera tracking technique adapted to dynamic 3D environments and does not require heavy roadmap precomputations.