ChatCam: Empowering Camera Control through Conversational AI

Authors: xinhang  liu
yu-wing  tai
chi-keung  tang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-25 10:22:38

Abstract: Cinematographers adeptly capture the essence of the world, crafting compelling visual narratives through intricate camera movements. Witnessing the strides made by large language models in perceiving and interacting with the 3D world, this study explores their capability to control cameras with human language guidance. We introduce ChatCam, a system that navigates camera movements through conversations with users, mimicking a professional cinematographer's workflow. To achieve this, we propose CineGPT, a GPT-based autoregressive model for text-conditioned camera trajectory generation. We also develop an Anchor Determinator to ensure precise camera trajectory placement. ChatCam understands user requests and employs our proposed tools to generate trajectories, which can be used to render high-quality video footage on radiance field representations. Our experiments, including comparisons to state-of-the-art approaches and user studies, demonstrate our approach's ability to interpret and execute complex instructions for camera operation, showing promising applications in real-world production settings. Project page: https://xinhangliu.com/chatcam.

Main paper: 
 introduction: 
 Cinematographers skillfully capture the essence of the 3D world by maneuvering their cameras, creating an array of compelling visual narratives [8] . Achieving aesthetically pleasing results requires not only a deep understanding of scene elements and their interplay but also meticulous execution of techniques. Our proposed ChatCam assists users in generating desired camera trajectories through natural language interactions. The generated trajectories can be used to render videos on radiance field representations such as NeRF [52] or 3DGS [36] .propose an Anchor Determinator, a module that identifies relevant objects within the 3D scene to serve as anchors, ensuring correct trajectory placement based on user specifications. Our LLM agent parses compositional natural language queries into semantic concepts. With these parsed sub-queries as inputs, the agent then calls our proposed CineGPT and Anchor Determinator. It composes the final trajectory with the outputs from these tools, which can ultimately be used to render a video that fulfills the user's request.With comprehensive evaluations and comparisons to other state-of-the-art methods, our method exhibits a pronounced ability to interpret and execute complex instructions for camera operation. Our user studies further demonstrate its promising application prospects in actual production settings. In summary, this paper's contributions are as follows:• We introduce ChatCam, a system that, for the first time, enables users to operate cameras through natural language interactions. It simplifies sophisticated camera movements and reduces technical hurdles for creators. • We develop CineGPT for text-conditioned camera trajectory generation and an Anchor Determinator for precise camera trajectory placement. Our LLM agent understands users' requests and leverages our proposed tools to complete the task. • Extensive experiments demonstrate the effectiveness of our method, showing how AI can effectively collaborate with humans on complex tasks involving multiple elements such as language, images, 3D assets, and camera trajectories. 
related work: 
 Multimodal Language Models. Large-scale language models (LLMs) [9, 19, 17, 1, 70] enabled by extensive datasets and model size, have demonstrated surprising emerging abilities. The emergence of multimodal models [43, 42, 33] is captivating as they can process text alongside other modalities such as images [25] , audio [27] , and videos [78] . Some unified models can perceive inputs and generate outputs in various combinations of text, images, videos, and audio [51, 68, 77, 86] . LLMs hold the potential to act as agents [76, 80, 62] , allowing them to be driven by goals, reason about their objectives, devise plans, utilize tools, and interact with and gather feedback from the environment. Our proposed method involves multiple modalities including language, images, 3D fields, and camera trajectories, and utilizes LLMs as agents to assist users in operating cameras.Radiance Field Representations. Utilizing continuous 3D fields modeled by MLPs and volumetric rendering, Neural Radiance Fields (NeRFs) [52] achieved breakthrough for novel view synthesis.From outside the window, enter the bedroom and explore its layout following an S-shaped path.The user is requesting a camera trajectory, starting from a location outside the window and then covering the bedroom's layout following an S-shaped path.Reasoning: I need to find a starting point outside the window, then move through the window into the room and capture the room using an S-shaped path. Additionally, I should use a smooth panning speed to better showcase the room's layout.1. Call Anchor Determinator with "capture the bedroom from outside the window".CineGPT with "pan straight forward, smooth panning speed". 
call: 
 CineGPT with "an S-shaped path, smooth panning speed".4. Connect output trajectories from steps 2 and 3, ensuring they pass through the anchor point determined in step 1.5. Ask the user for further clarification if needed.6. Render a video using the generated camera trajectory. Subsequent research has emerged to improve NeRFs and broaden their applications [69] , including enhancing rendering quality [5, 6, 7? , 47] , modeling dynamic scenes [84, 55, 58, 71, 74, 10, 22, 2? ], improving computational efficiency [83, 23] , and facilitating 3D scene editing [45, 84, 73, 34, 46, 40] .Replacing the deep MLPs with a feature voxel grid network has demonstrated enhancement in both training and inference speed [13, 66, 24, 54] . More recently, 3D Gaussian Splatting [36] has further advanced visual quality and rendering efficiency. Compared to traditional 3D representations, radiance field representations offer superior photorealistic rendering quality, therefore, this study focuses on camera manipulation upon mainstream radiance field representations such as NeRF or 3DGS.3D Scene Understanding. Early methods for 3D semantic understanding [32, 67, 79, 15] primarily focused on the closed-set segmentation of point clouds or voxels. NeRF's capability to integrate information from multiple viewpoints has spurred its application in 3D semantic segmentation [88, 20, 46, 53, 65, 26, 60, 31, 48, 49, 21] . Among these, [40, 37, 12] combine image embeddings from effective 2D image feature extractors [41, 11, 59, 39] to achieve language-guided object localization, segmentation, and editing. [21] proposes semantic anisotropic Gaussians to simultaneously estimate geometry, appearance, and semantics in a single feed-forward pass. Another line of research integrates 3D with language models for tasks such as 3D question answering [3] , localization [14, 57, 81] , and captioning [16] . Additionally, [29, 30, 87] propose 3D foundation models to handle various perception, reasoning, and action tasks in 3D environments. However, the AI-assisted operation of cameras within 3D scenes remains an unexplored area.Trajectory Control and Prediction. Controlling the trajectories of objects and cameras is crucial to advance current video generation approaches for practical usage. TC4D [4] incorporates trajectory control for 4D scene generation with multiple dynamic objects. Direct-a-Video [82] , MotionCtrl [75] , and CameraCtrl [28] manage camera pose during video generation; however, they are either limited to basic types or necessitate fine-tuning of the video diffusion model. Moreover, these approaches require user-provided trajectories, whereas we, for the first time, generate camera trajectories conditioned on text.Figure 2 provides an overview of our method's pipeline. ChatCam analyzes the user's camera operation instruction and devises a plan to generate a trajectory using our proposed CineGPT and Anchor Determinator. Finally, an AI agent utilizes the outputs from these tools to compose the complete trajectory.To enable text-conditioned trajectory generation, we collect a text-trajectory dataset and introduce CineGPT, a GPT-based autoregressive model integrating language and camera trajectories. Illustrated in Figure 3 (a), our method quantizes camera trajectories into a sequence of trajectory tokens using a trajectory tokenizer. Subsequently, a multi-modal transformer decoder is employed to convert input tokens into output tokens. Upon training, our model adeptly generates token sequences based on user-provided text prompts. These sequences are then de-quantized to reconstruct the camera trajectory.Camera Trajectory Parameterization. For each single frame, our camera parameters include rotation R ∈ R 3×3 , translation t ∈ R 3 , and intrinsic parameters K ∈ R 3×3 . We further convert the rotation matrix R into the S 2 × S 2 space [89] to facilitate computational efficiency and simplify the optimization process. The total M -frame camera trajectory is formulated as:EQUATIONTo additionally model the velocity of camera movement, we introduce a global parameter t representing the total duration. Consequently, the instantaneous velocity of each frame can be approximated by the relative translation and rotation to the previnous frame over unit time.Text-Trajectory Dataset. Given the scarcity of readily available data on camera operations, we manually constructed approximately 1000 camera trajectories using Blender [18] . These trajectories encompass a diverse range of movements, including various combinations of translations, rotations, focal lengths, and velocities. Each trajectory is accompanied by a human language description detailing the corresponding movements. This dataset spans various scenarios, capturing both simple pan-tilt-zoom motions and more complex trajectories mimicking real-world scenarios.Trajectory Tokenizer. We leverage a trajectory tokenizer based on the Vector Quantized Variational Autoencoders (VQ-VAE) architecture [72] to represent camera trajectories as discrete tokens. Our trajectory tokenizer consists of an encoder E and a decoder D. Given an M -frame camera trajectoryc 1:M = {c i } M i=1 , the encoder E encodes it into L trajectory tokens z 1:L = {z i } L i=1, where L = M/l and l is the temporal downsampling rate. The decoder D then decodes z 1:L back into the trajectory ĉ1:M = {ĉ i } M i=1 . Specifically, the encoder E first encodes frame-wise camera parameters c 1:M into a latent vector ẑ1:L = E(c 1:M ), by performing 1D convolutions along the time dimension. We then transform ẑ1:L into a collection of codebook entries z through discrete quantization. The learnable codebook Z = {z i } K i=1 consists of K latent embedding vectors, each with dimension d. The quantization process Q(•) replaces each row vector with its nearest codebook entry, as follows:EQUATIONwhere || • || 2 denotes the Euclidean distance. After quantization, the decoder projects z 1:L back to the trajectory space as the reconstructed trajectory ĉ1:M = D(z 1:L ). In addition to the reconstruction loss, we adopt embedding loss and commitment loss similar to those proposed in [85] to train our trajectory tokenizer. With a trained trajectory tokenizer, a camera trajectory c 1:M can be mapped to a sequence of trajectory tokens z 1:L , facilitating the joint representation of camera trajectory and natural language for text-conditioned trajectory generation.Cross-Modal Transformer. We utilize a cross-modal transformer decoder to generate output tokens from input tokens, which may consist of text tokens, trajectory tokens, or a combination of both. These output tokens are subsequently converted into the target space. To train our decoder-only transformer, we denote our source tokens as X s = {x i s } Ns i=1 and target tokens as X t = {x i t } Nt i=1 . We feed source tokens into it to predict the probability distribution of the next potential token at each step p θ (x t |x s ) = i p θ (x i t |x <i t , x s ). The objective function is formulated as:EQUATIONBy optimizing this objective, we aim to equip CineGPT with the ability to capture intricate patterns and relationships within the data distribution. We then fine-tune CineGPT on supervised trajectorylanguage translation leveraging our paired text-trajectory dataset, where the input for this stage can either be a camera trajectory or a text description, while the target is the opposite modality. During inference, CineGPT can generate camera trajectories solely from textual descriptions as inputs.While CineGPT enables text-conditioned trajectory generation, its generation process solely focuses on determining the camera's movements, without contextual connection to specific scenes. Consequently, CineGPT alone cannot effectively handle user prompts that involve object-centric descriptions, such as directives like "directly above the Sydney Opera House". In this light, we bridge trajectory generation with each underlying scene with "anchors" serving as reference points within the scene to achieve more accurate placement of trajectories, as illustrated in Figure 3 (b) .Our anchor determination procedure takes natural language descriptions of an image as input. This procedure identifies a set of camera parameters that can render an image that best matches the given description. Current 3D visual grounding approaches [57, 81] typically entail learning a 3D feature field [40, 37] and localizing objects within the scene, which often results in high computational costs. In contrast, our anchor determinator adopts a different strategy. Initially, it selects the input image that best matches the given text description as an initial anchor. Subsequently, an anchor refinement process is employed to iteratively improve upon this initial anchor, ultimately yielding the final anchor. This approach offers a more efficient alternative to traditional methods, reducing computational overhead while still achieving accurate scene anchoring.Initial Anchor Selector. Since our method leverages radiance field representations to render videos, we naturally have access to the input images for training the 3D scene representations. We utilize an initial anchor selector based on CLIP [59] to choose the image from these input images that best matches the text prompt. To be specific, for i-th input image I i , we extract their CLIP image features and convert the text prompt T into a CLIP text feature. Next, we compute the cosine similarity between the CLIP text feature vector and each of the CLIP image feature vectors. We select the best matching image with the highest cosine similarity score as the initial anchor. This can be formulated as:EQUATIONwhere f image (•) and f text (•) represent the image and text feature extractor, respectively.Anchor Refinement. Using the camera parameters c anchor associated with the selected image as initialization, we further minimize the following objective to obtain the final anchor camera parameters:EQUATIONwhere R(•) is the rendering function and c is initialized with c anchor . The optimization of c is performed using gradient descent, with the update rule given by:EQUATIONwhere η is the learning rate. The optimization typically achieves convergence within 100 to 1000 steps. This refinement process ensures that the camera parameters are adjusted to better match the text prompts, handling cases where the initial input images do not align well with the prompts.With our proposed CineGPT and anchor determination, a large language model acts as an agent to interpret the user's requests, generates a plan to use various tools, and composes a final camera trajectory. We adopt GPT-4 [1] to interpret users' natural language inputs and subsequently produce trajectory prompts. Specifically, we use a carefully designed prompt to instruct the LLM agent to reason about the user's requirements and devise a plan consisting of the following steps: 1) Break down the complex text query into sub-tasks that CineGPT and the Anchor Determinator can effectively handle. 2) Use these tools to generate atomic trajectories and determine anchor points. 3) Compose the final trajectory by concatenating atomic trajectories and ensuring they pass through the anchors.Observing, Reasoning, and Planning. Research indicates that LLMs can be prompted to decompose complex goals into sub-tasks, essentially thinking step-by-step [76] . As illustrated in Figure 2 , we begin by instructing the agent to describe its observations, providing a summary of the current situation. The agent then uses this summary to reason and develop a mental scratchpad for highlevel planning. Finally, it outlines specific steps to achieve the overarching goal of generating the user-required camera trajectory.Utilization of Proposed Tools. We inform our agent of the expected input and output format, i.e., the APIs, of our proposed CineGPT and Anchor Determinator, and instruct the agent to interact with them following the given format. In its outlined specific steps to generate the user-required camera trajectory, it first calls CineGPT and Anchor Determinator to obtain atomic trajectories and anchor points, respectively. Note that both tools can be called multiple times, and multiple atomic trajectories can later be concatenated into final trajectories that pass through all anchor points correctly.Here we explain how to combine atomic trajectories from CineGPT with anchor points to form the final trajectory. The agent first decides the role of the anchors in the ultimate trajectory, either as a starting point or an ending point of some atomic trajectory. Then affine transformations are applied to the respective atomic trajectories to ensure that their starting or ending points align with the anchor points. For the remaining atomic trajectories not controlled by anchor points, affine transformations are applied to make the endpoint of the previous trajectory align with the starting point of the subsequent trajectory. 
experiments: 
 We assess the performance of our proposed ChatCam for human language-guided camera operation across a series of challenging scenarios. Through ablation studies, we provide empirical evidence ofOpera House and roll the camera.Do a dolly zoom focusing on the chandelier.Pan the camera from left to right along the piano, then turn to look at the bicycle on the right.Figure 4 : Qualitative results on indoor and outdoor scenes. Visualizations of our generated trajectories from input text descriptions and the frames in the final rendered video. Our method is capable of understanding and executing instructions and providing correct translations, rotations, and camera focal lengths. Additionally, our method can comprehend more specialized terms such as "dolly zoom".the effectiveness of its fundamental components. We kindly refer the reader to our supplementary material for additional experimental results, including rendered videos.Implementation Details. We implement our approach using PyTorch [56] and conduct all the training and inference on a single NVIDIA RTX 4090 GPU with 24 GB RAM. The trajectory tokenizer has a codebook with K = 256 latent embedding vectors, each with dimension d = 256.The temporal downsampling rate of the trajectory encoder is l = 4. Our cross-modal transformer decoder consists of 24 layers, with attention mechanisms employing an inner dimensionality of 64.Sweep across the boy in black, the keyboard, and the boy in white, then zoom out to frame the boys and the white guitar together.Starting from the man playing cards, pan the camera along the corridor, then back up to the end and look to the right. The remaining sub-layers and embeddings have a dimensionality of 256. We train CineGPT using the Adam optimizer [38] with an initial learning rate of 0.0001. It takes approximately 30 hours to converge. Our anchor determination utilizes CLIP [59] with a ViT-B/32 Transformer architecture. The learning rate of anchor refinement is 0.002. By default, we use GPT-4 [1] as our LLM agent, and its prompt will be released with our codebase. We render final videos using 3DGS [36] as the 3D representation.Tested Scenes. We tested our method on scenes from a series of datasets suitable for 3D reconstruction with radiance field representations, including: (i) mip-NeRF 360 [6] , a real dataset with indoor and outdoor scenes. (ii) OMMO [50] , a real dataset with large-scale outdoor scenes. (iii) Hypersim [61] , a synthetic dataset for indoor scenes. (iv) MannequinChallenge [44] , a real dataset for human-centric scenes. If camera poses associated with images were not provided, we used COLMAP [63] for camera pose estimation. For each scene, we reconstructed using all available images without train-test splitting.Baselines. As the first method to enable human language-guided camera operation, there is no established direct baseline for comparison. Therefore, we adopt 3D understanding approaches based on radiance field representations to let the LLM agent attempt to select a series of images corresponding to the input text from input images and interpolate their camera poses to construct camera trajectories. These methods include LERF [37] , utilizing CLIP embeddings, and SA3D [12] , utilizing SAM embeddings.Evaluation Metrics. To evaluate the accuracy of the generated trajectories, we manually construct ground truth trajectories and compute the mean squared errors (MSEs) of translations and rotations relative to them. Additionally, we conduct a user study to evaluate the rendered videos using generated camera trajectories, where users are asked to select the video with the best visual quality and best alignment with the input text.Facing the piano, pull the camera back, then glance over at the TV on the left, and back to the piano.Figure 6 : Qualitative comparisons. Our approach avoids moving the camera to unreasonable positions such as inside objects, obtaining videos with better visual effects, and aligning best with input texts. As shown in Figure 4 , our method demonstrates the ability to understand and execute camera operation instructions on a range of complex indoor and outdoor scenes, giving appropriate translation, rotation, and focal length. Our method also understands more technical terms such as dolly zoom, which creates a special visual effect by zooming the camera out while adjusting the focus. In Figure 5 we further showcase the qualitative results of our method in human-centric scenes. Our method can correctly handle user instructions about specific people and create correct and vivid visual effects.Comparisons. In Figure 6 we qualitatively compare our method with LLM agents utilizing SA3D or LERF to locate target objects. The baselines do simple interpolation of keyframes because they have no knowledge about camera trajectories and tend to move the camera to unreasonable spots (such as entering an object). Therefore, the video rendered by baselines contains artifacts and is not correctly consistent with the input text. However, our method achieves better visual quality and alignment with input texts. Quantitative comparisons in Table 1 further prove that our method has better performance and is preferred by users.Ablation Study. We present our ablation study in Table 1 . We evaluate the performance of our method using different LLMs as agents. Our approach achieved the best accuracy using GPT-4 [1] as the agent, better than GPT-3 [9] and LLaMA-2 [70] .Without our proposed anchor determination, our method cannot correctly place trajectories within 3D scenes, thereby being less accurate than our full model. 
conclusion: 
 This paper presents ChatCam, a system designed for camera operation through natural language interactions. By introducing CineGPT, we bridge the gap between human language guidance and camera control, achieving text-conditioned trajectory generation. Our proposed anchor determination procedure further ensures precise camera trajectory placement. Our LLM agent comprehends users' requests and effectively utilizes our proposed tools to compose the final trajectory. Through extensive experiments, we demonstrate the effectiveness of ChatCam, showcasing its ability to collaborate with humans on complex tasks involving language, images, 3D assets, and camera trajectories. ChatCam has the potential to simplify camera movements and reduce technical barriers for creators.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: We will release the data and code upon acceptance.Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:Justification: Our paper includes details about its user study. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: This supplementary document presents additional qualitative results and discusses the limitations and societal impacts of our proposed approach.For better visualization of our reconstruction results, we create a set of video visualizations. We highly recommend to watch supplementary_video.mp4 for more results.Leap from the front of bulldozer to the back. Start from capturing everybody, then move closer to the man wearing the red shirt.Facing the piano, pull the camera back, then glance over at the TV on the left, and back to the piano. We present additional qualitative results in As the first AI-assisted system for language-guided camera operation, our method relies on LLMs as agents, and therefore its efficiency depends on LLMs. With the rapid development of the community, this limitation may be alleviated.Our current results are limited to static scenes due to the limited availability of high-quality 4D Dynamic NeRF/3DGS data. Extending our approach to a dynamic scene would be straightforward by introducing a timestamp in the Anchor Determinator. We leave this as one of our future work.Our approach has great potential to help creators in industries such as television, movies, games, etc. reduce their burden by simplifying the learning costs of utilizing 3D assets like radiance field reconstructions. This allows content creators to focus on their creations. We must also admit that as existing 3D assets become more and more abundant, it is inevitable that there will be harmful content in them, and our method may contribute to the creation of harmful content. We encourage the community to play wisely with ChatCam.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 


Decision: This paper proposes a novel method for camera trajectory via the language model. Reviewers have questions and doubts mainly about the experiments including the baseline and data being used.  The author did a good job of addressing the concerns of the reviewer with detailed explanations and clarifications. Although one reviewer is still concerned about the experiment setting, the overall rating from the rest of the three reviewers is positive. Therefore, I suggest to accept this paper as a poster.

Reviews: review: paper_summary: paper_summary: The paper proposes a method to generate camera trajectories from user prompts. The idea is to pass on the prompt to an LLM, find a starting anchor location (searched and refined using initial images used to construct the radiance field) and then use CineGPT (a cross-model transformer) trained for the next token prediction on quantized camera trajectories. Compared with some minimal baselines, the proposed approach attains favourable results in terms of MSE errors (on translation and rotation) and user-reviewed visual quality and alignment metrics.
strength_weakness: strength_weakness: - Baselines could be better; I feel the comparisons were extremely weak

- Many details in the paper are missing, making it difficult to comprehend the approach fully. The major misses are the details on the dataset construction, the proper explanation of baselines, and the LLM prompts. I give more detailed questions on the same in the next section.

- related work can be strengthened as well. They can cut a bit on the literature of radiance fields and 3D scene understanding and probably  add more on trajectory learning/optimization
questions: questions: - It appears a path prediction network might do well in this case. What is eventually needed is to select a set of anchor locations on the path and then interpolate between them. It is not clear which interpolation algorithm was used for the baselines in the paper, which I believe is a crucial detail. Also, please bring more insights on which aspect of the baselines actually fails, is it the keypoint/keylocation selection or the interpolation.

- I would argue for replacing CineGPT with a rule-based optimizer or an interpolation algorithm. Since it does not observe visual modality, why is it not possible to do it through rule-based system (quantizing key kinds of camera movements).

- It is not clear to me, how to select the extent of camera movement. For example, in Figure 2, after determining the initial anchor (outside the window), CineGPT is called to pan straight forward. What aspect of the algorithm decides how long this pan should go? What stops it from bumping into walls, if the visual modality is not seen or explicit start-end points are not given?

- It is not clear how the dataset was constructed. Was it done in scenes with objects/people present in the CG? Who collected the dataset, and what instructions were given to the collector? How the collection was planned? What kind of camera movements were included? All these details are extremely crucial. 

- There are open-source blender movies (e.g. https://spring-benchmark.org/) which are utilized in several benchmarks. Would it be a good idea to exploit these in the dataset construction? 

- I would argue for an ordered keypoint prediction network and then spline interpolation between them. It would be a well designed meaningful baseline in this case. 

- Not sure if clip can efficiently perform the task of location selection, would like to hear more from the users on the same

- Not fully clear, why would the method generalize to out-of-domain objects/places (for example, an opera house in this case. was it included in the train set?). That reduces the confidence on the final presented output. 

- It would be useful to share the used prompt into the author's response, which would help the reviewers to try it out themselves and observe its limitations

- Discussion on some works on camera trajectories would be useful. Professional camera paths are often composed of constant, linear and parabolic segments [A][B]? The static trajectories also play a key role in this? Did the authors think about the importance of static trajectories and did they include it in the dataset?

- The final trajectory composition step appears to be non trivial to me and not fully clear at this stage. It looks like a hack right now.


[A] Grundmann et al. Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths. CVPR 2011
[B] Gandhi et al. Multi-Clip Video Editing from a Single Viewpoint. CVMP 2014
limitations: limitations: The authors did not discuss the limitations properly. I do not see an explicit limitations section.
paper_summary: paper_summary: The paper proposes a method for generating camera trajectories for rendering a 3D scene, conditioned on natural language. This problem statement is novel and original, very useful, and the paper demonstrates convincing results. The method uses two components: a language-conditioned camera trajectory generator, and a language-conditioned camera anchor generator. An LLM takes a high-level language query as input and creates a plan based on these two components.
strength_weakness: strength_weakness: The paper fails to demonstrate the planning output of the LLM. Is Fig. 2 an actual output of the method, or just an illustration? I would expect to see many more results of the actual plan used to generate the trajectories. It is very difficult to understand the workings of the method without these. 

The method seems far from reproducible. Sec 3.3 talks about several important components of the method, but only at a very high-level that would not enable anyone to reproduce the results. The details of the finetuning proposed in L142-146 are also unclear. 

The quantitative evaluations rely on ground truth trajectories. Shouldn't there be many possible GT trajectories consistent with a language query? How would this diversity impact the evaluations? The same holds true for output of the system. Can the method generate diverse outputs? If so, the paper should include such results, and ideally, should also update the metrics to reflect this. 

In the output in Fig. 2, there does not seem to be an end anchor point. How would the method know the extent or volume of space the S-shaped path should cover? Are there many cases where the volume of the space is undetermined based on the plan? 

Comparisons to baselines should also be included as videos, especially as the paper talks about their trajectories.
questions: questions: What are the limitations of the approach? Does the LLM fail to produce reasonable plans at times? When does that happen? Is the scale of the trajectories always fully determined? Can you comment more on the failure cases?
limitations: limitations: Limitations are not mentioned
paper_summary: paper_summary: This paper propose ChatCam, a pipeline that utilizes GPT to translate the natural language into professional camera trajectory, which enhances the video production process for common users.
strength_weakness: strength_weakness: The design of this paper is clever and makes good use of the advantages of LLM. But I still has some concerns. If the authors can address my concerns, I would change my rate.
1. In the planning part of Figure 2, how can you ensure there is no collision with 3D scenes in every sub-step, such as “an S-shaped path, smooth panning speed”. If the  structure of the room is complicated, it is easy to collide with the scenes.
2. Is natural language really the best way for users to interact？Maybe user can import the 3D GS of scenes into Unreal Engine, where Luma AI provide this plugin (https://www.unrealengine.com/marketplace/en-US/product/luma-ai), and generate the camera trajectory by adding camera key points in the 3D space by UI interface, with position, rotation, camera intrinsics. The frames between key points can be obtained by using bilinear interpolation. You can preview whether collision has occurred in real time, and you can also adjust camera parameters by drag. (https://dev.epicgames.com/documentation/en-us/unreal-engine/creating-camera-cuts-using-sequencer-in-unreal-engine)
3. Can you show more complicated camera trajectories, such as dolly. The “straight forward and roll”, “S-shaped path”, “from left to right”,  which just need coarse control, seems can be easily obtained by the item 2 I propose above (UE).
questions: questions: If the generated trajectory is not satisfactory or contains collision, how can user interact with the ChatCam? Whether the pipeline can modify the generated trajectory?
limitations: limitations: The collision in the 3D scenes is not explicitly and hardly constrained in this pipeline.
paper_summary: paper_summary: This paper introduces ChatCam, a system that enables camera operation via natural language interactions. This system has two key components. CineGPT is proposed for text-conditioned trajectory generation and an Anchor Determinator for precise camera trajectory placement. Experimental results illustrate ChatCam’s effectiveness in text-conditioned trajectory generation and show the potential to simplify camera movements and lower technical barriers for creators.
strength_weakness: strength_weakness: The system is only verified on several limited 3D static scenes, and the generated trajectory is relatively short. 

Some components are not fully verified. For example, what’s the effect of the text prompt “smooth panning speed” in Fig. 3? How does this text prompt affect the trajectory? 

Since the method was only evaluated on a small dataset, and the trajectory seems short, it's not clear whether the generations overfit to a small set of text prompts. 

Some experimental details are unclear.
1.	How about the training dataset? L114 suggests there are 1000 trajectories to train the trajectory generation. I would suggest introducing the training dataset in detail in Sec 4.
2.	What’s the time cost of Anchor Determination in inference?
questions: questions: See limitations.

1. Reference information inaccurate: [32,34], NeurIPS 2023 paper, not NeurIPS 2024

2. For the same text prompt, ChatCam is expected to generate different trajectories for different scenes. A verification is interesting to illustrate the effectiveness of an Anchor Determinator.

3. How long is the trajectory sequence?

4. It’s not clear whether the approach works for dynamic scenes, but it seems the Anchor Determinator does not model the temporal relations.
limitations: limitations: Yes

Comments: title: Thanks for your response
On rule based trajectories
Thank you for your response
Thanks for your response
Thanks for your response
Thanks for your response
Facilitating the author-reviewer discussion
Thanks for your response
Confusion about the Figure C.
Trajectory Composition
Conversation Example
LLM Prompt (2/2)
LLM Prompt （1/2）
comment: Thank you for your feedback!

Your suggestion of "creating a dictionary of camera movements and giving them names (S-shaped, U-turn)" aligns with our understanding and implementation of the rule-based baseline during the rebuttal. However, as shown in the attached PDF and discussed with Reviewer **Shd5**, the results from this approach do not perform as well as our proposed method.

This rule-based approach relies on a predefined dictionary of camera movements, which is inherently **finite**, whereas our GPT-based text-to-trajectory translation model can handle theoretically infinite input prompts.

Additionally, we want to remind the reviewer that the rule-based approach cannot determine **anchor points** independently. The accurate determination of anchor points is a key part of our technical contribution. Moreover, the rule-based method lacks the ability to interact with users to modify or improve results, which is another significant advantage of our proposed approach.

Regarding your concerns about the reliability of our final results, we respectfully disagree with the assertion that there are "too many stochastic components" in our approach. We welcome the reviewers to point out specific stochastic elements in our method so we can engage in further discussion.

Before receiving your feedback shortly before the discussion period ends, we had already provided the LLM prompt as per your suggestion, allowing the conversational process in our method to be fully reproducible. **We encourage reviewers to try it out.** Additionally, any aspects of the final results that the user finds unsatisfactory can be adjusted through further interaction. Therefore, we believe that our results are reliable, especially when compared to rule-based approaches.
- As you created the trajectories in the dataset, in similar way you could create a dictionary of camera movements and give them a name (S-shaped, U-turn). Add few parameters to control and then use them across anchor points. Seems doable to me. Optionally, apply some optimization to post-process (example: L1 norm optimization or smoothing)

- There are too many stochastic components and the reliability of the final result is doubtful

I am still not convinced and I am staying with my original rating.
Thank you for the response and we will revise the submission according to the reviewers' concerns.
Thanks for the response. Most of my concerns have been solved.
Thank you for your response! We greatly appreciate the valuable discussion and will revise our work based on the reviewers' suggestions.
After reading other reviewers' comments and authors' response, most of my concerns are addressed. Conversation Example shows how to handle the collision by language in an iterative way, which is user-friendly. In the response to kHgE, they prove the efficiency of CineGPT compared to the interpolation and rule-based methods, which means this method requires fewer anchor points than existing tools. I hope authors could add these experiments and discussions in the final version. I would change my rate to borderline accept.
Thanks for your response. My concerns have been addressed. It's good to add the ablation of the CineGPT and rule-based/interpolation method in the final version to show the key design of your pipeline.
Thank you for your response and for appreciating our research! We will mention these issues and include the experimental results in the revised paper. We will also incorporate the reviewers' other suggestions to further strengthen our work.
Thank you for your detailed feedback!

Based on your comments, we believe you agree that the entire ChatCam pipeline is efficient. This reinforces the significance of our study on conservation AI in camera control.

We appreciate your suggestion to compare CineGPT with a rule-based approach while keeping the anchors the same, as this directly demonstrates the effectiveness of CineGPT. In the revised version of our paper, we will include both qualitative and quantitative results from this experiment.

Since we are unable to modify the PDF at this stage, we will verbally describe the results of the rule-based/interpolation method in this scenario: with accurate anchor points, the rule-based approach successfully goes through the tunnel with the "move forward" prompt. However, it is also important to emphasize that the accurate determination of anchor points is not trivial and is a significant part of our contribution.

Furthermore, the performance of the rule-based approach depends on the rule set, while the number of possible rules is theoretically infinite. If the user input includes more complex instructions beyond the predefined rule set, the rule-based method cannot perform effectively.

We hope this can address your remaining concern regarding whether CineGPT could be replaced by a rule-based method. We would be happy to engage in further discussion on this or any other aspect.
I appreciate the efficiency of your whole pipeline (user input -> LLM agent -> Anchor Determinator -> CineGPT). But while comparing the CineGPT with the rule-based/interpolation method, I think you should use the anchors generated from LLM agent and Anchor Determinator as the input of the rule-based and interpolation method, which can directly show the efficiency of the CineGPT. 

When the anchors are accurate, I am curious about the differences between CineGPT and the rule-based/interpolation method. For "move forward", I think it is easy to generate a straight trajectory based on rules and interpolation and adjust the scale to fit the start and end anchors.

I am not sure if the specific object-level prompts are useful for CineGPT, according to "Since our dataset is not attached to any objects or places, suggested movies cannot be directly used in the dataset construction. ". Is there difference between the prompt "move forward through the tunnel" and "move forward along the road". I think the key of success is the spatial accuracy of the anchors.
Thanks for the response. It addresses my concerns. The revised paper should clearly mention the issues with defining the scale of the trajectories and no collision handling, and show corresponding results.
Dear Reviewer of the submission 684,

Thank you for your time and effort so far in reviewing this paper. 

The author submitted the rebuttal to your review, and the author-review discussion period will end tomorrow (13th August). It would be great if you could check if the rebuttal addresses your concern or not. Essentially, we have mixed reviews on this paper and need your input. Thank you in advance.

Best regards,
Your AC of submission 684
Thank you for reading our rebuttal and for your question. 

In our implementation, we replaced the entire system (CineGPT + Anchor Determinator + LLM agent) with a rule-based algorithm, rather than just replacing CineGPT. Specifically, we used a 1-to-1 rule-based mapping to generate two sub-trajectories from the instructions "pan downwards" and "move forward." We then assembled these sub-trajectories and positioned them at the estimated first-frame key location ("top of the stairs"). This key location was estimated using LERF instead of our proposed Anchor Determinator. We made this choice as the rule-based system, without the LLM agent, lacks the capability to extract necessary inputs for anchor determination, such as "the entrance of the tunnel."

While the "move forward" instruction can be executed correctly if the anchor is accurate, this assumption does not hold for the rule-based baseline. In fact, the rule-based approach cannot understand specific object-level prompts unless those objects are covered by the predefined rule set. In other words, this rule-based baseline can only generate a trajectory based on the input but lacks the ability to accurately place it within the scene. In this case, human assistance was required to determine the first key location, whereas our approach is capable of doing this autonomously.

Additionally, we would like to note that different prompts were used for Figures A/B and Figure C. This was necessary because the input for Figures A/B would have led to meaningless results with the rule-based baseline. The rule-based baseline can only handle simple inputs like "pan forward," while our approach can manage much more complex inputs without human assistance to specify key locations. It can also collaborate with users to refine the results through conversation.

We hope this clears up your confusion. We welcome you or other reviewers to raise any further concerns so that we can address them in the discussion period.
In my opinion, you replacing CineGPT with a rule-based algorithm in the Figure C. So the anchor points should be same for these two methods. Figure C does not show the anchor 'entrance of the green tunnel'. CineGPT generate camera trajectories only based on text description without spatial information. How can you avoid collision when calling CineGPT with 'move forward through the tunnel'? I think if the anchor point is accurate, the collision will not occur with a high probability. In the same time, this will also be easy for rule-based method too.
The algorithm for trajectory composition works as follows:
1. It takes as input a list of trajectories and anchor points. If adjacent anchor points are encountered in the input, the algorithm will report this as illegal input.
2. Merge adjacent trajectories until there are no adjacent trajectories left in the list. During the merging process, calculate a Euclidean transformation (6 DoF) and apply it to the latter trajectory, ensuring its starting point coincides with the position and rotation of the endpoint of the previous trajectory.
3. For all trajectories, if a trajectory has two adjacent anchor points, find a similarity transformation (7 DoF) to make the position of its starting point and endpoint coincide with the two anchor points. If it has one adjacent anchor point, calculate a Euclidean transformation (6 DoF) to make the position and rotation of its starting or ending point coincide with this anchor point.

We will include pseudo-code for this algorithm in future versions to enhance clarity.
As suggested by reviewer **dvT5**, here we post the complete conversation between the user and ChatCam for the new result. Please refer to Figures A & B of the attached PDF for the visualization of trajectories and rendered video frames.

**User:** From the top of the stairs, pan downwards to the entrance of the green tunnel. Move forward to go through the tunnel. Make a u-turn and move forward to the yellow bulldozer on the table.

**ChatCam:** { "observation": "The user describes a camera trajectory starting from the top of the stairs, panning downwards to the entrance of a green tunnel, moving forward through the tunnel, making a U-turn, and moving forward to a yellow bulldozer on a table.", "reasoning": "I need to find anchor points for the top of the stairs, the entrance of the green tunnel, and the yellow bulldozer. Then, I'll generate trajectories for panning downwards, moving forward through the tunnel, making a U-turn, and moving forward to the bulldozer.", "plan": "1. Call Anchor Determinator with 'top of the stairs'. 2. Call Anchor Determinator with 'entrance of the green tunnel'. 3. Call Anchor Determinator with 'yellow bulldozer on the table'. 4. Call CineGPT with 'pan downwards'. 5. Call CineGPT with 'move forward through the tunnel'. 6. Call CineGPT with 'U-turn'. 7. Call CineGPT with 'move forward to the bulldozer'. 8. Connect the output trajectories, ensuring they pass through the anchor points. 9. Ask the user for further clarification if needed. 10. Render a video using the generated camera trajectory." }

**User:** This trajectory hits a wall! Starts all over again. But after you make a u-turn, try going to a colorful bulldozer, then to a white table with books on it, and finally to a yellow bulldozer.

**ChatCam:** {"observation": "The user indicates that the initial trajectory hits a wall and needs revision. The revised trajectory starts from the top of the stairs, pans downwards to the entrance of the green tunnel, moves forward through the tunnel, makes a U-turn, then moves to a colorful bulldozer, proceeds to a white table with books, and finally ends at a yellow bulldozer.","reasoning": "I need to find anchor points for the colorful bulldozer and the white table with books, in addition to the previously identified points. I'll regenerate the trajectory to include these new elements and ensure the path avoids any obstacles like walls.","plan": "1. Call Anchor Determinator with 'top of the stairs'. 2. Call Anchor Determinator with 'entrance of the green tunnel'. 3. Call Anchor Determinator with 'colorful bulldozer'. 4. Call Anchor Determinator with 'white table with books'. 5. Call Anchor Determinator with 'yellow bulldozer on the table'. 6. Call CineGPT with 'pan downwards to the entrance of the tunnel'. 7. Call CineGPT with 'move forward through the tunnel'. 8. Call CineGPT with 'U-turn'. 9. Call CineGPT with 'move to the colorful bulldozer'. 10. Call CineGPT with 'move to the white table with books'. 11. Call CineGPT with 'move to the yellow bulldozer'. 12. Connect the output trajectories, ensuring they pass through the anchor points and do not hit any walls. 13. Ask the user for further clarification if needed. 14. Render a video using the generated camera trajectory."}

**User:** Very good! After this, I hope to add a dolly zoom for the yellow bulldozer. Give me the final plan.

**ChatCam:** { "observation": "The user wishes to add a dolly zoom effect targeting the yellow bulldozer at the end of the revised camera trajectory.", "reasoning": "To incorporate the dolly zoom effect, I'll need to adjust the camera movement and focal length as it approaches the yellow bulldozer, ensuring that the size of the bulldozer remains consistent in the frame while creating a dramatic spatial effect.", "plan": "1. Call Anchor Determinator with 'top of the stairs'. 2. Call Anchor Determinator with 'entrance of the green tunnel'. 3. Call Anchor Determinator with 'colorful bulldozer'. 4. Call Anchor Determinator with 'white table with books'. 5. Call Anchor Determinator with 'yellow bulldozer on the table'. 6. Call CineGPT with 'pan downwards to the entrance of the tunnel'. 7. Call CineGPT with 'move forward through the tunnel'. 8. Call CineGPT with 'U-turn'. 9. Call CineGPT with 'move to the colorful bulldozer'. 10. Call CineGPT with 'move to the white table with books'. 11. Call CineGPT with 'move to the yellow bulldozer'. 12. Call CineGPT with 'dolly zoom on the yellow bulldozer'. 13. Connect the output trajectories, ensuring they pass through the anchor points, do not hit any walls, and incorporate the dolly zoom effect. 14. Ask the user for further clarification if needed. 15. Render a video using the generated camera trajectory." }
You should only respond in JSON format following the template below: 

RESPONSE TEMPLATE:
{
"observation": "observation",
"reasoning": "reasoning",
"plan": "a numbered list of steps to take that conveys the long-term plan"
}

In your response JSON, it is generated in the order of "observation", "reasoning", and "plan". Put any string in one line, do NOT include any new line character in observation, reasoning, plan, self-critique or speak.

Here are some examples you can refer to:
EXAMPLE 1: 
User Input: " Starting with a close-up of the man in the red T-shirt, the camera pans right to the woman in the white dress, and then looks around."
Your Output:
{
"observation”: “The user describes a camera trajectory starting with a close-up of a man in a red T-shirt, panning right to a woman in a white dress, and then looking around.",
"reasoning": " I need to find the anchor points for the close-up of the man in the red T-shirt and the woman in the white dress. Then, I will generate the trajectory for panning right and looking around.",
"plan": " 1. Call Anchor Determinator with 'close-up of the man in the red T-shirt'. 2. Call Anchor Determinator with 'the woman in the white dress'. 3. Call CineGPT with 'pan right'. 4. Call CineGPT with 'look around'. 5. Connect the output trajectories, ensuring they pass through the anchor points. 6. Ask the user for further clarification if needed. 7. Render a video using the generated camera trajectory."
}

EXAMPLE 2: 
User Input: "Take a dolly zoom shot of the woman's face."
Your Output:
{
"observation": "The user requests a dolly zoom shot of the woman's face.",
"reasoning": "I need to identify the woman's face as an anchor point and generate a detailed trajectory for the dolly zoom shot, which means moving forward while decreasing focal length to maintain the size of the woman's face in the frame.",
"plan": "1. Call Anchor Determinator with 'the woman's face'. 2. Call CineGPT with 'move forward while decreasing focal length to maintain the size of the woman's face in the frame'. 3. Connect the output trajectories, ensuring they pass through the anchor points. 4. Ask the user for further clarification if needed. 5. Render a video using the generated camera trajectory."
}
EXAMPLE 3: 
User Input: " Start from the iron gate and move forward to the computer monitor, then u-turn and walk out of the iron gate."
Your Output:
{
"observation": "The user describes a camera trajectory starting from an iron gate, moving forward to a computer monitor, then making a U-turn and walking out of the iron gate.",
"reasoning": "I need to find the anchor points for the iron gate and the computer monitor, then generate a trajectory that moves forward to the monitor, makes a U-turn, and moves back out of the iron gate.",
"plan": "1. Call Anchor Determinator with 'the iron gate'. 2. Call Anchor Determinator with 'the computer monitor'. 3. Call CineGPT with 'move forward'. 4. Call CineGPT with 'U-turn and move backward'. 5. Connect the output trajectories, ensuring they pass through the anchor points. 6. Ask the user for further clarification if needed. 7. Render a video using the generated camera trajectory."
}
As suggested by reviewer **kHgE**, here we post our designed prompt to instruct the LLM agent.

You are a dialog agent that helps users to operate cameras in 3D scenes using dialog. The user starts the conversation with a 3D scene represented by NeRF or 3DGS. The user will describe the camera trajectory in his mind in words, and you help him generate the camera trajectory. You have two useful tools, the first is CineGPT, which can help you translate text into trajectory. The second is Anchor Determinator, which can find anchor objects to correctly place the trajectory in the 3D scene.

Please act according to the following instructions:
INSTRUCTIONS:
1.	The user-provided description includes (a) descriptions of the trajectories, including the camera’s translation (i.e., “pan forward”), rotation (i.e., “turn left”), and camera parameters (i.e., “increasing focal length”) in the trajectory or the trajectories’ features like shape or speed, and (b) some specific descriptions of the scene, which we call anchor points, such as “starting with the close-up of the car” or “a bird’s-eye view of the temple”.
2.	For (a) descriptions of the trajectories, invoke the API of the CineGPT to translate human text into trajectory. When calling CineGPT, try to use the description of the trajectory itself without involving any specific scene information. 
i.	To summon CineGPT, the command is termed "infer_cinegpt". Its arguments are: "traj_description": "<traj_description>". 
ii.	This API returns a JSON containing the camera trajectory consisting of camera pose and camera intrinsics for each frame. 
3.	For (b) some specific descriptions of the scene, invoke the API of the Anchor Determinator to get anchors to place the trajectory in the 3D scene. You need to find a description of an object or an image from the user's words. Anchor Determinator will find the picture that best matches your input and return its camera pose as the anchor.
i.	To call Anchor Determinator, the command is termed "get_anchor". Its arguments are: "anchor_description": "<anchor_description>". 
ii.	This API returns a JSON containing the anchor camera pose and camera intrinsics. 
4.	When the user's description contains multiple stages, you need to learn to split it into units of trajectory and anchor points, and call CineGPT and Anchor Determinator accordingly. In this case, you would interleave calls to CineGPT and Anchor Determinator. 
5.	Invoke the API of trajectory composition to combine the obtained sub-trajectoires and anchor points.
i.	To call trajectory composition, the command is termed "traj_compose". Its arguments are: " compose ": "<list_of_traj_anchor>". 
ii.	This API returns a JSON containing the composed camera trajectory consisting of camera pose and camera intrinsics for each frame. 
iii.	When encountering illegal input, this API will raise an error.
6.	Your generated plan should follow these steps.
i. Call CineGPT with trajectory descriptions (one or more times).
ii. Call Anchor Determinator with anchor descriptions (one or more times).
iii. Connect output trajectories ensuring they pass through the determined anchor points.
iv. Ask the user for further clarification if needed.
v. Render a video using the generated camera trajectory.