Virtual Agent Positioning Driven by Personal Characteristics

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-30 09:31:22

Abstract: When people use agent characters to travel through different spaces (such as virtual scenes and real scenes, or different game spaces), it is important to reasonably position the characters in the new scene according to their personal characteristics. In this paper, we propose a novel pipeline for relocating virtual agents in new scenarios based on their personal characteristics. We extract the characteristics of the characters (including figure, posture, social distance). Then a cost function is designed to evaluate the agent's position in the scene, which consists of a spatial term and an personalized term. Finally, a a Markov Chain Monte Carlo optimization method is applied to search for the optimized solution. The results generated by our approach are evaluated through extensive user study experiments, verifying the effectiveness of our approach compared with other alternative approaches.

Main paper: 
 introduction: 
 As an emerging trend in the digital age, virtual characters can flexibly travel through different spaces. People can perform activities from locations mapped from the real world to virtual reality. The same game character can also travel freely between different game spaces. The emergence of virtual agents has changed the traditional social model and provides more possibilities for people to meet in different spaces. However, it brings up a new question: when the character travels to a new space, where should he/she be positioned? Especially when there are multiple agents in the scene that need to be positioned, the positioning of the characters will be affected by each other. Properly positioning the agent in the scene Figure 1 : Our approach learns the character's comfort zone in the scene through the user's position and posture in the original scene (left), thereby placing the virtual agent in an appropriate position in the new space to make the user comfortable and natural (right).greatly improves the user experience and allows the agent to better integrate into the new environment.When people orient themselves in a scene based on a given pose, they first sift through recalling similar positions from previous experiences. Then they use their own experience to find reasonable objects in the new scene that can interact with the given pose to help determine the appropriate location, such as key interactive objects, special terrain. They finally obtain information from social interactions, observing other's positions and behaviors to find the appropriate position for their own interactive behavior. However, these individual-related experiences are difficult to generalize so that can quickly predict the optimal position of an agent representing an individual in a scene.In recent years, some scholars have proposed approaches to reasonably locate agents in a scene [19, 38] . They locate agents in the scene based on the relevance of interaction behaviors, but do not consider the personalized characteristics of the agents themselves. The external characteristics, such as body shape and posture, will have an impact on their positioning in the scene. Agents cannot collide with other elements in the scene. Large one may sometimes have trouble standing in tight spaces. Intrinsic characteristics, such as personality and safe social distance, can also help determine the positioning of agents in a scene. For example, people tend to stand around people they are more familiar with and distance themselves from people they are unfamiliar with.Based on the personalized characteristics of the agent, different agents have different probabilities of appearing in the same position of a scene. With the development of 3D technology, information in the scene can be easily extracted, such as scene scanning and reconstruction, virtual equipment, etc. The personality characteristics of an agent can be divided into personal attributes and relationships between individuals. Personal attributes include the character's body shape, posture, and the positioning relationship between multiple people can also reflect the social relationship between them. To better model the agent's personalized characteristics, we define a cost function to measure these feature relationships. Then we regard the positioning problem as an optimization problem and use the Markov chain Monte Carlo optimization technology to perform personalized positioning of virtual agent.In this paper, we propose a new pipeline for positioning agents in new scenes based on personal characteristics in historical scenes, which can help agents migrate in different scenes automatically. Through the application of this framework, we can position the characters in the virtual game in VR games based on the positions of multiple people in the same real scene, giving them a more realistic immersive experience. In addition, multiple users in different spaces can also be transferred to a new virtual scene together according to each character's position in the historical space.The major contributions of our work are as follows: • Introducing a new problem of personalized positioning of virtual agents based on character characteristics to freely travel through different spaces.• Designing a computational framework to evaluate an agent's positioning in a scene, which combined agent characteristics and spatial constraints and then is applied to optimize the position of virtual agent.• Conducting an extensive user study to evaluate and validate the effectiveness of the proposed virtual agent positioning approach. 
related work: 
 In this section, we provide a succinct overview of the virtual agent positioning, and review the previous works in personalized characteristic modelling and social distance assessment.The core of virtual agent positioning is to confirm the position and direction of the virtual agent, which is particularly important in the field of robotics, such as path planning [2, 11] and navigation systems [25] . These systems rely on algorithms, such as A* [10, 18] and Dijkstra [27, 40] , to find optimal paths that enable virtual agents to move freely within the virtual environment. Ye et al. [41] present a position-aware virtual agent locomotion method, called PAVAL, that can perform virtual agent positioning in real time for roomscale VR navigation assistance.To position the agent accurately, some studies control the behavior of virtual agent through predefined rules [32, 36] , including avoiding obstacles [8, 14] , finding cover, following scene rules, etc. While these methods have some feasibility, they are often not flexible enough to cope with diverse scenarios. With the development of reinforcement learning, some works attempt to train the best behavior strategy of virtual agents based on reward information in the environment [9, 23] , which is more robust but also requires a large amount of training data and computing resources. Liang et al [19] also proposed a method to understand the scene with the help of scene semantics, allowing virtual agents to position themselves appropriately in the real world. Itsuki Noda [4] proposed an agent localization mechanism for dynamic environments using Delaunay triangulation to approximate the map. It can handle the positioning problem of multiple agents in the scene. However, these methods do not consider the personalized characteristics of virtual agents.Compared with previous works, our approach mainly focuses on the problem of positioning multiple agents in the scene, taking into account the personal characteristics of each agent.The comfort of the user experience is directly related to the fit of personal characteristics. With the upgrade of user experience, various fields have begun to work on personalized feature modeling, including interior design, education, human-computer interaction and so on. The most original method is to extract information directly from the communication of characters, such as face-to-face interviews, questionnaires, focus groups [1] , and so on. These methods are time-consuming and their effectiveness depends entirely on the experience of the professional. Some recent works extract personal habits through human daily life and activities in order to extract personality characteristics more realistically. Wang et al [34] constructed action graphs by observing videos of daily activities, which were used to train a generative model based on recurrent neural networks (RNN). This method models personal characteristics accurately, but requires significant storage costs. Some researchers propose that personal characteristics can be learned from virtual environments by simulating real activities. Wei et al. [21] proposed to a pipeline to learn personal preferences from virtual experiences, which was used in indoor furniture layout. Other studies look for clues of personal characteristics from historical scenes. For example, Wang proposed to use the previous home layout to layout furniture in a new scene.Inspired by these studies, we propose to extract the personality characteristics of the characters in the previous scene, which are then used in the personalized positioning optimization of the characters in the new scene.A safe social distance is a certain distance that people should maintain between themselves and others in different social and environmental situations to ensure physical health and social comfort [6, 22] . It plays an important role in social interactions in daily life. If two people stand too close together in a scene, each other will feel oppressed and tense. If the two people stand too far apart in the scene, it will affect communication and interaction with each other [30] .Traditional methods typically use mathematical distance measures, such as Euclidean distance [3] or Manhattan distance [29] , to measure the distance between people. For example, with the outbreak of the Corona virus in the past few years, more and more public areas have begun to pay attention to the safe distance between people, and have embedded cameras to detect whether the social distance between people reaches 1.2m [15] . These methods provide a basic way to assess social distance, but ignore the factors of individual differences and social comfort. Some studies have attempted to assess social distance through behavioral modeling. These methods take into account interactions and behaviors between individuals, such as posture [20] , facial expressions [5] , and how they move. Gao [20] conducted an experiment to investigate the effect of posture and embodiment on the social distance of participants and agents during interactive experiences in mixed reality. Leon [17] established personalized models of social safety distance by collecting users'subjective feelings and feedback in different scenarios, which may require large amounts of user participation and feedback data and are difficult to apply in real time. Different from the previous works, we model an individual's comfortable social zone through the agent's position in the virtual environment, which makes social distance assessments more personalized. 
overview: 
 Given a historical scene containing agents as input, our approach aims to optimize the agent's positioning in the new scene so that the agent can stand in a more reasonable position to perform tasks when transferred across scenes. It should be noted that the historical scene and the posture and information of agents can be obtained based on three-dimensional reconstruction [37, 42] , or can be directly captured based on the RGB camera of the VR device [39] , such as HoloLens.Inspired by the method of finding appropriate positioning in a scene, we designed a framework to automatically position multiple agents in new scenes based on personal characteristics in historical scenes. When a person tries to find a suitable position in a new scene, he must first call upon his previous memories to perceive the environment (what objects he is interacting with and how others around him relate to himself), and then find a position and direction in the scene that is suitable for the current action and task. Therefore, the first step of our approach is to model the environment (character actions, interactive objects, and relationships between characters in the scene) based on historical scenes. In this paper, we propose a method for encoding environments from historical scenes and then optimizing the position of each agent in new scenario through heuristic search.Assume that there are 𝑁 virtual agents in the scene, and their layout positioning in the scene is defined as 𝐿 = {𝑙 𝑖 |𝑙 𝑖 = (𝑥 𝑖 , 𝑦 𝑖 , 𝑧 𝑖 , 𝜃 𝑖 ), 𝑖 ∈ {1, 2, ...𝑁 }. Specifically, for the 𝑖th individual, (𝑥 𝑖 , 𝑦 𝑖 , 𝑧 𝑖 ) is the position of the agent, and 𝜃 𝑖 is the orientation of the agent. Based on the layout 𝐿 0 of characters in historical scenes, our goal is to automatically generate reasonable character layouts in new scenes. The framework is shown in Fig. 2 . Our approach includes two stages: preprocessing and optimization.Preprocessing. Using image processing algorithms, we preprocessed the information of individuals in historical environments, including their positions, postures, relationships with objects, and relationships with other individuals. Firstly, we employed the Mask R-CNN method [13] to detect agents and key objects in the scenes. Then we further utilized OpenPose [28, 33] to extract the poses of individuals, and employed Euclidean distance calculation to determine the distances between individuals as well as the angles between their directions. After these two steps of preprocessing, we modeled the effective information of the whole environment in the historical environment. We will discuss the details in Sec. 4.Optimization. The optimization stage considers the characteristics of the characters in the environment and optimizes the position and orientation of the virtual agent iteratively. The cost function with personalized constraints and spatial constraints is designed to evaluate how well the position and orientation of each virtual agent is. The personalized constraints ensure that the character's position and orientation maintain a reasonable social distance and relative relationship with others. The spatial constraints control the reasonable position of the virtual agent's posture in the scene, which is to achieve smooth interaction of characters in the scene. A Markov chain Monte Carlo optimization algorithm is applied to search for solutions, which is discussed in Sec. 5. 
preprocessing: 
 Following previous work, 3D scenes can be reconstructed through nerf [24] , 3D gaussion splatting [16] , differentiable rendering [26] and other method. To model the distribution of agents in a scene, it is necessary to extract the properties of the agents and the semantics of their interaction with the scene.In a given scene, the individual attributes of agents encompass their position, posture, as well as personal information such as height and body type.Position. The agent's position is where the user is standing. Therefore, we use the midpoint of the user's two foot coordinate points as the user's position. During motion capture, the positions of each agent are written sequentially as𝐿𝑜𝑐 = {𝑙𝑜𝑐 1 , 𝑙𝑜𝑐 2 , • • • , 𝑙𝑜𝑐 𝑁 },where 𝑙𝑜𝑐 𝑖 is the set of 3D coordinate of the agent 𝑖 in the original scene.Posture. The user's posture reflects the points at which the user needs support. We divide the postures according to the main supporting surface into standing, sitting, lying and leaning. To recognize the user's posture offline, we collected 1000 pictures containing various postures, about 250 pictures of each category, and trained an posture classification network based on VGG-16 [35] , with an accuracy of 98%. Each posture is coded as 1, 2, 3, 4 respectively. The posture of each agent is recorded as𝑃𝑜𝑠 = {𝑝𝑜𝑠 1 , 𝑝𝑜𝑠 2 , • • • , 𝑝𝑜𝑠 𝑁 }where 𝑝𝑜𝑠 𝑖 ∈ {1, 2, 3, 4}, and 𝑝𝑜𝑠 𝑖 is the posture type of the agent 𝑖.Body size. The body size reflects the space the agent needs in the scene. We use the IK algorithm to calculate and record the sequence of 16 key points(shoulders, elbows, torso, etc.) on the agent's body, with the foot posture as the origin of the coordinates. The Inverse Kinematics algorithm infers the positions of the other joints of the user by the Jacobian inverse technique real time [7] . The body size of agent 𝑖 is defined as 𝐽 𝑖 = { 𝑗 1 , 𝑗 2 ...𝑗 16 }, 𝑖 ∈ {1, 2, ...𝑁 }. 𝑗 𝑖 = (𝜃 𝑝 , 𝜃 𝑦 , 𝜃 𝑟 , 𝑑𝑖𝑠), where 𝜃 𝑝 , 𝜃 𝑦 , 𝜃 𝑟 is the vector direction from the joint point to the origin, representing pitch, yaw, and roll of the joint and 𝑑𝑖𝑠 is the distance from the joint point to the origin.It is worth noting that through the recording of the body size, we can calculate the 3D coordinates of each key point for each agent in the scene according to the position of them. The coordinates of the 𝑗 -𝑡ℎ key point of agent 𝑖 are recorded as 𝐽 𝑖 𝑗 , where 𝑖 ∈ {1, 2, ...𝑁 } and 𝑗 ∈ {1, 2, ...16} .It is crucial for the agent's positioning to complete the interaction between the character and the key objects in a reasonable position in the scene using corresponding postures. Key Object. Consider the closest object to each agent as the key object related to the agent. To make the environment in which the agent acts consistent in the new scene, the relative positional relationship between the agent and key objects should remain unchanged. As shown in Figure 3 , taking the nearest neighbor edge of the key object as the target, we calculate the vertical distance from the agent to the target and the relative angle between the agent and it of each agent. We denote the vertical distance of agent 𝑖 as 𝑑 𝑖 and the angle as 𝛼 𝑖 . Plane. Different planes in the scene can support different postures. We use the Mask RCNN algorithm [13] to detect the plane of each object in the scene. Figure 4 shows an example of a plane that can support various postures in a logo scene, where green is the place where you can stand, red is where you can sit, blue is where you can lie down, and yellow is where you can lean. 
optimization: 
 We randomly generate a set of localizations 𝐿 0 for each virtual agent in the new space. The problem of agent localization in new scenarios is then considered as an optimization problem, under personal and spatial constraints. In this section, we discuss the definition of the cost function and the optimization process in detail.During the optimization, we tried to enable the agent to be placed comfortably in the new scene by analyzing the relative relationship between the virtual agent and the objects in the new space, the social relationship between multiple agents, and the personal characteristics of the agent. We define the cost function considering personality constraints and spatial constraints as follows:𝐶 total (𝐿 𝑖𝑛 , 𝐿) = 𝜔 a 𝐶 a (𝐿 𝑖𝑛 , 𝐿) + 𝜔 s 𝐶 s (𝐿).(EQUATION𝐶 a (𝐿 𝑖𝑛 , 𝐿) is to assess the cost associated with personal characteristics extracted from the original scene. It penalize solutions that deviate from the expected interaction dynamics between the agent and scene elements, as well as the spatial distribution among all agents. 𝐶 s (𝐿) is to consider whether the agent is on a reasonable position in the scene, which is evaluated from the agent's admissibility and the distance between each agent. The 𝜔 coefficients determine the relative weighting of each item; in practice, we set 𝜔 a = 0.5 and 𝜔 s = 0.5.In personalized cost, we consider the relative relationships between human and scene, and the distributed relationships within agents. It is defined as follows:EQUATIONwhere 𝐶 r (𝐿 𝑖𝑛 , 𝐿) is the interaction cost, and 𝐶 d (𝐿 𝑖𝑛 , 𝐿) is the distribution cost within agents. 𝜔 r is set 0.6 and 𝜔 d is set 0.4.Interaction. When interacting in a new space, the relative relationship between the agent and the components of the scene with which it is interacting should remain as unchanged as possible. Once changed, it may affect the intent of the agent's posture. For example, it would make sense to raise a hand to try to grab something when the user is near a cupboard on the wall. Therefore, we consider the relative position of the user and key objects in the interaction cost. The cost is defined as follows:EQUATIONwhere 𝑑 𝑖 is the distance of the agent 𝑖 from the edge of the nearest interactive object. 𝑑 ′ 𝑖 is the distance from agent 𝑖 in the new space to the edge of the nearest object in the space. 𝑑 𝑚𝑎𝑥 is the maximum distance from all agents in the initial scene to key objects. 𝛼 𝑖 is the angle of the agent's orientation relative to the key edge. 𝛼 ′ 𝑖 is the angle of the agent's orientation in the new space relative to its key edge. It should be noted that 𝛼 𝑖 and 𝛼 ′ 𝑖 are constrained to 0-90 degrees. 𝑁 is the number of agents in the scene.Distribution. The new space is different in area from the original scene. The density distribution of agents in the scene is proportional to the scene area. In order to distribute the agents reasonably in the new space, we first use the mean of K nearest neighbor distances to calculate the density distribution of the agents in the original scene, where K is 3. For each agent, we find the three closest agents in the scene and calculate the Euclidean distance between it and these agents. Then we calculate the average of the three distances as the density of agent. We reposition the agent in the new space based on the area of the space. The cost is defined as:EQUATIONwhere 𝐾 𝑖 is the social density of the 𝑖-th agent with other agents in the scene. 𝐾 ′ 𝑖 is the social density of the 𝑖-th agent with other agents in the new scene. 𝑙 is the value of the diagonal length in the scene. 𝑁 is the number of agents in the scene.In the spacial cost, we consider the admissibility of people in the scene and the reasonable social distance around agents. It is defined as follows:EQUATIONwhere 𝐶 ad (𝐿) is the admissibility cost. 𝐶 sd (𝐿) is the social distance cost. The 𝜔 coefficients determine the relative weight of each item; in practice, we set 𝜔 ad = 0.5 and 𝜔 sd = 0.5.Admissibility. The positioning range of characters in the scene will vary depending on the character's posture and body size. For example, overweight people cannot complete some movements in some small places. The admissibility of an agent is mainly judged in two aspects: We detect the position of each key point of each agent in the scene to determine whether the positioning is feasible. In addition, it is also necessary to determine whether the user's posture is on a plane that matches the positioning. The cost is defined as:EQUATIONwhere 𝑙𝑜𝑐 𝑖 is the position of agent 𝑖. 𝑝𝑜𝑠 𝑖 is the posture type of agent 𝑖. 𝑉 (𝑝𝑜𝑠 𝑖 , 𝑙𝑜𝑐 𝑖 ) is to determine whether the user's posture 𝑝𝑜𝑠 𝑖 satisfies the scene plane label value at the location 𝑙𝑜𝑐 𝑖 . If it is satisfied, 𝑉 (𝑝𝑜𝑠 𝑖 , 𝑙𝑜𝑐 𝑖 ) is 0, if it is not satisfied, it is 1, which can be calculated based on the label map of the scene. 𝐽 𝑖 is the position of each joint point of agent 𝑖 in the scene. 𝑉 (𝐽 𝑖 ) is to determine whether the agent collides with an object in the scene. The value is 1 if the node in question is in an inappropriate position in the scene, 0 if not.Social distance. Social safe distance refers to the distance maintained between people in social situations to ensure that each other feels comfortable and safe. Positioning too close can cause agents to feel uncomfortable or even cause collisions between agents. In reality, the safe social distance should not be less than 1.2 meters [31] . The same as the virtual environment, where there should be reasonable social distance between agents in a multi-agent layout. The cost is defined as:EQUATION)where 𝑙𝑜𝑐 𝑖 is the position of agent 𝑖 in the scene, and 𝑙𝑜𝑐 𝑗 is the position of agent 𝑗 in the scene. 𝐷 (𝑙𝑜𝑐 𝑖 , 𝑙𝑜𝑐 𝑗 ) is the Euclidean distance between agent 𝑖 and 𝑗. 𝑁 is the number of agents in the scene.To steer clear of getting trapped in local minima within the solution space, we employ a simulated annealing algorithm to systematically probe the arrangement of the virtual agent. This algorithm is adept at embracing suboptimal choices according to the Metropolis criterion. To elaborate, starting with the initial placement of the agent denoted as 𝐿 0 within a fresh spatial context, we proceed by selecting a potential relocation and then assess its viability using the Metropolis-Hastings acceptance rule. This acceptance rule is delineated as follows:EQUATION𝑝 (•) is computed using by the defined cost:EQUATIONwhere 𝐶 total(•) is defined in Section 5.1. 𝑇 represents the temperature of the annealing process. Initially, 𝑇 assumes a large value, enabling the sampler to traverse the solution space with heightened. To speed up the optimization, three strategies is proposed to do an action with a respective probability, which modifies the position and orientation of each agent:Translation. Translation is the basic operation for changing the agent location in the scenario. Agent 𝑖 is selected and updated with the move. In the (𝑥, 𝑦)-space position move, a translation move generates a new position (𝑥 𝑘 + Δ𝑥, 𝑦 𝑘 + Δ𝑦) based on the current location (𝑥 𝑘 , 𝑦 𝑘 ). We sample the variation (Δ𝑥, Δ𝑦) using a Gaussian distribution.In the 𝑧-dimension, we set it in the new scene based on the height of the terrain of the (𝑥, 𝑦) plane. When there is no object at its location, that is, the agent is standing on the ground, 𝑧 is 0. Otherwise, the 𝑧 value is the height of the object in the corresponding position. For example, when the agent is placed on a bed, the 𝑧 value is the height of the bed.Rotation. Rotation changes the orientation of the agent. In the rotation move, the sampler generates a new orientation (𝜃 𝑖 + Δ𝜃 ) based on the current orientation 𝜃 𝑖 . The rotation change Δ𝜃 is generated from the Gaussian distribution.Swapping. Swapping refers to exchange the positions of any two agents in the scene, which can accelerate the exploration of the solution space. We choose two agents 𝑖and 𝑗 at random and swap their positions directly.It should be noted that we have enabled collision detection in the scene. If the agent and the scene collide through the model, the action is rolled back and the next action is selected. 
experiments: 
 In this section, we discuss several objective and subjective experiments conducted to evaluate the effectiveness of our pose synthesis approach. We implemented our approach using using C# and Unity 2021 and ran the optimization approach on a PC equipped with 32GB of RAM, a Nvidia Titan X graphics card with 12GB of memory, and a 2.60GHz Intel i7-5820K processor.To verify the proposed approach, we compared three approaches for virtual agent positioning:• Multiple agents are put into new scenes in turn through the POSA [12] . • Professionals put several virtual agents into new scenarios in sequence based on their professional experience. We recruited three professionals who have been engaged in animation production for 5 years. • Our approach automatically positioned the virtual agents in the new scene based on the character's characteristics of the virtual agents in the original scene.We compared results of these approaches in quantitative and qualitative experiments.Validation Dataset. The validation dataset consists of 25 scenes, both indoors and outdoors. We use the above three methods to place several agents for each scene. The number of agents in each scene is set to 5-10 depending on the size of the scene. First, we invite 5-10 people in each scene to simultaneously perform simulated actions in a scene similar to the operation of a simple social mobile game. The virtual agent of the new scene is then populated based on the scene positioning of the character in the original scene in our approach as well as that of the professionals. Compared with the method where professionals observe the scene, our method automatically analyzes the characteristics of the agent.we carried out user studies to evaluate the effectiveness of our approach and the aesthetic experience subjectively.Participants. We recruited 30 participants, with a diversity of backgrounds in terms of many aspects. The participants included 14 males and 16 females whose ages ranged from 12 to 55. Occupationally they range from unemployed or retired people to students, educators and business people. In addition, all the subjects reported normal or corrected-to-normal vision with no color-blindness.Procedure. The 25 scenes were randomly divided into 5 groups, with 5 scenes in each group. 30 participants were also randomly divided into 5 groups, with 6 people in each group. Each group of participants was randomly assigned to a set of scenarios to rate multiple agent positions generated by the three methods. Specifically, the score mainly consists of two parts: i) the rationality of a single character's position (to verify the rationality of each character's interaction with the scene) ii) the rationality of the overall character's position (to verify the rationality of the social distribution of all characters ). Ratings range from 1 to 5 on a 1-5 Liker scale, with 1 meaning inappropriate positioning and 5 meaning the opposite. Results generated by the three methods were presented to participants in random order to avoid bias. Not only that, we also conducted semi-structured interviews about user experience to further explore the factors that influence ratings. It should be noted that each set of scenarios is guaranteed to correspond to a set of participants.Results and Analysis. The average score statistics of the 25 scenes on the rationality of single-person postures and overall distribution of multi-person positions are shown in Figure 7 . From the statistical point of view, the positioning of professionals received the highest score in the two evaluations (single: 𝑀 = 4.10, 𝑆𝐷 = 0.76; overall: 𝑀 = 4.17, 𝑆𝐷 = 0.65), followed by Our approach (single: 𝑀 = 3.83, 𝑆𝐷 = 0.75; overall: 𝑀 = 3.89, 𝑆𝐷 = 0.71) and POSA (single: 𝑀 = 2.66, 𝑆𝐷 = 0.84 ; Overall: 𝑀 = 2.58, 𝑆𝐷 = 0.81). In all cases, our positioning approach are preferable to POSA and comparable to the Professional approach.To verify the validity of our results, we also performed a OneWay ANOVA test on the mean ratings of each scenario using a 𝛼 = 0.05 significance level. The results show that there is no significant difference between our method and the manual method of professionals pose (single: (𝐹 [1,49] = 3.21, 𝑝 = 0.07 > .05), overall: (𝐹 [1,49] = 2.25, 𝑝 = 0.134 > .05). However, compared with the POSA approach, both approaches achieved significantly higher scores than POSA: the Professional approach and POSA (single:(𝐹 [1, 49] )) It is proven that our approach can quickly learn character characteristics based on existing scenes and fill in new scenes, and the effect is comparable to that of professionals. This enhances the agent's scene switching experience and allows empty scenes to be quickly and appropriately filled.In our approach, the number of components in the original scene is 6. It is interesting that our method works better at positioning the agent in the new scene when it has the same number of components as the original scene.User Feedback. By asking users for their opinions, we learned that the suitability of the user's clothing and the environment will also affect the user's overall perception of the scene to a certain extent. For example, a user wearing a chef's uniform may feel strange if he or she is standing in a non-kitchen area. Some users said that virtual agents look a bit monotonous because their expressions are uniform. Because in daily interactions, characters may have a variety of expressions. These feedbacks have provided us with some interesting improvements, helping us to consider users' personalized information, such as clothing, expressions, etc., in the scene auto-fill problem.We verified the effectiveness of our approach quantitatively by comparing the number of collision points between the character's posture and the surrounding environment in each scene and the synthesis time of each approach.Collision Points. The reasonable posture and position of each virtual agent in the scene should not cause any unreasonable crossmodeling with any model in the scene. In each scene, we calculated the number of points where cross-mode collisions occurred between the virtual agent and the scene in each method generation scheme by turning on collision detection in the scene. Among a total of 202 characters in all scenes, our method did not cause improper crossmodeling between the characters and the scene. The professional method had 3 collision points with the scene, while POSA had 89 collision points with the scene. The results show that our approach has obvious advantages due to the collision loss in our approach. Compared with manual work by professionals, there are always small mold-piercing errors caused by carelessness.For each scene, we recorded the synthesis time for each method of automatically populating the new scene. Specifically, the time records for each method are as follows:• POSA: Place characters with fixed poses into the scene in sequence, starting from the time when the first character is placed until all characters are placed. • Professional: Professionals put characters with fixed poses into appropriate positions in the scene, but professionals can make fine adjustments to the characters' poses to fit the scene. The timing starts from when he sees all the characters until all characters are adjusted. • Ours: Our approach starts timing by extracting character relationships from the original scene until the repositioning and pose adjustment of all characters are completed in the new scene. The results show that POSA's method uses the least time to fill the scene (𝑀 = 0.3𝑠, 𝑆𝐷 = 0.02𝑠). Our method fills the scene much faster (𝑀 = 0.28𝑠, 𝑆𝐷 = 0.03𝑠) compared to the professional approach (𝑀 = 12.62𝑚𝑖𝑛, 𝑆𝐷 = 1.71𝑚𝑖𝑛). By asking professionals, we found that they would first spend 3 minutes viewing and analyzing the scene structure and each user's posture, then spend 1 minute initializing their position in the scene, and finally fine-tuning the character's posture according to the size of the scene. , to ensure the rationality of each pose in the scene. This process usually takes a long time, usually 8 minutes. The results shows that the positioning time of our method can realize real-time applications, such as rapid switching of multiple scenes during real live broadcasts or virtual games. 
conclusion: 
 In this paper, we propose a new problem of how to personalize characters based on their characteristics. We attempt to optimize the problem of agent positioning in new scenes based on character characteristics learned from historical scenes. To achieve this goal, we attempt to model human characteristics from information in historical scenes, which are then used in the optimization of agent positioning in new scenes. Specifically, the external characteristics (such as figure, posture) and internal characteristics (social distance) of the characters are first extracted from the historical environment.Then a cost function is designed to evaluate the reasonableness of the agent's position in the scene. Finally, a heuristic algorithm based on Monte Carlo criteria is applied to optimize the location of the agent in the scene, which make it more consistent with individual characteristics.Our approach leads to a variety of potential applications in both virtual and real worlds. For example, players often encounter scenarios where multiple characters need to seamlessly transition from one level or scene to another in virtual game development. By leveraging our approach, game developers can ensure that characters' positioning and interactions are consistent, enhancing the overall gaming experience and immersion for players. Another potential application area is in urban planning and architecture. For example, urban planners and architects can use virtual simulations of the distribution of crowds in a scene to visualize and analyze urban environments and building designs. This can help simulate human interaction and behavior in these environments, thereby informing more realistic and effective urban planning and building design.Limitation and Future Work. The analysis of specific inputs may be inaccurate due to the focus on human subjects in scene annotation data. For example, inaccuracies can occur when the proxy object is an animal such as a kitten. Various cartoon characters and avatars are becoming more and more popular in virtual applications, especially in games. Future work should involve collecting and integrating behavioral data from multiple organisms to improve the accuracy of agent localization.As characters transition from one scene to another, their attire should seamlessly complement the setting. For instance, when moving from a modern to an ancient-themed scene, it would be interesting if the character's wardrobe dynamically adjusts to align with the scene's style. Implementing such clothing adaptation not only ensures the logical positioning of characters during automatic scene transitions but also enhances their visual coherence with the new environment.While our current approach focuses on modeling the static posture of characters, a promising future direction involves seamlessly integrating dynamic user interactions into virtual scenes. This advancement could be achieved by swiftly mapping the real-time actions of users, captured through wearable MR devices like Microsoft HoloLens, into a dimensional space within the virtual environment. By accurately translating the user's dynamic action sequences into corresponding positions within the virtual scene, we can propel the evolution of AR animation, enhancing immersion and interactivity for users. 
 
 Appendix:  


Decision: The paper presents a new system for positioning virtual agents in different scenarios based on their characteristics. The method uses feature extraction and Markov Chain Monte Carlo optimization to find the best positions for the agents. The study includes qualitative and quantitative analyses, demonstrating the efficiency and effectiveness of the proposed method compared to existing techniques.

The paper has received mixed feedback from the reviewers but is leaning towards acceptance as 4 out of 3 reviewers rated it with weak acceptance. 

Reviewer nz1d: praises the motivation and efficiency of the work but noted limitations in the characteristic modeling and the scale of the user study. Reviewer 2FRs: highlights the thorough problem description and balanced evaluation despite the combination of existing methods. Reviewer 8jV3: describes concerns about the suitability for the multimodal community and the need for a more robust experimental evaluation. 
Reviewer SJiL: appreciates the innovative methodology, clear writing, and practical contributions.

While the paper is well written and the approach is innovative, it is important to focus on areas for improvement by addressing current limitations highlighted by the reviewers.

Several common limitations have been identified:
1) Historical Modeled Characteristics: The current approach may not fully represent human characteristics. Improvements in context extraction could enhance the modeling of human behavior.
2) Experimental Evaluation: Concerns have been raised about the limited baseline comparison (only with POSA) and the potential bias in user ratings.
3) Suitability for the MM Community: There is a minor concern about the paper's fit within the multimodal research community.

Based on the reviews, the paper is recommended for weak acceptance. The innovative approach and comprehensive evaluations are significant strengths, while the concerns regarding experimental robustness and multimodal suitability warrant attention. Addressing these concerns in future work could strengthen the paper's contributions and applicability to the broader MM community.

Reviews: review: paper_summary: paper_summary: This work introduces a system for positioning virtual agents in new scenarios based on their personal characteristics, using feature extraction and Markov Chain Mento Carlo optimization to find the optimal position.
strength_weakness: strength_weakness: 1. The motivation is reasonable.
2. Perform qualitative and quantitative analysis.
3. The proposed method is much more efficient than the previous works.
limitations: limitations: 1. The current historical modeled characteristics are limited and might not represent the human characteristics well. Developing a system that can extract various types of context to model human behavior might help.
2. The scale of the user study can be improved.

Considering the performance and efficiency of the proposed method, I think this work is borderline acceptable although some weaknesses occur.
paper_summary: paper_summary: This paper proposed a new pipeline to relocate virtual agents based on their personal characteristics in a new scenario. The features of proposed method were extracting the agent’s characteristics such as not only body shape and posture but also social distance based on past performance and preferences, and optimizing the positioning of agents in new scenes. The effectiveness of the proposed method was confirmed by comparing the positioning relationships through simulations for three virtual agents and by performing a usability testing with 30 participants.
strength_weakness: strength_weakness: The motivation for this study and the problems of existing studies were carefully described.
Although the proposed method was a combination of existing methods, the authors has carefully considered to establish the method.
This paper performed not only a simulation-based evaluation but also a large-scale user evaluation.
limitations: limitations: Although the effectiveness of this method could be confirmed based on the evaluation experiments, it was difficult to imagine the differences in positional relationships between proposed method reflecting individual characteristics and compared method. It would be better if an example could be shown in figures.
paper_summary: paper_summary: The paper introduces a pipeline designed to position virtual agents in scenarios that adapt to their specific characteristics. By utilizing the position and posture of an agent in an initial scene, the method determines the most suitable placement for the same agent in a new scene. The primary contribution is a cost function that evaluates the agent's position within the scenario, with the optimization of this cost function yielding the optimal positions for the agents.
strength_weakness: strength_weakness: The paper is easy to read and the proposed method is simple and straightforward. 
The addition of figures is useful for comprehending the approach and the setting. The related work seems complete and correctly frames the work in the current literature.

The cost function and its technical details are well presented and the related equations are sound.

Moving on to the experiments the proposed method is mainly compared to POSA and professionals who were asked to place the agents manually. The comparison to the professionals is very interesting and the proposed method returns results that are competitive.

Overall, the introduction of the novel cost function is interesting for the task and returns good results in the presented experiments.
limitations: limitations: The main limitations of this paper are related to the experimental evaluation of the proposed cost function. While the results could be interesting and comparable to those achieved by professionals manually placing the agents, the only baseline method used for comparison is POSA, and the sole evaluated metric is a potentially biased user rating. In terms of quantitative experiments, the author shows the time needed to process a scene for each method. However, a more understandable metric for evaluating task performance should be included, perhaps even using the cost function as a metric.

Additionally, the components of the cost function and their contributions to the final performance could be evaluated through an ablation study to demonstrate that each element of the cost function is functional for the final task.

Lastly, I have concerns regarding the suitability of the paper for the multimodal community, as it lacks a clear multimodal aspect.
paper_summary: paper_summary: The paper presents a novel approach for relocating virtual agents in new scenarios based on their characteristics. It outlines a pipeline involving the extraction of character traits, the design of a cost function evaluating the agent’s position, and applying a Markov Chain Monte Carlo optimization method. User study experiments confirm the effectiveness of the approach compared to alternatives.  The paper is accepted for publication pending minor typographical revisions.
strength_weakness: strength_weakness: The paper introduces an innovative methodology for optimizing agent positioning based on character characteristics in virtual environments. It addresses a pertinent problem in virtual scene navigation and offers a systematic solution. The proposed pipeline, comprising trait extraction, cost function design, and optimization, demonstrates a comprehensive understanding of the problem domain.
The writing style of the article is commendable, exhibiting clarity and coherence throughout. The use of language is precise, making complex concepts accessible to readers from diverse disciplines. The paper seamlessly integrates insights from various fields, highlighting its multidisciplinary nature.
The research contributes significantly to both theoretical and practical domains. By incorporating personal characteristics into agent positioning, the approach enhances realism and immersion in virtual environments. The potential applications, including virtual game development and urban planning, underscore the versatility and relevance of the proposed methodology.
Furthermore, the paper is well-structured, with each section logically progressing from problem formulation to solution implementation. The clarity of the presentation facilitates understanding and ensures a smooth reading experience for the audience.
In conclusion, the article offers valuable insights into personalized character positioning in virtual environments. Its rigorous methodology, clear exposition, and multidisciplinary relevance make it a valuable contribution to the scientific community. The paper is accepted for publication without revisions, except for two minor typographical errors and the suggestion to add some comment about the choice of the weights in equation 1 and 2. One of the typo mistake occurs in Section 6.2, "Results and Analysis," where "Our" is capitalized instead of lowercase as elsewhere in the manuscript. A missing space before "Then" in the conclusion has also been noted. These errors should be corrected before final publication.
limitations: limitations: One of the limitations highlighted in this scientific article concerns the choice of weights used in equations 1 and 2. It is emphasized the need to explore different combinations of weights to assess the impact on the performance of the proposed method. While choosing a weight of 0.5 may be reasonable, it would be desirable to conduct a more thorough analysis to understand how variations in weights affect the results. In particular, the article could benefit from a more robust justification regarding the selection of weights in equation 2. Exploring different combinations of weights could provide a better understanding of the model and allow for better optimization of the overall system performance.