Controllable Procedural Generation of Landscapes

Authors: jia-hong  liu
shao-kui  zhang
chuyue  zhang
song-hai  zhang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-01 07:47:47

Abstract: Landscapes, recognized for their indispensable role in large-scale scenes, are experiencing growing demand. However, the manual modeling of such content is labor-intensive and lacks efficiency. Procedural Content Generation (PCG) techniques enable the rapid generation of diverse landscape elements. Nevertheless, ordinary users may encounter difficulties controlling these methods for desired results. In this paper, we introduce a controllable framework for procedurally generating landscapes. We integrate state-of-the-art Large Language Models (LLMs) to enhance user accessibility and control. By converting plain text inputs into parameters through LLMs, our framework allows ordinary users to generate a batch of plausible landscapes tailored to their specifications. A parameter-controlled PCG procedure is designed to leverage optimization techniques and employ rule-based refinements. It achieves harmonious layering in terrains, zoning, and roads while enabling aesthetic arrangement of vegetation and artificial elements. Extensive experiments demonstrate our framework's effectiveness in generating landscapes comparable to those crafted by experienced architects. Our framework has the potential to enhance the productivity of landscape designers significantly.

Main paper: 
 introduction: 
 Recently, there is a growing interest in developing efficient approaches for generating 3D indoor [62, 63, 65] and outdoor [32, 43, 48] content. Landscapes are indispensable outdoor content that creates ecologically and aesthetically balanced living spaces, promoting healthy lifestyles in modern life [37, 64] . However, architects often rely on modeling software to craft landscapes, which lacks efficiency. Even with powerful tools [26] , the modeling process still requires many manual operations and much proficiency [18] .In the industry, generating landscapes is often assisted with Procedural Content Generation (PCG) techniques. PCG facilitates the efficient and large-scale creation of natural environment contents using mathematical rules, spatial patterns, or other techniques such as optimization and deep learning [6, 14, 35, 50] . Nevertheless, these approaches may have limitations in user control. Ordinary users may find it challenging to leverage the process to achieve their objectives fully. When confronted with multiple parameters, users may experience difficulty adjusting them for optimal results. Additionally, some PCG approaches require expertise in engineering or architectural techniques and compromise accessibility to the general audience. Addressing user control is essential for achieving more effective landscape scene generation.Large Language Models (LLMs) excel in natural language understanding and task performance [1, 52, 53] . Additional context or prompts enable more appropriate and accurate responses aligned with user specifications [52] . Therefore, LLMs are effective and friendly tools for ordinary users to control expert tasks.In this paper, we propose a controllable framework for procedurally generating landscapes, aiming to generate aesthetic landscapes effectively (Figure 1 ). We address user control by introducing a set of parameters and integrating an LLM into our framework. Utilizing our framework is simple for ordinary users, who merely need to describe their desired landscapes in plain text. The framework automatically engages the LLM to comprehend the input and provide responses that can be parsed into parameters, thereby guiding the generation process to align with user objectives.After determining the parameters, we generate the landscape plan leveraging optimization techniques, featuring an adapted genetic algorithm for terrain and vegetation types and a heuristic algorithm for road generation. We use a discrete grid to represent terrain, entrances, scenic spots, roads, and the layout scheme of the landscape. Subsequently, we refine the plan through mathematical smoothing for more natural transitions both horizontally and vertically. Finally, we arrange diverse landscape elements to complete an aesthetic landscape. Furthermore, the framework supports the batch generation of many diverse landscapes given a user input.We evaluate our framework through experiments. Performance analysis and ablation studies delve into several components and validate the overall plausibility of our method. The user study demonstrates that the landscapes generated by our framework are comparable to those crafted by experienced architects.The key contributions of this paper are outlined below:• We propose a framework facilitating the procedural generation of a large number of plausible landscapes. • Our framework converts text inputs to parameters using a Large Language Model, enhancing the generation process's controllability and accessibility to ordinary users. • Our framework follows a parameter-controlled procedure combining optimization and rule-based techniques, such as a 2D-adapted genetic algorithm, a heuristic road generation algorithm, and mathematical smoothing. 
related works 2.1 procedural generation of outdoor contents: 
 Procedural Content Generation (PCG) is a leading technique owing to its versatile applications across various industries (e.g., gaming) [4, 17, 20, 45] . It is particularly effective for generating modular, highly repetitive, or rule-based content [14] . Traditional methods often rely on rules and simple randomized algorithms [14] , such as grammar trees [56] and noise. Recently, more advanced rule-based methods have been proposed, such as binary [29] or multi-level [61] partitioning for urban space planning. Computational geometry [9] and example-based methods [66] are also widely employed. Infinigen [43] is a state-of-the-art approach that facilitates generating natural objects and scenes through randomized mathematical rules.Other PCG categories are also emerging. Optimization-based methods typically perform multi-objective optimization by integrating the problem into existing algorithmic frameworks. There are cases for integer programming [39] , Particle Swarm Optimization [51] , anisotropic shortest path algorithm [16] , and genetic algorithms [40] . Simulation-based methods simulate temporal changes in content influenced by the autonomous actions of creatures and natural phenomena to generate scenes iteratively. For instance, simulating the interaction of individuals [47, 55] or the natural weathering and battling events [28] . Learning-based methods apply frameworks such as fully connected neural networks [30] , Variational Autoencoders (VAEs) [57] , and Generative Adversarial Networks (GANs) [19, 23, 34] , etc.Our framework combines the rule-based and optimization-based approaches to leverage their respective advantages. Additionally, incorporating an LLM enhances controllability and accessibility to ordinary users. Recent studies [31, 48] also investigate the integration of LLM in PCG.Large language models (LLMs) have garnered extensive attention recently. Many models are introduced [5, 33, 42, 59 ]. Literature demonstrates their exceptional capabilities. Wei et al. [52] discover The framework involves three stages: In the first stage, the LLM is employed to convert the user input into parameters that control the generation. Next, we optimize the landscape's layout plan on a grid, focusing on the terrains, spots, roads, and attributes. Finally, we refine it by smoothing the resulting plan and arranging the elements to obtain an aesthetic and plausible landscape. that these models can handle complex tasks even in a zero-shot manner. They [53] also observe that larger scales empower LLMs with additional capabilities compared to smaller models. Bubeck et al. [1] evaluate GPT-4 and conclude that beyond mastering language, it can tackle diverse tasks spanning mathematics, coding, medicine, and more. Typical examples include the advances in reasoning tasks [54] and public health aiding [24] . Numerous applications have emerged based on LLMs, such as Codex [3] and Galactica [49] . This paper employs the GPT-4 [33] model (Section 4.1).Existing literature employs a diverse range of approaches to tackle landscape generation. For instance, PCG techniques are leveraged to automatically model urban forestry [32] and multi-biome landscapes [13] . The genetic algorithm is applied for systematic optimizations [12, 46] . Parametric design approaches [2, 58] are also explored. Simulations are performed to form ecoclimates [36] , terrain [7] , and glaciers [8] . Real-world data and machine learning [22] are combined to author large-scale ecosystems.There is growing interest in interactively editing landscapes. For instance, Emilien et al. [11] and Gain et al. [15] facilitate editing virtual worlds using "brush" operations. Schott et al. [44] enable user control over the simulation of large-scale erosion. Similar interactive simulations are also applied to meandering river networks [38] . Ecormier et al. [10] propose a method for interactively generating ecosystems containing flora and fauna. Makowski et al. [27] explore interactive multi-scale modeling of plant ecosystems.3D-GPT [48] has been proposed as an LLM-powered 3D procedural modeling system that enables the generation of natural scenes through simple instructions. Compared with it, our framework highlights arranging landscape elements rather than modeling them individually. We also support larger quantities of vegetation. 
overview: 
 Our framework comprises three stages: parameterization, optimization of a layout plan, and generation of refined features (Figure 2 ). In the first stage (Section 4.1), the LLM converts the user input into parameters. We design the context (prompt) of the queries to ensure accurate and robust responses, which are then parsed into parameters. These parameters cover diverse aspects and functionalities (see Table 1 ) and guide the subsequent generation.The second stage (Section 4.2) encompasses three optimization steps, deriving a "layout plan" including information of terrains, spots, roads, and attributes on a grid. The first and third steps employ an adapted genetic algorithm (Section 4.2.2), incorporating modified crossover/mutation operations and a novel "evolution" operation to produce solutions effectively. The first step (Section 4.2.3) assigns each grid cell of five terrain types to characterize its topography and surface features. The second step (Section 4.2.4) first determines entrances and points of interest. Then, a heuristic-based path-finding algorithm and evaluations are applied to generate primary and secondary roads. Like the first step, the third step (Section 4.2.5) assigns each cell one of five attribute types, describing what should be placed on the terrain.The third stage (Section 4.3) uses two steps to finalize the generation: smoothing (Section 4.3.1) and procedural arrangement of elements (Section 4.3.2). The smoothing step operates in both the horizontal and vertical aspects. Horizontally, it smooths the borders of the regions using cubic splines. Vertically, it generates a smooth height map employing interpolations and Perlin Noise. The second step arranges numerous landscape models based on specified rules and patterns, where factors like model categories, terrain, and location are considered for aesthetic arrangement. 
method 4.1 parameterization from the input text: 
 In this section, we introduce the methodology of utilizing the LLM to convert the user input into parameters. Specifically, we merge the input text with a predefined context, call the LLM API, and parse the response to extract parameters.We use the GPT-4's chat model [33] 𝐿, which supports a textbased context comprising one or multiple messages. According to the official guidance, each message is assigned one of three "roles". A "user" message 𝑚 𝑢 signifies the request from the user, and an "assistant" message 𝑚 𝑎 represents the response from the model. A "system" message 𝑚 𝑠 , or the system prompt, is an independent message providing instructions and defining the assistant's behavior. Recognizing that a comprehensive context enhances quality and robustness of the responses, we design a context that includes all three message types: a system prompt 𝑚 𝑠 , a sample input 𝑚 𝑢 , and a sample output 𝑚 𝑎 . Section A.1 of the supplementary material provides more details for designing the system prompt. For each query, this context is combined with user input text 𝑚 𝑖 and sent to the LLM. Therefore, calling the LLM can be expressed as 𝑚 𝑜 = 𝐿(𝑚 𝑠 , 𝑚 𝑢 , 𝑚 𝑎 , 𝑚 𝑖 ), where 𝑚 𝑜 is the response text from the model. The output 𝑚 𝑜 is then parsed into the parameters using a syntax tree, enforcing a dictionary-like format for successful parsing. For 𝑛 𝑝 parameters {𝑝 1 , 𝑝 2 , ..., 𝑝 𝑛 𝑝 }, the parsing process is formulated as {𝑝 1 , 𝑝 2 , ..., 𝑝 𝑛 𝑝 } = 𝑃 (𝑚 𝑜 ).Table 1 : The list of categorized parameters employed in the paper, with each row representing a category of parameters sharing similar functionalities. Despite some groups appearing similar, they exhibit distinct representations or objectives. For instance, 𝑃 𝐴 addresses all regions of the same type, whereas 𝑃 𝑎 targets individual regions.Brief Description Example Since this paper deals with many parameters, we categorize all parameters into distinct groups (Table 1 ), where parameters within a category share similar functionalities. Given the complexity of managing all these parameters in a single query, we opt for a multiagent approach. For every input, we construct multiple queries to the LLM, with each query focusing on parameters related to one specific aspect. Every context {𝑚 𝑠 , 𝑚 𝑢 , 𝑚 𝑎 } used in these queries contains similar background information and adheres to a standardized format. The task description within each context is customized to address the parameters associated with it. [60] and numerous natural/artificial elements to arrange, generating a landscape in a single-shot process is challenging. Therefore, we address the task in a general-to-specific manner, encompassing two stages. This section (Section 4.2) is the first stage, which focuses on devising a general plan. To reduce the search space, we employ a 𝑤 × ℎ grid to formulate the plan discretely.We adopt a three-step approach that addresses several layers to determine the grid contents. First, we generate the terrain, establishing it as the fundamental feature of the landscape. Second, we determine the layout of key spots (entrances and points of interest) and roads. Lastly, we assign the "attributes", guiding the placement of natural elements or artificial features.Algorithm on a Grid. The genetic algorithm is a versatile optimization technique commonly employed to address optimization problems. Each potential solution is encoded and treated as an individual. By evaluating these individuals using a fitness function and applying selection, crossover, and mutation operations, the algorithm iteratively fosters new generations of refined solutions to the given problem.We implement the genetic algorithm on a grid with 𝑤 × ℎ cells, where the "types" assigned to all cells represent a solution. Each cell's type is denoted by a non-negative integer, with a total of 𝐾 types available. Conventional practice employs a 1D binary string encoding for each individual [21] . However, when conforming the 1D encoding to 2D cells, the crossover/mutation operations may lead to fragmented and less optimal solutions due to mismatched spatial properties of 1D strings and 2D grids. Therefore, we introduce specialized modifications to these operations tailored to a grid. Additionally, we propose incorporating an "evolution" operation into the genetic algorithm to accelerate the iterations. These operations are illustrated in Figure 3 . Thus, the genetic algorithm follows an iterative selection-crossover-mutation-evolution procedure. Section A.3 of the supplementary material has more details.As the first step in Section 4.2, this part determines the terrain for the landscape. Specifically, we assign each cell a terrain type. Five distinct types are considered, each represented by an integer index: unused (0), aquatic (1), terrestrial (2), artificial (3), and elevated (4). These types account for the physical features of the land, encompassing its topography and surface attributes. To elaborate on each type:• An "unused" cell indicates that the cell is excluded from the site, allowing the landscape's shape to vary. • An "aquatic" cell signifies the presence of a water body.• "Terrestrial" and "artificial" cells represent flat areas. "Terrestrial" features land with natural elements (soil, rocks, etc.), and "artificial" features ground covered by artificial elements (bricks or cement). • An "elevated" cell specifies the presence of a highland or a hill mainly reserved for natural elements.As previously stated, the genetic algorithm is utilized for the optimization. The fitness function considers multiple factors (existence, numeracy, area, location, and compatibility), detailed in Section A.4 of the supplementary material. Figure 4(a) shows an optimized result in this step.In the second step, we determine the spots and generate the roads. The generation sequentially targets entrances, points of interest, and primary/secondary roads. A select-best structure is applied several times: Randomly or procedurally generate a total of 𝑁 𝑔 valid solutions, evaluate each of them, and then select the best-evaluated solution.An entrance is defined as a corner on the boundary of the landscape. In the select-best procedure to generate 𝑃 𝑁 (𝐸𝑛𝑡𝑟𝑎𝑛𝑐𝑒) entrances, solutions with entrances better adhere to the following criteria are more favored: (1) Adequate spacing between pairs, (2) Even distribution across all directions, and (3) Adjacency to flat lands. Similarly, we determine the 𝑃 𝑁 (𝑃𝑜𝑖𝑛𝑡𝑂 𝑓 𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡) points of interest (POIs), which serve as scenic spots or landmarks.Regarding road generation, we prioritize the primary roads, which serve as vital routes connecting key locations. In our context, we generate primary roads that connect all entrances and POIs. We start by connecting every entrance to its nearest POI, then proceed to repeatedly connect pairs of separated POIs with the smallest Euclidean distance until all POIs are connected. We aim for an approach that balances preserving the spatial structure and generating straightforward roads, guided by the following principles [25] :(1) Avoid self-intersections or loops, (2) Follow the region borders when feasible, (3) Minimize unnecessary twists and turns, and (4) Avoid running along the site boundary.Based on these principles, each road is generated using a randomized heuristic approach, similar to a path-finding process. A select-best structure is employed to evaluate and refine the generated roads. After determining all primary roads, we supplement the secondary roads to connect more regions and spots. Section A.5 of the supplementary material provides details for generating these roads. We conclude this part by providing an example of the optimized result, as shown in Figure 4 (b).The plan for vegetation and artificial elements has yet to be decided. In the final step, we address this by assigning each cell an "attribute" type to describe what should be placed on the terrain. Five types are designed: none (0), basic (1), low-growing (2), tall-growing (3), and architectural (4).• A "none" cell indicates that no explicit development is needed for the cell, leaving it as raw terrain. • A "basic" cell recommends fundamental and typical vegetation for the terrain, such as grass, weeds, and rocks. • "Low-growing" and "tall-growing" cells feature plants of different heights. Shrubs are typical low-growing vegetation, while trees and bamboo are tall-growing. • An "architectural" cell emphasizes artificial elements such as pavilions, corridors, and statues.The genetic algorithm is once again utilized. Compared with Section 4.2.3, we consider similar perspectives (existence, numeracy, area, location, and compatibility) but focus more on the attribute's compatibility with the terrain and key spots (Section A.6 of the supplementary material). The result is shown in Figure 4 (c).4.3.1 Smoothing. This section completes the generation by refining the features and determining all elements using two steps: smoothing and procedural arrangement. In the first step, we apply smoothing for a more natural landscape. In the vertical aspect, we generate a continuous height map that specifies the physical height of the ground at any location. The method involves assigning a relative height to each cell based on its terrain type, applying bilinear and bicubic interpolation techniques, and adding Perlin Noise [41] . We then proceed to the horizontal aspect and smooth the region borders by introducing cubic splines. These smoothing techniques are detailed in Section A.7 of the supplementary material. Figure 5 illustrates the results of this step.Table 2 : The representative elements for the "combined" types. A "/" indicates that the type is invalid or does not require vegetation. "(s)" implies only a few elements, while "(l)" for numerous elements. Note that more elements are supported when arranging the models (in Section 4.3.2). For instance, statues, hallways, and pavilions are all compatible with the type (𝑇 3 , 𝐴 4 ). 2 . A group of cells sharing the same combined type is defined as a "zone", a refined version of "region" that operates as a cohesive unit with unified functionality and characteristics. This step finalizes the generation by procedurally arranging landscape models. The arrangement starts with the instantiation of the zone borders, considering different cases for walls, bridges, and roads. Then, we place models within zones as the content, following specific "rules" and "patterns". In our context, a "rule" is represented by the spatial relationship among different models or between models and the zone, referring to spatial relationship explained in [62] . A "pattern" emphasizes particular structures (e.g., arrayed, random, clustering) for placing groups of models, similar to [32] . Rules are typically applied to artificial elements, whereas patterns are primarily used for natural elements. We can combine these rules and patterns for more complicated and appealing landscapes. See Section A.8 of the supplementary material for more details.The arrangements results are stored as files containing all necessary information (the height map, textures, and model placement) to construct the landscape scene. A Unity plug-in is developed to parse these files and convert them into Unity-native scenes. Each scene is automatically rendered with twenty-one images, comprising an overview from a bird's eye perspective, four large-scale views covering all sides, and sixteen small-scale views from predefined angles and locations. Examples of rendered results and viewpoints are presented in Figure 6 . Section B.1 of the supplementary material shows more results and implementation details.5 Experiments 5.1 Performance 5.1.1 Efficiency of the Framework. The grid size significantly influences the efficiency of our framework. Since the parameterization is LLM-related and unstable, we focus on assessing the optimization (Section 4.2) and refining (Section 4.3) sections. We conduct five hundred trials using widths and lengths from {10, 15, 20, 30}, with mean execution times in seconds displayed in Figure 7b . Larger grids like 30 × 30 require significant time, whereas lower-resolution grids may compromise generation quality. Therefore, we recommend a 20×20 grid, which takes about twenty seconds with parallel computing. Using a 20×20 grid, we analyze the efficiency of various framework components, illustrated in Figure 7a . The components with genetic algorithms and the procedural arrangement consume significant time due to the extensive iterations and computations. 
influence of the user input.: 
 In this part, we investigate the influence of user input on the results. First, we assess the LLM's ability to interpret user descriptions. We examine its accuracy in determining the existence of a lake (i.e., 𝑃 𝜖 (𝐿𝑎𝑘𝑒)) based on positive/neutral/negative descriptions (e.g., "The landscape has two lakes", "The site is mostly covered by trees", and "The landscape has no lakes"). Although the LLM may sometimes fail to accurately comprehend when disturbed by additional information, implicit references, or negative wording, the responses are generally accurate. Section B.2 of the supplementary material has more details on this case. For more complicated cases associated with many parameters, we find the LLM able to handle the preferences regarding each parameter accurately. We also test ambiguous or even unrelated inputs (e.g., "An economical landscape"), where parameters are mostly left as default. In general, we conclude that the LLM is effective in input comprehension. Then, we examine the generation process's ability to adhere to the parameter. For the case regarding the existence of lakes, we conduct one hundred trials and find that 98% of the resulting landscapes have no lakes when specifying 𝑃 𝜖 (𝐿𝑎𝑘𝑒) = 𝐹𝑎𝑙𝑠𝑒. Conversely, all landscapes contain at least one lake when 𝑃 𝜖 (𝐿𝑎𝑘𝑒) is set to True or left unspecified, due to the emphasis on lakes as essential elements in the fitness function. Figure 8 illustrates examples of landscapes with and without lakes. We also test cases for other landscape elements and obtain similar results, which indicates our process's high reliability.5.2.1 Effectiveness of the Genetic Algorithm. In this part, we evaluate the proposed generic algorithm from several aspects. First, we conduct tests with different crossover rates 𝜂 𝑐 ∈ [0.4, 1] and mutation rates 𝜂 𝑚 ∈ [0.2, 0.8]. The maximum fitness value for evaluating the terrains (Section 4.2.3) is used as the metric. For each combination of rates, we run the algorithm two hundred times under identical configurations. Figure 7c presents the results. The combination (𝜂 𝑐 = 0.9, 𝜂 𝑚 = 0.7) yields the highest fitness value, indicating that the algorithm is most effective under these values. These values are thereby chosen as the default hyperparameters.Next, we examine the novel "evolution" operation by comparing our algorithm with the baseline without this operation (i.e., evolution rate 𝜂 𝑒 = 0). Using the same experimental settings, the mean fitness value for the baseline yields 26.15, which is 22.5% lower than the value of 33.73 achieved with the evolution operation. This result is anticipated as the evolution operation actively raises fitness values. However, we observe that a high 𝜂 𝑒 may excessively reduce diversity due to extensive modifications. Thus, a default value of 𝜂 𝑒 = 0.1 is selected to balance effectiveness and diversity. Finally, we compare the proposed 2D-adapted algorithm with the classic 1D implementation. The classic approach achieves a mean fitness value of 12.74, which is significantly outperformed by our method. It demonstrates that the proposed genetic algorithm successfully adapts to the 2D case and produces better results.In this part, we compare the heuristic road generation approach in Section 4.2.4 with a baseline approach that connects corners with at most two straight segments (in an 'L' shape). Both methods are tested on random terrains, entrances, and POIs to generate the primary roads until all entrances and POIs are connected. The results demonstrate that although our method produces slightly longer roads, it significantly reduces the disruption to the region structure and results in more balanced region areas. See Section B.4 of the supplementary material for details.5.3.1 Task Specification. This section elaborates on the user study that evaluates our framework. We invite three experienced designers to craft landscapes manually within Unity. All landscapes are configured as 400 × 300 meters (equivalent to a grid size of 20 × 15), with the same set of assets utilized for design. Ten landscape scenes are obtained for comparison (Manual), each taking about two days to complete. Furthermore, we design two baseline approaches modified from our framework. The first baseline (Random1) replaces the optimization of attributes (Section 4.2.5) with a naive process that assigns each cell a random attribute. The second one (Random2) simplifies the procedural arrangement of elements (Section 4.3.2) by organizing all elements in a randomized manner. These baselines also serve as a supplement to the ablation study. One hundred landscapes are randomly generated for each baseline and our method (Ours).Six criteria with assigned weights, devised by professional landscape architects, are employed to facilitate a comprehensive evaluation (Table 3 ). These criteria are explained in Section B.5 of the supplementary material. Every participant evaluates twelve distinct landscapes, with three scenes randomly selected for each approach. Twenty-one images are generated from viewpoints in Figure 6 for every landscape. Using a 5-point Likert Scale, participants score each criterion from 1 (lowest) to 5 (highest) for all landscapes. Finally, participants are asked to select the best one among all landscapes. Figure 9b shows the frequency of each approach being selected as the best. Despite our approach not being as favored, it generates landscape scenes comparable to the manually crafted ones and exhibits satisfying plausibility.In Figure 9c , we analyze scores for each criterion. Landscapes from Ours and Manual receive higher overall ratings. Manual scenes demonstrate significant advantages in criteria T and N, attributed to exquisite landscaping techniques and outstanding naturality. However, they receive lower scores for D, primarily due to fewer elements compared to the generated scenes, given the substantial effort required for manual operations. Compared to the crafted landscapes, our results excel in diversity and richness but lack aesthetics in other aspects. However, feedback reveals that some users fail to notice that most scenes are generated or cannot distinguish them from crafted ones, even after completing the study. Despite room for improvement, the generated landscapes can already compete with the manually designed ones in terms of quality. 
conclusion and future work: 
 This paper introduces a framework leveraging the capabilities of LLMs and PCG for controllable landscape generation. Following a three-stage process, the framework can seamlessly convert any textual input into a batch of aesthetic landscapes tailored to user specifications. The performance analysis and ablation studies validate the plausibility of the proposed approaches, and the user study demonstrates that the generated landscapes stand on par with those crafted by professional designers. The framework serves as a tool that enables efficient and controllable landscape generation for ordinary users, potentially enhancing design productivity. However, we still have the following limitations. Firstly, the framework lacks dynamic effects and interactions in temporal and spatial dimensions. In real world, landscapes are vibrant components within a larger ecosystem. Landscapes are subject to dynamic influences and experience seasonal changes. Spatially, the landscape is intricately linked to its surroundings, encompassing natural factors like sunlight and artificial elements such as streets. We plan to contextualize the landscape within a broader framework and incorporate more dynamic interactions.Secondly, the currently generated results are not yet comparable to well-designed real-world landscapes. For instance, the artificial traces of a grid cannot be removed by smoothing. Additionally, the limited rules, patterns, and element categories compromise flexibility in landscape arrangement. Our future direction involves introducing refined approaches and expanding the range of available assets to achieve more realistic results.Finally, there remains room for enhancing controllability within the current framework. As discussed in Section 5.1.2, the conversion from user input to parameters occasionally suffers from inaccuracies. In the future, we will conduct more extensive experiments to understand the parameterization and enhance control. 
: 
 for procedurally generating landscapes. We integrate state-of-theart Large Language Models (LLMs) to enhance user accessibility and control. By converting plain text inputs into parameters through LLMs, our framework allows ordinary users to generate a batch of plausible landscapes tailored to their specifications. A parametercontrolled PCG procedure is designed to leverage optimization techniques and employ rule-based refinements. It achieves harmonious layering in terrains, zoning, and roads while enabling aesthetic arrangement of vegetation and artificial elements. Extensive experiments demonstrate our framework's effectiveness in generating landscapes comparable to those crafted by experienced architects. Our framework has the potential to enhance the productivity of landscape designers significantly 1 . 
 
 Appendix:  


Decision: This paper received three positive ratings and one negative rating as the final ratings. The major concerns of the negative comment are the novelty and insufficient evaluation. Regarding novelty, the reviewer asked for clarifications on the novelty in comparison to 3D-GPT, for which the authors provided reasonable and convincing responses. The AC agrees that these two works address different tasks and have different focuses. Regarding limited evaluations, the AC thinks that comprehensive evaluation remains a common challenge for generation problems, and the evaluation presented in the paper is sufficient to justify its effectiveness. Nevertheless, the authors should improve the clarity of the evaluation methods in the final version. Overall, the AC agrees with the majority and recommends acceptance.

Reviews: review: paper_summary: paper_summary: This paper presents a controllable framework for generating controllable and aesthetic landscapes using large language models and procedural content generation. ​The framework allows users to describe their desired landscapes in plain text, which is then converted into parameters for landscape generation using a large language model.​ It leverages rule-based and optimization techniques (such as 2-D genetic algorithm) and refinements (such as mathematical smoothing) to generate a general plan for terrains, roads, and various attributes. ​ The plan is refined through smoothing and arranging elements for aesthetic landscapes. ​ The framework supports batch generation of diverse landscapes and has been shown to produce landscapes comparable to those crafted by experienced architects validated by a user study. ​ The paper also discusses the limitations of the current framework as lacking dynamic interactions and suggests future improvements for higher controllability.
strength_weakness: strength_weakness: Following are the strengths of the paper:

1. Clarity and Thoroughness: The paper is easy-to-read and provides a clear and detailed explanation of the proposed framework for generating landscapes. ​ The methodology is well-described making it easy for readers to understand the framework.

2. Practical Application and Relevance: The paper addresses a practical problem in landscape design by proposing a controllable framework for procedurally generating landscapes. By integrating language models and optimization techniques, the framework offers a solution that enhances productivity and allows ordinary users to generate plausible landscapes tailored to their specifications. ​ The relevance of the research to the field of landscape design and the potential impact on the industry make this paper valuable.
limitations: limitations: Following are the limitations of the paper:
1. Novelty: While the proposed framework for generating landscapes is well-explained and the integration of language models and procedural content generation is interesting, I have some concerns regarding the novelty of the approach. ​ It appears that the framework mostly combines GPT4 and existing techniques such as genetic and heuristic algorithms, mathematical smoothing with the exception of a novel 'evolution' method adapted in the genetic algorithm. ​For ex:- It is not clear as to how this framework is different or better compared to 3D-GPT.  It would be helpful if you could further highlight the novel aspects or improvements over existing methods to clearly establish the novelty of your framework.

2. Insufficient evaluation: Secondly, although you mention extensive experiments in the paper, I believe there is room for further evaluation to strengthen the validity of your findings. 
  a. While the performance analysis and ablation studies provide valuable insights, it would be beneficial to include additional quantitative metrics or comparisons with other state-of-the-art methods to demonstrate the superiority of your framework (For ex:- 3D GPT). ​ 
  b. If the novelty lies in the framework, there needs to be quantitative evaluation with state-of-the-art. Also applicable If the novelty lies in the adapted genetic algorithm.
  c. To evaluate the LLM component, its ability to interpret user input is evaluated in terms of accuracy in determining the existence of a lake as per user input. 
  d. Additionally, the user study comparing the landscapes generated by your framework with those crafted by experienced architects is intriguing, but it would be valuable to provide more details on the methodology and results of this study. ​


Overall, I appreciate the clarity of your paper and the potential of your framework in addressing the challenges of landscape generation. ​ However, I believe further emphasis on the novelty of the approach and a more thorough evaluation would significantly strengthen the contribution of your work.
paper_summary: paper_summary: The paper proposes a controllable framework for the procedural generation of landscapes using Large Language Models (LLMs). The framework aims to enhance user accessibility and control by converting user text inputs into parameters that guide the generation process. The approach leverages optimization techniques and rule-based refinements to create landscapes that include terrains, roads, and various attributes. The framework is evaluated through extensive experiments and user studies, demonstrating its effectiveness in generating landscapes comparable to those crafted by experienced architects.
strength_weakness: strength_weakness: 1. Innovative Integration of LLMs: The framework integrates state-of-the-art LLMs to convert user text inputs into generation parameters, significantly enhancing user accessibility and control over the procedural generation process.
2. Optimization Techniques: The use of optimization techniques, including a genetic algorithm for terrain and vegetation types and a heuristic algorithm for road generation, ensures the generation of aesthetically pleasing and functional landscapes.
3. Comprehensive Evaluation: The framework is thoroughly evaluated through performance analysis, ablation studies, and user studies, providing a robust validation of its effectiveness and comparability to manually crafted landscapes.
4. Practical Applications: The ability to generate diverse landscapes from simple text inputs has practical applications in various fields, including urban planning, gaming, and virtual environment design.
limitations: limitations: 1. User Input Interpretation: The accuracy of converting user inputs into generation parameters can occasionally suffer, particularly when inputs are complex or negatively worded. This limitation impacts the overall controllability and precision of the generated landscapes.
2. Lack of Dynamic Effects: The current framework does not fully incorporate dynamic effects and interactions in temporal and spatial dimensions, limiting its realism in real-world scenarios where landscapes are subject to changes over time and interactions with surrounding elements.
3. Limited Rules and Patterns: The framework currently supports a limited set of rules, patterns, and element categories, which restricts the flexibility and diversity of landscape arrangements.
paper_summary: paper_summary: This paper presents a framework that uses advanced language models to help users easily generate customized landscapes. By inputting simple text, users can create landscapes that match those designed by professional architects.
strength_weakness: strength_weakness: The paper is well-structured, featuring clear motivation and solution. The methodology that employs a language model to convert linguistic contexts into parameters is not novel, yet it appears to be effective.
The evaluation of the proposed work is comprehensive and robust, encompassing quantitative analyses, ablation studies, and user experiments.  The results depicted in the figures looks good.
The accompanying video effectively demonstrates the performance of the proposed method, and the supplementary documents provide detailed insights into the technical processes and evaluations.
limitations: limitations: However, I have several concerns regarding the technical content of the draft:

<1> Sections 4.2 and 4.3 lack detailed descriptions of the techniques used, despite references to further details in the supplementary material. The paper should contain all essential information necessary to replicate and evaluate the work.

<2> I recommend summarizing the variables required for landscape generation—such as terrain types, artificial elements, roads, and spots—at the beginning of the solution section (perhaps Section 3). This summary would help readers better understand the design parameters involved in creating a landscape.

<3> Key concepts, such as "plan" or "layout plan," should be formally defined to aid comprehension among readers who are not familiar with landscape design.

<4> The unity plug-in mentioned in Section 5.1 should be elaborated upon and integrated into the overall workflow of the proposed solution. I think it is a crucial component necessary for completing the landscape generation process.
paper_summary: paper_summary: The manuscript presents a Procedural Content Generation (PCG) framework that simplifies the generation of tailored landscapes for ordinary users. The framework comprises three stages: parameterization, optimization of a layout plan, and generation of refined features.
In particular, it integrates LLM to input the description of the landscape from the user to control parameters that guide the generation process. It also leverages optimization and rule-based techniques and uses a grid to guide the landscape’s layout plan. Afterward, it mathematically smooths the transitions vertically and horizontally for a natural transition of the constituent landscape batches, i.e., terrains, vegetation, and roads, and arranges these elements to obtain an aesthetic and plausible landscape.
strength_weakness: strength_weakness: The paper is well-written, structured, and presented. The content includes cross-referencing, which enhances its coherence, facilitates understanding for readers, and makes it easy to follow
The authors conducted comprehensive experiments and evaluations
The authors concluded with insightful future works after identifying the limitations in their framework.
limitations: limitations: The visualization of Figure 2 can be improved for better understanding of the workflow in the framework