Abstract-Procedural modeling techniques can produce high quality visual content through complex rule sets. However, controlling the outputs of these techniques for design purposes is often notoriously difficult for users due to the large number of parameters involved in these rule sets and also their non-linear relationship to the resulting content. To circumvent this problem, we present a sketch-based approach to procedural modeling. Given an approximate and abstract hand-drawn 2D sketch provided by a user, our algorithm automatically computes a set of procedural model parameters, which in turn yield multiple, detailed output shapes that resemble the user's input sketch. The user can then select an output shape, or further modify the sketch to explore alternative ones. At the heart of our approach is a deep Convolutional Neural Network (CNN) that is trained to map sketches to procedural model parameters. The network is trained by large amounts of automatically generated synthetic line drawings. By using an intuitive medium, i.e., freehand sketching as input, users are set free from manually adjusting procedural model parameters, yet they are still able to create high quality content. We demonstrate the accuracy and efficacy of our method in a variety of procedural modeling scenarios including design of man-made and organic shapes. . A wide variety of 2D or 3D models can be created with PM, e.g., vases, jewelry, buildings, trees, to name a few [4] . PM frees users from direct geometry editing and helps them to create a rich set of unique instances by manipulating various parameters in the rule set. However, due to the complexity and stochastic nature of rule sets, the underlying parametric space of PM is often very high-dimensional and nonlinear, making outputs difficult to control through direct parameter editing. PM is therefore not easily approachable by non-expert users, who face various problems such as where to start in the parameter space and how to adjust the parameters to reach outputs that match their intent. We address this problem by allowing users to perform PM through freehand sketching rather than directly manipulating high-dimensional parameter spaces. Sketching is often a more natural, intuitive and simpler way for users to communicate their intent. We introduce a technique that takes 2D freehand sketches as input and translates them to corresponding PM parameters, which in turn yield detailed shapes. Examples of input sketches and output shapes are shown in Fig. 1 . The users of our technique are not required to have artistic and professional skills in drawing. We aim to synthesize PM parameters from approximate, abstract sketches drawn by casual modelers who are interested in quickly conveying design ideas. A main challenge in recognizing such sketches and converting them to high quality visual content is the fact that humans often perform dramatic abstractions, simplifications and exaggerations to convey the shape of objects [5] . Developing an algorithm that factors out these exaggerations, is robust to simplifications and approximate line drawing, and captures the variability of all possible abstractions of an object is far from a trivial task. Hand-designing an algorithm and manually tuning all its internal parameters or thresholds to translate sketch patterns to PM outputs seems extremely hard and unlikely to handle the enormous variability of sketch inputs. We resort to a machine learning approach that automatically learns the mapping from sketches to PM parameters from a large corpus of training data. Collecting training human sketches relevant to given PM rule sets is hard and time-consuming. We instead automatically generate synthetic training data to train our algorithm. We exploit key properties of PM rule sets to generate the synthetic datasets. We simplify PM output shapes based on structure, density, repeated patterns and symmetries to simulate abstractions and simplifications found in human sketches. Given the training data, the mapping from sketches to PM parameters is also far from trivial to learn. We found that common classifiers and regression functions used in sketch classification and sketch-based retrieval [6]
Huang et al. REF introduce deep convolutional neural network (CNN) for mapping 2D sketches to procedural model parameters.
Shape Synthesis from Sketches via Procedural Models and Convolutional Networks
IEEE Transactions on Visualization and Computer Graphics
IEEE Transactions on Visualization and Computer Graphics
Medicine Computer Science