Vitruvion: A Generative Model of Parametric CAD Sketches

1 INTRODUCTION . Parametric computer-aided design ( CAD ) tools are at the core of the design process in mechanical engineering , aerospace , architecture , and many other disciplines . These tools enable the specification of two- and three-dimensional structures in a way that empowers the user to explore parameterized variations on their designs , while also providing a structured representation for downstream tasks such as simulation and manufacturing . In the context of CAD , parametric refers to the dominant paradigm for professionals , in which the software tool essentially provides a graphical interface to the specification of a constraint program which can then be solved to provide a geometric configuration . This implicit programmatic workflow means that modifications to the design—alteration of dimensions , angles , etc.—propagate across the construction , even if it is an assembly with many hundreds or thousands of components . At an operational level , parametric CAD starts with the specification of two-dimensional geometric representations , universally referred to as “ sketches ” in the CAD community ( Shah , 1998 ; Camba et al. , 2016 ; Choi et al. , 2002 ) . A sketch consists of a collection of geometric primitives ( e.g. , lines , circles ) , along with a set of constraints that relate these primitives to one another . Examples of sketch constraints include parallelism , tangency , coincidence , and rotational symmetry . Constraints are central to the parametric CAD process as they reflect abstract design intent on the part of the user , making it possible for numerical modifications to coherently propagate into other parts of the design . Figure 2 , which demonstrates a capability made possible by our approach , illustrates how constraints are used to express design intent : the specific dimensions of the final model ( shown on the right ) can be modified while preserving the geometric relationships between the primitives . These constraints are the basis of our view of parametric CAD as the specification of a program ; indeed the actual file formats themselves are indistinguishable from conventional computer code . This design paradigm , while powerful , is often a challenging and tedious process . An engineer may execute similar sets of operations many dozens of times per design . Furthermore , general motifs may be repeated across related parts , resulting in duplicated effort . Learning to accurately predict such patterns has the potential to mitigate the inefficiencies involved in repetitive manual design . In addition , engineers often begin visualizing a design by roughly drawing it by hand ( Fig . 1 ) . The automatic and reliable conversion of such hand drawings , or similarly noisy inputs such as 3D scans , to parametric , editable models remains a highly sought feature . In this work , we introduce Vitruvion , a generative model trained to synthesize coherent CAD sketches by autoregressively sampling geometric primitives and constraints ( Fig . 3 ) . The model employs self-attention ( Vaswani et al. , 2017 ) to flexibly propagate information about the current state of a sketch to a next-step prediction module . This next-step prediction scheme is iterated until the selection of a stop token signals a completed sample . The resultant geometric constraint graph is then handed to a solver that identifies the final configuration of the sketch primitives . By generating sketches via constraint graphs , we allow for automatic edit propagation in standard CAD software . Constraint supervision for the model is provided by the SketchGraphs dataset Seff et al . ( 2020 ) , which pairs millions of real-world sketches with their ground truth geometric constraint graphs . In addition to evaluating the model via log-likelihood and distributional statistics , we demonstrate the model ’ s ability to aid in three application scenarios when conditioned on specific kinds of context . In autocomplete , we first prime the model with an incomplete sketch ( e.g. , with several primitives missing ) and query for plausible completion ( s ) . In autoconstrain , the model is conditioned on a set of primitives ( subject to position noise ) , and attempts to infer the intended constraints . We also explore image-conditional synthesis , where the model is first exposed to a raster image of a handdrawn sketch . In this case , the model is tasked with inferring both the primitives and constraints corresponding to the sketch depicted in the image . Overall , we find the proposed model is capable of synthesizing realistic CAD sketches and exhibits potential to aid the mechanical design workflow . 2 RELATED WORK . When discussing related work , is important to note that in computer-aided design , the word “ sketch ” is a term of art that refers to this combination of primitives and constraints . In particular , it should not be confused with the colloquial use of the word “ sketch ” as has sometimes appeared in the machine learning literature , as in Ha & Eck ( 2018 ) discussed below . CAD sketch generation The SketchGraphs dataset ( Seff et al. , 2020 ) was recently introduced to target the underlying relational structure and construction operations found in mechanical design software . Specifically , the dataset provides ground truth geometric constraint graphs , and thus the original design steps , for millions of real-world CAD sketches created in Onshape.1 Initial models are trained in Seff et al . ( 2020 ) for constraint graph generation using message-passing networks , but without any learned attributes for the positions of the primitives ; the sketch configuration is determined via constraints alone , limiting the sophistication of output samples . Following SketchGraphs , several works concurrent to ours explore generative modeling of CAD sketches utilizing transformer-based architectures ( Willis et al. , 2021 ; Ganin et al. , 2021 ; Para et al. , 2021 ; Wu et al. , 2021 ) . In Willis et al . ( 2021 ) , modeling is limited to primitives , without any learned specification of geometric constraints ; Vitruvion , however , enables edit propagation in standard CAD software by outputting constraints . Similar to our approach , both primitives and constraints are modeled in Ganin et al . ( 2021 ) ; Para et al . ( 2021 ) . In contrast to our work , these do not condition on hand-drawn sketches nor consider noisy inputs when training on the constraint inference task ( which proves crucial for generalization in Section 4.4 ) . Unlike the above works that solely target 2D sketches ( including ours ) , Wu et al . ( 2021 ) attempts to model sequences of both 2D and 3D CAD commands , but , similarly to ( Willis et al. , 2021 ) , does not output sketch constraints . Geometric program synthesis CAD sketch generation may be viewed via the broader lens of geometric program synthesis . This comprises a practical subset of program synthesis generally concerned with inferring a set of instructions ( a program ) for reconstructing geometry from some input specification . Recent examples include inferring TikZ drawing commands from images ( Ellis et al. , 2018 ) and constructive solid geometry programs from voxel representations ( Sharma et al. , 2018 ) . Related to these efforts is the highly-sought ability to accurately reverse engineer a mechanical part given underspecified and noisy observations of it ( e.g. , scans , photos , or drawings ) . One of the conditional generation settings we explore in this work is training a version of our model that generates parametric sketches when given a raster hand drawing . Note that previous work on geometric program synthesis have not generally considered constraints , which are critical to the CAD application . Vector graphics generation Generative models have been successfully applied to vector graphics in recent years . Sketch-RNN ( Ha & Eck , 2018 ) is trained on sequences of human drawing strokes to learn to produce vector drawings of generic categories ( dog , bus , etc. ) . Note the common meaning of the word sketch differs slightly from its usage as a term of art in mechanical design ( as we use it in this work ) . DeepSVG ( Carlier et al. , 2020 ) models SVG icons , demonstrating flexible editing with latent space interpolation . Less recent are tracing programs , which can convert raster drawings to vector graphics and have been widely available for decades ( e.g. , Selinger ( 2003 ) ) . None of this previous work considers constraints as a mechanism for specifying design intent as performed in 1https : //www.onshape.com CAD . The image-conditional version of Vitruvion also takes as input a raster image ( hand-drawn ) and can output vector graphics . Unlike tracing , which attempts to faithfully reproduce an input image as closely as possible , our model attempts to infer the sequence of geometric primitives and constraints intended by the user , learning to ignore noise in the input . Graph-structured modeling This work is also related to modeling of graph structures , via our use of geometric constraint graphs . Graphs constitute a natural representation of relational structure across a variety of domains , including chemistry , social networks , and robotics . In recent years , message-passing networks ( Gilmer et al. , 2017 ; Duvenaud et al. , 2015 ; Kipf & Welling , 2017 ) , building off of Scarselli et al . ( 2009 ) , have been extensively applied to generative modeling of graph-structured data , such as molecular graphs ( Jin et al. , 2018 ; Liu et al. , 2018 ) . These networks propagate information along edges , learning node- or graph-level representations . While message passing between adjacent nodes can be a useful inductive bias , it can be an impediment to effective learning when communication between distant nodes is necessary , e.g. , in sparse graphs ( Kurin et al. , 2021 ) . The constraint graphs in our work are sparse , as the constraints ( edges ) grow at most linearly in the number of primitives ( nodes ) . Recent work leverages the flexible self-attention mechanism of transformers ( Vaswani et al. , 2017 ) to generate graph-structured data without the limitations of fixed , neighbor-to-neighbor communication paths , such as for mesh generation ( Nash et al. , 2020 ) . Our work similarly bypasses edge-specific message passing , using self-attention to train a generative model of geometric constraint graphs representing sketches . 3 METHOD . We aim to model a distribution over parametric CAD sketches . Each sketch is comprised of both a sequence of primitives , P , and a sequence of constraints , C. We note that these are sequences , as opposed to unordered sets , due to the ground truth orderings included in the data format ( Seff et al. , 2020 ) . Primitives and constraints are represented similarly in that both have a categorical type ( e.g. , Line , Arc , Coincidence ) as well as a set of additional parameters which determine their placement/behavior in a sketch . Each constraint includes reference parameters indicated precisely which primitive ( s ) ( or components thereof ) must adhere to it . Rather than serving as static geometry , CAD sketches are intended to be dynamic—automatically updating in response to altered design parameters . Likewise , equipping our model with the ability to explicitly constrain generated primitives serves to ensure the preservation of various geometric relationships during downstream editing . We divide the sketch generation task into three subtasks : primitive generation , constraint generation , and constraint solving . In our approach , the first two tasks utilize learned models , while the last step may be conducted by any standard geometric constraint solver.2 We find that independently trained models for primitives and constraints allows for a simpler implementation . This setup is similar to that of Nash et al . ( 2020 ) , where distinct vertex and face models are trained for mesh generation . The overall sketch generation proceeds as : pθ ( P , C | ctx ) = pθ ( C | P ) pθ ( P | ctx ) S = solve ( P , C ) ( 1 ) where : • P is a sequence of primitives , including parameters specifying their positions ; • C is a sequence of constraints , imposing mandatory relationships between primitives ; • S is the resulting sketch , formed by invoking a solve routine on the ( P , C ) pair ; and • ctx is an optional context for conditioning , such as an image or prefix ( primer ) . This factorization assumes C is conditionally independent of the context given P . For example , in an image-conditional setting , access to the raster representation of a sketch is assumed to be superfluous for the constraint model given accurate identification of the portrayed primitives and their respective positions in the sketch . Unconditional samples may be generated by providing a null context .

Summary: The authors present a generator for constrained CAD sketches that can also be used to convert hand-drawn line drawings to constrained CAD sketches or to auto-complete partial sketches. Two transformer models are used, the first model generates CAD primitives, optionally conditioned on a line drawing image or on a partial CAD sketch, and the second model generates constraints conditioned on the generated primitives. The authors show good qualitative results and quantitatively improved CAD generation performance over two simple baselines.