text: 434,118,1390709745909211138,785763100712665088,Yaroslav Ganin,"New paper day. Our take on generation of structured objects: Protocol Buffers + Transformers + Pointer Nets We showcase the method on 2D CAD sketches (geometric primitives & relations between them). Should work for other domains too. <LINK> Mandatory samples: <LINK> A close-up view for those who got dizzy <LINK> Look at this guy - he is so happy to get vectorized! (we do bitmap to sketch translation too). This one is out of distribution - I just doodled it in an app (see the image in the bottom left corner) <LINK> (joint work w/ @sbos, @liyuajia, Ethan Keller and Stefano Saliceti) 1/ Some details. Thread 2D sketches are at the heart of mechanical CAD. Each sketch is a collections of entities (lines, arcs, splines) and constraints (""this line is parallel to that"", ""this point and that point are coincident"" and so on). The latter defines the design intent. <LINK> 2/ Both entities and constraints are structured objects and can be described using JSON, XML or Protocol Buffers (the path we take). Below are two examples. Since constraints are applied to entities they employ pointers to refer their arguments (that's why we need Pointer Nets). <LINK> 3/ Our approach is to let an external interpreter handle the structure and use Transformers only to generate missing bits (e.g., the field values). We encode each missing bit as a triplet: (discrete value, continuous value, boolean flag). The flag is needed to handle loops. <LINK> 4/ This Transformer+Interpreter tandem is flexible enough to generate pretty much anything that we can represent as a PB message. That's why we can synthesize both entities and constraints in the same sequence without resorting to using specialized multi-stage architectures.",http://arxiv.org/abs/2105.02769,"Computer-Aided Design (CAD) applications are used in manufacturing to model everything from coffee mugs to sports cars. These programs are complex and require years of training and experience to master. A component of all CAD models particularly difficult to make are the highly structured 2D sketches that lie at the heart of every 3D construction. In this work, we propose a machine learning model capable of automatically generating such sketches. Through this, we pave the way for developing intelligent tools that would help engineers create better designs with less effort. Our method is a combination of a general-purpose language modeling technique alongside an off-the-shelf data serialization protocol. We show that our approach has enough flexibility to accommodate the complexity of the domain and performs well for both unconditional synthesis and image-to-sketch translation. ",Computer-Aided Design as Language,8,"['New paper day.\n\nOur take on generation of structured objects:\nProtocol Buffers + Transformers + Pointer Nets\n\nWe showcase the method on 2D CAD sketches (geometric primitives &amp; relations between them). Should work for other domains too.\n\n<LINK>\n\nMandatory samples: <LINK>', 'A close-up view for those who got dizzy https://t.co/myjvPfhO2s', 'Look at this guy - he is so happy to get vectorized! (we do bitmap to sketch translation too). This one is out of distribution - I just doodled it in an app (see the image in the bottom left corner) https://t.co/CsGlVLa8qv', '(joint work w/ @sbos, @liyuajia, Ethan Keller and Stefano Saliceti)', '1/ Some details. Thread\n\n2D sketches are at the heart of mechanical CAD. Each sketch is a collections of entities (lines, arcs, splines) and constraints (""this line is parallel to that"", ""this point and that point are coincident"" and so on). The latter defines the design intent. https://t.co/cPCg6QODBg', ""2/ Both entities and constraints are structured objects and can be described using JSON, XML or Protocol Buffers (the path we take). Below are two examples. Since constraints are applied to entities they employ pointers to refer their arguments (that's why we need Pointer Nets). https://t.co/a9hjG5fk6f"", '3/ Our approach is to let an external interpreter handle the structure and use Transformers only to generate missing bits (e.g., the field values). We encode each missing bit as a triplet: (discrete value, continuous value, boolean flag). The flag is needed to handle loops. https://t.co/2t231CdtXq', ""4/ This Transformer+Interpreter tandem is flexible enough to generate pretty much anything that we can represent as a PB message. That's why we can synthesize both entities and constraints in the same sequence without resorting to using specialized multi-stage architectures.""]",21,05,1718