A New Approach to Defining 3 D Models with 2 D Sketches Interim Report

While computers have been extensively applied to the later stages of the architectural design process, early conceptual sketching still occurs primarily on paper. This project explores how rough sketching might be brought into a virtual environment and develops a sketch-based computer aided design (CAD) system for the middle ground between 2D concept sketches and 3D presentation models. This CAD system takes a qualitatively different approach to 3D representation and design than do the current standard CAD applications. Rather than using a conventional “top-down” approach, defining large masses in 3D and refining and decorating these masses, the system develops designs as the synthesis of sketches and photographs of varying levels of detail arranged by the user on planes in a 3D environment. The system allows the user to tour the 2D sketches and photographs in a manner that lets the model emerge by the fusion of images by the user’s own visual system. The advantage of the proposed approach is that the user works in a familiar and flexible 2D mode for generating ideas, avoiding the limitations and time delays of a full and consistent 3D model being defined at each step in the design process. On top of this CAD system, I am working on an interface and suite of tools for ‘meta-sketching’. This work includes concepts for sketch layout and organization, sketch annotation, simplified and summary representations of sketches, and sketch search. When complete, such a system will provide architects and designers a comprehensive tool for generating and organizing conceptual level sketches of an emerging design. In the second semester of work on this project, I plan to refine the ideas explored here, and to finish my in-progress implementation of the most promising of these ideas. Introduction CAD has become essential to the architectural design process, yet its use remains generally reserved for late stages in the design process, such as for presentation drawings and technical drafts. This is largely due to emphasis on exacting precision and technical accuracy in conventional architectural CAD systems. Such precision and accuracy do not exist at early stages of the design process. So, architects continue to rely on pencil and paper sketches and physical massing models to work out the conceptual levels of design. There has been much recent enthusiasm in developing virtual systems to break into these early stages of design. However, most approaches have been limited by the fact that they maintain at all times a complete and consistent 3D model of the design being worked out. Certain of these systems, such as Google’s Sketchup [10], place emphasis on quick design mockups by giving the user simple, intuitive controls. A consequence of this simplicity is a lack of complete flexibility in defining forms. Sketchup models tend to be blocky and formulaic looking. Other systems attempt to infer implied 3D form from simple strokes and sketches. For example, Ijiri et al.’s flower modeling system attaches intricate flower forms to gestural strokes [12]. As with Sketchup, this system is highly limited in its flexibility – it is only effective at generating models of certain varieties of flowers. The Teddy system [11] similarly infers structure from strokes, except in a much more general purpose setting. These approaches attempt to tackle the complex and often intractable problem of reading a user’s mind to automatically infer full 3D models from a simple and imprecise set of input strokes. In the current project I am exploring an alternative to this paradigm: instead of relying on automation to piece together a design, give the user complete control and let the design cognitively fuse in the user’s mind. This is a two semester project. As such, the current report outlines background theory and motivation, describes the preliminary implementation, and outlines plans and ideas further development next semester. Parameters of the Design Problem The architectural design process, and in particular the sketching domain, has its own particular set of parameters and characteristics that limit the scope of the problem to be dealt with. Firstly, sketching is very intuitive, physical, and immediate. The traditional tools for sketching reflect this – they are simple and straightforward: pencil, paper, eraser, etc. Virtual tools for sketching should also be based on these criteria. Controls should be few but flexible. The process should feel physical. A sketch should take little time to make and require little maintenance to store. Previous work has offered insight into how such goals might be met. Gestural and context sensitive controls have been explored extensively as a means of making controls simpler yet still highly descriptive. Researchers have tapped into humans’ innate comfort with physical manipulation with systems such as BumpTop [1] for organizing files on a desktop and Teddy [11] for sketching 3D shapes. Other systems have successfully used more abstracted gestural controls ([12], [4]). Much work has also been done on mimicking the look and feel of physical sketches with non-photorealistic rendering techniques. Such methods increase users’ comfort level with the virtual medium while at the same time making the sketches more descriptive (with taper, translucency, texture, etc) and look better as pieces of presentable art [21]. One other way to make the virtual sketching process more physical is to give users appropriate input devices. Sketching with a mouse is difficult, both because users are most often unfamiliar with this technique and because mice generally lack the critical level of friction that keeps pencil and pen movements under control (mouse-sketched drawings tend to look jittery and imprecise). Additionally, traditional mice fail to replicate the pressure sensitive nature of physical drawing. Tablets have been widely employed as a way to make virtual sketching feel more natural. The SketchPad+ system [18] demonstrates the effectiveness of pen and tablet based input for architectural perspective drawing. Another obvious but important observation is that sketching is traditionally done in two dimensions. Humans tend to conceptualize designs most easily and naturally when working in two dimensions. 3D structure is implicitly understood by the mind, but it is easier to think of novel forms and movements in terms of images (2D projections) than models. The ease of conceptualizing in 2D motivates us to center our design on the principle that the actual 3D environment should be interacted with as little as possible during the design phase. Many previous research projects have explored ways to limit dimensionality in design. Kaillo’s 3D6B editor [13] projects all strokes onto 2D planes (as does our system in development). 3D structure emerges only as these planes are arranged in 3D space. 2D projective drawing has also been explored by Tolba et al. ([19], [20]). In their system, strokes are projected onto a unit sphere surrounding the viewer. Reprojecting these strokes allows for certain changes of the viewpoint, simulating 3D form without ever explicitly representing it. Core Design Principles The overarching design goal of this project is to build a qualitatively different type of CAD system that is intimately informed by the specific criteria of conceptual sketching for architectural design. Thus, the design parameters outlined above imply certain guidelines for this project. First, whereas most previous sketch-based CAD systems have relied heavily on automatic interpolation, we aim to as much as possible put the user in complete control. We thus simply provide convenient, flexible tools and a clear and modifiable framework in which to sketch. Second, while traditional 3D modeling requires a complete model to be maintained at all times, we explore the alternative of letting ambiguities and imprecision exist in the early stages of design. This is achieved by representing 3D data as a collage of discrete parts rather than as a consistent whole. This technique fits the rough nature of conceptual sketching. An architect might have inconsistent ideas for how various views onto a building should look. Our system will allow these inconsistencies to exist and hopefully will help architects to refine their designs with reference to such visualization. As a rendering technique, this collage process can be thought of as incorporating the user into the rendering loop. All the pieces of the collage are precisely arranged by the software, but the user’s mind is what glues them together. Sketching in 3D I am currently working on implementing a successor to the Mental Canvas [7] system for 3D sketching. Mental Canvas is a sketch-based CAD system that represents 3D form as a collection strokes projected onto planes arranged in 3D space. Users may either sketch directly on previously positioned planes or may draw from a particular viewpoint (as if the lines are attached to the surface of the camera lens) and subsequently push these lines out to their various intended locations in depth. In my implementation, I have placed emphasis on limiting the cognitive load of dimensionality. As discussed above, sketching is most natural in two dimensions. To this end, I have separated the 3D sketching process into two cognitively separate modules. First the user sketches a perspective drawing in 2D from particular view point. This specifies the u-v coordinates of a set of strokes. Next, the user pushes each of these strokes, or whole groups of strokes, back into the scene in the direction perpendicular to the image plane on which they were drawn. This second step specifies the z coordinate of the strokes. This process can be thought of as first drawing a projective cone of rays passing through the points drawn on the image plane in u-v coordinates. Then, in the second step, a two dimensional slice (or a more complex surface) of this cone is selected as the