M-Reps: A New Object Representation for Graphics

M-reps are a multiscale approach to the modeling and rendering of 3D solid geometry. Traditional geometric models, whether b-reps or CSG, are represented at infinitesimal spatial scale and then require simplification to meet needs requiring coarser scale or smaller data sets. We have developed a model that is designed at successively smaller scales and supports a coarse-to-fine hierarchy in design, rendering, physical deformation, and other graphics operations. We base our representation on figural models, defined at coarse scale by a hierarchy of figures – protrusions, indentations, corners, neighboring figures, and included figures – which simultaneously represent solid regions and their boundaries. To capture local shape at scale and thus local zoom-invariance, the figural components imply a fuzzy, i.e., probabilistically described boundary position with a widthand scale-proportional tolerance. At small scale these figures are made precise by displacement maps, geometric textures, or image textures. While these models can exist in 2D, we focus on models of 3D objects. We note the needs of a variety of graphics operations and thus motivate the following definition of an m-rep. A model for a single figure is made from a mesh of medial atoms (hence the name m-reps), each atom describing not only a position and width, but also a local figural frame implying figural directions, and an object angle between opposing, corresponding positions on the implied boundary. In addition, width proportionality constants indicate mesh link length, boundary tolerance, boundary curvature limits, and, for 3D image visualization, an interrogation aperture. Thus, a figural mesh defines the figural boundary to within a widthproportional tolerance and provides a width-proportional sampling of the figure’s implied medial surface. A figural model, then, consists of a directed acyclic graph of figure meshes, with interfigural, intramesh, and interscale links capturing information about differences in position (and thus figural length and subfigural offset), figural width, narrowing rate, boundary curvature, and figural orientation. The leaves of the figural graph also contain the displacement maps, geometric textures, or image textures that are used to give infinitesimal scale to the model where fine detail is needed. We describe methods for building m-rep models by a design tool, from b-rep or CSG representations, or from a 3D-intensity dataset. We present a method for deforming m-reps into image data, allowing model-directed visualization of objects in volume data. We describe algorithms for rendering m-reps, and show two uses in graphics for them – rendering for 3D visualization and computer-aided design – with preliminary results. Algorithms for other graphics objectives are sketched. February 8, 2000 Submitted to ACM TOG 2 1. The Need for a New Model Computer graphics modeling in 3D rests on five basic techniques and their variations: • boundary representations (b-reps) by polygons, higher-order patches, or subdivision meshes; • constructive solid geometry (CSG), which uses Boolean operations on simple solid primitives such as spheres, cylinders, blocks, and tori; • implicit surfaces, which define object boundaries as isosurfaces of density functions; • skeletal surfaces, which define object boundaries as offset or convolution surfaces; and • image-based rendering (IBR), where geometry is reduced to a depthor disparity-map over a pixel image. We will see that most current techniques of computer graphics share the following problems: lack of multilocal and multiscale information with the consequence of inefficiency or inability in handling an object or a significant component as a unit; over-precision with the consequence of inflexibility, ill conditioned computation, and serious inefficiency; and inability to handle objects simultaneously as surfaces, which is needed for rendering, and as solids, which is needed for physically based modeling and is convenient for design. We motivate m-reps by examining these weaknesses of current graphics techniques regarding (1) object design, (2) rendering, (3) physically based deformation, and (4) fusion of real-world visualizations. Object design. With b-reps, nonlocal properties are often hard to specify – for example, that a portion of a surface is (and perhaps should remain) a section of a cone. While subdivision surfaces allow multiscale modeling at different subdivision levels, they still lack such multilocal information. On the other hand, object design via CSG gives too little flexibility in the vocabulary of primitive shapes. Implicit models have problems of local control and unpredictable blending. The restricted, projective geometry of IBR and its discrete pixelsampling make it ill-suited to object design. Skeletal methods provide better local and multilocal control over implicit object shape but have problems with pruning and bifurcations of their medial structures. Rendering. The rendering of nearby objects requires detailed boundary information; while b-reps provide this directly, CSG models require considerable computation. Efficient rendering of distant objects by b-reps or by implicit representations requires a simplification that is computationally complex. The source of the complexity is the unavailability of multilocal information, e.g., as to the way in which an object is formed by a hierarchy of protrusions and indentations. While IBR techniques were created for fast, O(1) rendering, they don’t easily allow dynamic scenes because the multilocal properties of occlusion are not represented and their pixel-based nature creates sampling problems for reconstruction and display. Skeletally-based objects are generally rendered as implicit, blobby models or by tessellation of an implied-boundary isosurface, which allows level-ofdetail (LOD) control of the blobs or tessellation; however, they typically do not carry tolerance information that would support an optimal choice of this LOD. Physically based operations. These operations depend on having a mechanical model on a solid representation, which CSG provides directly but b-reps, implicit representations, and IBR do not. Moreover, even when a solid 1 Multilocal is defined as “involving multiple separated positions. ” 2 Multiscale is defined as “at multiple levels of detail. ” February 8, 2000 Submitted to ACM TOG 3 representation is derived from b-reps, it is dealt with mechanically only at small scale, e.g., by finite element techniques, resulting in a large computational load. An ability to work in coarser units seems necessary, but the locality of the boundary patches does not yield such mechanically sensible units without lengthy computation. Skeletal methods have advantages in this regard and have found common usage in animation of articulated figures. Fusion of images. The fusion of images from nearby viewpoints in image-based rendering depends on object correspondence determinations, given the typical inability to measure the viewpoint from each image with adequate precision. The information in b-reps, implicit representations, or pixel maps is too local to determine the correct way to fuse scenes where changes in the inter-object occlusion from one image to the next reveal previously hidden image regions or occlude previously visible ones. To overcome these drawbacks, we need a representation that is simultaneously easy to view as boundaries or as solid, that is easy to view at multiple levels of scale, and that can handle the multilocal aspects of objects and the solid figures that make them up. The very precision of standard object representations is an impediment. B-rep and CSG models create a Boolean division of modeling space: for every real position, one can determine unequivocally whether the position is inside or outside the represented object. Even if the representation is an approximation and thus not necessarily accurate – as with an object boundary represented by a polygonal mesh or a spline – the representation itself is seen as precise. As a result, computing Boolean operations on objects is a difficult, often ill-conditioned problem. Also, when an object is to be viewed or manipulated in a situation requiring only large-scale information, either the calculation is quite inefficient or simplification must be carried out to reduce the object to a coarser approximation. Examples of such applications include perspective rendering of objects very distant from the viewer, occlusion culling in complex models, and cases in mechanical modeling where properties must be based on the whole object but need only be grossly correct. An alternative approach is to associate explicit boundary tolerances with various stages and locations of the representation and to represent the object successively from large scale to small, with a larger tolerance associated with the larger scale components. But for the representation to be locally magnification invariant – and thus a valid representation for shape not only globally but also locally – the acceptable tolerance for a component of an object must be proportional to its scale. This scale may vary across a component as well. A useful and longstanding measure of the object size at a position of a figure is the figural width, an aspect of a multilocal, solid representation. This measure is based on the understanding that the medial relationship between points on opposite sides of a figure is an important factor in the object's shape description (see Fig. 1). Biederman [1987], Marr [1978], Burbeck [1996], Leyton [1992], Lee [1995], and others have produced psychophysical and neurophysiological evidence for the importance of medial relationships (in 2D projection) in human vision. The relation has also been explored in 3D by Nackman [1985] and Vermeer [1994], and medial axis and modeling techniques have been applied by many researchers, including Bloomenthal [1991], Wyvill [1986], Sin