74

JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, VOL. 5, NO. 3, AUGUST 2014

A Method of Reconstructing 3D Models from
Sketches by Extracting Features
Masaji Tanaka and Toshiaki Kaneeda
Department of Mechanical Systems Engineering, Okayama University of Science, Okayama, JAPAN
Email: {tanaka, kaneeda}@mech.ous.ac.jp

Abstract—The automatic reconstruction of a 3D object froma sketch has been very crucial research area in ComputerAided Design and Computer Vision since decades.
Conventionally most of proposed methods have applied thetechnique of line labelling. However, generally line labelingtechnique could be difficult for the reconstruction when
input sketches include many curves. In this paper, a method
for the reconstruction of mechanical parts by extracting
features repeatedly is proposed. In the method, the sketchesof cubes, cylinders, holes and fillets are applied as features.
Four examples that are difficult for line labelling techniqueare indicated in this paper.

a hole. In contrast, Example 1 could be also recognized asa strange solution as in Fig. 2(b) when line labelling
technique is only applied. The reason why Example 1 is aplate with a hole would be because he/she has alreadyknown the sketches of plates and holes respectively.

Index Terms—sketch, line drawing, reconstruction, CAD,
feature, mechanical parts

(a)

(b)

Figure 1. (a) A sketch of a cube, (b) line labelling in the cube.

I. INTRODUCTION
Generally line drawings as sketches of 3D objects arewidely used in designing mechanical products. The
automatic reconstruction of a 3D object from a sketch has
been very crucial research area in Computer-Aided
Design and Computer Vision since decades.
Conventionally most of proposed methods have applied
the technique of line labelling. However, generally line
labeling technique could be difficult for thereconstruction when input sketches include many curves.
In this paper, a method of reconstructing a 3D modelfrom a sketch of a mechanical part by extracting featuresfrom the sketch repeatedly is proposed. This method is anextension of the author’s past work, e.g. [1-2]. Generallyline labelling technique would be effective to recognize
3D convex or concave shapes from sketches. For example,
Fig. 1(a) illustrates a sketch of a cube. In line labelling
technique, firstly each of straight lines that form an outer
loop of the cube is named an occluding edge and its label
is an arrow. The directions of the arrows of all occludingedges are clockwise. Then three W junctions each ofwhich consists of two occluding edges and an edge
labeled “+” are recognized as in Fig. 1(b). Finally a Y
junction consisting of three “+” edges can be found. A Y
junction indicates a 3D convex shape in a sketch in line
labelling technique. Therefore, a model of a 3D cube of
the sketch can be generated automatically.
However, it seems to be difficult to suppose that ahuman understands sketches by only line labelling. For
example, when a human looks at Example 1 illustrated inFig. 2(a), he/she would recognize a sketch of a plate with© 2014 ACADEMY PUBLISHER
doi:10.4304/jait.5.3.74-78

(a)

(b)

Figure 2. (a) Example 1, (b) a strange solution.

Originally it could be assumed that a child firstly
learns how to sketch primitives and features of 3D objects.
In this method, primitives are included to features. They
could correspond to simpler tools for a human. Thenhe/she could draw sketches of tools whose shapes are
more complex by applying sketches of features. On theother hand, cubes, cylinders, holes, etc. are the features ofmechanical parts and they are modeled by combining thefeatures step by step in solid modelers. This combining
process is similar to the learning of how to draw sketchesof tools in a human.
In contrast, the recognition processes of the sketches ofmechanical parts could be considered as the reverse
processes of drawing sketches of them in a humanbecause generally the shapes of mechanical parts are too
various. If popular tools are sketched, they would berecognized at a glance in a human. This method applies
the recognition processes of the sketches of mechanicalparts in a human. In the present step of this method, thesketches of cubes, cylinders, holes and fillets could be
defined as in Section III. When a sketch of a mechanicalpart is input to this method, these features would beJOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, VOL. 5, NO. 3, AUGUST 2014

recognized and extracted step by step from the sketch.
The 3D model as the solution of the sketch could beobtained from combining extracted features. Four
examples that are difficult for line labelling technique areindicated in this paper.
II. RELATED WORKS
There are a great many papers for the reconstruction.
The classification of them is indicated such as in [3].
When designers draw a sketch of a 3D object, morenumber of visible edges and less number of hidden edges
would be drawn to express the object effectively. Moreeffective positions of 3D objects projected to sketches arecalled general positions. When designers draw sketchesof mechanical parts, it is natural that they draw only
visible lines. So in this method, hidden lines are notdrawn in sketches, and 3D objects are viewed in general
positions. Under these constraints, line labelling
technique has been developed and many kinds ofjunctions such as Y junction and W junction weresummarized into junction catalogs, e.g. [4-12]. Althoughthese references handled only polyhedrons as 3D objects,
moreover, sketches of curved 3D objects have beenhandled and the junction catalogs of curved lines weresummarized, e.g. [13-15].
On the other hand, Robert [16] attempted to extract
polyhedral primitives such as cubes from the sketches ofpolyhedrons to obtain 3D models as solutions. Also,
Wang et al. [17-19] attempted to extract cylinders fromsketches including curved lines for the reconstruction.
However, their methods did not handle machining
features such as holes and fillets in 3D objects.
III. THE ALGORITHM OF THIS METHOD
In this method, sketches are perfectly drawn in 2D
CAD systems as the orthogonal projections of 3D objects
placed in general position. Hidden lines of the 3D objects
are not drawn in the sketches. Also, each of input
sketches to this method consists of ellipses, arcs andstraight lines. Each of straight lines does not intersect toany other lines. A point is defined as an intersection oflines. A region is defined as a closed loop of lines. They
correspond to an edge, a vertex, and a face in 3D models.
The flowchart of the algorithm in this method is
illustrated in Fig. 3. Six types of sketches of features aredefined in this method as follows.
• CUBE
There are three parallelograms connecting to eachother at their three straight lines and their endpoints
are converged to a point.
• CYLINDER
There are an ellipse, an arc, and two parallel lines.
These lines connect to the ellipse and the arc
respectively and tangentially, and two connecting
regions are formed.
• HOLE
There are an ellipse and an arc. The arc is a part ofthe ellipse. The arc is placed in the inside of the© 2014 ACADEMY PUBLISHER

75

ellipse and they are connected to each other. The
ellipse is placed in the inside of some region.
• GENERAL FILLET
There are two arcs and four straight lines. The two
arcs are placed like a part of the sketch of a cylinder.
Each pair of parallel lines connects to the arcs
individually and tangentially.
• PARTIAL FILLET
There are two arcs and four straight lines. The two
arcs are placed like a part of the sketch of a cylinder.
Three lines connect to the end points of the arcs
individually and tangentially. One line connects toboth of the arcs tangentially.
• HIDDEN FILLET
There are an arc and two straight lines. Each of theselines connects to the arc tangentially.
Input a sketch
Recognize lines and regions
Draw additional lines
Search a feature

Are there any features?

noSearch a featureby the other ways

yes
Recognize and extract a feature
Remove lines forming T junctions

Are there any lines?

noyes
Restore isolated lines
Make 3D features
Combine 3D features
Output a 3D model
Figure 3. Flowchart of algorithm in this method.

When some cube, cylinder or hole is recognized in a
sketch, the hidden lines of its 3D model are drawn as
additional lines because they can form contact faces
described as follows. Additional lines are drawn asdashed lines. Also, in the recognition of a cube, threelines forming a Y junction are drawn as solid lines.
Additionally the following three operations are applied inthis method.
• Extension of lines as additional lines:
Two straight lines forming a L junction are extended

76

JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, VOL. 5, NO. 3, AUGUST 2014

within the nearest region of a sketch. Two straightlines in both sides of a W junction are extended
within the nearest region of a sketch and the W
junction is changed into a junction consisting of fivelines. The extended parts of the lines are additionallines. The reason why the lines are extended is
explained in Fig. 4. When a work shaping a cube ismachined, two samples in Fig. 4(a), (b) are basic. In
this figure, when the extensions are applied as in Fig.
4(c), (d), it becomes easy to recognize the feature of
the sketch of a cube for this method.
• Restoration of lines:
When a feature is extracted from a sketch, someisolated line that does not form any regions can exist.
They are extended until forming a region or
removed from the sketch.
• Shaping features:
If only two parallelograms that become parts of thesketch of a cube can be searched in a sketch,
additional lines are drawn to the sketch to shape acomplete sketch of the cube.

(a)

(b)

(c)

(d)

Figure 4. Samples of extending lines.

When a feature is extracted from a sketch, to searchanother feature clearly, each line forming a T junctionthat consists of two straight lines shaping “T” is removed
repeatedly, and then additional lines are drawn again.
Basically solid lines are prior to dashed lines in therecognition of features. Features are extracted from asketch repeatedly until there are no lines in the sketch.
Basically the extractions of holes and fillets are prior to
the extraction of cubes and cylinders. Also, general fillets,
partial fillets and hidden fillets are extracted in this order.
After all extractions are finished, extracted features areconverted to 3D features and the 3D model of the sketch
can be generated by combining them.
In the combinations, a contact face between two 3Dfeatures could be searched as follows. In the sketch of acube, regions corresponding to contact faces are formedby additional lines. In the sketch of a cylinder, a region
corresponding to a contact face is formed by its arc and afillet is the same. In the sketch of a hole, its cylindricalface becomes the contact face of the 3D hole. Also, cubes
and cylinders are prior to holes and fillets in their
combinations.
The ways to extract 3D features from sketches are as
follows. General fillets and partial fillets are changed into
sketches of 3D corners when they are extracted fromsketches. Each of hidden fillets is changed into a Ljunction. The 3D shape of a hidden fillet is made after allof the other 3D features are combined. The way to makethat is the same way as solid modelers. All lines ofcylinders and holes are removed. When a 3D cube is© 2014 ACADEMY PUBLISHER

extracted from a sketch, all of its solid lines are removed
because they are not needed to search another feature.
IV. EXAMPLESIn Example 1, firstly a sketch of a hole can berecognized as in Fig. 5(a) and extracted as in Fig. 5(b) inthis method. Then a sketch of a cube can be recognized asin Fig. 5(c). Since the cylindrical face of the hole is thecontact face to combine to the cube, they can be
combined correctly and the solution of Example 1 can beobtained as in Fig. 5(d). This method never outputs Fig.
2(b) because only a sketch of a hole can be recognized in
Example 1.
Fig. 6(a) illustrates Example 2 that is a sketch of asimplified and clarified motor bracket. In this method,
firstly three holes can be recognized as in Fig. 6(b) andextracted as in Fig. 6(c). In this figure, two general fillets
drawn as bold lines can be recognized, and each of themis extracted as in Fig. 6(d). In this figure, a pair of general
fillets drawn as bold lines can be recognized. If the rightfillet of them is extracted as in Fig. 6(e), two partial fillets
and a hidden fillet can be extracted respectively as in thisfigure. However, no more features can be recognized inthis figure. So another general fillet is extracted as in Fig.
6(f). In this figure, two partial fillets and a hidden fillet
are extracted respectively. So a cube can be recognized asin Fig. 6(g). When the solid lines of the cube are removed,
an isolated line and a line forming two T junctions arerecognized as in Fig. 6(h). Since they can be removed, acube can be recognized finally. Fig. 6(i) illustrates eachof extracted features (f1, f2, … , f11) in Example 2. They
are combined as f9+f10-f1-f2-f3-f4-f5+f6-f7-f8-f11. The
way of combining f11 is the same way as solid modelers.
Fig. 6(j) illustrates the overviews of the solution in
Example 2.

(a)

(b)

(c)

(d)

Figure 5. (a) Recognition of a hole in Example 1, (b) extractionof the hole, (c) recognition of a cube, (d) overview of thesolution in Example 1.

Fig. 7(a) illustrates Example 3 that is a sketch of abubble wrap. In this method, firstly four cylinders can berecognized individually as in Fig. 7(b) and they are
extracted as in Fig. 7(c). In this figure, all of isolated lines
are extended and four cylinders can be recognized as in

JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, VOL. 5, NO. 3, AUGUST 2014

Fig. 7(d). In the same way, when these cylinders are
extracted, two cylinders can be recognized as in Fig. 7(e).
When these cylinders are extracted, finally a cube can berecognized. The solution in Example 3 can be obtained asin Fig. 7(f).

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

77

Fig. 8(a) illustrates Example 4 that is referred from[10]. In this figure, a fillet is added to a figure of [10]. In
this method, firstly a hidden fillet is recognized andextracted as in Fig. 8(b). Then a cube is recognized as inFig. 8(c). In Fig. 8(d), the solid lines of the cube areremoved. In Fig. 8(e), all of lines forming T junctions are
removed. In this figure, an isolated line can be extended.
So Fig. 8(f) can be drawn. In Fig. 8(g), a cube isrecognized. In Fig. 8(h), the solid lines of the cube areremoved. In Fig. 8(i), all of lines forming T junctions are
removed. In Fig. 8(j), additional lines are drawn. In Fig.
8(k), a cube is recognized. In Fig. 8(l), the solid lines of
the cube are removed. In this figure, since two isolatedlines can be removed, a cube can be recognized finally.
When all of extracted features are combined except thehidden fillet, Fig. 8(m) can be obtained. When the fillet isadded to this figure, the solution of Example 4 can beobtained as in Fig. 8(n).

f7
f5

f1

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

f9
f6
f10

f8

f11

f4
f3

f2
(i)

(j)

Figure 6. (a) Example 2, (b) recognition of three holes, (c)extraction of the holes and recognition of two general fillets, (d)extraction of the fillets and recognition of a pair of general
fillets, (e) the case to extract the right bold fillet in (d), (f) thecase to extract the left bold fillet in (d), (g) recognition of a cube,(h) removal of solid lines in the cube, (i) each of extracted
features, (j) overviews of the solution in Example 2.

(a)

(b)

(c)
(m)

(d)

(e)

(f)

Figure 7. (a) Example 3, (b) recognition of four cylinders, (c)extraction of the cylinders, (d) recognition of four cylinders
from (c), (e) recognition of two cylinders, (g) overviews of thesolution in Example 3.
© 2014 ACADEMY PUBLISHER(n)Figure 8. (a) Example 4, (b) recognition and extraction of ahidden fillet, (c) recognition of a cube, (d) removal of solid lines
in the cube, (e) removal of lines forming T junctions, (f)restoration of an isolated line, (g) recognition of a cube from (f),(h) removal of solid lines in the cube in (g), (i) removal of lines
forming T junctions in (h), (j) drawing of additional lines, (k)recognition of a cube from (j), (l) removal of solid lines in the
cube in (k), (m) combination of extracted features except thehidden fillet, (n) overview of the solution in Example 4.

78

JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, VOL. 5, NO. 3, AUGUST 2014

V. CONCLUSION
In this paper, a method of reconstructing a 3D modelfrom a sketch of a mechanical part by extracting features
is proposed. The effectiveness of this method isdemonstrated by four examples in this paper. Since all offeatures applied in this method are adapted to machining
features of mechanical parts, obviously this method
cannot apply to 3D objects whose shapes are difficult to
be machined such as pyramids and prisms. To handle
them, the definitions of new features are required in thismethod. Also, when sketches of 3D objects whose shapes
are very complex are input, the search of features would
become more complex.
However, it seems to be difficult to handle examples ofthis paper by conventional methods using line labelling
technique. Especially it is easier to recognize Example 3
by recognizing each of cylinders than by recognizing
each of junctions of lines. The recognition of thecylinders is similar to the way of a human to understand
Example 3. As the result, this method would be different
from conventional methods using line labelling techniquein the directions of their extensibility, and when thismethod is extended to more various kinds of sketches, it
might become more human.
REFERENCES[1] M. Tanaka, K. Iwama, A. Hosoda and T. Watanabe,“Decomposition of a 2D Assembly Drawing into 3D Part
Drawings,” Computer-Aided Design, 30(1), pp. 37-46,
1998, doi:10.1016/S0010-4485(97)00051-1[2] M. Tanaka, L. Anthony, T. Kaneeda and J. Hirooka, “ASingle Solution Method for Converting 2D Assembly
Drawings to 3D Part Drawings,” Computer-Aided Design,
36(8), pp. 723-734, 2004, doi:10.1016/j.cad.2003.08.003[3] P. Company, A. Piquer, M. Contero, and F. Naya, “Asurvey on geometrical reconstruction as a core technology
to sketch-based modeling,” Computers & Graphics, 29(6),pp. 892–904, 2005, doi:10.1016/j.cag.2005.09.007.
[4] M. B. Clowes, “On seeing things,” Artificial Intelligence,2(1), pp.79–116, 1971, doi:10.1016/0004-3702(71)90005-1[5] D. A. Huffman, “Impossible Objects as Nonsense
Sentences,” Machine Intelligence 6, New York American
Elsevier, pp. 295–323, 1971.
[6] T. Kanade, “Recovery of the Three-Dimensional Shape ofan Object from a Single View, Artificial Intelligence,” 17,
1981, pp. 409–460. doi:10.1016/0004-3702(81)90031-X[7] K. Sugihara, “Machine Interpretation of Line Drawings,
MIT Press,” 1986.
[8] L. M. Kirousis and C.H. Papadimitriou, “The complexityof recognizing polyhedral scenes,” Journal of Computer
System Sciences, 37(1), pp. 14–38, 1988, doi:
10.1016/0022-0000(88)90043-8© 2014 ACADEMY PUBLISHER[9] I. J. Grimstead and R. R. Martin, “Creating solid models
from single 2D sketches,” Proceedings of the third ACM
symposium on SMA ’95, pp. 323-337, 1995, doi:10.1145/
218013.218082[10] L. Cao, J. Liu and X. Tang, “What the back of the object
looks like: 3D reconstruction from line drawings without
hidden lines,” IEEE Transaction on Pattern Analysis andMachine Intelligence, 30(3), pp. 507–517, 2007, doi:
10.1109/TPAMI.2007.1185[11] P. A. C. Varley and R. R. Martin, The Junction Catalogue
for Labelling Line Drawings of Polyhedra with Tetrahedral
Vertices,” Shape Modeling, 7(1), pp. 23-44, 2001, doi:
10.1142/S0218654301000035[12] P. A. C. Varley, R. R. Martin and H. Suzuki, “Frontal
geometry from sketches of engineering objects: is line
labelling necessary?,” Computer-Aided Design, 37(12), pp.
1285–1307, 2005, doi:10.1016/j.cad.2005.01.002[13] J. Malik, “Interpreting Line Drawings of Curved Objects,”
International Journal of Computer Vision, 1, pp. 73–103,
1987, doi:10.1007/BF00128527[14] M. C. Cooper, "Linear-time algorithms for testing therealisability of line drawings of curved objects," Artificial
Intelligence, 108(1-2), pp. 31-67, 1999, doi:10.1016/S0004-3702(98)00118-0[15] M. C. Cooper, “Wireframe Projections: PhysicalRealisability of Curved Objects and Unambiguous
Reconstruction of Simple Polyhedra,” Computer Vision64(1), pp. 69–88, 2005, doi:10.1007/s11263-005-1087-9[16] L. G. Robert, “Machine Perception of Three-Dimensional
Solids,” Ph.D. thesis, MIT Dep. Of Electrical Engineering,1963.
[17] W. Wang and G. G. Grinstein, “A polyhedral object's
CSG-Rep reconstruction from a single 2D line drawing,”
Proceedings of SPIE, 1192(1), pp. 230-238, 1989.
[18] W. Wang, “A Regular Curved Object's CSG-Rep
Reconstruction from a Single 2D Line Drawing,”
Proceedings of SPIE, 1608(1), pp. 119-127, 1991.
[19] W. Wang and G. G. Grinstein, “A Survey of 3D Solid
Reconstruction from 2D Projection Line Drawings,”
Computer Graphics Forum, 12(2), pp. 137-158, 1993 doi:
10.1111/1467-8659.1220137

Masaji Tanaka received his Doctor of Engineering inmechanical engineering from Ritsumeikan University, Japan.
He is currently a professor of mechanical systems engineeringat Okayama University of Science, Japan. His current researchinterests include machine learning, computer vision, CAD/CAM.
Toshiaki Kakeeda received his Doctor of Engineering inprecision engineering from Osaka University, Japan. He iscurrently a professor of mechanical systems engineering at
Okayama University of Science, Japan. His current researchinterests include ultra precision machining, artificial joints.