977

Abstraction of Sketch Features for Predicting Hidden Shapes ofSketches for The Automatic Conversion into 3D Models
Masaji Tanaka1, Tetsuya Asano2

and Chiharu Higashino2

1Okayamaity University of Science, tanaka@mech.ous.ac.jp
2Aikoku Alpha Corporation, {higashino, t-asano}@aikoku.com

Corresponding author: Masaji Tanaka, tanaka@mech.ous.ac.jp
Abstract. Sketches in the form of line drawings are important illustrations to directly
express the overviews of objects, especially mechanical objects. Numerous methods
that automatically convert sketches into 3D models have been proposed. However,no practical system for conversion has been developed till now. Consequently, we
have developed a method as SFBCM (Sketch Feature-Based Conversion Method) for
this conversion. In SFBCM, when a sketch is input, SFs (Sketch Features) thatindicate simple sketches of objects such as cubes, cylinders are detected andextracted as 3D features step by step. Consequently, a 3D model can be obtained by
combining them based on the sketch. However, several issues have remained in
SFBCM. Especially, to handle hidden shapes in sketches was an important issue. In
this paper, ASF (Abstract Sketch Feature) is introduced for predicting hidden shapesof sketches. Each ASF is a part of an SF. This prediction is based on human perception
and enables SFBCM to be simplified drastically. For example, when people look at asketch of a pole standing on a plate, they think the two objects are contacted
centrally, but they can be uncontacted and placed near their edges theoretically. The
introduction of ASF can clarify this contact problem. In this paper, ASF for SFBCM is
explained in detail with an example, and many other complex examples are describedfor indicating its effectiveness.
Keywords: Abstraction, SFBCM, Abstract Sketch Feature, ASF, Human Perception.
DOI: https://doi.org/10.14733/cadaps.2022.977-987

1

INTRODUCTION

Sketches in the form of line drawings are commonly observed in magazines, books, manuals, etc.
Sketches are also important for designers, especially mechanical designers, when they invent newideas of products and their parts. The automatic conversion of sketches into 3D models will beadvantageous for several applications. Moreover, it is expected that robots will be able to understandsketches by their converted 3D models in the future. In the last fifty years, numerous methods toautomatically convert sketches into 3D models have been considered and developed. However, noComputer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

978real system for the conversion has been developed till now. We have been developing methods forthe conversion of sketches to 3D models for approximately eight years. Consequently, we proposeda method as SFBCM (Sketch Feature-Based Conversion Method) to achieve this conversion [16].
Fig. 1 shows three basic SFs (Sketch Features) that indicate a cuboid, cylinder, and round hole.

Figure 1: Basic three SFs: (a) Cuboid, (b) Cylinder, and (c) Round hole.
In SFBCM, when a sketch is input, the 3D model of it can be obtained by detecting and extracting
SFs step by step and combining them. Fig. 2(a) shows Example 1 that is a sketch of a mechanicalpart. When Example 1 is input to SFBCM, first a cylinder is detected and extracted. Second, a cuboid
is detected and extracted. If remained lines can be restored into a cuboid, the 3D model of Example
1 can be obtained. This scene can be seen in Fig. 2(b). However, several issues have remained in
SFBCM. Obviously, to handle hidden shapes in sketches is an important issue as shown in therestoration of the cuboid in Example 1. Also, when a person looks at Example 1, he/she thinks thata cylinder and a rectangular column are placed symmetrically and vertically on a plate. However,many solutions as 3D models can exist in theory from that. For example, 3D models of Fig. 2(c)which is a screen shot of solid modeler (SolidWorks) can correspond to Fig. 2(a).

(a)

(b)

(c)

Figure 2: Example 1: (a) Example 1, (b) 3D model of (a), and (c) Another 3D models corresponding
to (a).
Therefore, it is difficult or impossible to predict exact shapes from hidden shapes in sketches because
many solutions can exist despite that there are many hidden shapes in sketches. In our past methods,
inductive learning systems were applied to the prediction as restoration processes of SFs [19].
However, generally learning systems seem to be difficult for applying to the prediction because there
are too many types of hidden shapes in sketches. Moreover, to collect big data of that will be wastefuland not effective because each data will have to be input by user(s) manually until all target hidden
shapes can be predicted exactly. On the other hand, human perception will be effective such as therecognition of Example 1. Therefore, it might be possible to construct a theory of the prediction
based on human perception. If the theory is constructed, learning systems will be useless.
In this paper, human perception is applied to the prediction of hidden shapes in sketches for
SFBCM. For this application, we attempt to abstract SFs, and ASF (Abstract Sketch Feature) isdefined. Each ASF is a part of an SF. Fig. 3 shows two SFs and an ASF separately in Example 1.
There are an SF of a cylinder (green), a cuboid (pink), and an ASF (blue). Here, two dotted lines ofthe pink cuboid are easy to predict in SFBCM. The blue ASF can be predicted as an SF of a cuboid
by applying human perception. Consequently, we propose our improved method as ASF for SFBCM
in this paper. The rest of this paper is organized as follows. In Section 2, related works of this method
is explained. In Section 3, the algorithm of this method is explained in detail with Example 1. In
Section 4, three examples are indicated. In Section 5 and Section 6, several issues and the possibilityof this method to the future are discussed and conclusion is described.

Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

979

Figure 3: Two SFs (cylinder and cuboid) and an ASF in Example 1.
2

RELATED WORKS

There are numerous papers on the automatic recognition of sketches in the form of line drawings
and/or their conversion into 3D models. Their classification is presented in [6]. The original line
labeling technique was developed as Huffman-Clowes labeling, [5],[12]. In this labeling process, theobjects of sketches were limited to opaque trihedral polyhedrons, and each sketch was an orthogonal
projection of an object viewed from a general position. Each line segment of a sketch was labeled as“+” (convex line), “-” (concave line), or with an arrow (occluding line). From the labeling, the vertices
were classified into the following four types of junctions: L, W, T, and Y-junctions. This naming wasderived from the shapes of the alphabets, i.e., “L,” “W,” “T,” and “Y,” respectively. The relationships
between the labeling and the junctions were summarized as a junction dictionary. Moreover, Varley
et al. [21] handled high-order junctions such as K- and X-junctions. Although these labelingtechniques were applied to sketches consisting of only straight lines, Malik [14] created a junction
dictionary for curved lines in sketches. For example, if a straight line segment expresses a limb lineof a cylindrical face of an object, it was labeled with double arrows.
Fig. 4 shows the line labeling in Example 1. In Fig. 4(a), each line segment is labeled. Arrowed,
double arrowed, “+,” and “-“ lines are colored blue, pink, red, and green, respectively. From thisfigure, each junction can be recognized, as shown in Fig. 4(b), by using junction dictionaries. Here,four blue points are L-junctions, and two light blue points are Curvature-L-junctions [14]. A red pointis a Y-junctions, and two pink points are Three-Tangent points [14]. In our method, these threepoints are regarded as Y-junctions. Also, four green points are W-junctions, and five brown points
are T-junctions. The red Y-junction expresses a convex corner because it is formed from three “+”lines. Varley et al. [21] identified a “cubic corner” based on [15] from a convex Y-junction. In their
methods, when a Y-junction is given in an x-y coordinate system and three lines form a cubic corner,
their equation can calculate all z values of terminals in the lines. Consequently, Y-junctions
expressing cubic corners in sketches could be converted to 3D cubic corners in 3D models. However,it cannot be applied to ambiguous cuboid sketches such as shown in Fig. 5(a) although people canunderstand it at a glance. We have searched this human perception for recognizing this figure, andfound that people draw sketches as isometric and symmetrical as they can [16]. For example, whena person draw a sketch of a long pole, he/she usually draw it as shown in not Fig. 5(b) but Fig. 5(c)because it is easy to understand the pole exactly for people. Also, people do not usually understand
Example 1 as Fig. 2(c). In general, hidden shapes of sketches occur at T-junctions. There are several
researches for the prediction of the hidden shapes from T-junctions. For instance, Cao et al. [2]attempted topological approach although they handled only polyhedrons.
L
W

L
W
L

T

T

L
Y

W
T

T

Y

L
T Y

L
W

Figure 4: Line labeling in Example 1: (a) Line labeling and (b) Junctions.
Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

980

(a)

(b)

(c)

Figure 5: Three types of sketches: (a) Ambiguous but understandable cuboid sketch, (b) A sketchof a long pole, and (c) Another sketch of the pole.
In recent years, Company et al. [7] investigated techniques to detect junctions from sketches using
human perception. From this analysis, it can be determined that human perception is important for
conversion, which is incorporated in our method. Interactive or semiautomatic systems forconversion were previously proposed, [1],[24]; however, there are no fully automatic systems forconversion. Further, 3D sketching systems were also developed [10-11]. Although these systems
might be useful for beginners, they are only a few types of solid modelers, for example, CATIA and
SolidWorks. Neural network techniques especially deep learning techniques have also been actively
used for conversion [3-4],[9],[13],[20],[22-23]. Although these learning techniques will be effective
to find 3D objects from sketches automatically, it will be difficult to convert sketches into 3D models
geometrically and precisely. Moreover, the learning will be suitable for known objects such as tables,
chairs, cups; however, it is not suitable for different types of mechanical and creative objects because
each of them must be learned individually and repeatedly for different cases. The learning of theseobjects seems to be wasteful. In summary, although all the studies described above are effective forconversion, the types of convertible objects are strictly limited. Our proposed method in this paper
aims to extend the limitation of convertible objects significantly more than conventional techniquesby predicting hidden shapes of sketches in accordance with human perception.
3

ALGORITHM OF THIS METHOD

In this method, all SFs are initially defined. They are quite different from “machining features” in
CAD techniques. Firstly, we found that people can recognize and draw simple sketches as shown in
Fig. 1, and they can be recognized as 3D models. So, we got an idea that if a complex sketch canbe disassembled into these simple sketches, it can be converted to a 3D model. Therefore, we have
not referred to machining features and their recognition techniques. This is because finding more
effective SFs for the conversion is a more important problem so that the abstraction of SFs is themain theme in this paper. However, abstracted machining features would be meaningless because
they cannot be machined. In this method, a sketch is drawn in 2D drawing systems on a PC, tablet,
etc. Also, a sketch is an orthogonal and opaque projection of a 3D object placed in a general position.
In the present step of this method, main targets of sketches are mechanical parts. In addition, eachsketch consists of ellipses, elliptical arcs, and straight lines. Images and freehand sketches are not
be handled in this method because this problem can be separated from the conversion. For example,
the problem was handled in [1],[8].
The algorithm of this method is shown in Fig. 6. Here, this method is explained using Example
1 shown in Fig. 2(a) in accordance with the algorithm. When Example 1 is input, in Step 1 of thealgorithm, straight lines are divided at their intersections. In Step 2, additional lines are drawn as
dotted lines. They can be drawn from T-, W-, and L-junctions, such as extending their lines within
the input sketch. If a curved line is an element of a T-junction, it is extended in a manner similar toforming an oval. If an additional line cannot become a part of a closed loop of lines, it is cut until
the loop is formed. In Example 1, ten additional lines (brown) can be drawn as shown in Fig. 7(a).
In Step 3, each region forming a closed loop of line(s) is recognized for detecting an SF. In Step 4,a (red) cylinder sketch can be detected as shown in Fig. 7(b). In Step 6, all useless additional lines
are removed, and a hidden (red) elliptical arc can be drawn as an additional line in Step 7 as shownin this figure. Step 8 is omitted at this time. In Step 9, the cylinder sketch can be extracted as a 3D
cylinder model (f1) as shown in Fig. 7(c). The dimensions of the f1 are corresponded to the cylindersketch because of the assumption that sketches are drawn as isometric as possible. In this method,
Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

981

each extracted SF can be called a 3D feature. After this extraction, the contact face between f1 andthe other 3D feature become an (blue) ellipse as shown in this figure. There is an assumption that
f1 contacts at its bottom face to the other 3D feature. This assumption can be almost correct inhuman perception because people usually do not predict that they are separated such as Fig. 2(c).
In Step 10, this process goes back to Step 1 because many lines are remained. Continuously, a (red)cuboid sketch can be detected as shown in Fig. 7(d), and a 3D coordinate system can be defined.
This definition is explained in the next paragraph. When the cuboid is extracted as a 3D feature (f2),five (green) lines are remained as shown in Fig. 7(e). These lines cannot form any SFs. So, we
attempt to abstract SFs. In this method, ASF (Abstract Sketch Feature) is introduced as the firststep of this abstraction. Each ASF becomes a cut SF because it is usually occurred at T-junction(s).
In this figure, five (green) lines can become an ASF of a cuboid sketch in this method. However, thelength of the cuboid to f2 cannot be decided at this time. Therefore, we analyze that as follows.
Input a sketch

1) Recognize each line segment
2) Draw additional lines
3) Recognize each region of lines

yes4) Can detect a SF?

5) Can predict SF(s)from ASF(s)?

no

yes
no

6) Remove useless addtional lines7) Draw hidden line(s) of the SF8) Decide a 3D coordinate system

9) Extract the SF as a 3D featureand recognize contact face(s)yes10) Are there any lines
in the sketch?

no11) Combine all extracted
3D features

12) Output a 3D model

Figure 6: The algorithm of this method.
The 3D coordinate system of Example 1 (omitting f1) can be defined from the convex Y-junction of
f2 such as shown in Fig. 8(a). In this figure, P1P2 is one of the lines in the ASF. In Fig. 8(b), P1P2 isextended as P1P3. In this figure, the ASF contacts to f2 but their two T-junctions are removed.
Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

982

Therefore, it is found that if P1P3 is shorter than this figure, they are not contacted. In the sameway, if P1P2 is extended as P1P4 as shown in Fig. 8(c), the ASF contacts to f2 but if P1P4 is longer
than this figure, they will be overlapped. Therefore, it is found that the length of the line is morethan P1P2 and less than or equal to P1P4. Here, when a person looks at Example 1, he/she may
predict that the ASF contacts to f2 centrally. It is a human perception. In Fig. 8(d), P1P2 is extended
as P1P5. In this figure, the (brown) contact face of the ASF is placed centrally to f2. This prediction
can correspond to that P5 is the middle point between P3 and P4. In general, people prefer to selectcenter/middle points in the range of selective areas because of the human perception. Consequently,
an SF of a cuboid as f3 can be defined from the ASF in Step 5. Finally, when three 3D features (f1,
f2, f3) can be combined at their contact faces in Step 11, the 3D model of Example 1 can be obtainedin Step 12 such as shown in Fig. 2(b). In Fig. 8(e), another SF of a cuboid is drawn to Fig. 8(a). In
this case, it is found that the length of f3 is decided as P6P7 by seeing from y axis. Therefore, it isfound that many problems still exist to decide the length of f3.

ASF

f2

f1

(a)

(b)

(c)

(d)

(e)

Figure 7: The process to convert Example 1 into its 3D model: (a) Drawing of ten additional lines,(b) Detection of an SF of a cylinder, (c) Extraction of the cylinder (f1) and recognition of the contact
face of it, (d) Detection of a cuboid, and (e) Extraction of the cuboid (f2) and detection of an ASF ofa cuboid.
z
z

P1

z

x

P1

P2

y

z

x

P1

P3

y

P1

P4

y

z

f3

x

x

x

P5

y

y

P6

O

O

O

O

(a)

(b)

(c)

(d)

P7

O

(e)

Figure 8: The prediction of the length in the ASF: (a) 3D coordinate system, (b) Minimum contact
between f2 and ASF, (c) Maximum contact between them, (d) Center contact between them, and(e) A case that ASF contacts to both two cuboids.
Lastly, defined SFs and ASFs in our present step are indicated in Fig. 9 and Fig. 10 respectively. In
SFs, three of them are shown in Fig. 1. The other eight SFs are shown in Fig. 9. The detailed
explanation of them can be referred in [17-18].

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 9: Eight SFs: (a) Polygonal extrusion, (b) Multi-extrusion, (c) Rib, (d) Round rib, (e) Pipe,(f) Front fillet, (g) Side fillet, and (h) Hidden fillet.

Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

983

For example, Fig. 9(a) shows a polygonal extrusion. It consists of a polygon and parallelograms. Fig.
9(b) shows a multi-extrusion. It consists of a multi-region shape, parallelogram(s), and curved
parallelogram(s) similar to the polygonal extrusion. Fig. 9(f) shows a front fillet. It consists of four
straight lines and two curved lines, and they are contacted tangentially. Also, two green dotted lines
can be drawn as additional lines among them. Then four green solid lines can be drawn, and a redline can be drawn as shown in this figure. Consequently, a front fillet sketch can be changed into a
polygonal corner sketch.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 10: Seven types of ASFs: (a) Partial cuboid, (b) Partial cuboid made with an additional line,(c) Partial cylinder, (d) Two types of partial polygonal extrusion, (e) Two types of partial multiextrusion, (f) Two types of partial rib, and (g) Partial pipe.
Fig. 10(a) shows a partial cuboid. In this figure, each isolated terminal of a line is marked as a red
point. Fig. 10(b) shows a partial cuboid with an additional line. In this figure, two terminals of theadditional line are blue points because the line becomes an element of a contact face to the other
SF. So, these blue points and additional lines can be applied to the other ASFs. Fig. 10(c) shows apartial cylinder. Fig. 10(d) shows two types of partial polygonal extrusion. Fig. 10(e) shows two
types of partial multi-extrusion. Fig. 10(f) shows two types of partial rib. Fig. 10(g) shows a partial
pipe. These ASFs are representative because obviously there are many more types of ASFs.
4

EXAMPLES

Fig. 11(a) shows Example 2 that is a cylindrical mechanical part. First, an SF of a (red) cylinder canbe detected and extracted as f1 as shown in Fig. 11(b). In this case, only z axis can be defined fromthe axis of f1 as this figure. Second, an SF of a (blue) cylinder can be detected and extracted as f2
as shown in Fig. 11(c). Consequently, a partial (green) cylinder as an ASF can be detected as shownin this figure. When it is assumed that the axis of f2 is the same as the ASF with the human perception,an (green) SF of a cylinder can be predicted as f3 as shown in Fig. 11(d). Consequently, the 3Dmodel of Example 2 can be obtained by combining these three 3D features.

f1
z
(a)

(b)

f2

ASF

(c)

f2

f3

(d)

Figure 11: Example 2: (a) Example 2, (b) Extraction of f1 and z axis, (c) Extraction of f2 anddetection of a partial cylinder as an ASF, and (d) Prediction of f3.
Fig. 12(a) shows Example 3, which is an arm part of a mechanical product. This example was an
issue in our last paper [16]. In an overview, this arm seems a cylinder with two ribs. Fig. 12(b)shows a prototype of that. The top view of the prototype can be drawn as shown in Fig. 12(c) in
human perception. Therefore, it is found that each rib is placed radially from the axis of the cylinder.
In other words, people would not imagine Fig. 12(d) instead of Fig. 12(c). Consequently, people
would think that two ribs are placed to the cylinder radially in Example 3. When Example 3 is input
to this method, first, three round holes can be detected and extracted as shown in Fig. 12(e).
Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

984

Second, in this figure, a round (red) rib can be detected and extracted as f1 and then a (blue) multiextrusion can be detected and extracted as f2 and also a (green) 3D coordinate system can be placed
from f1. After f1 and f2 are extracted, a (pink) cylinder can be detected and extracted as f3 as shownin Fig. 12(f). After f3 is extracted, a (brown) partial rib as an ASF can be detected from a (pink) limb
line of f3 such as shown in Fig. 12(g). The length of the rib can be predicted by applying the humanperception that ribs are placed to a cylinder radially such as Fig. 12(c). The direction of the rib fromf3 can be regarded as y axis. So, in Fig. 12(h), the center point of a blue ellipse parallel to two
ellipses of f3 is the intersection between the (red) axis of f3 and (green) y axis corresponding to thecenter line of the bottom face in the rib. Therefore, it is found that the rib can extend until the blue
ellipse as this figure so it can become a 3D feature as f4. In the same way, an ASF of a multiextrusion can be extended in accordance with f4, and it can become f5 as shown in Fig. 12(i).
Consequently, the 3D model of Example 3 can be obtained by combining all 3D features as shownin Fig. 12(j).

z
y
f3

f1
f2
(a)

(b)

ASF

(c)

f4

(d)

x

(e)

(f)

y

f5
(g)

(h)

(i)

(j)

Figure 12: Example 3: (a) Example 3, (b) Prototype of a cylinder with ribs, (c) Top view of (b), (d)Unimaginable top view, (e) Detected f1, f2 and placed 3D coordinate system, (f) Extraction of f1, f2
and detected f3, (g) Detection of a partial rib as an ASF, (h) Prediction of f4, (i) Prediction of f5, and(j) Overview of the 3D model of Example 3.
Fig. 13(a) shows Example 4 that is a round table. Obviously, a disk-shaped cylinder can be detected
and extracted firstly as shown in Fig. 13(b). However, the lengths of three partial cylinders as ASFs
cannot be predicted in the present step of this method. Especially, the prediction of the length in the
middle partial cylinder is difficult. In Fig. 13(c), the cylinder is placed at the center of the disk like
Example 1, but it is obvious that this placement is a mistake because people know the shapes oftables. It is a normal idea for people that the lengths of the ASFs are the same and they are arrangednear the edges of the disk radially in tables. It is difficult to apply this inference to the present stepof this method. Although this problem might be solved in pattern recognition techniques of images
and/or deep learning techniques, we want to solve that as smarter and more theoretical as possible.
One of the measures is a higher abstraction of ASF. If Fig. 13(b) can be defined as a complex ASF,
this problem might be solved. So, it would become a next issue for this method.

Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

985

(a)

(b)

(c)

Figure 13: Example 4: (a) Example 4, (b) Extracting a disk and appearance of three partial cylinders(ASFs), and (c) Case that middle ASF is contact to the center of the disk.
Fig. 14 shows the algorithm of automatic converting ASFs to SFs in this method. When an ASF in a
sketch is input, in Step 1, if there is some contact line, the contact face and the length of the ASF
can be confirmed in Step 2, so an SF can be output in Step 7. In Step 3, if some contact face to the
ASF can be detected, its length can be decided centrally such as Fig. 8(d) in Step 5 and then Step 7
can be executed. In Step 4, adaptive prototypes such as Fig. 12(b) are searched and applied to findsome contact face to the ASF. To find more prototypes and how to apply prototypes to ASFs are thenext issues of this method. If this step is failed, “ERROR” is output in Step 6.
Input an ASF in a sketch

2) Make an SF fromthe line(s)yes1) Are there any
contact lines?

noyes3) Can detect acontact face?

yes5) Decide the length of the ASF
centrally from the faceno4) Can any prototypes
be applied?

no6) Output "ERORR"

7) Output an SF

Figure 14: The algorithm of automatic converting ASFs to SFs in this method.
5

DISCUSSION

Although many issues still exist in the prediction of ASFs, several cases of ASFs can be predictedwith human perception in this method in this paper. In Example 1, the length of a partial cuboid as
an ASF can be predicted as shown in Fig. 8(d) because of a human perception that people prefer toselect center/middle points in the range of selective areas. In Example 3, although the prototype in
Fig. 12(b) is introduced, if it is a mistake, it becomes impossible to convert this example into a 3Dmodel such as shown in Fig. 12(j). Therefore, more abstraction (such as Fig. 13(b)) of SFs and clues
will be required for this issue in this method. Prototypes such as Fig. 12(b) will become a clue but
more adaptive clues would become different views of objects drawn in sketches. The views can be
pictures and/or Engineering drawings. In general, the prediction of hidden shapes made from Tjunctions has been a difficult issue for the researches to automatically convert sketches into 3Dmodels. However, if the issue may be ignored, practical systems for the conversion would never be
Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

986

developed. Therefore, it is important to research the prediction continuously. On the other hand, it
is necessary to make dictionaries of SFs and ASFs. In addition, to handle simplified sketches ofmechanical parts will become an issue. For example, Fig. 15 shows sketches of a screw and springs.
It is still impossible to handle these sketches in this method despite that there are many mechanicalsketches including screws and springs. For the conversion, epochal ASFs would be required.

(a)

(b)

(c)

Figure 15: Sketches of a screw and springs: (a) Screw, (b) Two types of springs, and (c)Symbolic sketches from (b).
6

CONCLUSION

In this paper, a method to predict hidden shapes making from T-junctions in sketches by applying
human perception is proposed. The results of this method are summarized as follows:• For this prediction, sketch features (SFs) can be abstracted to ASFs, and seven types of ASFs canbe defined, and the algorithm of this method is explained in detail by solving Example 1.
• In Example 1, a partial cuboid as an ASF can be predicted as a SF by human perception that people
prefer to select center/middle points in the range of selective areas. Also, a partial cylinder as an
ASF can be predicted in Example 2.
• In Example 3, a partial rib and a partial multi-extrusion made from a limb line can be predicted byusing a prototype of a cylinder with ribs. Also, it is found that more abstraction of ASF is necessary
in Example 4. From these examples, the algorithm of automatic converting ASFs to SFs can beindicated. Consequently, more abstract ASFs, prototypes, and clues such as different views will be
required for more exact predictions of hidden shapes in sketches.
• The importance of the prediction is discussed, and to handle simplified sketches such as screws
and springs will become an issue.
Masaji Tanaka, https://orcid.org/0000-0002-5266-9182
Tetsuya Asano, https://orcid.org/0000-0002-4988-6928
REFERENCES
[1]
[2]
[3]

[4]

Bobenrieth, C.; Cordier, F.; Habibi, A.; Seo, H.: Descriptive: Interactive 3D Shape Modelingfrom A Single Descriptive Sketch, Computer-Aided Design, 128(102904), 2020,
https://doi.org/10.1016/j.cad.2020.102904
Cao, L.; Liu, J.; Tang, X.: What the back of the object looks like: 3D reconstruction from line
drawings without hidden lines, IEEE Transaction on Pattern Analysis and Machine Intelligence,30(3), 2007, 507–517, https://doi.org/10.1109/TPAMI.2007.1185
Chen, Q.; Nguyen, V.; Han, F.; Kiveris, R.; Tu, Z.: Topology-Aware Single-Image 3D Shape
Reconstruction, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition(CVPR)
Workshops,
2020,
270-271,https://doi.org/10.1109/CVPRW50498.2020.00143
Choy, C; Xu, D.; Gwak, J.; Chen, K.; Savarese, S.: 3D-R2N2: A Unified Approach for Single
and Multi-view 3D Object Reconstruction, Computer Vision – ECCV, 2016, 628-644,
https://doi.org/10.1007/978-3-319-46484-8_38

Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net

987

[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]

[23]
[24]

Clowes, M.B.: On seeing things, Artificial Intelligence, 2(1), 1971, 79–116. https://doi.org/
10.1016/0004-3702(71)90005-1
Company, P.; Piquer, A.; Contero, M.; Naya, F.: A survey on geometrical reconstruction as acore technology to sketch-based modeling, Computers & Graphics, 29(6), 2005, 892–904,
https://dx.doi.org/10.1016/j.cag.2005.09.007
Company, P.; Plumed, R.; Varley, P. A. C.; Camba, J. D.: Algorithmic Perception of Vertices in
Sketched Drawings of Polyhedral Shapes, ACM Transactions on Applied Perception, 16(3),2019, Article 18, https://doi.org/10.1145/3345507
Dematapitiya, S.; Kawazoe, M.; Nishikawa, A; Sakurai, M.; Saga, S.: Snapping of fuzzy objects
using the multi-resolution fuzzy grid snapping technique, Information and Media Technologies,
4(2), 2009, 463-474. https://doi.org/10.2197/ipsjjip.17.47
Guo, T.; Cui, R.; Qin, X.; Wang, Y.; Tang, Z.: Bottom-up/top-down geometric object
reconstruction with CNN classification for mobile education, PG '18: Proceedings of the 26th
Pacific Conference on Computer Graphics and Applications, 2018, 13-16,
http://help.solidworks.com/2021/English/SolidWorks/sldworks/c_3d_sketching_top.htm
https://www.sketchup.com/
Huffman, D. A.: Impossible objects as nonsense sentences, Machine Intelligence 6, 1971, 295–
323.
Jiang, L.; Shi, S.; Qi, X.; Jia, J.: GAL: Geometric Adversarial Loss for Single-View 3D-Object
Reconstruction, Proceedings of the European Conference on Computer Vision (ECCV), 2018,802-816, https://doi.org/10.1007/978-3-030-01237-3_49
Malik, J.: Interpreting line drawings of curved objects, International Journal of Computer
Vision, 1, 1987, 73–103. https://dx.doi.org/10.1007/BF00128527
Perkins, D. N.: Cubic Corners, Quarterly Progress Report 89, MIT Research Laboratory ofElectronics, 1968, 207–214.
Tanaka, M.; Asano, T.; Higashino, C.: Isometric Conversion of Mechanical Sketches into 3D
Models, 18(4), 2021, 772-785, https://doi.org/10.14733/cadaps.2021.772-785
Tanaka, M.; Kaneeda, T.: Feature extraction from sketches of objects, Computer-Aided Design& Applications, 12(3), 2014, 300-309. https://dx.doi.org/10.1080/16864360.2014. 981459
Tanaka, M.; Terano, M.; Asano, T.; Higashino, C.: Method to Automatically Convert Sketchesof Mechanical Objects into 3D Models, Computer-Aided Design & Applications, 17(6), 2020,
1168-1176, https://doi.org/10.14733/cadaps.2020.1168-1176
Tanaka, M.; Terano, M.; Higashino, C.; Asano, T.; Takasugi, K.: A Learning Method for
Reconstructing 3D Models from Sketches, Computer-Aided Design & Applications, 16(6), 2019,1158-1170, https://doi.org/10.14733/cadaps.2019.1158-1170
Tatarchenko, M.; Richter, S.; Ranftl, R.; Li, Z.; Koltun, V.; Brox, T: What Do Single-View 3D
Reconstruction Networks Learn?, Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition (CVPR), 2019, 3405-3414, https://doi.org/10.1109/CVPR.2019.00352
Varley, P. A. C.; Martin, R. R.; Suzuki, H.: Frontal geometry from sketches of engineering
objects: is line labelling necessary?, Computer-Aided Design, 37(12), 2005, 1285–1307,
https://doi.org/10.1016/j.cad.2005.01.002
Yang, B.; Wen, H.; Wang, S.; Clark, R.; Markham, A.; Trigoni, N.: 3D Object Reconstruction
from a Single Depth View with Adversarial Learning, Proceedings of the IEEE International
Conference on Computer Vision (ICCV), 2017, 679-688, https://doi.org/10.1109/
ICCVW.2017.86
Yang, B.; Rosa, S.; Markham, A.; Trigoni, N.; Wen, H.: Dense 3D Object Reconstruction froma Single Depth View, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(12),2019, https://doi.org/10.1109/TPAMI.2018.2868195
Zou, C.; Peng, X.; Lv, H.; Chen, S.; Fu, H.; Liu, J.: Sketch-based 3-D modeling for piecewise
planar objects in single images, Computers & Graphics, 46, 2015, 130-137,
https://doi.org/10.1016/j.cag.2014.09.031

Computer-Aided Design & Applications, 19(5), 2022, 977-987© 2022 CAD Solutions, LLC, http://www.cad-journal.net