Press Release

MAKING IT EASIER TO TRANSFORM FREEFORM 2D SKETCHING INTO 3D MODELS
Novel method for modeling diverse freeform shapes by drawing intuitive andexpressive sparse 2D sketches
TOKYO, Japan, 30 November 2018 – A new computational approach, built on data-driven techniques, ismaking it possible to turn simple 2D sketch into a realistic 3D shape, with little or no user input necessary.
A global team of computer scientists from University of Hong Kong, Microsoft and University of British
Columbia, have developed a novel framework, combining CNNs, or convolutional neural network techniques,with geometric inference methods to compute sketches and their corresponding 3D shapes, faster and moreintuitively than existing methods.
“Our tool can be used by artists toquickly and easily draft freeform
shapes that can serve as base shapes
and be further refined with advanced
modeling software,” says Hao Pan,
researcher at Microsoft Research Asia
who led the work. “Novice users may
also use it as an accessible tool tomake stylish contents for their 3D
creations and virtual worlds.”
Freeform 3D shapes with compelling details and variations are easily
sketched. Sketches are gray scale images and support different views,
strokes marked in red have been re-sketched for bending the local shape.

The novel method builds on the team’s previous sketch-based 3D modeling work—BendSketch: Modeling
Freeform Surfaces through 2D Sketching*—a purely geometric-based method which requires annotatedsketch inputs. Adds Pan, “We want to develop a method that is both intelligent—in the sense of requiring as
few user inputs as possible and doing all needed inference automatically, and generic—so that we do nothave to train a new machine learning model for each specific object category.”
Pan’s collaborators Changjian Li and Wenping Wang at University of Hong Kong; Yang Liu and Xin Tong atPage 1 of 4

Microsoft Research Asia; and Alla Sheffer at University of British Columbia, will present at SIGGRAPH Asia
2018 in Tokyo 4 December to 7 December. The annual conference features the most respected technical andcreative members in the field of computer graphics and interactive techniques, and showcases leading edge
research in science, art, gaming and animation, among other sectors.
It is a challenge to easily, automatically transform a sparse, imperfect 2D sketch to a robust 3D shape. The key
challenge the researchers addressed in this work is the significant information disparity presented between
the stages of ‘sketch’ and ‘3D’ model. This is where the artist would fill in features such as bumps, bends,creases, valleys, ridges on the surface drawings to create a 3D design. For each sketch, demonstrated in theirpaper, the method uses a CNN to predict the 3D surface patch.
The researchers’ CNN is trained on a large dataset generated by rendering sketches of various 3D shapes using
non-photorealistic line rendering, or NPR, method that automatically mimics human sketching of freeform
shapes – shapes, including that of a teddy bear, a bird or a fish, for example. The researchers validated their
new approach, compared it with existing methods and evaluated its performance. Results demonstrated that
their method provides a new way to produce 3D modeling from 2D sketches in a user-friendly and morerobust manner.
One example, included in their paper, involved novice users who were tested on the new method. The users
have little knowledge or training about sketching or 3D modeling. They were given a 20-minutes training
demo on the new method. They then were asked to create three target shapes—bird, teddy bear anddolphin—of varying complexity. The users managed to complete the tasks, ranging from just 5 minutes to 20
minutes total. As noted in the paper, the novice participants found the sketches easy to produce using theresearchers’ tool, and some also wanted to explore with it more.
In future work, the researchers intend to explore several possibilities and extensions of their algorithm. For
example, currently their method models a surface patch each time by sketching in view, but it would be
beneficial to be able to model a complete shape by sketching both front and back patches in a single view.
They also hope to enhance the tool so that he can handle multiple scale modeling, so that large-scale shapes
and fine-level details can be sketched with ease.
For the authors’ paper, titled “Robust Flow-Guided Neural Prediction for Sketch-based Freeform SurfaceModeling”, and video, visit the team’s project page.
*published at SIGGRAPH 2017
Page 2 of 4

More details on this Technical Papers Session at SIGGRAPH Asia 2018:
Session: Fabulously Computed Fashion
Date: 5 December 2018, Wednesday
Time: 4:15pm – 6:00pm
Venue: Hall B5 (1), 5F, B BlockTo register for SIGGRAPH Asia, and for more details of the event, visit http://sa2018.siggraph.org.
– end –Notes to editors:•

For more information on the Technical Papers Program, click here.

•For media accreditation to SIGGRAPH Asia 2018, please register here.

About SIGGRAPH Asia 2018
The 11th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
(SIGGRAPH Asia 2018) will be held in Tokyo, Japan at the Tokyo International Forum from 4 to 7 December
2018. The annual event held in Asia attracts the most respected technical and creative people from all over
the world who are excited by research, science, art, animation, gaming, interactivity, education and emerging
technologies.
The four-day conference will include a diverse range of juried programs, such as the Art Gallery, Computer
Animation Festival, Courses, Emerging Technologies, Posters, Technical Briefs, Technical Papers and Virtual &Augmented Reality. A three-day exhibition held from 5 to 7 December 2018 will offer a business platform forindustry players to market their innovative products and services to the computer graphics and interactive
techniques professionals and enthusiasts from Asia and beyond. For more information, please visit
http://sa2018.siggraph.org. Find us on: Facebook, Twitter, Instagram and YouTube with the official event
hashtag, #SIGGRAPHAsia.

About ACM SIGGRAPH
The ACM Special Interest Group on Computer Graphics and Interactive Techniques is an interdisciplinary
community interested in research, technology, and applications in computer graphics and interactivetechniques. Members include researchers, developers, and users from the technical, academic, business, andart communities. ACM SIGGRAPH enriches the computer graphics and interactive techniques community
year-round through its conferences, global network of professional and student chapters, publications, andeducational activities. For more information, please visit www.siggraph.org.
Page 3 of 4

About Koelnmesse
Koelnmesse Pte Ltd is one of the world's largest trade fair companies. Its more than 80 trade fairs andexhibitions have the broadest international scope in the industry, as 60 percent of the exhibitors and 40
percent of the visitors come from outside Germany. The Koelnmesse events include leading global trade fairs
for 25 sectors, such as Imm Cologne, Anuga, IDS, INTERMOT, Interzum Cologne, Photokina, Gamescom, andthe International Hardware Fair Cologne. Koelnmesse is ACM SIGGRAPH’s event organizer for the last 10editions of SIGGRAPH Asia. For more information, please visit www.koelnmesse.com.sg.

Media Contacts
Hitoshi Kasai
Head of Communications, SIGGRAPH Asia 2018
MIACIS Associates
hitoshi.kasai@siggraph.org
Jamie Huang
Marketing & Media Partnerships, SIGGRAPH Asia 2018
Koelnmesse Pte Ltd
jamie.huang@siggraph.org

Page 4 of 4