Original: T RAINING D ATA G ENERATING N ETWORKS : S HAPE R ECONSTRUCTION VIA B I - LEVEL O PTIMIZATION
Revised: T RAINING D ATA G ENERATING N ETWORKS : S HAPE R ECONSTRUCTION VIA B I - LEVEL O PTIMIZATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We propose a novel 3d shape representation for 3d shape reconstruction from a single image
Revised: We propose a novel 3d shape representation for 3d shape reconstruction from a single image

Original: Rather than predicting a shape directly
Revised: Rather than predicting a shape directly

Original: , we train a network to generate a training set which will be fed into another learning algorithm to deﬁne the shape
Revised: , we train a network to generate a training set which will be fed into another learning algorithm to deﬁne the shape

Original: The nested optimization problem can be modeled by bi- level optimiza- tion
Revised: The nested optimization problem can be modeled by bi- level optimiza- tion

Original: Speciﬁcally
Revised: Speciﬁcally

Original: , the algorithms for bi-level optimization are also being used in meta learning approaches for few
Revised: , the algorithms for bi-level optimization are also being used in meta learning approaches for few

Original: -shot learning
Revised: -shot learning

Original: Our framework establishes a link between 3D shape analysis and few
Revised: Our framework establishes a link between 3D shape analysis and few

Original: -shot learning
Revised: -shot learning

Original: We combine training data generating networks with bi level optimization algorithms to obtain a complete framework for which all components can be jointly trained
Revised: We combine training data generating networks with bi level optimization algorithms to obtain a complete framework for which all components can be jointly trained

Original: We improve upon recent work on standard benchmarks for 3d shape reconstruction
Revised: We improve upon recent work on standard benchmarks for 3d shape reconstruction

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Neural networks have shown promising results for shape reconstruction (Wang et al., 2018;
Revised: Neural networks have shown promising results for shape reconstruction (Wang et al., 2018;

Original: Groueix et al., 2018; Mescheder et al., 2019;
Revised: Groueix et al., 2018; Mescheder et al., 2019;

Original: Genova et al., 2019).
Revised: Genova et al., 2019).

Original: Different from the image domain, there is no universally agreed upon way to represent 3d shapes.
Revised: Different from the image domain, there is no universally agreed upon way to represent 3d shapes.

Original: There exist many explicit and implicit representations.
Revised: There exist many explicit and implicit representations.

Original: Explicit representations include point clouds (Qi et al., 2017a;b; Lin et al., 2017), grids (Wu et al., 2015; Choy et al., 2016; Riegler et al., 2017; Wang et al., 2017; Tatarchenko et al., 2017), and meshes (Wang et al., 2018; Hanocka et al., 2019).
Revised: Explicit representations include point clouds (Qi et al., 2017a;b; Lin et al., 2017), grids (Wu et al., 2015; Choy et al., 2016; Riegler et al., 2017; Wang et al., 2017; Tatarchenko et al., 2017), and meshes (Wang et al., 2018; Hanocka et al., 2019).

Original: Implicit representations (Mescheder et al., 2019; Michalkiewicz et al., 2019; Park et al., 2019) deﬁne shapes as iso-surfaces of functions.
Revised: Implicit representations (Mescheder et al., 2019; Michalkiewicz et al., 2019; Park et al., 2019) deﬁne shapes as iso-surfaces of functions.

Original: Both types of representations are important as they have different advantages.
Revised: Both types of representations are important as they have different advantages.

Original: In this work, we set out to investigate how meta-learning can be used to learn shape representations.
Revised: In this work, we set out to investigate how meta-learning can be used to learn shape representations.

Original: To employ meta-learning, we need to split the learning framework into two (or more) coupled learning problems.
Revised: To employ meta-learning, we need to split the learning framework into two (or more) coupled learning problems.

Original: There are multiple design choices for exploring this idea.
Revised: There are multiple design choices for exploring this idea.

Original: One related solution is to use a hypernetwork as the ﬁrst learning algorithm that produces the weights for a second network.
Revised: One related solution is to use a hypernetwork as the ﬁrst learning algorithm that produces the weights for a second network.

Original: This approach (Littwin & Wolf, 2019), is very inspiring, but the results can still be improved by our work.
Revised: This approach (Littwin & Wolf, 2019), is very inspiring, but the results can still be improved by our work.

Original: The value of our work are not mainly the state-of-the-art results, but the new framework that enables the ﬂow of new ideas, techniques, and methods developed in the context of few-shot learning and meta-optimization to be applied to shape reconstruction.
Revised: The value of our work are not mainly the state-of-the-art results, but the new framework that enables the ﬂow of new ideas, techniques, and methods developed in the context of few-shot learning and meta-optimization to be applied to shape reconstruction.

Original: In order to derive our proposed solution, we draw inspiration from few-shot learning.
Revised: In order to derive our proposed solution, we draw inspiration from few-shot learning.

Original: The formulation of few-shot learning we use here is similar to Lee et al.
Revised: The formulation of few-shot learning we use here is similar to Lee et al.

Original: ; Bertinetto et al.
Revised: ; Bertinetto et al.

Original: (2019), but also other formulations of few shot learning exist, e.g ., MAML (Finn et al., 2017).
Revised: (2019), but also other formulations of few shot learning exist, e.g ., MAML (Finn et al., 2017).

Original: In few-shot learning for image classiﬁcation, the learning problem is split into multiple tasks (See Fig. 1).
Revised: In few-shot learning for image classiﬁcation, the learning problem is split into multiple tasks (See Fig. 1).

Original: For each task, the learning framework has to learn a decision boundary to distinguish between a smaller set of classes.
Revised: For each task, the learning framework has to learn a decision boundary to distinguish between a smaller set of classes.

Original: In order to do so, a ﬁrst network is used to embed images into a high-dimensional feature space.
Revised: In order to do so, a ﬁrst network is used to embed images into a high-dimensional feature space.

Original: A second learner (often called base learner in the literature of meta learning approaches for few-shot learning) is then used to learn decision boundaries in this high-dimensional feature space.
Revised: A second learner (often called base learner in the literature of meta learning approaches for few-shot learning) is then used to learn decision boundaries in this high-dimensional feature space.

Original: A key point is that each task has separate decision boundaries.
Revised: A key point is that each task has separate decision boundaries.

Original: In order to build a bridge from fewshot learning to shape representations, we identify each shape as a separate task.
Revised: In order to build a bridge from fewshot learning to shape representations, we identify each shape as a separate task.

Original: For each task, there are two classes to distinguish: inside and outside.
Revised: For each task, there are two classes to distinguish: inside and outside.

Original: The link might seem a bit unintuitive at ﬁrst, because in the image classiﬁcation case, each task is deﬁned by multiple images already divided into multiple classes.
Revised: The link might seem a bit unintuitive at ﬁrst, because in the image classiﬁcation case, each task is deﬁned by multiple images already divided into multiple classes.

Original: However, in our case, a task is only associated with a single image.
Revised: However, in our case, a task is only associated with a single image.

Original: We therefore introduce an additional component, a training data generating network, to build the bridge to fewshot learning (See Fig. 2).
Revised: We therefore introduce an additional component, a training data generating network, to build the bridge to fewshot learning (See Fig. 2).

Original: This training data generating network takes a single image as input and outputs multiple 3D points with an inside and outside label.
Revised: This training data generating network takes a single image as input and outputs multiple 3D points with an inside and outside label.

Original: These points deﬁne the training dataset
Revised: These points deﬁne the training dataset

Original: D train...
Revised: D train...

Original: D test
Revised: D test

Original: Task
Revised: Task

Original: D train...
Revised: D train...

Original: D test
Revised: D test

Original: Task...
Revised: Task...

Original: D trainT ...
Revised: D trainT ...

Original: D testT
Revised: D testT

Original: Task T
Revised: Task T

Original: D train
Revised: D train

Original: D test
Revised: D test

Original: Shape
Revised: Shape

Original: D train
Revised: D train

Original: D test
Revised: D test

Original: Shape...
Revised: Shape...

Original: D trainT
Revised: D trainT

Original: D testT
Revised: D testT

Original: Shape T for one task (shape).
Revised: Shape T for one task (shape).

Original: We adopt this data set format to describe a task, because points are a natural shape representation.
Revised: We adopt this data set format to describe a task, because points are a natural shape representation.

Original: Similar to the image classiﬁcation setting, we also employ an embedding network that maps the original training dataset (a set of 3D points) to another space where it is easier to ﬁnd a decision boundary (a distorted 3D space).
Revised: Similar to the image classiﬁcation setting, we also employ an embedding network that maps the original training dataset (a set of 3D points) to another space where it is easier to ﬁnd a decision boundary (a distorted 3D space).

Original: In contrast to the image classiﬁcation setting, the input and output spaces of the embedding network have a lot fewer dimensions, i.e. only three dimensions.
Revised: In contrast to the image classiﬁcation setting, the input and output spaces of the embedding network have a lot fewer dimensions, i.e. only three dimensions.

Original: Next, we have to tackle the base learner that takes a set of points in embedding space as input.
Revised: Next, we have to tackle the base learner that takes a set of points in embedding space as input.

Original: Here, we can draw from multiple options.
Revised: Here, we can draw from multiple options.

Original: In particular, we experimented with kernel SVMs and kernel ridge regression.
Revised: In particular, we experimented with kernel SVMs and kernel ridge regression.

Original: As both give similar results (In meta learning (Lee et al., 2019) we can also ﬁnd the same conclusion), we opted for ridge regression in our solution due to the much faster processing time.
Revised: As both give similar results (In meta learning (Lee et al., 2019) we can also ﬁnd the same conclusion), we opted for ridge regression in our solution due to the much faster processing time.

Original: The goal of ridge regression is to learn a decision boundary (shape surface) in 3D space describing the shape.
Revised: The goal of ridge regression is to learn a decision boundary (shape surface) in 3D space describing the shape.

Original: Our intermediate shape representation is a set of points, but these points cannot be chosen arbitrarily.
Revised: Our intermediate shape representation is a set of points, but these points cannot be chosen arbitrarily.

Original: The set of points are speciﬁcally generated so that the second learning algorithm can process them to ﬁnd a decision boundary.
Revised: The set of points are speciﬁcally generated so that the second learning algorithm can process them to ﬁnd a decision boundary.

Original: The second learning algorithm functions similar to other implicit shape representations discussed above.
Revised: The second learning algorithm functions similar to other implicit shape representations discussed above.

Original: The result of the second learning algorithm (SVM or ridge regression) can be queried as an implicit function.
Revised: The result of the second learning algorithm (SVM or ridge regression) can be queried as an implicit function.

Original: Still the second learning algorithm cannot be replaced by other existing networks employed in previous work.
Revised: Still the second learning algorithm cannot be replaced by other existing networks employed in previous work.

Original: Therefore, among existing works, only our representation can directly use few-shot learning.
Revised: Therefore, among existing works, only our representation can directly use few-shot learning.

Original: It does not make sense to train our representation without a few-shot learning framework and no other representation can be directly trained with a few-shot learning framework.
Revised: It does not make sense to train our representation without a few-shot learning framework and no other representation can be directly trained with a few-shot learning framework.

Original: Therefore, all components in our framework are intrinsically linked and were designed together.
Revised: Therefore, all components in our framework are intrinsically linked and were designed together.

Original: Contributions.
Revised: Contributions.

Original: • We propose a new type of shape representation, where we train a network to output a training set (a set of labeled points) for another machine learning algorithm (Kernel Ridge Regression or Kernel-SVM).
Revised: • We propose a new type of shape representation, where we train a network to output a training set (a set of labeled points) for another machine learning algorithm (Kernel Ridge Regression or Kernel-SVM).

Original: • We found an elegant way to map the problem of shape reconstruction from a single image to the problem of few-shot classiﬁcation by introducing a network to generate training data.
Revised: • We found an elegant way to map the problem of shape reconstruction from a single image to the problem of few-shot classiﬁcation by introducing a network to generate training data.

Original: We combined training data generating networks with two meta learning approaches (R2D2 (Bertinetto et al., 2019) and MetaOptNet (Lee et al., 2019)) in our framework.
Revised: We combined training data generating networks with two meta learning approaches (R2D2 (Bertinetto et al., 2019) and MetaOptNet (Lee et al., 2019)) in our framework.

Original: Our work enables the application of ideas and techniques developed in the few-shot learning literature to the problem of shape reconstruction.
Revised: Our work enables the application of ideas and techniques developed in the few-shot learning literature to the problem of shape reconstruction.

Original: FeatureNetwork λ
Revised: FeatureNetwork λ

Original: R
Revised: R

Original: PointGenNetwork { x i ,y i } Ni =
Revised: PointGenNetwork { x i ,y i } Ni =

Original: X ∈ R N ×(cid:74)
Revised: X ∈ R N ×(cid:74)

Original: EmbeddingNetwork { e i ,y i } Ni =
Revised: EmbeddingNetwork { e i ,y i } Ni =

Original: E ∈ R N ×
Revised: E ∈ R N ×

Original: BaseLearner α
Revised: BaseLearner α

Original: Weights x Query (cid:74)
Revised: Weights x Query (cid:74)

Original: EmbeddingNetwork e
Revised: EmbeddingNetwork e

Original: P ( · ) y
Revised: P ( · ) y

Original: Label
Revised: Label

Original: Few-Shot Learning Data Generation
Revised: Few-Shot Learning Data Generation

Original: • We validate our model using the problem of 3d shape reconstruction from a single image and improve upon the state of the art.
Revised: • We validate our model using the problem of 3d shape reconstruction from a single image and improve upon the state of the art.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Neural implicit shape representations.
Revised: Neural implicit shape representations.

Original: We can identify two major categories of shape representations: explicit representations, where a shape can be explicitly deﬁned; and implicit representations, where a shape can be deﬁned as iso-surface of a function (signed distance function or indicator function).
Revised: We can identify two major categories of shape representations: explicit representations, where a shape can be explicitly deﬁned; and implicit representations, where a shape can be deﬁned as iso-surface of a function (signed distance function or indicator function).

Original: In the past decade, we have seen great success with neural network based explicit shape representations: voxel representations (Wu et al., 2015; Choy et al., 2016; Riegler et al., 2017; Wang et al., 2017;
Revised: In the past decade, we have seen great success with neural network based explicit shape representations: voxel representations (Wu et al., 2015; Choy et al., 2016; Riegler et al., 2017; Wang et al., 2017;

Original: Tatarchenko et al., 2017), point representations (Qi et al., 2017a;b; Fan et al., 2017; Lin et al., 2017), and mesh representations (Wang et al., 2018; Hanocka et al., 2019)).
Revised: Tatarchenko et al., 2017), point representations (Qi et al., 2017a;b; Fan et al., 2017; Lin et al., 2017), and mesh representations (Wang et al., 2018; Hanocka et al., 2019)).

Original: On the other hand, modeling implicit representations with neural networks has been a current trend, where usually a signed distance function or indicator function is parameterized by a neural network (Mescheder et al., 2019; Chen & Zhang, 2019;
Revised: On the other hand, modeling implicit representations with neural networks has been a current trend, where usually a signed distance function or indicator function is parameterized by a neural network (Mescheder et al., 2019; Chen & Zhang, 2019;

Original: Michalkiewicz et al., 2019; Park et al., 2019).
Revised: Michalkiewicz et al., 2019; Park et al., 2019).

Original: More recent works learn a network that outputs intermediate parameters, e.g. CvxNet (Deng et al., 2019) and BSP-Net (Chen et al., 2019) learns to output half-spaces.
Revised: More recent works learn a network that outputs intermediate parameters, e.g. CvxNet (Deng et al., 2019) and BSP-Net (Chen et al., 2019) learns to output half-spaces.

Original: We propose a novel type of shape representation, where the model outputs a training set of labeled points.
Revised: We propose a novel type of shape representation, where the model outputs a training set of labeled points.

Original: Few-shot learning.
Revised: Few-shot learning.

Original: There are two common meta learning approaches for few-shot learning: metric-based (Koch et al., 2015; Vinyals et al., 2016;
Revised: There are two common meta learning approaches for few-shot learning: metric-based (Koch et al., 2015; Vinyals et al., 2016;

Original: Snell et al., 2017), which aims to learn a metric for each task; optimization-based (Ravi & Larochelle, 2017;
Revised: Snell et al., 2017), which aims to learn a metric for each task; optimization-based (Ravi & Larochelle, 2017;

Original: Finn et al., 2017; Nichol et al., 2018), which is designed to learn with a few training samples by adjusting optimization algorithms.
Revised: Finn et al., 2017; Nichol et al., 2018), which is designed to learn with a few training samples by adjusting optimization algorithms.

Original: These approaches commonly have two parts, an embedding model for mapping an input to an embedding space, and a base learner for prediction.
Revised: These approaches commonly have two parts, an embedding model for mapping an input to an embedding space, and a base learner for prediction.

Original: Qiao et al.
Revised: Qiao et al.

Original: ; Gidaris & Komodakis (2018) train a hypernetwork (Ha et al., 2016) to output weights of another network (base learner).
Revised: ; Gidaris & Komodakis (2018) train a hypernetwork (Ha et al., 2016) to output weights of another network (base learner).

Original: R2D2 (Bertinetto et al., 2019) showed that using a light-weight and differentiable base learner ( e.g. ridge regression) leads to better results.
Revised: R2D2 (Bertinetto et al., 2019) showed that using a light-weight and differentiable base learner ( e.g. ridge regression) leads to better results.

Original: To further develop the idea, MetaOptNet (Lee et al., 2019) used multi-class support vector machines (Crammer & Singer, 2001) as base learner and incorporated differentiable optimization (Amos & Kolter, 2017; Gould et al., 2016) into the framework.
Revised: To further develop the idea, MetaOptNet (Lee et al., 2019) used multi-class support vector machines (Crammer & Singer, 2001) as base learner and incorporated differentiable optimization (Amos & Kolter, 2017; Gould et al., 2016) into the framework.

Original: Lee et al.
Revised: Lee et al.

Original: (2019) also shows it can outperform hypernetwork-based methods.
Revised: (2019) also shows it can outperform hypernetwork-based methods.

Original: In our work, we propose a shape representation that is compatible with a few-shot classiﬁcation framework so that we can utilize existing meta learning approaches.
Revised: In our work, we propose a shape representation that is compatible with a few-shot classiﬁcation framework so that we can utilize existing meta learning approaches.

Original: Speciﬁcally, we will use ridge regression and SVM as the base learner.
Revised: Speciﬁcally, we will use ridge regression and SVM as the base learner.

Original: The most relevant method to ours is Littwin & Wolf (2019) which adapts hypernetworks.
Revised: The most relevant method to ours is Littwin & Wolf (2019) which adapts hypernetworks.

Original: However, as we discussed above, differentiable optimization methods (Bertinetto et al., 2019; Lee et al., 2019) are generally better than hypernetworks.
Revised: However, as we discussed above, differentiable optimization methods (Bertinetto et al., 2019; Lee et al., 2019) are generally better than hypernetworks.

Original: Besides that, meta learning has been applied to other ﬁelds in shape analysis, e.g ., both Sitzmann et al.
Revised: Besides that, meta learning has been applied to other ﬁelds in shape analysis, e.g ., both Sitzmann et al.

Original: (2020) and Tancik et al.
Revised: (2020) and Tancik et al.

Original: (2021) propose to use MAML-like algorithms (Finn et al., 2017) to learn a weight initialization over  implicit shape spaces.
Revised: (2021) propose to use MAML-like algorithms (Finn et al., 2017) to learn a weight initialization over an implicit shape spaces.

Original:  M ETHOD
Revised:  M ETHOD

Original: The framework is shown in Fig.
Revised: The framework is shown in Fig.

Original: The network is mainly composed of 3 sub-networks.
Revised: The network is mainly composed of 3 sub-networks.

Original: The Feature
Revised: The Feature

Original: Network maps an input image to feature space.
Revised: Network maps an input image to feature space.

Original: The resulting feature vector λ is then decoded by the Point Generation Network to a labeled point set { x i , y i } Ni =1 .
Revised: The resulting feature vector λ is then decoded by the Point Generation Network to a labeled point set { x i , y i } Ni =1 .

Revised: The point set will be used as training data in our base learner later.

Original: After that the Embedding Network projects the point set into embedding space.
Revised: After that the Embedding Network projects the point set into embedding space.

Original: The projected points e i and the labels are taken as the input of a binary classifer (ridge regression or SVM) parameterized by α .
Revised: The projected points e i and the labels are taken as the input of a binary classifer (ridge regression or SVM) parameterized by α .

Original: Finally, the framework is able to output the inside/outside label y of a query point x by projecting it into the embedding space and feeding it to the binary classiﬁer.
Revised: Finally, the framework is able to output the inside/outside label y of a query point x by projecting it into the embedding space and feeding it to the binary classiﬁer.

Original: In the following subsections, we describe our method in more detail.
Revised: In the following subsections, we describe our method in more detail.

Original: First, we introduce the background of meta learning approaches for few-show learning (Sec. 3.1) and establish a link between single image 3D reconstruction and few-shot learning (Sec. 3.2).
Revised: First, we introduce the background of meta learning approaches for few-show learning (Sec. 3.1) and establish a link between single image 3D reconstruction and few-shot learning (Sec. 3.2).

Original: We propose a problem formulation inspired by few-shot learning (Sec. 3.3) and propose a solution in the following subsections.
Revised: We propose a problem formulation inspired by few-shot learning (Sec. 3.3) and propose a solution in the following subsections.

Original: Speciﬁcally, we apply recently developed differentiable optimization.
Revised: Speciﬁcally, we apply recently developed differentiable optimization.

Original: 3. B ACKGROUND
Revised: 3. B ACKGROUND

Original: Supervised learning.
Revised: Supervised learning.

Original: Given training set D train = { x i , y i } Ni =1 , supervised learning learns a predictor y
Revised: Given training set D train = { x i , y i } Ni =1 , supervised learning learns a predictor y

Original: = P ( x ) which is able to predict the labels of test set D test =
Revised: = P ( x ) which is able to predict the labels of test set D test =

Original: { x i , y i } Mi =1 (assuming both
Revised: { x i , y i } Mi =1 (assuming both

Original: D train and D test are sampled from the same distribution).
Revised: D train and D test are sampled from the same distribution).

Original: Few-shot learning.
Revised: Few-shot learning.

Original: In few-shot learning, the size N of the training set is typically small.
Revised: In few-shot learning, the size N of the training set is typically small.

Original: The common learning algorithms on a single task usually cause problems like overﬁtting.
Revised: The common learning algorithms on a single task usually cause problems like overﬁtting.

Original: However, we are given a collection of tasks, the meta-training set D meta − train =
Revised: However, we are given a collection of tasks, the meta-training set D meta − train =

Original: {D traint , D testt } Tt =1 , on which we train a meta-learner which produces a predictor on every task {D traint , D testt } and generalizes well on the meta-testing D meta − test =
Revised: {D traint , D testt } Tt =1 , on which we train a meta-learner which produces a predictor on every task {D traint , D testt } and generalizes well on the meta-testing D meta − test =

Original: {D trains , D tests } Ss =1 .
Revised: {D trains , D tests } Ss =1 .

Original: Consider a K -class classiﬁcation task, each training set D train consists of N/K labelled examples for each of K classes.
Revised: Consider a K -class classiﬁcation task, each training set D train consists of N/K labelled examples for each of K classes.

Original: The meta-training task is often referred to as N/K -shot- K -way .
Revised: The meta-training task is often referred to as N/K -shot- K -way .

Original: Refer to
Revised: Refer to

Original: Figure 1 for an example visualization of 2-shot-3-way few-shot image classiﬁcation.
Revised: Figure 1 for an example visualization of 2-shot-3-way few-shot image classiﬁcation.

Original: Meta learning approaches for few-shot learning often involve an embedding network, and a base learner (learning algorithm).
Revised: Meta learning approaches for few-shot learning often involve an embedding network, and a base learner (learning algorithm).

Original: The embedding network maps training samples to an embedding space.
Revised: The embedding network maps training samples to an embedding space.

Original: We explain in later subsections how the 3d reconstruction is connected to meta learning in these two aspects.
Revised: We explain in later subsections how the 3d reconstruction is connected to meta learning in these two aspects.

Original: 3. S INGLE IMAGE 3D RECONSTRUCTION
Revised: 3. S INGLE IMAGE 3D RECONSTRUCTION

Original: A watertight shape can be represented by an indicator (or occupancy)
Revised: A watertight shape can be represented by an indicator (or occupancy)

Original: function O : R 3 → { 0 , 1 } .
Revised: function O : R 3 → { 0 , 1 } .

Original: We deﬁne O ( x ) = 

Revised: We deﬁne O ( x ) =

Original: if x ∈ R 3 is inside the object, O ( x ) = 0 otherwise.
Revised: if x ∈ R 3 is inside the object, O ( x ) = 0 otherwise.

Original: We can sample a set of points in R 3 and evaluate the indicator O , then we have the labeled point set { x i , y i } M i =1 where y i ∈ { 0 , 1 } .
Revised: We can sample a set of points in R 3 and evaluate the indicator O , then we have the labeled point set { x i , y i } Mi =1 where y i ∈ { 0 , 1 } .

Original: The number M needs to be large enough to approximate the shape.
Revised: The number M needs to be large enough to approximate the shape.

Original: In this way, we rewrite the target ground-truth as a point set.
Revised: In this way, we rewrite the target ground-truth as a point set.

Original: This strategy is also used by Mescheder et al.
Revised: This strategy is also used by Mescheder et al.

Original: and
Revised: and

Original: Deng et al.
Revised: Deng et al.

Original: Also see Figure 1 for an illustration.
Revised: Also see Figure 1 for an illustration.

Original: The goal of single image 3D reconstruction is to convert an input image I to the indicator function
Revised: The goal of single image 3D reconstruction is to convert an input image I to the indicator function

Original: O .
Revised: O .

Original: Previous work either directly learns O (Mescheder et al.
Revised: Previous work either directly learns O (Mescheder et al.

Original: , 2019))or trains a network to predict an intermediate parametric representation ( e.g. collection of convex primitives (Deng et al., 2019), half-spaces (Chen et al., 2019)).
Revised: , 2019))or trains a network to predict an intermediate parametric representation ( e.g. collection of convex primitives (Deng et al., 2019), half-spaces (Chen et al., 2019)).

Original: Different from any existing methods, our shape representation is to generate training data for a few-shot classiﬁcation problem.
Revised: Different from any existing methods, our shape representation is to generate training data for a few-shot classiﬁcation problem.

Original: In order to make the connection clear, we denote the ground-truth { x i , y i } Mi =1 as D test .
Revised: In order to make the connection clear, we denote the ground-truth { x i , y i } Mi =1 as D test .

Original: The training data of single image 3D reconstruction are a collection of images I t and their corresponding shapes D test t which we denote as D meta − train =
Revised: The training data of single image 3D reconstruction are a collection of images I t and their corresponding shapes D testt which we denote as D meta − train =

Original: { I t , D testt } T t =1 .
Revised: { I t , D testt } Tt =1 .

Original: The goal is to learn a network which takes as input an image I and outputs a functional (predictor) P ( x ) which works on
Revised: The goal is to learn a network which takes as input an image I and outputs a functional (predictor) P ( x ) which works on

Original: D test .
Revised: D test .

Original: We summarize the notation and the mapping of few-shot classiﬁcation to 3D shape reconstruction in Table 1.
Revised: We summarize the notation and the mapping of few-shot classiﬁcation to 3D shape reconstruction in Table 1.

Original: Using the proposed mapping, we need to ﬁnd a data generating network (Fig. 2 left) to  convert the input I to a set of labeled points D train =
Revised: Using the proposed mapping, we need to ﬁnd a data generating network (Fig. 2 left) to [Table] convert the input I to a set of labeled points D train =

Original: { x i , y i } N i =1 (usually N is far smaller than M ).
Revised: { x i , y i } Ni =1 (usually N is far smaller than M ).

Original: [Table]

Original: Then the D meta − train can be rewritten as {D traint , D testt } Tt =1 .
Revised: Then the D meta − train can be rewritten as {D traint , D testt } Tt =1 .

Original: It can be seen that, this formulation has a high resemblance to few-shot learning.
Revised: It can be seen that, this formulation has a high resemblance to few-shot learning.

Original: Also see Figure 1 for a visualization.
Revised: Also see Figure 1 for a visualization.

Original: As a result, we can leverage techniques from the literature of few-shot learning to jointly train the data generation and the classiﬁcation components.
Revised: As a result, we can leverage techniques from the literature of few-shot learning to jointly train the data generation and the classiﬁcation components.

Original: 3. F ORMULATION
Revised: 3. F ORMULATION

Original: Similar to few-shot learning, the problem can be written as a bi-level optimization.
Revised: Similar to few-shot learning, the problem can be written as a bi-level optimization.

Original: The inner optimization is to train the predictor P ( x ) to estimate the inside/outside label of a point,
Revised: The inner optimization is to train the predictor P ( x ) to estimate the inside/outside label of a point,

Original: [Equation]
Revised: [Equation]

Original: (1) where L ( · , · ) is a loss function such as cross entropy.
Revised: (1) where L ( · , · ) is a loss function such as cross entropy.

Original: While in few-shot learning D train is provided or sampled, here D train is generated by a network f , D train =
Revised: While in few-shot learning D train is provided or sampled, here D train is generated by a network f , D train =

Original: f ( I ) .
Revised: f ( I ) .

Original: To reconstruct the shape ( I , D test ) , the predictor P should work as an approximation of the indicator O and is expected to minimize the term,
Revised: To reconstruct the shape ( I , D test ) , the predictor P should work as an approximation of the indicator O and is expected to minimize the term,

Original: [Equation]
Revised: [Equation]

Original: This process is done by a base learner (machine learning algorithm) in some meta learning methods (Bertinetto et al., 2019; Lee et al., 2019).
Revised: This process is done by a base learner (machine learning algorithm) in some meta learning methods (Bertinetto et al., 2019; Lee et al., 2019).

Original: The ﬁnal objective across all shapes (tasks) is
Revised: The ﬁnal objective across all shapes (tasks) is

Original: [Equation]
Revised: [Equation]

Original: which is exactly the same as for meta learning algorithms if we remove the constraint D train
Revised: which is exactly the same as for meta learning algorithms if we remove the constraint D train

Original: =
Revised: =

Original: f ( I ) .
Revised: f ( I ) .

Original: Point Embedding.
Revised: Point Embedding.

Original: In meta learning approaches for few-shot classiﬁcation, an embedding network is used to map the training samples to an embedding space, g ( x ) =
Revised: In meta learning approaches for few-shot classiﬁcation, an embedding network is used to map the training samples to an embedding space, g ( x ) =

Original: e , where e is the embedding vector of the input x .
Revised: e , where e is the embedding vector of the input x .

Original: We also migrate the idea to 3d shape representations, g ( x | I ) =
Revised: We also migrate the idea to 3d shape representations, g ( x | I ) =

Original: e , where the embedding network is also conditioned on the task input I .
Revised: e , where the embedding network is also conditioned on the task input I .

Original: In later sections, we use e i and e to denote the embeddings of point x i and x , respectively.
Revised: In later sections, we use e i and e to denote the embeddings of point x i and x , respectively.

Original: In addition to the objective Eq equation 3, we add a regularizer,
Revised: In addition to the objective Eq equation 3, we add a regularizer,

Original: [Equation]
Revised: [Equation]

Original: where w is the weight for the regularizer.
Revised: where w is the weight for the regularizer.

Original: There are multiple solutions for the Embedding Network, thus we want to use the regularizer to shrink potential solutions, which is to ﬁnd a similar embedding space to the original one.
Revised: There are multiple solutions for the Embedding Network, thus we want to use the regularizer to shrink potential solutions, which is to ﬁnd a similar embedding space to the original one.

Original: The w is set to 0 .
Revised: The w is set to 0 .

Original: 01 in all experiments if not speciﬁed.
Revised: 01 in all experiments if not speciﬁed.

Original: 3. D IFFERENTIABLE LEARNER
Revised: 3. D IFFERENTIABLE LEARNER

Original: The Eq.
Revised: The Eq.

Original: equation 1 is the inner loop of the ﬁnal objective Eq.
Revised: equation 1 is the inner loop of the ﬁnal objective Eq.

Original: equation 3.
Revised: equation 3.

Original: Recent meta learning approaches use differentiable learners, e.g ., R2D2 (Bertinetto et al., 2019)
Revised: Recent meta learning approaches use differentiable learners, e.g ., R2D2 (Bertinetto et al., 2019)

Original: uses Ridge Regression and MetaOptNet (Lee et al., 2019)
Revised: uses Ridge Regression and MetaOptNet (Lee et al., 2019)

Original: uses Support Vector Machines (SVM).
Revised: uses Support Vector Machines (SVM).

Original: We describe both cases here.
Revised: We describe both cases here.

Original: Different than the learner in R2D2 and MetaOptNet, we use kernelized algorithms.
Revised: Different than the learner in R2D2 and MetaOptNet, we use kernelized algorithms.

Original: Ridge Regression.
Revised: Ridge Regression.

Original: Given a training set D train = { x i , y i } Ni =1 , The kernel ridge regression (Murphy, 2012) is formulated as follows,
Revised: Given a training set D train = { x i , y i } Ni =1 , The kernel ridge regression (Murphy, 2012) is formulated as follows,

Original: [Equation]
Revised: [Equation]

Original: where K ∈ R N × N is the data kernel matrix in which each entry K i,j is the Gaussian kernel K ( e i , e j ) =
Revised: where K ∈ R N × N is the data kernel matrix in which each entry K i,j is the Gaussian kernel K ( e i , e j ) =

Original: exp( − (cid:107) e i − e j (cid:107) 22 / (2 σ 2 )) .
Revised: exp( − (cid:107) e i − e j (cid:107) 22 / (2 σ 2 )) .

Original: The solution is given by an analytic form α =
Revised: The solution is given by an analytic form α =

Original: ( K + λ I ) − 1 y .
Revised: ( K + λ I ) − 1 y .

Original: By using automatic differentiation of existing deep learning libraries, we can differentiate through α with respect to each x i in D train .
Revised: By using automatic differentiation of existing deep learning libraries, we can differentiate through α with respect to each x i in D train .

Original: We obtain the prediction for a query point x via
Revised: We obtain the prediction for a query point x via

Original: [Equation]
Revised: [Equation]

Original: SVM.
Revised: SVM.

Original: We use the dual form of kernel SVM, minimize
Revised: We use the dual form of kernel SVM, minimize

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where K ( · , · ) is the Gaussian kernel. .
Revised: where K ( · , · ) is the Gaussian kernel. .

Original: The discriminant function becomes,
Revised: The discriminant function becomes,

Original: [Equation]
Revised: [Equation]

Original: Using recent advances in differentiable optimization by Amos & Kolter (2017), the discriminant function is differentiable with respect to each x i in D train .
Revised: Using recent advances in differentiable optimization by Amos & Kolter (2017), the discriminant function is differentiable with respect to each x i in D train .

Original: While the deﬁnitions of both learners are given here, we only show results of ridge regression in our main paper.
Revised: While the deﬁnitions of both learners are given here, we only show results of ridge regression in our main paper.

Original: Additional results of SVM are provided in the supplementary material.
Revised: Additional results of SVM are provided in the supplementary material.

Original: In traditional machine learning, SVM is better than ridge regression in many aspects.
Revised: In traditional machine learning, SVM is better than ridge regression in many aspects.

Original: However, we do not ﬁnd
Revised: However, we do not ﬁnd

Original: SVM shows signiﬁcant improvement over ridge regression in our experiments.
Revised: SVM shows signiﬁcant improvement over ridge regression in our experiments.

Original: The conclusion is also consistent with Lee et al.
Revised: The conclusion is also consistent with Lee et al.

Original: 3. I NDICATOR APPROXIMATION
Revised: 3. I NDICATOR APPROXIMATION

Original: We clip the outputs of the ridge regression to the range [0 , 1] as an approximation ˆ O RR of the indicator O .
Revised: We clip the outputs of the ridge regression to the range [0 , 1] as an approximation ˆ O RR of the indicator O .

Original: For the SVM, the predictor outputs a positive value if x is inside the shape otherwise a negative value.
Revised: For the SVM, the predictor outputs a positive value if x is inside the shape otherwise a negative value.

Original: So we apply a sigmoid function to convert it to the range [0 , 1] , ˆ O SV M ( x ) =
Revised: So we apply a sigmoid function to convert it to the range [0 , 1] , ˆ O SV M ( x ) =

Original: Sigmoid( β SVM( e ; D train )) , where β is a learned scale.
Revised: Sigmoid( β SVM( e ; D train )) , where β is a learned scale.

Original: Then Eq.
Revised: Then Eq.

Original: equation 2 is written as follows:
Revised: equation 2 is written as follows:

Original: [Equation]
Revised: [Equation]

Original: where the minimum squared error (MSE) loss is also used in CvxNet (Deng et al., 2019).
Revised: where the minimum squared error (MSE) loss is also used in CvxNet (Deng et al., 2019).

Original:  I MPLEMENTATION
Revised:  I MPLEMENTATION

Original: Base learner.
Revised: Base learner.

Original: We use λ = 0 .
Revised: We use λ = 0 .

Original: 005 for ridge regression and C = 1 for SVM in all experiments.
Revised: 005 for ridge regression and C = 1 for SVM in all experiments.

Original: The parameter σ in the kernel function K ( · , · ) is learned during training and is shape-speciﬁc, i.e., each shape has its own σ .
Revised: The parameter σ in the kernel function K ( · , · ) is learned during training and is shape-speciﬁc, i.e., each shape has its own σ .

Original: Similar to recent works on instance segmentation (Liang et al., 2017; Kendall et al., 2018; Novotny et al., 2018; Zhang & Wonka, 2019), we also ﬁnd that a simple R 3 spatial embedding works well, i.e., x ∈ R 3 and σ ∈ R 3 .
Revised: Similar to recent works on instance segmentation (Liang et al., 2017; Kendall et al., 2018; Novotny et al., 2018; Zhang & Wonka, 2019), we also ﬁnd that a simple R 3 spatial embedding works well, i.e., x ∈ R 3 and σ ∈ R 3 .

Original: Another reason for choosing R 3 embedding is that we want to visualize the relationship between the original and embedding space in later sections.
Revised: Another reason for choosing R 3 embedding is that we want to visualize the relationship between the original and embedding space in later sections.

Original: We compare our results with
Revised: We compare our results with

Original: Pixel2Mesh(P2M) (Wang et al., 2018),
Revised: Pixel2Mesh(P2M) (Wang et al., 2018),

Original: AtlasNet(AN) (Groueix et al., 2018),
Revised: AtlasNet(AN) (Groueix et al., 2018),

Original: OccNet(ON)
Revised: OccNet(ON)

Original: (Mescheder et al., 2019), SIF (Genova et al., 2019), CvxNet(CN)
Revised: (Mescheder et al., 2019), SIF (Genova et al., 2019), CvxNet(CN)

Original: (Deng et al., 2019) and Hypernetwork(HN)
Revised: (Deng et al., 2019) and Hypernetwork(HN)

Original: (Littwin
Revised: (Littwin

Original: Wolf,
Revised: Wolf,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Networks.
Revised: Networks.

Original: Our framework is composed of three sub-networks: Feature Network, Point Generation Network and Embedding Network (see Fig. 2).
Revised: Our framework is composed of three sub-networks: Feature Network, Point Generation Network and Embedding Network (see Fig. 2).

Original: For the Feature Network, we use EfﬁcientNetB1 (Tan & Le, 2019) to generate a 256 -dimensional feature vector.
Revised: For the Feature Network, we use EfﬁcientNetB1 (Tan & Le, 2019) to generate a 256 -dimensional feature vector.

Original: This architecture provides a good tradeoff between performance and simplicity.
Revised: This architecture provides a good tradeoff between performance and simplicity.

Original: Both the Point Generation Network and Embedding
Revised: Both the Point Generation Network and Embedding

Original: Network are implemented with MLPs (see the supplementary material for the detailed architectures).
Revised: Network are implemented with MLPs (see the supplementary material for the detailed architectures).

Original: The Point Generation Network outputs σ and points { x i } Ni =1 (half of which have +1 inside label and the other half have − 1 outside label) where N = 64 .
Revised: The Point Generation Network outputs σ and points { x i } Ni =1 (half of which have +1 inside label and the other half have − 1 outside label) where N = 64 .

Original: The Embedding Network takes as input the concatenation of both the point x and the feature vector λ .
Revised: The Embedding Network takes as input the concatenation of both the point x and the feature vector λ .

Original: Instead of outputting e directly, we predict the offset o =
Revised: Instead of outputting e directly, we predict the offset o =

Original: e − x and apply the tanh activation to restrict the output to lie inside a bounding box.
Revised: e − x and apply the tanh activation to restrict the output to lie inside a bounding box.

Original: The training batch size is 32 .
Revised: The training batch size is 32 .

Original: We use Adam (Kingma & Ba, 2014) with learning rate 2 e − 4 as our optimizer.
Revised: We use Adam (Kingma & Ba, 2014) with learning rate 2 e − 4 as our optimizer.

Original: The learning rate is decayed with a factor of 0 .
Revised: The learning rate is decayed with a factor of 0 .

Original: 1 after 500 epochs.
Revised: 1 after 500 epochs.

Original: Data.
Revised: Data.

Original: We perform single image 3d reconstruction on the ShapeNet (Chang et al., 2015) dataset.
Revised: We perform single image 3d reconstruction on the ShapeNet (Chang et al., 2015) dataset.

Original: The renderedRGB images anddata split aretaken from (Choy et al., 2016).
Revised: The rendered images and are

Original: Wesample 100k points uniformly from the shape bounding box as in OccNet (Mescheder et al., 2019) and also 100k “near-surface” points as in CvxNets (Deng et al., 2019) and SIF (Genova et al., 2019).
Revised: We 100k points uniformly from the shape bounding box as in OccNet (Mescheder et al., 2019) and also 100k “near-surface” points as in CvxNets (Deng et al., 2019) and SIF (Genova et al., 2019).

Original: Along with the corresponding inside/outside labels, we construct D test for each shape ofﬂine to increase the training speed.
Revised: Along with the corresponding inside/outside labels, we construct D test for each shape ofﬂine to increase the training speed.

Original: At training time, 1024 points are drawn from the bounding box and 1024 ”nearsurface”.
Revised: At training time, 1024 points are drawn from the bounding box and 1024 ”nearsurface”.

Original: This is the sampling strategy proposed by CvxNet.
Revised: This is the sampling strategy proposed by CvxNet.

Original:  R ESULTS
Revised:  R ESULTS

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We use the volumetric IoU, the Chamfer-L1 distance and FScore (Tatarchenko et al., 2019) for evaluation.
Revised: We use the volumetric IoU, the Chamfer-L1 distance and FScore (Tatarchenko et al., 2019) for evaluation.

Original: Volumetric IoU is obtained from 100k uniformly sampled points.
Revised: Volumetric IoU is obtained from 100k uniformly sampled points.

Original: The Chamfer-L1 distance is estimated by randomly sampling 100k points from the groundtruth mesh and predicted mesh which is generated by Marching Cubes (Lorensen & Cline, 1987).
Revised: The Chamfer-L1 distance is estimated by randomly sampling 100k points from the ground-truth mesh and predicted mesh which is generated by Marching Cubes (Lorensen & Cline, 1987).

Original: FScore is calculated with d = 2% of the side length of the reconstructed volume.
Revised: F-Score is calculated with d = 2% of the side length of the reconstructed volume.

Original: Note that following the discussions by Tatarchenko et al.
Revised: Note that following the discussions by Tatarchenko et al.

Original: (2019), F-Score is a more robust and important metric for 3d reconstruction compared to IoU and Chamfer.
Revised: (2019), F-Score is a more robust and important metric for 3d reconstruction compared to IoU and Chamfer.

Original: All three metrics are used in CvxNet (Deng et al., 2019).
Revised: All three metrics are used in CvxNet (Deng et al., 2019).

Original: Competing methods.
Revised: Competing methods.

Original: The list of competing methods includes Pixel2Mesh (Wang et al., 2018),
Revised: The list of competing methods includes Pixel2Mesh (Wang et al., 2018),

Original: AtlasNet (Groueix et al., 2018), SIF (Genova et al., 2019), OccNet (Mescheder et al., 2019),
Revised: AtlasNet (Groueix et al., 2018), SIF (Genova et al., 2019), OccNet (Mescheder et al., 2019),

Original: CvxNet (Deng et al., 2019) and Hypernetwork (Littwin & Wolf, 2019).
Revised: CvxNet (Deng et al., 2019) and Hypernetwork (Littwin & Wolf, 2019).

Original: Results are taken from these works except for (Littwin & Wolf, 2019) which we provide re-implemented results.I npu t I m ag e s G r o und
Revised: Results are taken from these works except for (Littwin & Wolf, 2019) which we provide re-implemented results.

Original: T r u t h

Original: O cc N e t

Original: P r o p o s e d

Original: Quantitative results.
Revised: Quantitative results.

Original: We compare our method with a list of state-of-the-art methods quantitatively in Table 2.
Revised: We compare our method with a list of state-of-the-art methods quantitatively in Table 2.

Original: We improve the most important metric, F-score, from 51 .
Revised: We improve the most important metric, F-score, from 51 .

Original: 75% to 59 .
Revised: 75% to 59 .

Original: 66% compared to the previous state of the art OccNet (Mescheder et al., 2019).
Revised: 66% compared to the previous state of the art OccNet (Mescheder et al., 2019).

Original: We also improve upon OccNet in the two other metrics.
Revised: We also improve upon OccNet I npu t I m ag e s G r o und

Original:  According to the L1-Chamfer metric, AtlasNet (Groueix et al., 2018) has a slight edge, but we would like to reiterate that this metric is less important and we list it mainly for completeness.
Revised: T r u t h O cc N e t P r o p o s e d in the two other metrics. According to the L1-Chamfer metric, AtlasNet (Groueix et al., 2018) has a slight edge, but we would like to reiterate that this metric is less important and we list it mainly for completeness.

Original: Besides that, we re-implemented OccNet (Mescheder et al., 2019) using our embedding network and Hypernetwork (Littwin & Wolf, 2019) under our training setup.
Revised: Besides that, we re-implemented OccNet (Mescheder et al., 2019) using our embedding network and Hypernetwork (Littwin & Wolf, 2019) under our training setup.

Original: For
Revised: For

Original: OccNet, the re-implemented version has better results than the original one.
Revised: OccNet, the re-implemented version has better results than the original one.

Original: For Hypernetwork, we carefully choose the hyper-parameters as the paper suggested.
Revised: For Hypernetwork, we carefully choose the hyper-parameters as the paper suggested.

Original: Overall, we can observe that our method has the best results compared to all other methods.
Revised: Overall, we can observe that our method has the best results compared to all other methods.

Original: Qualitative results.
Revised: Qualitative results.

Original: We show qualitative reconstruction results in Fig.
Revised: We show qualitative reconstruction results in Fig.

Original: We compare our results with re-implemented OccNet.
Revised: We compare our results with re-implemented OccNet.

Original: We also show shape interpolation results in Fig.
Revised: We also show shape interpolation results in Fig.

Original: Interpolation is done by acquiring two features λ 1 and λ 2 from two different images.
Revised: Interpolation is done by acquiring two features λ 1 and λ 2 from two different images.

Original: Then we reconstruct meshes while linearly interpolating the two features.
Revised: Then we reconstruct meshes while linearly interpolating the two features.

Original: Ablation study.
Revised: Ablation study.

Original: We show the metrics under different hyper-parameter choices in Table 3, including different N and w .
Revised: We show the metrics under different hyper-parameter choices in Table 3, including different N and w .

Original: We ﬁnd that generally smaller w and larger N gives better results.
Revised: We ﬁnd that generally smaller w and larger N gives better results.

Original: However when w is small enough ( w = 0 .
Revised: However when w is small enough ( w = 0 .

Original: 01 , 0 . 1 ), the metrics are very close no matter how large N is.
Revised: 01 , 0 . 1 ), the metrics are very close no matter how large N is.

Original: Visualization of embedding space.
Revised: Visualization of embedding space.

Original: In Fig.
Revised: In Fig.

Original: 5, we ﬁrst show D train along with the corresponding meshes.
Revised: 5, we ﬁrst show D train along with the corresponding meshes.

Original: Since our embedding space is also R 3 , we can visualize the embedding space.
Revised: Since our embedding space is also R 3 , we can visualize the embedding space.

Original: When the number of points N in D train is small, the embedded shape looks more like an ellipsoid.
Revised: When the number of points N in D train is small, the embedded shape looks more like an ellipsoid.

Original: In this case, the Embedding Network maps inside points into an ellipsoid-like shape.
Revised: In this case, the Embedding Network maps inside points into an ellipsoid-like shape.

Original: When N is getting larger, the embedded mesh has a shape more similar to the original one.
Revised: When N is getting larger, the embedded mesh has a shape more similar to the original one.

Original: Thus, the Embedding Network only shift points by a small distance.
Revised: Thus, the Embedding Network only shift points by a small distance.

Original: This shows how the Embedding Network and the Point Generation
Revised: This shows how the Embedding Network and the Point Generation

Original: Network collaborate.
Revised: Network collaborate.

Original: Sometimes the Embedding Network does most of the work when points are difﬁcult to classify, while sometimes the Point Generation Network needs to generate structured point sets.
Revised: Sometimes the Embedding Network does most of the work when points are difﬁcult to classify, while sometimes the Point Generation Network needs to generate structured point sets.

Original: In Fig.
Revised: In Fig.

Original: 6, we show more examples of D train given N = 64 and w = 0 .
Revised: 6, we show more examples of D train given N = 64 and w = 0 .

Original: Relation to hypernetworks.
Revised: Relation to hypernetworks.

Original: The approach of hypernetworks (Ha et al., 2016), uses a (hyper- )network to generate parameters of another network.
Revised: The approach of hypernetworks (Ha et al., 2016), uses a (hyper- )network to generate parameters of another network.

Original: Our method has a relation to hypernetworks.
Revised: Our method has a relation to hypernetworks.

Original: The ﬁnal (decision) surface is decided by e i
Revised: The ﬁnal (decision) surface is decided by e i

Original: = g ( x i ) ,
Revised: = g ( x i ) ,

Original: i = 1 , . .
Revised: i = 1 , . .

Original: , N where x i is a point in D train .
Revised: , N where x i is a point in D train .

Original: According to Eq.
Revised: According to Eq.

Original: equation 6, the output is (cid:80) Ni
Revised: equation 6, the output is (cid:80) Ni

Original: =

Revised: =

Original: α i K ( g ( x i ) , g ( x )) .
Revised: α i K ( g ( x i ) , g ( x )) .

Original: The points
Revised: The points

Original: D train =
Revised: D train =

Original: { x i , y i } Ni =1 is obtained by the Point Generation Network, which we can treat as a hypernetwork , and the points are generated parameters .
Revised: { x i , y i } Ni =1 is obtained by the Point Generation Network, which we can treat as a hypernetwork , and the points are generated parameters .

Original: Thus our method can also be interpreted as a hypernetwork.
Revised: Thus our method can also be interpreted as a hypernetwork.

Original: Littwin & Wolf (2019) proposes an approach for shape reconstruction with hypernetworks but with a very high-dimensional latent vector (1024) and generated parameters (3394), while we only use a 256 dimensional latent vector and 3 × 64 = 192 parameters.
Revised: Littwin & Wolf (2019) proposes an approach for shape reconstruction with hypernetworks but with a very high-dimensional latent vector (1024) and generated parameters (3394), while we only use a 256 dimensional latent vector and 3 × 64 = 192 parameters.

Original: We still have better results according to Table 2.
Revised: We still have better results according to Table 2.

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: O
Revised: O

Original: [Figure]
Revised: [Figure]

Original: j ec t
Revised: j ec t

Original: E
Revised: E

Original: [Figure]
Revised: [Figure]

Original: dd e d
Revised: dd e d

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we presented a shape representation for deep neural networks.
Revised: In this paper, we presented a shape representation for deep neural networks.

Original: Training data generating networks establish a connection between few-shot learning and shape representation by converting an image of a shape into a collection of points as training set for a supervised task.
Revised: Training data generating networks establish a connection between few-shot learning and shape representation by converting an image of a shape into a collection of points as training set for a supervised task.

Original: Training can be solved with meta-learning approaches for few-shot learning.
Revised: Training can be solved with meta-learning approaches for few-shot learning.

Original: While our solution is inspired by few-shot learning, it is different in: 1) our training datasets are generated by a separate network and not given directly; 2) our embedding network is conditioned on the task but traditional few-shot learning employs unconditional embedding networks; 3) our test dataset is generated by sampling and not directly given.
Revised: While our solution is inspired by few-shot learning, it is different in: 1) our training datasets are generated by a separate network and not given directly; 2) our embedding network is conditioned on the task but traditional few-shot learning employs unconditional embedding networks; 3) our test dataset is generated by sampling and not directly given.

Original: The experiments are evaluated on a single image 3D reconstruction dataset and improve over the SOTA.
Revised: The experiments are evaluated on a single image 3D reconstruction dataset and improve over the SOTA.

Original: A A PPENDIX A. N ETWORKS
Revised: A A PPENDIX A. N ETWORKS

Original: We show the architecture of the embedding network and the points generator in Fig.
Revised: We show the architecture of the embedding network and the points generator in Fig.

Original: The embedding network is similar to DeepSDF (Park et al., 2019).
Revised: The embedding network is similar to DeepSDF (Park et al., 2019).

Original: [Figure]
Revised: [Figure]

Original: A. V ISUALIZATION OF FEATURE SPACE
Revised: A. V ISUALIZATION OF FEATURE SPACE

Original: We visualize the feature space in Fig.
Revised: We visualize the feature space in Fig.

Original: A. S TATISTICS OF METRICS
Revised: A. S TATISTICS OF METRICS

Original: Here we show more detailed statistics in Fig. 9.
Revised: Here we show more detailed statistics in Fig. 9.

Original: Comparing to IoU and F-score, most of the values of Chamfer concentrate around the mean value and rare values very far from the mean.
Revised: Comparing to IoU and F-score, most of the values of Chamfer concentrate around the mean value and rare values very far from the mean.

Original: The “peakedness” property of Chamfer implies it is unstable to a certain extent.
Revised: The “peakedness” property of Chamfer implies it is unstable to a certain extent.

Original: The conclusion is also consistent with Tatarchenko et al.
Revised: The conclusion is also consistent with Tatarchenko et al.

Original: A. S HAPE I NTERPOLATION
Revised: A. S HAPE I NTERPOLATION

Original: In Fig.
Revised: In Fig.

Original: 10, we show reconstructed shapes with bilinear interpolation.
Revised: 10, we show reconstructed shapes with bilinear interpolation.

Original: Given 4 shape latent feature vectors λ 0 , 0 , λ 0 , 1 , λ 1 , 0 and λ 1 , 1 , we show interpolated shapes with the bilinear interpolation equation
Revised: Given 4 shape latent feature vectors λ 0 , 0 , λ 0 , 1 , λ 1 , 0 and λ 1 , 1 , we show interpolated shapes with the bilinear interpolation equation

Original: [Equation]
Revised: [Equation]

Original: where v ∈ [0 , 1] and h ∈ [0 , 1]
Revised: where v ∈ [0 , 1] and h ∈ [0 , 1]

Original: A. E FFECTS OF w
Revised: A. E FFECTS OF w

Original: In the main paper, we have a regularizer to force the embedding space to be similar to the original space,
Revised: In the main paper, we have a regularizer to force the embedding space to be similar to the original space,

Original: [Equation]
Revised: [Equation]

Original: When w is very large, the embedding space is almost the same as the original space, especially when N is large.
Revised: When w is very large, the embedding space is almost the same as the original space, especially when N is large.

Original: The visualization is shown in Fig.
Revised: The visualization is shown in Fig.

Revised: We set w = 10 .

Revised: This is much higher than w = 0 .

Revised: 01 recommended in the paper.

Revised: In this case, we expect the embedding network to do less work and as reported for w = 10 the method mainly works for a large number of points.

Original: Recall that, with the kernel ridge regression as our base learner, the ﬁnal shape (decision) surface is decided by,
Revised: We can observe that indeed the embedding space (bottom) is similar to the original space (top) especially when N is large. Positive points are shown in red and negative points in blue. Top row: meshes are reconstructed with our regular pipeline. Generated points are fed into the

Revised: [Figure]

Revised: [Figure]

Revised: [Table]

Revised: the embedding network without re-training .

Revised: This isn’t expected to work well.

Original: [Equation]
Revised: Generated points are directly used in generating classiﬁcation boundaries. Visualized points are in the original space. We can see that even without the embedding network the results are surprisingly reasonable for a large number of points. Bottom row: meshes in the embedding space.

Original: [Figure]
Revised: Visualized points are in the embedding space.

Original: [Figure]
Revised: We can observe that the meshes look similar to the correct chair, especially for large N . This also validates that the embedding network does not have a large inﬂuence on the result for large w . If the regularizing weight w is set to 0, the shape in embedding space will be more ellipsoid like. Recall that, with the kernel ridge regression as our base learner, the ﬁnal shape (decision) surface is decided by,

Original: [Table]
Revised: [Equation]

Original: If the original and embedding space are close, we can even reconstruct meshes without the Embedding Network g ,
Revised: If the original and embedding space are close, we can even reconstruct meshes without the Embedding Network g ,

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: A. T HE CHOICE OF BASE LEARNERS
Revised: A. T HE CHOICE OF BASE LEARNERS

Original: In Table [Table] we show results of different base learners, SVM and ridge regression.
Revised: In Table 4, we show results of different base learners, SVM and ridge regression.

Original: The difference between the two methods is very small.
Revised: The difference between the two methods is very small.

Original: N (cid:88) i =α
Revised: Note that our model is trained on a synthetic dataset (ShapeNet). Here we test how our model generalizes to real world datasets. Without retraining and ﬁne-tuning, we apply our trained model to

Revised: Stanford Online Products.

Revised: The qualitative results are shown in Fig.

Revised: We believe that the results are good, but there is no ground truth data for evaluation.

Revised: A. I NTER -C ATEGORIES I NTERPOLATION

Revised: Similar to Fig.

Revised: 4, we also show inter-categories interpolation in Fig.

Revised: We take samples from two different categories, and reconstruct shapes with interpolated latents.

Revised: Although the interpo-

Revised: N (cid:88) i =α

Original: i K ( g ( x i ) , g ( x ))
Revised: i K ( g ( x i ) , g ( x ))

Original: N (cid:88) i =α
Revised: N (cid:88) i =α

Original: i K ( x i , x )
Revised: i K ( x i , x )

Original: N (cid:88) i =α
Revised: N (cid:88) i =α

Original: i K ( e i , e ) { x i , y } N i = { x i , y } N i = { e i , y } N i = 
Revised: i K ( e i , e ) { x i , y } N i = { x i , y } N i = { e i , y } N i = N = N = N =

Original: N = N =
Revised: N = N =

Original: N = 
Revised: N = lated shapes are not as high quality than the originals, they still show some characteristics of both categories, e.g ., sofas which are as tall as tables, and cars with chair-leg-like tires. A. T RAINING AND I NFERENCE We describe the detailed training process in Algorithm 1. This algorithm can be viewed along with

Original: N =
Revised: Fig. We also show the full process how we generate a triangular mesh given an input image in Algorithm 2. Algorithm 1 Training for number of training iterations do

Revised: • Sample a minibatch of m training sample pairs { I b , M b } Bb =1 , where I b is an image and M b is the target ground-truth mesh.

Revised: • Get the latent representations for images with a feature backbone Feature

Revised: [Equation]

Revised: • Get the labeled point sets (the ﬁrst half are marked as positive and the other half are marked as negative)

Revised: [Equation]

Revised: • Embed the point sets X to another space

Revised: [Equation]

Revised: • Optimize differentiable base learner P ( ·| α ) which deﬁnes binary classiﬁcation boundaries,

Revised: [Equation]

Revised: • Sample labeled point sets from ground-truth meshes { M i } Bb =1 ,

Revised: [Equation]

Revised: • Convert X t to the embedding space Embedding

Revised: [Equation]

Revised: • Predict labels by using base learner P ( ·| α )

Revised: [Equation]

Revised: • Update Feature( · ) , PointGen( · ) and Embedding( · ) by descending stochastic gradients

Revised: [Equation]

Revised: end for

Revised: • Prepare an input image I ∈ R 3 × H × W , • Get the latent representations for the image

Revised: [Equation]

Revised: [Equation]

Revised: • Get the labeled point sets (the ﬁrst half are marked as postive and the other half are marked as negative)

Revised: [Equation]

Revised: • Embed points to the embedding space

Revised: [Equation]

Revised: • Optimize the differentiable base learner P (

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: • Prepare a point set on a 3D grid with arbitrary resolution G × G × G ,

Revised: [Equation]

Revised: • Convert X t to embedding space

Original: N =
Revised: [Equation] • Predict labels by using base learner P ( α ) [Equation]

Revised: • Apply marching cubes algorithm to ¯ y t ,

Revised: [Equation]

Revised: [Equation]

Revised: [Table]

Revised: [Table]

Revised: A.

Revised: C HOICE OF BACKBONES

Revised: In Table

Revised: [Table]

Revised: we show the performance of our method with different backbones.

Revised: We keep all of the setup the same except for the feature backbone.

Original: N =
Revised: A. E MBEDDING DIMENSIONS In Table [Table]

Revised: we show the performance of our method with different embeddings.

Revised: Note that different from Table 5, the results shown here are trained with N = 256 .