Weakly Supervised Label Learning Flows
Authors: You Lu
Chidubem Gibson Arachie
Bert Huang
Venue: ICLR 2022 Submitted (2022)
Keywords: Weakly Supervised Learning
Deep Generative Flows
Deep Learning
Deep Generative Models
Machine Learning

Abstract: Supervised learning usually requires a large amount of labelled data. However, attaining ground-truth labels is costly for many tasks. Alternatively, weakly supervised methods learn with only cheap weak signals that only approximately label some data. Many existing weakly supervised learning methods learn a deterministic function that estimates labels given the input data and weak signals. In this paper, we develop label learning flow (LLF), a general framework for weakly supervised learning problems. Our method is a generative model based on normalizing flows. The main idea of LLF is to optimize the conditional likelihoods of all possible labelings of the data within a constrained space defined by weak signals. We develop a training method for LLF that trains the conditional flow inversely and avoids estimating the labels. Once a model is trained, we can make predictions with a sampling algorithm. We apply LLF to three weakly supervised learning problems. Experiment results show that our method outperforms many state-of-the-art alternatives.

Under review as a conference paper at ICLR 2022
WEAKLY SUPERVISED LABEL LEARNING FLOWs
Anonymous authors
Paper under double-blind review
ABSTRACT
Supervised learning usually requires large amount of labelled data. However. attaining ground-truth labels is costly for many tasks. Alternatively, weakly su- pervised methods learn with cheap weak signals that only approximately label some data. Many existing weakly supervised learning methods learn determin- istic function that estimates labels given the input data and weak signals. In this paper, we develop label learning flows (LLF), a general framework for weakly supervised learning problems. Our method is a generative model based on nor- malizing flows. The main idea of LLF to optimize the conditional likelihoods of all possible labelings of the data within a constrained space defined by weak signals. We develop a training method for LLF that trains the conditional flow inversely and avoids estimating the labels. Once a model is trained, we can make predictions with sampling algorithm. We apply LLF three weakly supervised learning problems. Experiment results show that our method outperforms many state-of -ftt---fthhaaaee art alternatives.
INTRODUCTION
Machine learning has achieved great success in many supervised learning tasks. However, in prac- tice, data labeling is usually human intensive and costly. To address this problem, practitioners are turming to weakly supervised learning (Zhou 2018), which trains machine learning models with only noisy labels that are generated by human specified rules or pretrained models for related tasks. Recent research shows that these models trained with weak supervisions can also perform well.
Many existing weakly supervised learning methods (Ratner et al. 2016 2017 Bach et al. 2019 Arachie & Huang 2021b) learn a deterministic function that estimates the unknown labels y given input data x and weak signals Q. Since the observed information is incomplete, the predictions based on can be varied. However, these methods ignore this uncertainty between x and y. this paper, we develop label learning flows (LLF),, general framework for weakly supervised learning problems, LLF is a flow-based generative model (Dinh et al. 2014 Rezende & Mohamed] 2015 Dinh et al. 2016 Kingma& & Dhariwal 2018). The main idea behind LLF is that define the rela- tionship between x and y with probability distribution p |x). which is modeled by conditional use the weak signals Q define constrained space for y and then optimize the likelihood of all possible that are within this constrained space. Therefore, this model captures all possible relationships between the input x and output y. Learning LLF can be defined as a con- strained optimization problem. We develop a learning method for LLF that trains the conditional flow inversely and avoids estimating the labels. For prediction, we use sample- based method (Lu & Huang 2020) to estimate the labels.
flow. In training,
We apply LLF to three weakly supervised learning problems: weakly supervised classifica- tion (Arachie & Huang 2021b) Mazzetto et al. (021b), weakly supervised regression, and unpaired cloud completion (Chen al. [019 Wu et al.1) These three problems have very different label types and weak signals. Our method outperforms all other state-of-the-art methods on weakly supervised classification and regression, and can perform comparably to other recent methods on unpaired point cloud completion. These experiments show that LLF is versatile and powerful.
BACKGROUND
In this section, we introduce weakly supervised learning and conditional normalizing flows.

---

Under review as a conference paper at ICLR 2022
Weakly Supervised Learning. Given dataset D {x1...,, XN J. and weak signals Q. weakly supervised learning finds a model that can predict the unknown label y; for each input data x,- Some methods, e.g., ALL (Arachie & Huang 2021b), define set of constrained functions based on Q and x. These functions form space of possible y. The methods then look for one possible y within this constrained space. In this work, follow this idea and use constrained functions to restrict the predicted y.
Conditional Normalizing Flows. A normalizing flow (Rezende & Mohamed] 2015 is series of invertible functions f fi f2 ..... o fx that transform the probability density of output variables y to the density of latent variables z. conditional flows (Trippe Turner 2018), flow layer function f; is also parameterized by the input variables x, i.e., f, fe.o.. where Ø; is the parameters
of f. With the thange-of-variable formula, the log conditional distribution log p(y (x) can be exactly and tractably computed as
K Σ
[fxxtr Ōri-1
logp(y|x) = 1z + log det
(I)
af is the Jacobian or
where pz (z) is tractable base distribution, e.g. Gaussian distribution. The matrix of f The r, = fx.e. (ri-1), Fo = and rą = z.
To normalizing flows, need to develop flow layers that invertible and have tractable Jacobian determinant. this paper, we use affine coupling layer (Dinh et al.) 2014 2016) to form normalizing flows. It splits the input two parts, and force the first part only relate the second part, so that the Jacobian is triangular matrix. For conditional flows, we can define conditional affine coupling layer as
Yo: Ys
= split(y),
Zか s(x, @ + b(x,ya),
z = concat(ya: 2n),
where s and b are two neural networks. The split() and the concat() functions split and concatenate the variables.
PROPOSED METHOD
In this section, we introduce the label learning flows (LLF) framework for weakly supervised learn- ing. Given Q and x, we define a set of constraints to restrict the predicted label y. These constraints can be inequalities, formatting like f(x, y, Q) b, equations. formatting like f(x, y, Q) b. For simplicity, we represent this set of constraints as C(x,y, y, Q). Let Ω be the constrained space of all possible y defined by C(x,y, Q). Previous methods only look for one possible y within Ω. In contrast, LLF optimizes the conditional log- likeehodd of all possible within Ω. resulting in the following objective
max B B (x) (()) [log ply Φ)],
where U(2) is a uniform distribution within Ω and p(y xx) a continuous density model of y.
Let ý be the true label. We assume that each data point x; only has one unique label ý,, so that Paata (x, 5) = Paata (x). Let q(El x) be a certain model of y. Traditional supervised learning learns a 4(5) x) that maximizes the cross entropy (x..) [log g |x, Φ)]. Following theorem indicates that maximizing log p(y) x) can be interpreted as maximizing lower bound of log g(51x).
Theorem Let 2 C Ω is a tight enough space satisfying that ý E Q and for two different y; and 5,, the Si and S23 are non-overlapped. The volume of S is bounded such that p S M. The relationship between p(y|x) and q(5 xx) can be defined as: q(5(x) = Syear ply|x)dy. Then maximizing log p(y|x) can be interpreted maximizing the lower bound of loologg() (()(x)) That is,
(xy
p(y xx Φ)] S MB g[}{( +)]
The complete proof is in appendix. Theorem T indicates that, when Q is well-defined, learning LLF is analogous to dequantization (Theis et al. 2015 Ho al. 2019). That is, the method optimizes the
2

---

Under review as a conference paper at ICLR 2022
likelihood of dequantized true labels. In practice, the real constrained space, i.e., Q, may be loose and does not fulfill the conditions in Theorem [ which will result inevitable errors that come from the weakly supervised setting. Moreover, for some regression problems, the ideal Ω only contains a single point: the ground truth label, i.e., S2 = {
3.1 LEARNING AND PREDICTION
Since y is unobserved, directly optimizing the conditional likelihood, i.c.. Equation 2 is impossible. Using the invertibility of normalizing flows, we can rewrite log p(y x) as
K
0fx,0x
logp(y)x) = log pz (y), +
Σ
log det
dr,-1
1=1
@Bgx.ds Ør,
lz)
log det
where 8x fx is the inverse flow.
With the inverse flow, Equation can be interpreted as a constrained optimization problem
dgx.o. Yr;
)
max E pa (x) Epz(z)
logpz(z)-
Σ log det
(5)
Φ
s.t. C(x,Bx,(z),0
Note that in Equation 5 the original constraint for is (x.o(z) Ω, and this constraint can be replaced with C(x, gx.o Q). For efficient training, this constrained optimization problem can be approximated with the penalty method, resulting in the objective
K
Öggx.>
) + (x) Q)
max E Paence (x) Ep
logpz(z))
Σ log det
(6)
Ör,
where is the penalty coefficient, and C,O means we reformulate the constraints to be penalty losses. For example, an inequality constraint will be redefined as hinge loss.
In practice the expectation with respect to pz(z) can be approximated with Monte Carlo esti- mate with samples. Since we only need to obtain stochastic gradients, we follow previous works (Kingma & Welling 2013) and set L ==
Given a trained model and a data point x,, prediction requires outputting a label y, for x,. We follow (Lu & Huang, 2020) and use sample average, i.e., yt = Σラ= ( as the prediction. In our expermeens, we lound that L = 10 enough for generating high-quality labels.
4
CASE STUDY
In this section, we illustrate three scenarios of using LLF to address weakly supervised learning problems.
4.1 WEAKLY SUPERVISED CLASSIFICATION
We follow previous works Arachie & Huang 2021b Mazzetto et al. 2021b) and consider binary classification. For each example, the label y two-dimensional vector within one-simplex. That is, the y y = [y E [0, 1 Σ. yli] = 1}. where y(i] is the ith dimension of y. Each ground truth label ý E {0. 1}2 is a two-dimennt vio-dimensional one-hot vector. We have M weak labelers, which will generate M weak signals for each data point x;, i.e., qi [q.. Mi Each weak signal qi.m Q {q E [0, 1|2 Σ, q 1} is soft labeling of the data. In practice, if weak labeler m fails to label data point x,, the qi. can be null, i.c., q.m. (Arachie & Huang 2021a)) Following Arachie & Huang (2021b), assume have access expected rate bounds of

---

Under review as a conference paper at ICLR 2022
these weak signals b [i,... by] Therefore, the weak signals imply constraints
(i- O qim + yı O qm.m) S No Dm 4m (I,,,, M).
(7)
m
where Ni, is the number of data points that are labeled by weak labeler m.
This problem can be solved with LLF, i.c., Equation by defining C(x, gx,o Q) to be a com- bination of weak signal constraints, i.e., Equation 7 and simplex constraints, i.e,, y. The objective function of LLF for weakly supervised classification is
K
(88x.ds rr;
max
(z)
log det
1=1
2
+A1 (Bxxo(z)) + A2 Bx,0z)]] + 2
EExxdl 8x.o 8x,o(2)11
+ Σ
Σ
(1 Ex (z),) qim (xx(Zz) @ qi.)) Nm by
(8)
9..m7 70
where the second row describes the simplex constraints, and the last row is the weak signal con- straints. The [+]+ is hinge function that returns its input if positive and zero otherwise. We omit the expectation terms for simplicity.
4.2 WEAKLY SUPERVISED REGRESSION
For weakly supervised regression, predict one-dimensional continuous labels y [0, 1] given input dataset D [x,,.., x.) } and weak signals Q. We define the weak signals as follows. For the with feature input data, have access to threshold cm, which splits D two parts, i.e., D1, D2, such that for each X; E DD,, the Xim 2 and for each Xj 'D2, the Xj.m < cm We also have estimated values of labels for subsets D1 and D2, i.e. bm.. and 6m.2- This design of weak signals tries to mimic that in practical scenarios, human experts can design rule-based methods for predicting labels for given data. For example, in disease prediction, medical doctor can predict the disease rates for patients based on their ages. For group of people whose age is greater than threshold, an experienced physician would know an estimate of their average disease rate. Assuming that we have M rule-based weak signals, the constraints can be mathematically defined as follows:
Σ
= bm,
Σ
= bm... m E 1,... M.
(9)
D
[Dm.2l jEDm,
IE D
Plugging in Equation 9 to Equation o we have
log det Ögx.s Or;
)
max
lzg
dr [x), + de [x)(
M + +\g
Σ
(x)0
bm,1
+
Σ Bxxo(2); - bm,
(Dm,1
[Dm,a jEDm.
maml
iEDm..
(10)
4.3 UNPAIRED POINT CLOUD COMPLETION
Unpaired point cloud completion (Chen al. 2019 Wu al. 2020 is practical problem in 3D scanning. Given a set of partial point clouds X,
{x]]]
..... x N }. and set complete point
clouds X. {x(0) x (c)
want restore each x! Xr. Each point cloud is set of points,
4

---

Under review as a conference paper at ICLR 2022
Le., x, {x1,1; ... X,r),, where each Xi,t E R2 is 3D point, and the counts T represent the number points in a point cloud.
Note that the point clouds in Xp and Xe are unpaired, directly modeling the relationship between x(() and x(6) is impossible. This problem be interpreted weakly supervised learning prob- lem, in which the weak signals Q are derived from the referred complete point clouds X. We predict complete point clouds y y for partial point clouds in Xp- The conditional distribution p(y xp) is an exchangable distribution. We follow previous works (Yang et al., 2019; Klokov et al. 2020) and use De Finetti's representation theorem and variational inference to compute lower bound as the objective.
Te log log p(y,uu, xp) (L(g(()x,)lll(u))
logp(y|xp)
Bx(())pp)
(1I)
where ((((pp) is variational distribution of latent variable In practice, it can be represented by an encoder, and uses the reparameterization trick (Kingma & Welling 2013) to sample u. The pf (u) standard Gaussian prior. The p(y;|u, (,) is defined by conditional flow. We follow Chen al. ;
Wu et al. (2020) and use the adversarial loss and Hausdorff distance loss as constraints. The final objective function is
To Pz (2 (z+)
K
log det. (Buxxens )|
max @
(u(xp)))
E 1=1
Öre
(l.(g(ul)x))lp(uu)
+Ea(u/,,)) (D (2)) 1)a + Azdz
(12)
where D() represents the discriminator of least square GAN (Mao et al. 2017). and dx(() repre- sents the Haudorsff distance, which measures the distance between a generated complete point cloud its corresponding input partial point cloud. For clarity, we use and r, to represent variables of the Ith point point cloud, and gu.x,.. (z) to represent a generated point cloud. Detailed derivations of Equation 12 are in appendix.
Training with Equation 12 different from the previous settings, because we also need train the discriminator of the GAN. The objective for D() is
min E Punn (x..) [(((x)) 1)²] + Epaots (2)
(z))
(13)
The training process is similar traditional GAN training. The inverse flow gu.x,. be roughly seen as the generator. In training, we train the flow to optimize Equation [2]and the discriminator to optimize Equation 13, alternatively.
RELATED WORK
In this section, we introduce the research that most related work.
Weakly Supervised Learning. For weakly supervised classification, we use the same strategy as adversarial label learning (ALL) (Arachie Huang, (2221) to define constraint functions based on weak signals. ALL then uses a min-max optimization to learn the model parameters and estimate alternatively. In contrast ALL, LLF is generative model, so it can learn the model parameters and output y simultaneously, and it does not need min-max optimization. Moreover, LLF optimizes the likelihoods of all possible ys within Ω while ALL only estimates one possible y. Other methods also constrain the label space of the predicted labels using weak supervision (Arachie & Huang. 2021a Mazzetto et al.) 2021a (). These methods are deterministic and developed for classification tasks. However, LLF can be used for regression and uses sampling during inference.
Non-constraint based weak supervision methods typically assume a joint distribution for the weak signals and the true labels of the data. These methods use a latent variable model to estimate the labels of the data while accounting for the dependency among the weak signals (Ratner et al. 2016 2019 Fu et al. 2020). Like these methods, assume family of distributions for the label space of the data. This space is defined by the constraints of the weak supervision and the data. Unlike these methods, we use a flow network rather than a graphical model to solve for the label of the
5

---

Under review as a conference paper at ICLR 2022
data. Additionally, do not solve for the dependence amongst the weak signals thereby avoiding the need for making extra assumptions.
Normalizing Flows. Normalizing flows (Dinh et al., 2014 Rezende & MMohamed 2015 Dinh et al, 2016 Kingma & Dhariwal 2018) have gained recent attention because of their advantages of ex- act latent variable inference and log-likelihood evaluation. Specifically, conditional normalizing flows have been widely applied to many supervised learning problems (Trippe & Turnerr 2018 Lu & Huang. 2020 Lugmayr al. 2020; Pumarola et al.) (020) and semi-supervised classifica- tion (Atanov et al. 2019 Izmailov et al. 2020). However, normalizing flows have not previously been applied to weakly supervised learning problems.
Our inverse training method for LLF is similar to that of injective flows (Kumar ct al. 2020). Injec- tive flows are used model unconditional datasets. They use an encoder network map the input data x to latent code z, and then they use an inverse flow to map z back to x, resulting in an autoen- coder architecture. Different from injective flow, LLF directly samples z from a prior distribution and uses conditional flow map z back to y conditioned on We use constraint functions to restrict y to be valid, so that does not need an encoder network.
Point Cloud Modeling. Recently, Yang et al. (2019) and Tran et al. (2019) combine normaliz- ing flows with variational autoencoders (Kingma Welling (013) and developed continuous and discrete normalizing flows for point clouds. The basic idea of point normalizing flows is to use conditional flow to model each point in a point cloud. The conditional flow is conditioned on a la- tent variable generated by encoder. To guarantee exchangeability, the encoder uses PointNet (Qi
ct al. 2017) to extract features from input point clouds.
The unpaired point cloud completion problem is first defined by Chen et al. 2019). They develop pel2pel-a GAN (Goodfellow et al., 2014) based model-to solve it. Their method is two-staged. In the first stage, trains autoencoders space. In the second stage, a GAN is used to transform the latent features of partial point clouds to latent features of complete point clouds. In their follow-up paper (Wu et al., 2020). they develop a variant of pel2pel, called multi-modal pel2pel (mm-pcl2pel), which incorporates random noise to the generative process, so that can capture the uncertainty reasoning.
map partial and complete point clouds their latent
In contrast to pel2pel, LLF can be trained end-to-end When applying LLF to this problem, LLF has similar framework VAE-GAN (Larsen al., 2016). The main differences are that LLF models a conditional distribution of points, and its encoder is point normalizing flow.
6 EMPIRICAL STUDY
In this section, we evaluate LLF on the three weakly supervised learning problems.
Model architecture, For weakly supervised classification and unpaired point cloud completion, the labels y are multi--immesional variables. We follow Klokov al. (2020) and use flows with only conditional coupling layers. We use the same method as Klokov et al. (2020) to define the condi- tional affine layer. Each flow model contains 8 flow steps. For unpaired point cloud completion, each flow step has 3 coupling layers. For weakly supervised classification. each flow step has 2 coupling layers. For weakly supervised regression, since y is a scalar, we use simple conditional affine transformation as flow layer, which is defined as:y = sxx) s z + b(x), where s and b are two neural networks that take x as input and output parameters for y. The flow for this problem contains 8 conditional affine transformations.
For the unpaired point cloud completion task, we need also use an encoder network, i.e., q( x,) and a discriminator D() We follow Klokov et al. (2020); Wu et al. (2020) and use PointNet (Qi et al. 2017) in these two networks to extract features for point clouds.
Experiment setup. In weakly supervised classification and regression experiments, we assume that the ground truth labels are inaccessible, so tuning hyper-parameters for models are impossible. We use default settings for all hyper-parameters of LLF, e.g., As and learning rates. We fix X 10 and use Adam (Kingma & Ba 2014) with default settings, i.e., 7 = 0.001, 31 = 0.9 and 82 = 0.999. We use an exponential learning rate scheduler with decreasing rate of 0.996 to guarantee convergence. We track the decrease of loss and when the decrease is small enough, the training stops. Following previous works (Arachie & Huang 2021b a), we use full gradient optimization

---

Under review as a conference paper at ICLR 2022
to train the models. For fair comparison, we run each experiment 5 times with different random seeds {0. 10, 100, 123, 1234). We split each dataset to training, simulation, and test sets. We follow Arachie & Huang (2021b and create weak signals with randomly chosen features, and estimate error bounds and thresholds on simulation set.
For experiments with unpaired point cloud completion, the datasets contain validation sets, tune the hyper-parameters using these. We use Adam with an initial learning rate 1j 0.0001 and default As.. The best coefficients for the constraints in Equation [2 are A 10, A2 100. We use stochastic optimization to train the models, and the batch size is 32. Each model is trained for at most 2000 epochs.
6.1 WEAKLY SUPERVISED CLASSIFICATION
Datasets. We follow Arachie & Huang 202 la b and conduct experiments on 12 datasets. Specif- ically, the Breast Cancer, OBS Network, Cardiotocography, Clave Direction, Credit Card, Statlog Satellite, Phishing Websites, Wine Quality tabular datasets from UCI repository (Dua & Graff 2017). The Mashion-MNIST (Xiao et al. 2017) is an image set with 10 classes of clothing types. We choose 3 pairs of classes, i.e., dresses/sneakers (DvK), sandals/ankle boots (SvA), and coats/bags (CvB), to conduct binary classification. We follow Arachie & Huang (2021b) and create 3 synthetic weak signals for each dataset. The IMDB (Maas al. 20111), SST (Socher al. 2013) and YELP are real text datasets. We follow Arachie & Huang (2021a) and use keyword-based weak super- vision. Each dataset has more than 10 weak signals. For the experiments on tabular datasets, we set the maximum epochs to 2000. Since the experiments on real text datasets are larger, we set the maximum epochs 500.
Baselines. We compare our method with state-of-the-ee methods for weakly supervised classifi- cation. For the experiments on tabular datasets and image sets, we use ALI. (Arachie & Huang. 2021b), generalized expectation (GE) (Druck et al.) 20088 Mann & McCallum (2010) and averaging of weak signals (AVG), For experiments on text datasets, we use CLL (Arachie & Huang 2021a),
Snorkel Me (Ratner et al. 2019). Data Programming (DP) (Ratner et al. 2016)) regularized min- imax conditional entropy for crowdsourcing (MMCE) (Zhou et al. 2015). and majority-vote, We
also show supervised learning (SL) results for reference.
We report the mean and standard deviation of accuracy on test sets in Table I and Table LLF outperforms all baseline methods on almost all datasets. On some datasets, L.F. can perform as well as supervised learning methods. These results prove that LLF is powerful and effective. In our experiments, we also found that the performance of LLF will also be impacted by different initialization of weights. This is why LLF has relatively larger variance on some datasets.
Results.
Table I: Test set accuracy on tabular and image datasets. We report the mean accuracy of 5 experi- ments, and the subscripts are standard deviation. LLF outperforms other baselines 10 datasets.
LLF
ALL
GE
AVG
SL
Fashion MNIST (DvK)
1.000p.00m
0.995
0.979..0n0
0.835g.000
1.000p.a00
Fashion MNIST (SvA)
0.944p.om 001
0.908..mm one
0.501g.om
0.79lg.om
0.972o.000
Fashion MNIST (CvB)
0.916p 038
0.805g 000
0.501..000
0.740g..0
0.988p.000
Breast Cancer
0.968..0o 008
0.937 o19
0.933g.016 016
0.91 023
0.973p.007
OBS Network
0.684o.006 006
0.691..00 011
0.676g.00 010
0. 0.7009.022
0. 0.704p.032
Cardiotocography
0.931p.01n
0.795g.mm
0.6630.06
0.902a.047
0.941p.om
Clave Direction
0.858n.011
0.750
0.7566..2z
0.707
0.963o.om1
Credit Card
0.680 p220p.022
0.678o.021
0.4922.o5s
0.602g.010
0. 0.7177.05 70.031
Statlog Satellite
0.997o.00r
0.959u.uus
0.987a.u12
0.915g.011
0.999p.001
Phishing Websites
0.906u.om
O. 0.896g..um
0.870g.up9
0.848g.002
0.929p.a01
Wine Quality
0.647u.017
0.623
0.445o.014
0.555g.000
0.685p.000
Ablation Study. We can directly train the model using only the constraints as the objective function. In our experiments, we found that training LLF without likelihood (LLF-w/o-nll) will still work. However, the model performs worse than training with likelihood. We believe that this is because the likelihood helps accumulate more probability mass to the constrained space ?, so the model will more likely generate y samples within Ω and the predictions are more accurate.
7

---

Under review as a conference paper at ICLR 2022
Table 2: Test set accuracy on real text datasets. LLF outperforms all other baselines.
LLF
CLL
MMCE
DP
MV
McTal.
SL
SST IMDB
0.766p.om 002
0.729b.00 001
0.727
0.720o 001
0.720o.om
0.728p.m 001
0.792a.00
0.804v.or 000
0.740u.00.
0.551
0.6233 007
0.724g.um 004
0.742..u o00
0.820u.um 0003
YELP
0.861u 8.6100000
0.840a.091
0.680
0. 0.760a.om
0.798a.007
0.780g.usz
0.879u.u01
Accuncy of MUST
MLL MNIST
Violation HMSTT
0.100
0.3
0,075
0.3
0.050
0.7
0,225
0.6-
0.000
Epoch
Spech
Spech
Accurccy of Clasn
WLL of Clave
Violatin of Clasn
0.8
0.775
0,050
0.5
0.5
1500
1006
1500
0.000
1006
2006
Epech
muthed Daa thuind
Har baad wi--- (bain)
Figure 1: Evolution of accuracy, likelihood and violation of weak signal constraints. Training with likelihood makes LLF accumulate more probability mass to the constrained space, so that the gen- erated y are more likely to be within Ω and the predictions are more accurate.
6.2 WEAKLY SUPERVISED REGRESSION
Datasets. We use 3 tabular datasets from the UCI repository Dua & Graff (017): Air Quality, Temperature Forecast, and Bike Sharing dataset. For each dataset, we randomly choose features to develop the rule based weak signals. We split each dataset to training, simulation, and test sets. The simulation set is then used to compute the threshold es, and the estimated label values bs. Since we do not have human experts to estimate these values, we use the mean value of a feature as its threshold, i.e, 6m = |aaul EieD... x; [mm We then compute the estimated label values 6m,1 and 6m.2 based on labels in the valid set. Note that the labels in the simulation set are only used for generating weak signals, simulating human expertise. In training, we still assume that we do not Myy. have access to labels. The original label is within an interval [y). We normalize the original label to within [0, 1] by computing y = (y (,)/(u,, 1,,). In prediction, we recover the predicted label to original value by computing y u(u, (y) + b1y
Baselines. To the best of our knowledge, there are no methods specifically designed for weakly supervised regression of this form. We use average of weak signals (AVG) and LFFF-//-nll as basclines. We also report the supervised learning results for reference.
Results. We use root square mean error (RSME) metric. The results of test set are in Table 3 In general, LLF can predict reasonable labels. Its results are much better than AVG or any of the weak signals alone. Similar to the classification results, training LLF without using likelihood will reduce its performance.
Table 3: Test RMSE of different methods. The numbers in brackets indicate the label's range. LLF outperforms other baselines on all datasets.
LLF
LLF-w/o-nll
AVG
SL
Air Quality (0.1847 ~ 2.231)
0.211o.009
0.2666.004
0.373
0.123u.v02
Temperature Forecast (17.4 ~ 38.9) Bike Sharing (1 ~ 999)
2.552. 050
2.656o oss
2.827 027
1.465 631
157.348, 541
162.6971.583
171.3381.900
141.920, 280
8

---

Under review as a conference paper at ICLR 2022
6.3 UNPAIRED POINT CLOUD COMPLETION
Datasets. We use the Partnet (Mo et al. 2019) dataset our experiments. We follow Wu et al. 2020) and conduct experiments on the 3 largest classes of PartNet: Table, Chair, and Lamp. We treat each class as a dataset, which is split to training, validation, and test sets based on official splits of PartNet. For each point cloud,
remove points of randomly selected parts to create partial point cloud. We follow Chen et al.] (2019); Wu et al. (2020) and let the partial point clouds have 1024 points, and the complete point clouds have 2048 points. We let the latent variable of the VAE to be a 128-dimensional vector.
Metrics. We follow Wu al. (2000) and use minimal matching distance (MMD) (Achlioptas al., 2018), total mutual difference (TMD), and unidirectional Hausdorff distance (UHD) as metrics. MMD measures the quality of generated. A lower MMD better. TMD measures the diversity of samples. A higher TMD is better. UHD measures the fidelity of samples, A lower UHD is better.
Baselines. We compare our method with pel2pel (Chen et al. 2019), mm-pel2pel (Wu al. 2020), and LLF-w/o-nll. We use two variants of mm- mm-pelppel. Another variant is called mm--pel num-pel2pel-ing which different from the original model in that jointly trains the encoder of modeling multi- modality and the GAN.
Table 4: Evaluation results on three classes of PartNet. LLF performs comparable to baselines.
PartNet
Chair
Lamp
Table
MMD
TMD*
UHD1
MMDJ
TMD1
UHD
MMD
TMD1
UHDI
LLF
1.72
0.63
5.74
2.11
0.37
4.71
1.57
0.55
5.42
LLF-w/o-nll
1.79
0.47
5.49
2.21
0.41
4.61
1.57
0.43
5.13
pel2pel
1.90
0.00
4.88
2.50
4.64
1.90
0.00
4.78
mm-pcl2pel
1.52
2.75
6.89
1.97
3.31
5.72
1.46
3.30
5.56
mm-pel2pel-im
1.90
1.01
6.65
2.55
0.56
5.40
1.54
0.51
5.38
Results. We list the test set results in Table general, pel2pel has the best fidelity, i.e., lowest UHD. This is because pcl2pel is discriminative model, and will only predict one certain sample for each input. This is also why pel2pel has the worse diversity as measured by TMD. Mm-pel2pel has the best diversity. However, we found our experiments that some samples generated by mm- pel2pel are invalid, i.c., they are totally different from the input partial point clouds. Therefore, mm-pcl2pel has the worse fidelity. LLF scores between pcl2pel and mm-pel2pel. has better UHD than mm-pe/2pel, and better TMD and MMD than pel2pel. The LLFF-/O-nll has slightly better UHD than LLF. We believe this is because, without using the likelihood, L.LF-w/o-ll is trained directly by optimizing the Hausdorff distance. However, the sample diversity and quality, i.e., TMD and MMD, are worse than LLF. As argued by Yang et al. (2019), the current metrics for evaluating point cloud samples all have flaws, so these scores cannot be treated as hard metries for evaluating model performance. We therefore visualize some samples in appendix, which show that LLF can generate samples that comparable to mm-pel2pel.
CONCLUSION
In this paper, we propose label learning flows, which represent general framework for weakly supervised learning. LLF uses a conditional flow to define the conditional distribution p(y[x), so that can model the uncertainty between input x optimization problem that optimizes the likelihood of all possible y within the constrained space defined by weak signals. We develop a specific training method to train LLF inversely, avoiding the need of estimating results show that our method outperforms many state-of oftte-of-the-se art baselines on the weakly supervised classification and regression problems, and performs comparably to other methods for unpaired point cloud completion. These results indicate that LLF a powerful and effective tool for weakly supervised learning problems.
possible y. Learning LLF is a constrained
We apply LLF to three weakly supervised learning problems, and the

---

Under review as a conference paper at ICLR 2022
REFERENCES
Panos Achlioptas, Olga Diamanti, loannis Mitliagkas, and Leonidas Guibas. Learning representa- tions and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR, 2018.
Chidubem Arachie and Bert Huang. Constrained labeling for weakly supervised learning. In Inter- national Conference in Uncertainty Artificial Intelligence. 2021a.
Chidubem Arachie and Bert Huang. A general framework for adversarial label learning. Journal of Machine Learning Research, 22(1 22(118)) 11)::13,,, 2021b.
Andrei Atanov, Alexandra Volokhova, Arsenii Ashukha, Ivan Sosnovik, and Dmitry Vetrov, Semi- conditional normalizing flows for semi-supervised learning. arXiv preprint 2019.
Stephen H Bach, Daniel Rodriguez, Yintao Liu, Chong Luo. Haidong Shao, Cassandra Xia, Souvik Sen, Alex Ratner, Braden Hancock, Houman Alborzi, et al. Snorkel drybell: A case study in de- ploying weak supervision at industrial scale. In Proceedings of the 2019 International Conference on Management of Data, pp.
2019.
Xuelin Chen, Baoquan Chen, and Niloy J Mitra. Unpaired point cloud completion on real scans using adversarial training. arXiv preprint arXiv: 1904.0009, 2019.
Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-linear independent components
estimation. arXiv preprint arXiv: 141.08516. 2014.
Laurent Dinh, Jascha Sohl- Dickstein, and Samy Bengio. Density estimation using real NVP. arXiv 2016.
preprint
Gregory Druck, Gideon Mann, and Andrew McCallum. Learning from labeled features using gen- eralized expectation criteria. Proceedings s of the 31st annual international ACM SIGIR confer- ence on Research and development in information retrieval, pp. 595-602, 2008
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL /ttp::////archteee ics. uci. sedu/m)
Daniel Fu, Mayee Chen, Frederic Sala, Sarah Hooper, Kayvon Fatahalian, and Christopher Ré. Fast and three-rious: Speeding up weak supervision with triplet methods. In International Conference on Machine Learning, pp. 3280-3291. PMLR, 2020.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Infor- mation Processing Systems, pp. 2672-2680, 2014.
Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++: Improving flow- based generative models with variational dequantization and architecture design. arXiv preprint 10Xiv:1902.00025, 2019.
Pavel Izmailov, Polina Kirichenko, Mare Finzi, and Andrew Gordon Wilson. Semi-supervised learn- ing with normalizing flows. In International Conference on Machine Learning, pp. 4615-4630. PMLR, 2020.
Diederík Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. 1rXiv:1312.61 14, 2013.
arXiv preprint
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible IxI convolutions. In Advances in Neural Information Processing Systems, pp. 10215-10224, 2018.
Roman Klokov, Edmond Boyer, and Jakob Verbeek. Discrete point flow networks for efficient point cloud generation. In Computer Vision-ECCe 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII 16, pp. 694-710. Springer, 2020.
10

---

Under review as a conference paper at ICLR 2022
Abhishek Kumar, Ben Poole, and Kevin Murphy. Regularized autoencoders via relaxed injective probability flow. In International Conference on Artificial Intelligence and Statistics, pp. 4292- 4301. PMLR. 2020.
Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther, Autoen- coding beyond pixels using a learned similarity metric. In International conference on machine learning, pp. 1558-1566. PMLR, 2016.
You Lu and Bert Huang. Structured output leamning with conditional generative flows. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5005-5012, 2020.
Andreas Lugmayr, Martin Danelljan, Lue Van Gool, and Radu Timofte. Srflow: Learning the super- resolution space with normalizing flow. In European Conference on Computer Vision, pp. 715- 732. Springer, 2020.
Andrew Maas, Raymond Daly, Peter Pham, Dan Huang, Andrew Y Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings the 49th annual meeting of the association for computational linguistics: Human language technologies, pp. 142-150, 2011.
Gideon S Mann and Andrew McCallum. Generalized expectation criteria for semi-supervised learn- ing with weakly labeled data. Journal of machine learning research, 11(2), 2010.
Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE international confer- ence on computer vision, pp. 2794-2802, 2017.
A. Mazzetto, C. Cousins, D. Sam, S. H. Bach, and E. Upfal. Adversarial multiclass learning under weak supervision with performance guarantees. In International Conference on Machine Learn- ing (ICML), 2021a.
Alessio Mazzetto, Dylan Sam, Andrew Park. Eli Upfal, and Stephen Bach. Semi-supervised ag- gregation of dependent weak supervision sources with performance guarantees. In International Conference Artificial Intelligence and Statistics, pp. 3196-3204 -204. PMLR, 2021b.
Kaichun Mo. Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas Guibas, and Hao Su. Partnet: A scale benchmark for fine-grained and hierarchical part-level 3d object under- standing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 909-918, 2019.
Albert Pumarola, Stefan Popov, Francese Moreno-Noguet and Vittorio Ferrari. C-flow: Condi- tional generative flow models for images and 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7049-99999999 -958, 2020.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Alexander Ratner, Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. Snorkel: Rapid training data creation with weak supervision. In Proceedings of the VLDB En- dowment. International Conference on Very Large Data Bases, volume 11, pp. 269. NIH Public Access, 2017.
Alexander Ratner, Braden Hancock, Jared Dunnmon, Frederic Sala, Shreyash Pandey, and Christo- pher Ré. Training complex models with multi-task weak supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 4763-4771, 2019.
Alexander J Ratner, Christopher M De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. Data programming: Creating large training sets, quickly. Advances in neural information processing systems, 29:3567-3575, 2016.
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint / 2015.
11

---

Under review as a conference paper at ICLR 2022
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language pro- cessing, pp. 1631-1642. 2013.
Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. arXiv preprint arXiv:1511.0184. 2015.
Dustin Tran, Keyon Vafa, Kumar Krishna Agrawal, Laurent Dinh, and Ben Poole. Discrete flows: Invertible generative models of discrete data. arXiv preprini arXiv: 2019.
Brian Trippe and Richard E Turner. Conditional density estimation with Bayesian normalising flows. arXiv preprint vrXiv:1802.049000 2018.
Rundi Wu, Xuelin Chen, Yixin Zhuang, and Baoquan Chen. Multimodal shape completion via
conditional generative adversarial networks. arXiv preprint arXiv:2003.07717, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint WXXvv1708.007477 2017.
Guandao Yang, Xun Huang, Zekun Hao, Ming-Yo Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4541-444 4541-4550, 2019.
Dengyong Zhou, Qiang Liu, John C Platt, Christopher Meek, and Nihar B Shah. Regularized minimax conditional entropy for crowdsourcing. arXiv preprint arXiv:1503.0722 2015.
Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National science review, 5(1):
44-53, 2018.
A PROOF OF THEOREM
The proof of Theorem D is similar the proof of dequantization Theis et al. 2015 Ho al. 2019). The complete proof is as follows.
Proof.
Σ Pdasa(x)) / Jyerr S2-
E Exacca (x) (xy [logply|x,d)) 4))
logp(y(x)ds
M Σ Patata (x) log
plylx)dt
yenr
M
Σ Pattaa (x. log ((}1x)
M Papaac (log [(}[x)]
(14)
In the second row, we use the properties that [32- < M, and the Jensen's 's inequality. In the third row, we use the assumption that Panaa = Paata (x, 8), and the relationship between p(y) (x) and q()lx))))
B LABEL LEARNING FLOW FOR UNPAIRED POINT CLOUD COMPLETION
In this section, provide complete derivations of LLF for unpaired point cloud completion. The conditional likelihood log p(y xx,) is an exchangeable distribution. We use De Finetti's representa- tion theorem and variational inference derive tractable lower bound for it.
12

---

Under review as a conference paper at ICLR 2022
/ plyy, ulxp)du
logply lyyp(y) =
/ plylu, xp)p(udd
Bx())),))
Hogg(y/uu xx,)) ] kL(g(u\>,)ll((u))
AI (
log p(y: xp) <L(((uxx;))ll(u))
(15)
where in the third row, we use Jensen's inequality to compute the lower bound, and in the last row, use De Finetti's theorem factorize p(y [, xp) to the distributions of points.
The least square GAN discriminator and the Hausdorff distance for generated complete point clouds can be treated two equality constraints
D(y) = 1 dh (y,y)xp)
Note that the dx is non-negative. Convert these two constraints to penalty functions, we have
Tc
K
(0BI))Ar))) ){
max Ø
(u(xp)
Σ
log det
Öri,
([((((()))))l)(())
1=1
+ {()()) (D (z)) 1)2 A2d (gux (Bux
(16)
C EXPERIMENT DETAILS
In this section, provide more details experiments to help readers reproduce our results.
C.I MODEL ARCHITECTURE
For experiments of weakly supervised classification, and unpaired point cloud completion, we use normalizing flows with only conditional affine coupling layers Klokov al. (2020). Each layer is defined as
Ya Y; split(y)
m, (w, O (x) wx((x) b m. (C,(Va) 3 (x) + (x())
2₁ yo b
concat a. 2b),
(17)
where m, are all small neural networks.
For LLF, we only need the inverse flow, i.e., gx for training and prediction. so in our experiments, we actually define 8x.o as the forward transformation, and let = s @ + b. We do this because multiplication and addition are more stable than division and subtraction.
C.I.1 WEAKLY SUPERVISED CLASSIFICATION
In this problem, layers. These two layers will transform different dimensions. Each w and e are small MLPs with two linear layers. Each has one linear layer. The hidden dimension of linear layers is fixed 64.
flow with flow steps, and each step has conditional affine coupling
C.1.2 WEAKLY SUPERVISED REGRESSION
In this problem. since the label y is scalar, use conditional affine transformation introduced in Section 6 as a flow layer. A flow has 8 flow layers. The and b in flow layer are three layer MIPs. The hidden dimension of linear layers is 64.
13

---

Under review as a conference paper at ICLR 2022
C.1.3 UNPAIRED POINT CLOUD COMPLETION
The model architecture used LLF used for this problem illustrated Figure 2 We use the same architecture DPF (Klokov al.] 2020) for point flow. Specifically, the flow has 8 flow steps, and each step has 3 conditional affine coupling layers, i... Equation [7 Slightly different from the original DPF, the conditioning networks Was Ca. Wb. and O₁ will take the latent variable u and the features of partial point cloud Xp input. The ws and cs are MLPs with two linear layers, whose hidden dimension is 64. The ms are one layer MLPs.
We use a PointNet (Qi et al. 2017) to extract features from partial point cloud Xp. Following Klokov
al. (020), the hidden dimensions of this PointNet is set 64 124- 256 256-512. Given the features of Xp. the encoder E then uses the reparameterization trick (Kingma & Welling 2013) to generate
latent variable u. The encoder has two linear layers, whose hidden dimension is 512.
The GAN discriminator uses another PointNet to extract features from (generated) complete point clouds. We follow Wu al. (2020) and set the hidden dimensions this PointNet as 64-664422 1-8--28-11 1228- 256-128 The discriminator 1) is three layer MIP, whose hidden dimensions are 128 256 165-51 512.
c
PointNet
D
0 or
y
u
Point
X
PointNet
Flow
Z
Figure 2: Model architecture of LLF for unpaired point cloud completion. The E represents the encoder, and the D represents the GAN discriminator.
C.2 MORE EXPERIMENT DETAILS
C.2.1 WEAKLY SUPERVISED CLASSIFICATION
We use the same way as Arachie & Huang (021b to split each dataset to training, simulation, and test sets. We use the data and labels simulation sets create weak signals, and estimated bounds. We train models on training sets and test model on test sets. We assume that the models do have access any labels. The labels in simulation sets are only used to generated weak signals and estimate bounds.
For experiments on Fashion-MNIST and the tabular datasets, we follow Arachie Huang (202 and choose 3 features to create weak signals. We train logistic regression with each feature on the simulation set, and use the label probabilities predicted by this logistic regression as weak signals. We compute the error of this trained logistic regression simulation set estimated error bound.
For experiments on real text datasets, we use the same keyword- based method as Arachie & Huang (221a)) to create weak supervision. Specifically, choose key words that weakly indicate positive and negative sentiments. Documents containing possitive words will be labeled as positive, and vice versa.
We list some main features of these datasets in Table 5] We refer to their original papers for more details. For those datasets without official splits, randomly split them with ratio of : 3.
14

---

Under review as a conference paper at ICLR 2022
Table 5:
Summary of datasets used in weakly supervised classification experiments. indicates this dataset does not have a official split
The "_"
Dataset
Size
Train Size
Test Size
No. features
No. weak signals
Fashion MNIST (DvK)
14, 000
12, 000
2, 000
784
3
Fashion MNIST (SvA)
14, 000
12,
2, 2,0000
784
3
Fashion MNIST (CvB)
14,000 000
12, 100
2, 2,000
784
3
Breast Cancer
569
30
3
OBS Network
795
21
3
Cardiotocography
963
21
3
Clave Direction
8, 606
16
3
Credit Card
1,000
24
3
Statlog Satellite
041
36
3
Phishing Websites
15,
30
3
Wine Quality
4,974
11
3
IMDB
49.574
182
20, 392
300
10
SST
5,819
3,998
1,821
300
14
YELP
57,
45,
10, 000
300
14
C.2.2 WEAKLY SUPERVISED REGRESSION
We use three datasets from the UCI repository. For each dataset, randomly split it to training, simulation, and test sets with a ratio of 4 3 3. We use the simulation set to create weak signals and estimated label values. We choose 5 features to create weak signals for each dataset. The detailed introduction of these datasets are as follows. Table summarize the statistical results of them.
Air Quality. In this task, predict the absolute humidity in air, based other air quality features such as hourly averaged temperature, hourly averaged NO₂ concentration etc. The raw dataset has 358 instances. We remove those instances with Nan values, resulting in a dataset with 8, 991 in- stances. We use hourly averaged concentration CO, hourly averaged Benzene concentration, hourly averaged NO, concentration, tungsten oxide hourly averaged sensor response, and relative humidity as features for creating weak signals.
Temperature Forecast. In this task, we predict the next day maximum air temperature based on current day information. The raw dataset has 7, 750 instances, and we remove those instances with Nan values, resulting 7, 588 instances. We use present max temperature, forecasting next day wind speed, forecasting next day cloud cover, forecasting next day precipitation, solar radiation as features for creating weak signals.
Bike Sharing. In this task, we predict the count of total rental bikes given weather and date in- formation. The raw dataset has 17, 389 instances, and we remove those instances with Nan values, resulting in 17, 379 instances. We use season, hour, if is working day, normalized feeling tempera- ture, and wind speed as features for creating weak signals.
Table 6: Summary of datasets used in weakly supervised regression experiments
Dataset
Size
No. features
No. weak signals
Air Quality
8,991
12
Temperature Forecast
7,588
24
Bike Sharing
17, 379
12
C.2.3 UNPAIRED POINT CLOUD COMPLETION
Datasets. We use the same way as Wu et al. 2020 to process PartNet Mo al. (209)) PartNet provides wisnt-wise semantic labels for point clouds. The original point clouds are used as complete point clouds. To generate partial point clouds, randomly removed parts from complete point
15

---

Under review as a conference paper at ICLR 2022
clouds, based the semantic labels. We use Chair, Table, and Lamp categories. The summary of these three subsets are in Table
Table 7: Summary of datasets used unpaird point cloud completion experiments
Dataset
Train Size
Valid Size
No. Test Size
Chair
4, 489
617
1,217
Table
5, 707
843
1, 668
Lamp
1,545
234
416
Metrics. Let X. be the set of referred complete point clouds, and Xp be the set of input par- tial point clouds. For each partial point cloud x (?) we generate M complete point cloud samples (M) All these samples form
y; (1) ..... y y(M)
set of complete point clouds y. In our experiments, we follow Wu et al. (2020) and set M 10.
The MMD Achlioptas et al. 2018) is defined as
MMD
Σ X:EX
(18)
X
where NN(x) is the nearest neighbor of in V. The de represents Chamfer distance. MMD) computes the distance between the set of generated samples and the set of target complete shapes, so it measures the quality of generated.
The TMD is defined as
TMD
X
2
Σ
M
Σ kmj+1
dcly
(cly!".!!o) x(0))
(19)
X
M
1=1
TMD measures the difference of generated samples given an input partial point cloud, so it measures the diversity samples.
The UHD is defined as
Xp
M dn (x. dn(x))))))
UHD
(20)
M
J=1
where du represents the unidirectional Hausdorff distance. UHD measures the similarity between generated samples and input partial point clouds, so it measures the fidelity of samples.
Samples. We compare LLF to mm-pc2pe in Figure 3 and more samples of LLF in Figure 4 Figure and Figure 6 LLF can generate samples that are as good as mm-pe2pe. Mm-pc2pe has higher diversity in samples, but some generated samples may be unreasonable.
16

---

Under review as a conference paper at ICLR 2022
Partial
LLF
mm mm- -ccppc
Figure 3: Random sample point clouds generated LLF and mm-pe2pe. The point clouds generated by LLF are as realistic as mm-pe2pe. Mm-pc2pe has a higher diversity in samples. However, sometimes it may generate unreasonable or invalid shapes.
Figure 4: Random chair samples generated by LLF, The first row is partial point clouds, and the second row is generated complete point clouds.
Figure 5: Random lamp samples generated by LLF.
17

---

Under review as a conference paper at ICLR 2022
Figure 6: Random table samples generated by LLF.
18


Reviews:
[{"review_id": "qV6ZBWVrDII", "reviewer": "Reviewer_W3Vi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major Points:\n- Clarity regarding formal definition of the model.  Overall, the paper does not sufficiently describe some important technical details, especially with regards to prior work.  In many places, informal language is used instead of specific, formal definitions of key aspects.  This makes it difficult to fully grasp the work without having read numerous papers on the subject if certain points can be gleaned at all.  I am not suggesting that the work has to extensively review the methods LLF is based on, but the paper could be greatly improved simply by fully elaborating on the construction of the method and taking the time to formally define important concepts.  Some examples include:\n1. 3. “In this paper, we use affine coupling layer (Dinh et al., 2014; 2016) to form normalizing flows. It splits the input to two parts, and force the first part only relate to the second part, so that the Jacobian is a triangular matrix”.   The equation below this elaborates further by introducing a “split” function, but it isn’t clear 1) How the vector “y” is split 2) Whether the particular split is important (can you split them any way?) 3) How this results in a triangular Jacobiam 4) Why a triangular Jacobian is important here.\n2. “Previous methods only look for one possible y within Ω.” This needs a citation.  What methods?\n3. Theorem 1:\n    A.  “tight enough space” is too informal.  What is meant here?\n    B. It is stated that for two different true labels their constrained spaces are “non-overlapped”:\n        I. Does this mean “disjoint”?\n        II. It is stated without direct relationship to the rest of the theorem.  Is this assumption?  Is it for all pairs of labels or just two?  \n        III.  I think the main takeaway from this theorem is to directly compare LLF to a technique that fix labels derived from weak supervision, namely the LLE objective lower bounds the traditional cross-entropy objective, which would be used in the case where weak supervision is used to obtain hard labels for training.  Not only does  I think not only does this point need to be more explicit, but if there is a technique in the literature that does the latter, it should be cited.  Further, the intuitive benefit or insight this theorem shows should be stated more explicitly and clearly.  Later it is written: “learning LLF is analogous to dequantization (Theis et al., 2015; Ho et al., 2019).”.  This is helpful, but elaborating on what dequantization is and why this relationship is important would be better.\n4. It’s not clear to me how Equations 2 and 4 result in the constrained optimization in Equation 5.  Is it that you’re simply replacing the sampling assumption in Equation 2 with a constraint?\n5. “L” is introduced without definition.  Later, through context, I was able to understand it as the number of samples of z ~ p_Z, but it should be made clearer.\n6. Equation 7 and Equation 8: What does the inequality mean between the two vector operands?  Is it element wise?\n7. The problem definition of the unpaired point cloud completion problem (Section 4.3) is rather informal.  What specifically does it mean to “restore” an incomplete point cloud?  How is it measured?\n8. It is unclear from Section 6 what the various hyper parameters for LLF are set to.  Namely, the paper states that that “\\lambda = 10”.  Does this indicate that for all experiments and all different \\lambda hyper parameters across all learning problems (this encapsulates 9 different hyper parameters) are set to 10?  If this is true, this is rather remarkable due to the variety of influences each constraint term has on the proposed objectives (discussed later)\n\n- Practicality. Some of the motivation for key assumptions made in the paper lack argumentation for their practicality.  In my mind this hurts the potential impact of the paper.\n1. Why is it a practical assumption that the error rate for a weak labeler to be known a priori in the weak classification case?  How does one obtain this in practice?\n2. There should be some discussion on the kinds of problems the weakly supervised setting proposed in this work applies to.  The case studies help, but those seem highly specialized and weakly motivated.  For instance, the weakly supervised regression seems somewhat impractical. In general, humans struggle to provide real valued labels such as those suggested int he given example.    I think such single-variate, rule-based assessments make sense for classification problems, where humans can more easily make class assessments based on thresholds (such as “patient has a disease if lab test x_i is over threshold e).   Even in the given example, the underlying problem is better modeled as classification problem (disease versus not-disease), where the labels are noisy assessments of confidence in class membership.\n\n- Other technical questions that need to be addressed:\n1. “Since y is unobserved, directly optimizing the conditional likelihood, i.e., Equation 2, is impossible.”  I don’t understand this.  If p(x) and \\Omega are given, you should be able to compute Equation 2.  Sampling from p(x) for the outer expectation gives you, x.  Sampling from U(\\Omega) for the inner expectation gives you y.  The optimization variable is \\phi.  Why can’t I compute p(y|x,\\phi)?  Is it rather that sampling within the constraint set is impossible or that it’s simply computationally infeasible in practice?  Clarification is needed.\n2. In Equation 8, it seems very important to ensure g(x) is producing valid soft-labels in the simplex and, similarly, in Equation 9 it seems important to ensure the regression labels are in [0.1] (which is an assumption I would like to see explicitly stated in the paper).  The reason I believe this is that other constraint terms could easily push the objective to infinity (namely, the lambda_1 terms).  How is this managed practically?  Would projected gradient ascent be more appropriate to ensure model outputs are on the proper simplex to alleviate this issue?\n3. It is difficult to interpret the performance of the proposed methods as well as the baselines if there is assumed to be no ground truth available for validation and all hyperparameters are default.  How can one be sure that the performance of LLF relative to the baselines cannot be explained simply by assuming bad default parameters for the baselines?  \n\nMinor Points:\n1. “attaining” in the second sentence in the abstract seems like the wrong word.  Perhaps “obtaining”?\n2. “Therefore, this model captures all possible relationships between the input x and output y”.  Seems dubious.  Certainly, this is subject to the class of functions used to learn p(y|x).\n3. Consider using a different function name for “f” (constraint functions) in section 3, as “\\textbf{f}” is used as a the normalizing flow vector function above.  Having them be the same character might make readers try to understand them as directly related, when they are not.\n4. I believe in equation 2 “p(y|x,\\phi)” is the same as equation 1’s p(y|x).  If so, they should be consistent (either they both should be conditioned on \\phi, or neither)\n", "summary_of_the_paper": "This work focuses on the problem of learning predictive models where no ground truth labels are provided.  Instead a set of weak constraints over instances that restrict the space of possible target values are provided.  The proposed method, LLF, is based on the core idea that instead of determining a single true weak label for each instance, a model can be learned by considering the expectation over all possible labels in the feasible set defined by the given constraints.  The authors utilize a conditional normalizing flow model as a generative process for labels and constrain it's output by imposing regularization terms on the training objective that represent different constraints imposed by the weak supervision.  The authors show how their framing of weakly supervised learning can be applied go specific settings in classification, regression, and unpaired point cloud completion.  In their evaluation, the authors compare LLF to a number of weakly supervised baselines and show superior performance on classification and regression tasks, while largely being outperformed by one method in point cloud completion.", "summary_of_the_review": "Overall, the idea of modeling the range of possible labels for instances based on weak supervision instead of simply using a single estimate is a very interesting one.  However, the paper suffers from a number of major issues.  1) The experiments use default hyperparameters for all techniques, thus it is difficult to tell whether the relative performance of methods is indicative of their quality or the quality of the particular default values of their hyperparameters. 2) The normalizing flow component is not well motivated nor is it clearly presented in the context of learning from weak supervision. 3) Some of the proposed used cases lack strong practical motivation.  Many of the major issues I present in the main review seem difficult to remedy in a short period of time, though, if they are the result of a misunderstanding on my end and small edits to the paper could alleviate future readers from similar confusion, I would be willing to change my score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lhmK128uje", "reviewer": "Reviewer_mY6j", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I like the paper and vote to accept. The conversion of the traditional use of normalizing flows is clever and appears to be effective. I really appreciate the variety of the experiments, e.g., a classification problem, a regression problem, and a generative problem. That the generative problem is additionally performed within the constraints of point clouds/sets is very impressive and really helps to illustrate the versatility of the idea.\n\nThat said, I had to read section 4 several times before I understood how the constraints were constructed for a particular problem and am not sure I could construct them on a different problem set. The paper would be considerably more accessible with a simple toy problem with figures/cartoons. This could also be used to help better motivate \"weak signals.\"\n\nI'm also curious why the paper (exclusively?) utilizes affine coupling layers to construct the normalizing flow when more advanced methods exist (e.g., spline coupling).\nThe use of a GAN as part of the constraints alongside the flow-model appears to be a bit of a contradiction but I think helps to illustrate the variability that is allowed within the framework.", "summary_of_the_paper": "The paper proposes to utilize conditional normalizing flows to exploit weak supervisory signals by converting the conventional likelihood maximization problem into a constrained optimization problem. The paper develops three (very) different constrained problems and demonstrates the method on each.", "summary_of_the_review": "The paper proposes a clever twist on conditional normalizing flows and shows the versatility of the idea on several different weakly supervised problems. An illustration would help to make the paper more accessible which would increase it's impact.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hJE_XDAg6Ka", "reviewer": "Reviewer_fLEx", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Section 3 gives a theoretical underpinning to the procedure, claiming that learning LLF is analogous to dequantization, i.e., learning LLF optimizes the likelihood of dequantized true labels. While I can see the similarity in Theorem 1, I fail to see how this motivates the use of LLF as an appropriate method for learning the true label's generating process, which ultimately should be our target in weakly supervised learning. Dequantization is typically considered to simplify the fitting of discrete observations, whereas in this paper we do not have any observations from the true generating process.\n\nSection 4 gives instantiations of LLF to three different scenarios: weakly supervised classification, weakly supervised regression, and unpaired point cloud completion. The formulation for classification is from previous work (Arachie & Huang, 2021), with LLF as the solution instead of min-max optimization (Equation 8). For regression, the paper assumes the existence of M rule-based weak signals (Equation 9), which again enter the optimization problem as constraints (Equation 10). Lastly, the formulation for unpaired point cloud completion is presented, which is set-up differently from the classification and regression cases. It is more similar to a VAE-GAN formulation (Larsen et al., 2016) where the encoder is the conditional normalizing flow. Alternating steps are required during training as per traditional GAN.\n\n### Pros\n\n- Creating a unified framework to solve both (inaccurate) weakly supervised classification and regression is an important research question as collecting data sampled from the true data generating process can be an expensive process and while a lot of research has been done on solving weakly supervised classification, the regression case is under-studied. Having one paradigm can reduce the entry barrier for users who have to solve both problems.\n\n- Formulating weak supervision, where there is no access to true labels, as an inverse flow which sidesteps the need to estimate the labels prior to training the model is an interesting formulation. This work makes use of the same weak supervision setting as ALL (Arachie & Huang, 2021), but the training is simpler as it does not require min-max optimization.\n\n### Cons\n\n- In unpaired point cloud completion, the complete clouds, which are true observations from the true data generating process, are considered as weak supervision, and the partial clouds are the covariates that we seek to map to being complete. In typical weak supervision scenarios, weak signals are generated in a paired manner to their covariates. Even though I can see how Equation 12 can be cast as Equation 6, I fail to see how unpaired point cloud completion is a weakly supervised learning problem. I was wondering if we can reformulate this problem such that the labels that are being used are truly weak, i.e., not coming from the true conditional p(y|x)? Moreover, just like the typical weakly supervised learning set-up, we should not tune on true validation sets.\n\n- The paper states \"However, these methods ignore this uncertainty between x and y\", and \"LLF uses a conditional flow to define the conditional distribution p(y|x), so that can model the uncertainty between input x and all possible y\". However, the experiments only measure test set accuracies and not goodness-of-uncertainty metrics like NLL or calibration. If we want to make this claim, can we show that LLF performs good uncertainty quantification?\nWe are comparing approaches that only use weak signals against LLF (and ALL) which require the error rate bounds of the weak signals, an extra piece of information. In this paper, it is not clear to me how the error rate bounds are computed — in order to keep things fair, error rate bounds should be determined solely based on the weak signals. Is it possible to clarify this?\n\n- Inference requires sampling and summarizing which can be slow and limits the use of LLF as a final predictor. On the other hand, two-stage approaches like Data Programming (DP) uses the label model to output probabilistic labels, which is then used to train a downstream neural network. As it is right now, if we compare their inference speeds, DP seems to be better. Can we also use LLF in a two-stage manner like in DP? How does it perform then?\n\n### Additional Points for Discussion:\n- Can you handle the case where you know of the dependence/correlational structure between the weak signals?\n\n### Additional Experiments:\nSee Cons\n", "summary_of_the_paper": "This article proposes **label learning flows (LLF)**, a general framework for weakly supervised learning tasks. The modelling framework is primarily driven by 1) conditional normalizing flows (Trippe & Turner, 2018), which is used as a flexible conditional likelihood model, with an affine coupling flow layer, and 2) constrained optimization formulation of maximum likelihood, where weak signals/error rate bounds enter as constraints (Arachie & Huang, 2021). The main practical formulation uses Lagrange multiplier to define an unconstrained stochastic loss function that requires Monte-Carlo samples during training (Equation 6). Ultimately, they obtain a trained inverse flow, which conditional on a covariate, can produce prediction samples.", "summary_of_the_review": "Conceptually, constrained flow formulation of LLF is interesting and novel. However, its numerical results are not too convincing and there are some questions related to the practicality of this method.  I also do not think that unpaired point cloud completion should be an example of weakly supervised learning.  As it stands right now, I do not think that this paper meets the bar of \"powerful and effective tool for weakly supervised learning problems\". The paper can be much improved (especially for weak supervision practitioners audience) by addressing the questions under Cons and additional points above.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "DzUmWY4IesV", "reviewer": "Reviewer_ii99", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n1. This paper presents a new method based on normalising flow for weakly supervised learning problems. It seems to be novel in this specific setting.\n\n2. The paper has a comprehensive set of experiments in a variety of applications.\n\n3. Overall, the paper is well written and easy to follow.\n\nWeakness:\n\n1. It is a bit unclear why normalising flow is used. Specifically, in the weakly supervised learning setting of the paper, $y$ is unobserved but we know a set of constraints of $y$, i.e., $C$. It is unclear why directly parameterising $\\log{p(y|x)}$ and optimising with $C$ is impossible and why normalising flow makes it possible. I feel that the context around Eq (4) needs more explanation.\n\n2. It seems that there are not many existing methods that exactly follow the setting of the paper. Therefore, it's better to have a more comprehensive comparison with the latest related methods, such as PGMV and its variants in [1] and AMCL and its variants in [2].\n\n[1] Mazzetto, Alessio, et al. \"Semi-supervised aggregation of dependent weak supervision sources with performance guarantees.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2021.\n\n[2] Mazzetto, Alessio, et al. \"Adversarial Multi-Class Learning under Weak Supervision with Performance Guarantees.\" International Conference on Machine Learning. PMLR, 2021.", "summary_of_the_paper": "This paper proposes a new method for weakly supervised learning where the true label of a data sample is unknown but there are known constraints on the label. The proposed method is based on conditional normalising flows, where the inverse flow is used to model the conditional label distribution and optimised with the label constraints. The proposed method is used in three applications including classification, regression, and point cloud completion. The proposed method is shown to have better performance in these applications in the comparison with several baseline methods.\n\n", "summary_of_the_review": "The idea of the paper is interesting and seems to be effective. But more explanations and comparisons are needed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Dv7Q2ux_mln", "reviewer": "Reviewer_tWTj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper has some positive aspects as follows:\n1. The proposed method is novel. This method proposes to optimize the conditional likelihoods of all possible labeling of the data within a constrained space defined by weak signals, which can alleviate the issue that the existing methods ignore the uncertainty between x and y.\n2.  The experiment is abundant. The proposed method is validated in three different tasks, including weakly supervised classification, weakly supervised regression and unpaired point cloud completion, which shows its effectiveness.\n3.  The paper is well written, the arguments are clear, and the methodology is well presented.\n\n The authors need to deal with some issues as follows:\n\n4.  In Table 1,  for OBS Network, the performance of LLF is poor than other methods, while on other datasets can obtain the obvious improvement.  the authors should provide an explanation for this result.\n5. For the results in Table 4, it is better, if the authors can give a comprehensive evaluation protocol, which can mix MMD, TMD and UHD together, thus it can obviously highlight the superiority of LLF in unpaired point cloud completion.\n", "summary_of_the_paper": "Different from many existing weakly supervised learning methods, which learn a deterministic function that estimates labels given the input data and weak signals,  this paper proposes label learning flows (LLF), a general framework for weakly supervised learning problems, and it is a generative model based on normalizing flows. The main idea of LLF is to optimize the conditional likelihoods of all possible labeling of the data within a constrained space defined by weak signals.  The proposed method is applied to three weakly supervised learning problems. Experimental results show that this method outperforms many state-of-the-art alternatives.", "summary_of_the_review": "The proposed method is novel and the experiment is sufficient. More experimental analysis can further improve the quality of this paper.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new approach for weakly supervised learning, based on conditional normalizing flows. Reviewers generally found the paper to have an interesting, novel proposal with empirical promise. However, some concerns were raised: to name a few,

(1) _Clarity._ Several reviewers found portions of the technical content hard to follow, e.g., the description of constraints in Sec 4.

(2) _Scalability compared to data programming._ One reviewer was unsure of how the present approach compares in terms of inference time and/or accuracy to a two-stage data programming approach.

(3) _Infeasibility of sampling from Equation 2._ One reviewer suggested the paper discuss and compare to a simpler baseline, which is to perform rejection sampling from the constraint set.

(4) _Suitability of point cloud problem._ One reviewer was unsure of whether the point cloud problem, considered as an experimental setting in this paper, is reflective of weakly supervised learning.

(5) _Practicality of knowing weak labeler error rates._ The paper assumes knowledge of the weak labeler error rates in constructing constraints. Some reviewers raised concerns on the practical viability of this assumption.

For point (2), the relevant reviewer was not convinced following the discussion. The suggestion is to treat LLF as a label model, which serves as input to a non-MC predictor. The question then is what the predictive performance of this combined approach looks like, as opposed to the LLF's themselves.

For point (3), the response clarified that the number of constraints might make rejection sampling infeasible. This appears to be true, but it is suggested that the paper at a minimum discuss this, and ideally also clarify claims about the general-purpose need for the proposed approach (since in some cases one might be able to do rejection sampling).

For point (4), the discussion was somewhat inconclusive. It is suggested that the authors explicitly discuss some of the points brought up in the response.

For point (5), while the assumption not wholly uncommon in the literature, it would be better for the authors to perform some sensitivity analysis against misspecification of the error rates.

Overall, the paper has some interesting ideas that are well worth exploring. The present execution appears to have some scope for improvement, with the reviews providing a range of suggestions of areas of the paper that could be made clearer or strengthened. The paper would be best served by incorporating these comments and undergoing a fresh review.
Decision: Reject