Multiple-sample Neural Image Compression
Anonymous Author(s)
AffiliationAddress email
Abstract
This paper considers the problem of lossy neural image compression (NIC).
Currentstate-of-the-art (sota) methods adopt uniform posterior to approximate quantization noise, and single-sample pathwise estimator to approximate the gradientof evidence lower bound (ELBO).
In this paper, we propose to train NIC withmultiple-sample importance weighted autoencoder (IWAE) target, which is tighterthan ELBO and converges to log likelihood as sample size increases.
First, weidentify that the uniform posterior of NIC has special properties, which affect thevariance and bias of pathwise and score function estimators of the IWAE target.
Moreover, we provide insights on a commonly adopted trick in NIC from gradientvariance perspective.
Based on those analysis, we further propose multiple-sample
NIC (MS-NIC), an enhanced IWAE target for NIC.
Experimental results demonstrate that it improves sota NIC methods.
Our MS-NIC is plug-and-play, and canbe easily extended to neural video compression.
Introduction
Latent variable-based lossy neural image compression (NIC) has witnessed significant success.
Themajority of NIC follows the framework proposed by Ballé et al.
[2017]: For encoding, the originalimage x is transformed into y by the encoder.
Then y is scalar-quantized into integer ¯ y , estimatedwith an entropy model p ( ¯ y ) and coded.
For decoding, ¯ y is transformed back by the decoder to obtainreconstructed ¯ x .
The optimization target of NIC is R-D cost: R + λD .
R denotes the bitrate of ¯ y , Ddenotes the distortion between x and ¯ x , and λ denotes the hyper-parameter controlling their trade-off.
During training, the quantization ¯ y =
⌊ y ⌉ is relaxed with ˜ y
= y + ϵ to simulate the quantizationnoise.
And ϵ is fully factorized uniform noise ϵ ∼ p
( ϵ ) =
(cid:81) U ( − 12 , + 12 ) .
Ballé et al.
[2017] further recognises that such training framework is closely related to variationalinference.
Indeed, the above process can be formulated as a graphic model x ← ˜ y .
During encoding,x is transformed into variational parameter y by inference model (encoder), and ˜ y is sampled fromvariational posterior q ( ˜ y | x ) , which is a unit unifrom distribution centered in y .
The prior likelihoodp ( ˜ y ) is computed, and ˜ y is transformed back by the generative model (decoder) to compute thelikelihood p ( x | ˜ y ) .
Under such formulation, the prior is connected to the bitrate, the likelihood isconnected to the distortion, and the posterior likelihood is connected to the bits-back bitrate (See
Appendix), which is 0 in NIC.
Finally, the evidence lower bound (ELBO) is the negative R + λDtarget (Eq. 1).
Denote the transform function ˜
y
( ϵ ; ϕ ) =
y + ϵ , and sampling ˜
y ∼ q ( ˜ y
| x ) is equivalentto transforming ϵ through ˜ y
( ϵ ; ϕ ) .
Then the gradient of ELBO is estimated via pathwise estimatorwith single-sample Monte Carlo (Eq. 2).
This is the same as SGVB-1 [Kingma and Welling, 2013].
[Equation] [Equation]
[Equation]
Ballé et al.
[2018] further extends this framework into a two-level hierarchical structure, withgraphic model x ← ˜ y ← ˜ z .
The variational posterior is fully factorized uniform distribution
U ( y − 12 , y + 12 ) U ( z − 12 , z + 12 )
To simulate the quantization noise.
And y , z denote outputs oftheir inference networks.
[Equation]
The majority of later NIC follows this hierarchical latent framework [Minnen et al., 2018, Chenget al., 2020].
Some focus on more expressive network architectures [Zhu et al., 2021, Xie et al.
, 2021],some stress better context models [Minnen and Singh, 2020,
He et al., 2021, Guo et al., 2021a], andsome emphasize semi-amortization inference [Yang et al., 2020].
However, there is little research onmultiple-sample methods, or other techniques for a tighter ELBO.
On the other hand, IWAE [Burda et al., 2016] has been successful in density estimation.
Specifically,
IWAE considers a multiple-sample lowerbound L k (Eq. 4), which is tighter than its single-samplecounterpart.
The benefit of such bound is that the implicit distribution defined by IWAE approachestrue posterior as k increases [Cremer et al., 2017].
This suggests that its variational posterior is lesslikely to collapse to a single mode of true posterior, and the learned representation is richer.
Thegradient of L k is computed via pathwise estimator.
Denote the exponential ELBO sample as w i , itsreparameterization as w ( ϵ i ; ϕ ) , and its weight ˜ w i = w i (cid:80) w j .
Then ∇ ϕ L k has the form of importanceweighted sum (Eq. 5).
[Equation]
In this paper, we consider the problem of training NIC with multiple-sample IWAE target (Eq.
4),which allows us to learn a richer latent space.
First, we recognise that NIC’s factorized uniformvariational posterior has impacts on variance and bias properties of gradient estimators.
Specifically,we find NIC’s pathwise gradient estimator equivalent to an improved STL estimator [Roeder et al.
,2017], which is unbiased even for the IWAE target.
However, NIC’s IWAE-DReG estimator [Tuckeret al., 2018] has extra bias, which causes performance decay.
Moreover, we provide insights on acommonly adopted but little explained trick of training NIC from gradient variance perspective.
Basedon those analysis and observations, we further propose MS-NIC, a novel improvement of multiplesample IWAE target for NIC.
Experimental results show that it improves sota NIC methods [Balléet al., 2018, Cheng et al., 2020] and learns richer latent representation.
Our method is plug-and-play,and can be extended into neural video compression.
To wrap up, our contributions are as follows:
• We provide insights on the impact of the uniform variational posterior upon gradientestimators, and a commonly adopted but little discussed trick of NIC training from gradientvariance perspective.
• We propose multiple-sample neural image compression (MS-NIC).
It is a novel enhancementof hierarchical IWAE [Burda et al., 2016] for neural image compression.
To the best of
our knowledge, we are the first to consider a tighter ELBO for training neural imagecompression.
• We demonstrate the efficiency of MS-NIC through experimental results on sota NIC methods.
Our method is plug-and-play for neural image compression and can be easily applied toneural video compression.
Gradient Estimation for Neural Image Compression
The common NIC framework (Eq. 1, Eq 3) adopts fully factorized uniform distribution q
( ˜ y , ˜
z | x ) = 74 (cid:81)
U ( y i − 12 , y i + 12 ) (cid:81) U ( z j − 12 , z j + 12 ) to simulate the quantization noise.
Such formulation hasthe following special properties:
• Property I: q ( ˜ z | ˜ y ) and q ( ˜ y
| x ) ’s support depends on the parameter.• Property II: log q ( ˜ z | ˜ y )
=
log q
( ˜ y
| x ) = 0 on their support.
The impacts of these two properties are frequently neglected in previous works, which does notinfluence the results for single-sample pathwise gradient estimators (a.k.a. reparameterization trick in
Kingma and Welling [2013]).
In this section, we discuss the impacts of these two properties uponthe variance and biasness of gradient estimators.
Our analysis is based on single level latent (Eq.
1)instead of hierarchical latent (Eq. 3) to simplify notations.
2.1 Impact on Pathwise Gradient Estimators
First, let’s consider the single-sample case.
We can expand the pathwise gradient of ELBO in Eq.
2into Eq. 6.
As indicated in the equation, ϕ contributes to L in two ways.
The first way is through thereparametrized ˜ y
( ϵ ; ϕ ) (pathwise term), and the other way is through the parameter of log q ( ˜ y | x )(parameter score term).
Generally, the parameter score term has higher variance than the pathwiseterm.
The STL [Roeder et al., 2017] reduces the gradient by dropping the score.
It is unbiased sincethe dropped
term
’s expectation E q ( ˜ y
| x ) [ ∇ ϕ log q ϕ ( ˜ y | x )] is 0 .
[Equation]
Now let’s consider the STL estimator of multiple-sample IWAE bound (Eq. 4).
As shown in Tuckeret al.
[2018], the STL estimation of IWAE bound gradient is biased.
To reveal the reason, considerexpanding the gradient Eq.
5 into partial derivatives as we expand Eq.
2 into Eq.
Unlike singlesample case, the dropped parameter score term E p ( ϵ 1: k )
[ (cid:80) ˜
w i ( −∇ ϕ log q ϕ ( ˜ y | x ))] is no longer
dueto the importance weight ˜ w i .
This means that STL loses its unbiasness in general IWAE cases.
Regarding NIC, however, the direct pathwise gradient for IWAE bound is automatically an unbiased
STL estimator.
Property II means that variational posterior has constant entropy, which further meansthat the parameter score gradient is 0 .
So, NIC’s pathwise gradient of IWAE bound is equvailent toan extended, unbiased STL estimator.
2.2 Impact on Score Function Gradient Estimators
In previous section, we show the bless of NIC’s special properties on pathwise gradient estimators.
In this section, we show their curse on score function gradient estimators.
Sepcifically, Property Iimplies that q ( ˜ z | ˜ y ) and q ( ˜ y | ˜ x ) are not absolute continuous, and hence the score function gradientestimators of those distributions are biased.
For example, consider a univariate random variable x ∼ p θ ( x )
=
U ( θ − 12 , θ + 12 ) .
Our task isto estimate the gradient of a differentiable function f ( x ) .
And consider the θ -independent randomvariable ϵ ∼ p ( ϵ )
=
U ( − 12 , + 12 ) , the transform x ( ϵ ; θ ) = θ + ϵ .
Under such conditions, the Monte
Carlo estimated pathwise gradient and score function gradient are:
[Equation] [Equation]
Eq.
7 does not equal to Eq. 8, and Eq.8 is wrong.
The score function gradient is only unbiasedwhen the distribution satisfies absolute continuity [Mohamed et al., 2020].
This reflects that underthe formulation of NIC, the equivalence between the score function gradient (a.k.a.
REINFORCE [Table]
[Williams, 1992]) and pathwise gradient (a.k.a reparameterization trick in [Kingma and Welling,2013]) no longer holds.
Such equivalence is the cornerstone of many gradient estimators, and IWAE-DReG [Tucker et al.
,2018] is one of them.
IWAE-DReG is a popular gradient estimator for IWAE target (Eq. 4) asit resolves the vanish of inference network gradient SNR (signal to noise ratio).
However, thecorrectness of IWAE-DReG depends on the equivalence between the score function gradient andpathwise gradient, which does not hold for NIC.
As we show empirically in Tab. 1, blindly adopting
IWAE-DReG estimator for multiple-sample NIC brings evident performance decay.
Other than IWAE-DReG, many other graident estimators such as NVIL [Mnih and Gregor, 2014],
VIMCO [Mnih and Rezende, 2016] and GDReG [Bauer and Mnih, 2021] do not apply to NIC.
Theyeither bring some extra bias or are totally wrong.
2.3 The direct-y Trick in Training NIC
In NIC, we feed deterministic parameter y into z inference model instead of noisy samples ˜ y .
Thisimplies that ˜ z is sampled from q ( ˜ z | y ) instead of q ( ˜ z | ˜ y ) .
This trick is initially adopted in Balléet al.
[2018] and followed by most of the subsequent works.
However, it is little discussed.
In thispaper, we refer it to direct-y trick.
Yang et al.
[2020] observes that feeding ˜ y instead of y causessevere performance decay.
We confirm this result in Tab. 2.
Thus, direct-y trick is essential to trainhierarchical NIC.
[Table]
[Table]
One explanation is to view q ( ˜ z | y ) as q ( ˜ z | x ) , and q ( ˜ y , ˜
z | ˜ x ) factorized as q ( ˜ y | x ) q ( ˜ z | x ) (See Fig.1(a)-(c)).
A similar trick of feeding mean parameter can be traced back to the Helmholtz machine[Dayan et al., 1995].
However, this provides a rationale why direct-y is fine to be adopted but does notexplain why samping ˜ z from q ( ˜ z | ˜ y ) fails.
We provide an alternative explanation from the gradientvariance perspective.
Specifically, q ( ˜ z | ˜ y ) has two stochastic arguments that could cause high variancein the gradient of z inference model, and make its convergence difficult.
To verify this, we follow
Rainforth et al.
[2018] to compare the gradient SNR, which is the absolute value of the empiricalmean divided by standard deviation.
We trace the gradient SNR during different training stages asmodel converges (See Sec.
5.1 for detailed setups).
As demonstrated in Tab. 3, the gradient SNR of z inference model of standard 2-level VAE (withoutdirect y ) is indeed significantly lower than Ballé et al.
[2018] (with direct y ) during all 3 stage oftraining.
This result reveals that the z inference model is more difficult to train without direct-y .
Andsuch difficulty could be the source of the failure of NIC without direct-y trick.
Multiple-sample Neural Image Compression
In this section, we consider the multiple-sample approach based on the 2-level hierarchical frameworkby Ballé et al.
[2018], which is the de facto NIC architecture adopted by many sota methods.
Tosimplify notations, log q ( ˜ z | ˜ y ) and log q ( ˜ y | x ) in ELBO are omitted as they are 0 .
The inference models show how we sample from variational posterior duing training.
n is the number of data points in dataset, k, l is the sample size of multiple-sample approaches.
The generative model of (b), (c) is (a).
The generative model of (d)-(f) is shown in Appendix.
For clarity, we omit the parameters.
First, let’s consider directly applying 2-level IWAE to NIC without direct-y trick (See Fig.
1 (d)).
Regarding a k sample IWAE, we first compute parameter y of q ( ˜ y | x ) and sample ˜ y 1: k from it.
Then,we compute parameter z 1: k of q ( ˜ z 1: k | ˜ y 1: k ) and samples ˜
z 1: k from it.
Afterward, ˜ y 1: k and ˜ z 1: k arefed into the generative model and compute w 1: k .
Finally, we follow Eq 5 to compute the gradient andupdate parameters.
In fact, this is the standard 2-level IWAE in the original IWAE paper.
However, the vanilla 2-level IWAE becomes a problem for NIC with direct-y trick.
Concerning a ksample IWAE, we sample ˜ y 1: k from q ( ˜ y | x ) .
Due to the direct-y trick, we feed y instead of ˜ y 1: k intoz inference network, and our q ( ˜ z | y ) has only one parameter z other than k parameter z 1: k .
If wefollow the 2-level IWAE approach, only one sample ˜ z is obtained, and w 1: k can not be computed.
One method is to limit the multiple-sample part to ˜ y related term only and optimize other parts viasingle-sample SGVB-1, which produces our MS-NIC-MIX (See Fig 1 (e)).
Another method is tosample another l samples of ˜ z
j from q ( ˜ z | y ) and nest it with MS-NIC-MIX, which generates our
MS-NIC-DMS (See Fig 1 (f)).
3.1 MS-NIC-MIX: Multiple-sample NIC with Mixture
One way to optimize multiple-sample IWAE target of NIC with direct-y trick is to sample ˜
y k timesto obtain ˜ y 1: k and ˜ z only 1 time.
Then we perform k sample log mean of p ( x | ˜
y i ) p
( ˜ y i | ˜ z ) to obtain a multiple-sample estimated log p ( x | ˜ z ) , add it with single-sample log p ( ˜ z ) .
This brings a L MIXk withthe form of a mixture of 1-level VAE and 1-level IWAE ELBO:
[Equation]
Moreover, L MIXk is a reasonably preferable target over ELBO as it satisfies the following properties(See Appendix for proof):
[Equation]
Although L MIXk does not converge to true log p ( x ) as k grows, it is still a lower bound of log p ( x )and tighter than ELBO (as L MIX= ELBO).
Its gradient can be computed via pathwise estimator.
Denote the per-sample integrand p ( x | ˜ y i ) p
( ˜ y i | ˜ z ) as w MIXi , and its relative weight as ˜ w MIXi , thenthe gradient ∇ ϕ L MIXk can be estimated as Eq. [Equation]
[Equation]
Another way to understand MS-NIC-MIX is to view the y inference/generative model as a single level
IWAE, and the z inference/generative model as a large prior of ˜ y which is optimized via SGVB-1.
This perspective is often taken by works in NIC context model [Minnen et al., 2018,
He et al., 2021],as the context model of NIC is often limited to ˜ y .
3.2 MS-NIC-DMS: Multiple-sample NIC with Double Multiple Sampling
An intuitive improvement over MS-NIC-MIX is to add another round of multiple-sample over ˜ z .
Specifically, we sample ˜
z l times, nest it with L MIXk to obtain L DMSk,l
[Equation]
[Equation]
And we name it MS-NIC-DMS as it adopts multiple sampling twice.
Moreover, L DMSk,l is a reasonablybetter target for optimizaion over ELBO and L MIXk , as it satisfies the following properties (See proofin Appendix):
1. log p ( x ) ≥ L DMSk,l
L DMSk,l ≥ L DMSm,n for k ≥ m, l ≥ n
L DMSk,l ≥ L MIXk
L DMSk,l → log p ( x ) as k, l → ∞ , under the assumption that log p ( x | ˜ y i ) p ( ˜ y
i | ˜
z j ) q ( ˜ y i | x )
and
log p ( x | ˜ z j ) p ( ˜ z j ) q ( ˜ z
j | x
) are asymptotically
uniformly integrable
In other words, the target L DMSk,l is a lowerbound of log p ( x ) , converging to log p ( x ) as k, l → ∞ ,tighter than L MIXk and tighter than ELBO (as L DMS 1 ,= ELBO).
However, its Monte Carlo estimationis biased due to the nested transformation and expectation.
Empirically, we find that directly adopting biased pathwise estimator works fine.
And its gradient can be estimated by pathwise estimator similarto original IWAE target (See Eq. 5).
[Equation]
Related Work
4.1 Lossy Neural Image and Video Compression
Ballé et al.
[2017] and Ballé et al.
[2018] formulate lossy neural image compression as a variationalinference problem, by interpreting the additive uniform noise (AUN) relaxed scalar quantizationas a factorized uniform variational posterior.
After that, the majority of sota lossy neural imagecompression methods adopt this formulation [Minnen et al., 2018, Minnen and Singh, 2020, Chenget al., 2020, Guo et al., 2021a, Gao et al., 2021, He et al., 2022].
And Yang et al.
[2020], Guo et al.[2021b] also require a AUN trained NIC as base.
Moreover, the majority of neural video compressionalso adopts this formulation [Lu et al., 2019, 2020, Agustsson et al., 2020, Hu et al., 2021, Li et al.
,2021], implying that MS-NIC can be extended to video compression without much pain.
Other approaches to train NIC include random rounding [Toderici et al., 2015, 2017] and straightthrough estimator (STE)
[Theis et al., 2017].
Another promising approach is the VQ-VAE [Van
Den Oord et al., 2017].
By the submission of this manuscript, one unarchived work [Zhu et al., 2022]has shown the potential of VQ-VAE in practical NIC.
Our MS-NIC does not apply to the approachesmentioned in this paragraph, as the formulation of variational posterior is different.
4.2 Tighter Lower Bound for VAE
IWAE [Burda et al., 2016] stirs up the discussion of adopting tighter lower bound for training VAEs.
However, at the first glance it is not straightforward why it might works.
Cremer et al.
[2018]decomposes the inference suboptimality of VAE into two parts: 1)
The limited expressivenessof interence model.
The gap between ELBO and log likelihood.
However, this gap refers toinference not training.
The original IWAE paper empirically shows that IWAE can learn a richerlatent representation.
And Cremer et al.
[2017] shows that the IWAE target converges to ELBO underthe expectation of true posterior.
And thus the posterior collapse is avoided.
From the information preference [Chen et al., 2017] perspective, VAE prefers to distribute informationin generative distribution than autoencoding information in the latent.
This preference formulatesanother view of posterior collapse.
And it stems from the gap between ELBO and true log likelihood.
There are various approaches alleviating it, including soft free bits [Theis et al., 2017] and KLannealing [Serban et al., 2017].
In our opinion, IWAE also belongs to those methods, and it isasymptotically optimal.
However, we have not found many works comparing IWAE with thosemethods.
Moreover, those approaches are rarely adopted in NIC community.
Many follow-ups of IWAE stress gradient variance reduction [Roeder et al., 2017, Tucker et al.
,2018, Rainforth et al., 2018], discrete latent [Mnih and Rezende, 2016] and debiasing IWAE target[Nowozin, 2018].
Although the idea of tighter low bound training has been applied to the field ofneural joint source channel coding [Choi et al., 2018, Song et al., 2020], to the best of our knowledge,no work in NIC consider it yet.
4.3 Importance Weighted Neural Compression
Theis and Ho [2021] considers the similar topic of importance weighted NIC.
However, it doesnot consider training of NIC.
Instead, it focuses on achieving IWAE target with an entropy codingtechnique named softmin , just like BB-ANS
[Townsend et al., 2018] achieving ELBO.
It is alluring to apply softmin to MS-NIC, as it closes the multiple-sample training and inference gap.
However, itrequires large number of samples (e.g. 4096 ) to achieve slight improvement for 64 × 64 images.
Thepotential sample size required for practical NIC is forbidding.
Experimental Results
5.1 Experimental Settings
Following He et al.
[2022], we train all the models on the largest 8000 images of ImageNet [Denget al., 2009], followed by a downsampling according to Ballé et al.
And we use Kodak [Kodak,1993] for evaluation.
For the experiments based on Ballé et al.
[2018] (include Tab. 1, Tab. 2), wefollows the setting of the original paper except for the selection of λ s, For the selection of λ s, we setλ ∈ { 0 . 0016 , 0 . 0032 , 0 . 0075 , 0 . 015 , 0 .
08 } as suggested in Cheng et al.
And forthe experiments based on Cheng et al.
[2020], we follows the setting of original paper.
More detailedexperimental settings can be found in Appendix.
And when comparing the R-D performance of models trained on multiple λ s, we use Bjontegaardmetric (BD-Metric) and Bjontegaard bitrate (BD-BR)
[Bjontegaard, 2001], which is widely appliedwhen comparing codecs.
More detailed experimental results can be found in Appendix.
[Table]
IWAE [Burda et al., 2016]
[Table]
[Table]
5.2 R-D Performance
We evaluate the performance of MS-NIC-MIX and MS-NIC-DMS based on sota NIC methods [Balléet al., 2018, Cheng et al., 2020].
Empirically, we find that MS-NIC-MIX works best with sample size [Table] and MS-NIC-DMS with sample size 16.
The experimental results on sample size selection canbe found in Appendix.
Without special mention, we set the sample size of MS-NIC-MIX to 8 and
MS-NIC-DMS to 16.
For Ballé et al.
[2018], MS-NIC-MIX saves around 4% of bitrate compared with single-samplebaseline (See Tab. 4).
And MS-NIC-DMS saves around 5% of bitrate.
On the other hand, the original
IWAE suffers performance decay as it is not compatible with direct-y trick.
For Cheng et al.
[2020], we find that both MS-NIC-MIX and NS-NIC-DMS suppress baseline in PSNR.
However, it is notas evident as Ballé et al.
Moreover, the MS-SSIM is slightly lower than the baseline.
Thisis probably due to the auto-regressive context model.
Besides, the original IWAE without direct-ytrick suffers from severe performance decay in both cases.
The BD metric of IWAE on Cheng et
al.[2020] can not be computed as its R-D is not monotonous increasing, we refer interested readers to
Appendix for details.
5.3 Latent Space Representation of MS-NIC
To better understand the latent learned by MS-NIC, we evaluate the variance and coefficient ofvariation (Cov) of per-dimension latent distribution mean parameter y
( i ) ,
z ( i ) , with regard to inputdistribution p ( x ) .
As we are also interested in the discrete representation, we provide statistics ofrounded mean ¯ y ( i ) , ¯ z ( i ) .
These metrics show how much do latents vary when input changes, and alarge variation in latents means that there are useful information encoded.
A really small variationindicates that the latent is "dead" in that dimension.
As shown in Tab. 6, the latent of multiple-sample approaches has higher variance than those ofsingle-sample approach.
Moreover, the Cov ( y ) of multiple-sample approaches is around 4 − 5 timeshigher than single-sample approach.
Although the Cov ( z ) of multiple-sample approaches is around2 times lower, the main contributor of image reconstruction is y , and z only serves to predict y ’sdistribution.
Similar trend can be concluded from quantized latents ¯ y , ¯ z .
From the variance and Covperspective, the latent learned by MS-NIC is richer than single-sample approach.
It is also noteworthythat although the variance and Cov of y , ¯ y of MS-NIC is significantly higher than single-sampleapproach, the bpp only varies slightly.
[Table]
[Table]
Conclusion
In this paper we propose MS-NIC, a multiple-sample importance weighted target for training NIC.
Itimproves sota NIC methods and learns richer latent representation.
A known limitation is that its R-Dperformance with regard to MS-SSIM is limited when applied to models with spatial context models(e.g.
Cheng et al.
Further work could consider improving this and extend it into neural videocompression.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data are
proprietary.• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeAppendix(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] In Appendix3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No](b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes](c)
Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[No] It is really expensive to train a NIC model.
The error baris rarely reported out of toy size model and dataset.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[Table]
(b) Did you mention the license of the assets?
[N/A] [Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]