An Invisible Issue of Task Underspeciﬁcation in Deep Reinforcement Learning
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Performance evaluations of Deep Reinforcement Learning (DRL) algorithms are anintegral part of the scientiﬁc progress of the ﬁeld.
However, standard performanceevaluation practices in evaluating algorithmic generalization within a task of DRLmethods can be unreliable and misleading if not careful.
An important sourceof possible error lies in the reliance of the reported outcomes on often arbitrarilyselected point Markov decision processes (point MDPs), stemming from taskunderspeciﬁcation.
A large class of DRL tasks, particularly in real-world decisionproblems, induce a family of MDPs, which—perhaps confusingly—each has thesame high-level problem deﬁnition.
As a demonstrative example, consider that aclassic pendulum control task could be represented by a family of possible MDPs,each with a different pendulum mass, but is typically represented as a single MDP.
This article argues that for reliable downstream decision-making, performanceevaluations on a task in DRL should be carried out over a family of MDPs ratherthan a point MDP, which may be subject to bias.
This article ﬁrst illustrates thepitfalls of point MDP based evaluations through benchmark DRL control tasks anda real-world case study in trafﬁc signal control.
Then, signiﬁcant inconsistenciesbetween conclusions derived from point MDP based evaluations and MDP familybased evaluations are presented.
Subsequently, to overcome the prohibitive costof training DRL models on entire families of MDPs, a series of recommendationsis provided to perform accurate yet efﬁcient performance evaluations under acomputational budget.
This work contributes to bolstering the empirical rigor ofreinforcement learning, especially as the outcomes of DRL trickle into downstreamdecision-making in real-world contexts.
Introduction
Deep reinforcement learning (DRL) research has progressed rapidly in recent years, achieving superhuman level performance in many applications.
At the core of DRL research lies the need to performrigorous quantitative performance evaluations (PE).
PE not only disclose the strengths and limitationsof a method but also impact the progress of the ﬁeld at large.
PE of DRL methods are typicallycarried out by solving the corresponding Markov Decision Process (MDP) and building performanceproﬁles of the method according to some performance metrics.
However, we ﬁnd that if not careful,typical practice can lead to unreliable PE.
The real-world induces many complexities for control tasks and one major complexity is the existenceof multiple task instances of the same problem. For example, to reliably claim a DRL methodsolves trafﬁc signal control problem, one needs to show that the proposed method sufﬁcientlyworks for all or majority of the signalized intersection instances [27]. However, we ﬁnd that manystudies evaluate DRL methods for trafﬁc signal control on a few arbitrary selected intersection instances [5, 38, 3, 20, 39].
Similar issues can also be seen in other applications [37, 41, 43]. As DRL trickles into real-world implementation and critical use cases, such practices can have profoundimplications, ranging from the opportunity cost of employing a lower quality method to hurtingconﬁdence in the ﬁeld. To formally state, consider the classic pendulum control task, which is typically represented asa single MDP.
At the same time, the task is underspeciﬁed and could easily be represented by afamily of MDPs, each with a different pendulum mass or length (speciﬁed using some environmentparameters denote by  , which induces different MDPs).
Standard performance evaluation practicefor assessing algorithmic generalization of DRL methods within the pendulum task is to train andevaluate DRL methods on a point MDP (or a subset of MDPs) chosen from an implicit family of
MDPs (e.g., selecting a speciﬁc  in the example).
We refer to such evaluation approaches as point MDP based evaluations 1 .
Unfortunately, such point MDPs often appear to be selected arbitrarily or ina way that inadvertently simpliﬁes the task.
It is well known that the performance of a DRL methodis sensitive to the underlying MDP [30, 18, 2].
In this work, we hypothesize that the coupling of anarbitrary selection of point MDPs for evaluation and the general sensitivity of DRL methodscan result in substantial error in the true performance of DRL methods over the implicit MDPfamily of a task.
This, in turn, harms empirical rigor in DRL.
We refer this phenomenon as taskunderspeciﬁcation problem in DRL performance evaluations.
While evaluating DRL methods on a sufﬁciently large and carefully chosen family of MDPs canalleviate the issues and enable reliable PE, it often can be prohibitively costly to execute computationally.
Many of the problems can have an excessive amount of possible MDPs that one needs toevaluate.
Moreover, multiple runs per single MDP are needed to achieve statistical conﬁdence.
Buteven training a single model for multiple MDPs can be prohibitively expensive, let alone multipleruns per MDP.
In this work, we take a deep dive into this matter.
In particular, we reveal potential pitfalls of point MDP based PE when DRL methods are evaluated for algorithmic generalization within a task. We useclassic and modern control tasks frequently used in the DRL literature for this purpose.
Furthermore,we conduct a case study on a benchmark DRL trafﬁc signal control suite to highlight the limitations ofthese pitfalls in real-world applications.
Afterward, we provide a series of recommendations to avoiddrawbacks of point MDP based evaluations by evaluating on a family of MDPs while subjected to acomputational budget.
Finally, we re-evaluate the trafﬁc signal control benchmark accommodatingthe recommendations and show discrepancies in what has been reported previously.
This articlecontributes to bolstering the practical accuracy of DRL, when employed for a complex downstreamdecision (e.g., whether to adopt an DRL method for a real-world system) and calls for a change onhow PE are performed at present.
Important distinctions: First, we identify and substantiate an emerging problem in DRL performanceevaluation, where a single or small set of arbitrarily selected point MDPs is used to evaluate algorithmic generalization of DRL methods within a task.
Such requirements can be frequently seen inreal-world problems where there is a necessity to achieve economies-of-scale in order to deploy themethod cost-effectively in practice.
To put this into context of related DRL practices, in this workwe neither address the evaluation of algorithmic generalization across task families (standard DRLevaluations using task suites), nor the evaluation of policy generalization within a task family (asaddressed in multi-task, robust RL).
Second, we focus on problems where a separate individual DRLmodel can be trained for each MDP ( i.e., to achieve overall better performance).
Another parallelline is in which a single agent is designed to perform well on all MDPs of the family ( e.g., domainrandomization in robotics).
However, in the scope of this work, we do not consider such problems.
Inthe following sections, unless otherwise stated, by performance evaluation , we mean performanceevaluations in DRL when evaluating algorithmic generalization of DRL methods within a task.
Pitfalls of Point MDP based Performance Evaluations
We illustrate the pitfalls of point MDP based PE by considering an experiment derived from popular
DRL benchmarks.
We use three popular control tasks (Pendulum, Quad [19], Swimmer) as exampleunderspeciﬁed tasks.
For each task, we augment the nominal task to induce a family of MDPs.
A summary of the tasks is given in Table 1, along with its augmentation.
For each task family, wethen specify ﬁve point MDPs.
More details of the tasks and their corresponding modiﬁcations aregiven in the Appendix B.1.
We consider three popular DRL algorithms (PPO [32], TRPO [31] and
TD3 [16]) for performance evaluation on the tasks.
We have veriﬁed that each modiﬁed point MDPis not under-actuated and that given actuator limits, all are solvable.
Observation 1: Reporting performance evaluations based on point MDPs can be misleading .
Previous works commonly use three selection criteria for selecting a point MDP for PE. 1) random
MDP : a random MDP is used to model a somewhat arbitrary selection of an MDPs from the family( e.g., a random selection of a signalized intersection for training a trafﬁc signal control agent [5]) 2)generic MDP : an MDP that represents the key characteristics the MDPs the family ( e.g., the use of ageneric cancer progression model for training a chemotherapy designing agent [41]) and 3) simpliﬁed
MDP : an MDP that simpliﬁes the modeling ( e.g., zero-error instrument modeling when training anagent for chemical reaction optimization [43]).
[Table]
•2D quadcopter in an obstacle course.
• Goal : maneuver the 2D quadcopter through an obstacle course using its vertical acceleration as the control action.
•MDPs with varying the obstacle course lengths (upper obstacle length and lower obstacle length).
[Table]
•A pendulum that can swing.
• Goal : swing up the pendulum.
•MDPs with varying masses and lengths of the pendulum.
[Table]
•MuJoCo 3-link swimming robot in a viscous ﬂuid.
• Goal : make the robot swim forward as fast as possible by actuating the two joints.
•MDPs with varying capsule sizes for the segments comprising the robot swimmer
In Figure 1, we report the PE of the DRL methods on each point MDP described in Table 1 for thethree control tasks.
That is, each method is trained and evaluated on each individual point MDP.
Given a MDP family, all child MDPs are considered possible random MDPs.
For each task, ?
denotesthe generic MDP, and † represents the simpliﬁed MDP.
The simpliﬁed MDP is chosen as the “easiest”
MDP, i.e. simpliﬁes the transition dynamics.
The generic MDP is the parametric mean MDP of other
MDPs of the family.
We observe a few interesting phenomena: (1) the same method yield signiﬁcant differences inperformance under different point MDPs for all three tasks, (2) simpliﬁed MDPs generally achievebetter performance than other point MDPs in the family, and (3) comparing DRL methods basedon point MDPs can provide conﬂicting conclusions (e.g., conﬂicting relative performance beneﬁts)based on which point MDP is used for evaluations. For example, for Quad, point MDP (0,6) indicatesboth PPO and TRPO are equally well-performing. However, under point MDP (0.5,3), we see that PPO outperforms TRPO with a ⇠ 0.75 point difference.
These observations highlight the variability ofpoint MDP based evaluations and the uncertainties involved.
Such PE could mislead the communityto incorrectly conclude that one method is better than another and thereby hinder the scientiﬁcprogress of the ﬁeld.
Observation 2: DRL training can be sensitive to the selected point MDP properties .
It isgenerally known that DRL training is sensitive to the underlying MDP.
Therefore, selection of a point
MDP from a family can demonstrate a training impact that does not generalize to other MDPs in thefamily.
Subsequently, the performance of the DRL method could be overestimated or underestimated.
We demonstrate this phenomenon using our three control tasks in Figure 2 when using the PPOalgorithm for training 2 .
First, in Quad (Figure 2a), reaching the goal means achieving a normalizedscore of 1.
However, only under a few of the point MDPs, the agent reaches the goal during training.
Speciﬁcally, 2 out of 5 settings converge to a local minimum (crashing on the same obstacle as [Figure]
training progress). Similarly, in Figure 2b for the pendulum, under one of the MDPs the agent doesnot achieve the goal (stuck at the 200 steps mark) while under the other four the agent achieves thegoal.
Finally, we see similar behavior in Swimmer task in Figure 2c where under one point MDP theagent fails to achieve the goal (stuck at the 5000 steps mark) during training and succeed under others.
Further examples of training complications under other DRL algorithms are given in Appendix B.3.
[Figure]
We hypothesize one of the root causes of this phenomenon is the complexity differences in point
MDPs.
For example, in Quad, making the obstacle-free course narrower requires the DRL agent toexplore some speciﬁc actions different from what it would otherwise need to explore when the courseis wider.
This effectively makes the training process harder for point MDPs with the narrower paths.
Another potential cause is MDP designers over-ﬁtting MDP design to point MDPs.
For example, inpendulum, the default reward function penalizes higher torque values while encouraging the reachingthe goal with a weighted composite reward.
MDP designers may over-ﬁt the the weighting values toa selected point MDP which does not generalize to the entire family.
This can result in point MDPswith higher pendulum mass to fail if the weight on the torque limits are higher.
Notation and Formalism
Given a sequential decision-making task T and a reinforcement learning method R to be evaluated,we consider the setting where there are M possible point MDPs in the family of MDPs.
In general, despite the illustrative examples in the previous section, M can be quite large.
Let N denote acomputational budget in terms of the number of models that can be trained 3 .
We assume that
M  N ; that is, we cannot evaluate the given RL method on all MDPs in the family.
As we demonstrated in Section 2, the performance of a DRL method R can signiﬁcantly depend onthe choice of the point MDP.
As suggested by previous work [1], it is therefore warranted to modelthe performance of R on a given point MDP as a real-valued random variable X .
This means anormalized performance score s R,i for a given R and a point
MDP i is a realization of the randomvariable X .
We normalize per point MDP performance scores by linearly rescaling scores based ona given baseline.
For example, scores in Atari games are typically normalized with respect to anaverage human [25, 1].
Given a family of MDPs, the a point MDP i may be more important or common than another point
MDP j .
This importance score p T,i of a point MDP on task T can be considered as a real-valuedrandom variable Y .
This means depending on the importance scores of each point MDP, a distributioncan be generated for the MDP family.
For ⌧ 2 R n where ⌧ represents the point MDP characterizationfeatures, we therefore deﬁne the point MDP distribution as F ( ⌧ ) =
P ( Y ) .
Assumption : Given a task, we assume that MDP designers are aware of the importance scores foreach MDP according to some predeﬁned criterion for building the point MDP distribution.
Deﬁnition
The overall performance of a DRL method R on task T is deﬁned as E TR =
P | U | i =1 s R,i ⇥ 162 p
T,i where U is the set of point MDPs.
However, obtaining E TR is not always possible because of the budget constraint M  N .
Therefore,we focus on selecting a subset of point MDPs V from the MDP family to perform a performanceevaluation.
Accordingly, the estimated performance of the method R on task T is ˆ E TR =
P | V | i
=
1 s R,i ⇥ 166 p
T,i .
Clearly, if not careful, selecting different subsets can greatly affect the accuracy of the PE.
An intuitive approach to selecting a subset of point MDPs is to assess
the contribution c
i of each point
MDP i to overall performance evaluation.
Contribution depends both on the importance score, whichis given, and performance score, which incurs a computational cost to evaluate, and can be deﬁned asc i
=
s R,i ⇥ p T,i .
Thus, we seek to ﬁnd the set of point MDPs that has the highest contributions tothe overall performance evaluation.
Given that we wish to estimate the contribution of individualpoint MDPs without assessing the overall performance E TR , this poses a chicken-and-egg problem.
In the subsequent sections, we propose approximation techniques that one can employ to identify asubset V based on the approximated contributions.
Case Study: Trafﬁc Signal Control
To validate the pitfalls of point MDP based evaluations, we consider an established benchmarkthat exhibits a large implicit family of MDPs describing a single task and wherein there could besigniﬁcant real-world implications.
In particular, we consider the PE of DRL methods on the trafﬁcsignal control task, leveraging the RESCO benchmark [5].
We use six algorithms from RESCO in ourcase study, namely: IDQN , IPPO [4], MPLight [11], MPLight ⇤ , Fixed time and Max pressure [36].
Further details can be found in Appendix C.1.
Naturally, trafﬁc signal control should be considered on multiple intersection layout and vehicleﬂow rates, hence a family of MDPs.
We base our importance score p T,i of each intersection on thefrequency of occurrence within a geographic region and the performance score s R,i as the normalizedper vehicle average delay.
Scores are normalized based on an untuned yet sufﬁciently performantﬁxed time controller baseline.
The importance scores and the intersections used to build the point MDP distribution of the intersections are taken from Salt Lake City in Utah.
We use 164 uniqueintersections and refer the reader to Appendix C.2 for more details on building this distribution.
In Figure 3, we report the performance of each of six methods on 164 unique point MDPs and thereported performance as per [5] 4 .
First, we observe signiﬁcant variations in the performance based on the point MDP used for evaluations.
Speciﬁcally, all DRL methods demonstrate a signiﬁcant variationwhile non-DRL methods demonstrate comparatively low yet considerable variations.
Second, weobserve that the reported performances in related literature can be signiﬁcantly biased.
As an example,
IDQN, IPPO and MPLight performances are clearly overestimated.
To quantitatively analyze thepotential pitfalls, we denote the performance of each method over the entire MDP family in Figure 4.
Interestingly, we see a signiﬁcant result change from previously reported results.
Although the Fixed time controller is regarded as an underperforming method as per reported performance in [5], we ﬁnd that under the family of MDP based evaluations, the non-RL Fixed time controller and Max pressure controller perform signiﬁcantly better than all the four DRL controllers.
We note that ﬁne-tuning a non-RL Fixed time controller is simple enough that it does not pose a computational burden and can be done easily, even on a regular computer.
While reported performances ranked IPPO as a well-performing model with a normalized score of 0.84, our analysis shows that it, in fact, is the lowest-performing method and that the revised normalized score is as high as 1.7 (even cannot outperform an untuned ﬁxed time controller).
It is thus clear that point MDP based PE can be misleading and may pose performance beneﬁts that do not generalize to the MDP family.
Further Evidence on Pitfalls of Point MDP based Evaluations
To further validate the pitfalls of point MDP based evaluations and the impact, we look at threepopular DRL control tasks: cartpole (discrete actions), pendulum (continuous actions) and halfcheetah (continuous actions). For each task, we devise a family of MDPs as described in Table 4 in Appendix D.1.
Our MDP families consist of 576, 180 and 120 point MDPs for cartpole, pendulum and half cheetah respectively.
Due to space limitations, we provide in-depth analysis of performancevariations in the Appendix D.2 and only provide a summary of analysis in this section.
In Figure 5, weshow the signiﬁcant discrepancies in overall performance and the reported performances.
Reportedperformance of each task is measured by training and evaluating DRL methods on commonly usedsingle point-MDP given in the common benchmark suites.
We see signiﬁcant result changes when evaluated on the point MDP family.
For example, in cartpole(Figure 5a), all reported performances achieve the best score of 1.0 while we see a signiﬁcantlydifferent overall performance.
Although under reported performance, all four DRL methods forcartpole are ranked as equally well performing, we see signiﬁciant rank change as some methodssigniﬁcantly underperform when considering their overall performance.
Similar insights can be seenin
Pendulum (Figure 5b) and in half cheetah (Figure 5c). [Figure]  Recommendations for Reliable Evaluation Section 2, 4 and 5 demonstrate the pitfalls of point MDP based evaluations.
In this section, we presentrecommendations to alleviate the pointed pitfalls and perform reliable PE under budget.
We providethree main recommendations to the research community as summarized in Table 2. Challenge
Our recommendation
Lack of benchmarks
•Create benchmarks that depict MDP families.
•Publish datasets of MDP families of control tasks including point-MDP distributions.
•Incentivize publication of such datasets and control task at leading conferences.
Large families of MDPs with limited computational budgets
•Adopt performance approximations using clustering and random sampling under a computational budget.
•Standardize the evaluations by making the selected pointMDPs public.
Lack of emphasize on all point -MDP performances
•Use performance proﬁles to show a detailed view of how overall performance changes with point MDPs
6.1 Create data sets of MDP families for benchmark RL tasks for performance evaluations
The use of benchmark datasets for PE is well established in supervised learning.
Ranging fromcomputer vision, biological data to natural language processing, standard datasets are being used for PE of deep learning methods.
In DRL, instead of published datasets, researchers use standardizedcontrol tasks like Atari 100k [22] and DeepMind control suite [33] for PE.
However, we recognize that current state-of-the-art task-speciﬁc benchmarks alone are not sufﬁcientto standardize the performance evaluations in DRL.
The main limitation of current suits is that they only provide an arbitrary-created (or simpliﬁed) point MDPs and do not consider the family ofpossible MDPs.
As many real-world applications inherently demonstrate a family of MDPs, thecurrent DRL standardization of PE may steer the research community in a vacuum, while realisticallyuseful DRL methods may get rejected or not even developed.
Therefore, our ﬁrst recommendation is to create/or update task-speciﬁc benchmarks which demonstrate the necessity of incorporating a family of MDPs in the DRL method design process.
Second,we also encourage the research community to actively create MDP families for existing controltasks (e.g., different signalized intersections as trafﬁc signal control MDP family) and publish thempublically.
Finally, we also encourage main artiﬁcial intelligence conferences with datasets andbenchmark tracks like NeurIPS to encourage the community to publish such datasets and tasks and toinclude necessary check-ins in the paper submission [-checklists.5.
-]checklists.6.2 Evaluate DRL methods on a family of MDPs instead of point MDPs
Evaluating performance over an entire family of MDPs can often be computationally expensive tocarry out in practice due to the large cardinality of the family.
Therefore, it is often will require toconduct performance approximations.
While more sophisticated methods like active learning approaches are possible, we resort to effective yet straightforward techniques to bolsters the adoptabilityof the techniques within a wider community.
We present three techniques: (1) M1 : random sampling with replacements from the point MDPdistribution, (2) M2 : random sampling without replacements, and (3) M3 : clustering point MDPsusing k-means and assigning probability mass of all point MDPs that belong to same cluster to itscentroid.
More details of each method and more results can be found in Appendix E.1.
[Figure] In Figure6, we denote the approximate PE of IDQN, IPPO and Fixed time in comparison to the groundtruth performance while varying the budget size.
In general, k-means clustering-based approximationproduces better estimates than the other two techniques with smaller standard deviations.
Speciﬁcally,if the budget size is half the total MDP family size, k-means clustering demonstrates a reasonablyaccurate performance estimates.
Also, by publishing the selection of point MDPs as data sets canenable reproducibility and standardization of the PE of the tasks.
A sensitivity analysis of theproposed techniques to the underlying point MDP distribution is also given in Appendix E.2.
Remarks.
We acknowledge that there are other factors to consider for a computational budget,including hyperparameter tuning [21] and training using multiple random seeds within each point MDPs [1].
Here, we focus on the computational budget allotted for the task underspeciﬁcation issue.
6.3 Use performance proﬁle of the MDP family instead of point MDP performance
Inspired by the idea of performance proﬁling for DRL evaluations [1] and in optimization software [12], our ﬁnal recommendation is to report the performance of a DRL method over a family of
MDPs as a performance proﬁle.
Although the overall performance of a DRL method over a familyof MDPs can yield more reliable evaluations than evaluating on a point MDP, it may encapsulatefurther insights into the method’s performance.
By representing the performance of a method as aperformance proﬁle, a more detailed visualization of the performance over the point MDPs can beillustrated.
A performance proﬁle illustrates what point MDPs are most probable in the distribution and how each of the point MDP contributes to the ﬁnal estimate of the performance.
It enables direct comparison of methods.
In particular, if the cumulative performance curve of method A is strictly above method
B, method A is said to stochastically dominate method B [1].
Furthermore, point MDPs in which a method seemingly under-performs is easily visible, giving a clear overview of where the limitations and strengths are originating from.
Related Work
Deep reinforcement learning algorithms have notoriously high variance [1, 21, 10, 10, 9], resultingin reliability and reproducibility issues when applying them to real world applications.
Whilestandardized control benchmarks [6, 34] are proposed to benchmark existing and newly proposedalgorithms, often times ﬂawed evaluation metrics such as point estimates are used.
Agrawal etal.
[1] advocates for using better evaluation metrics such as interval estimates, performance proﬁles,and more robust interquantile mean while Jordan et al.
[21] proposes a performance percentilesmeasure.
Additionally, Chan et al.
[10] studies the behaviors of DRL algorithms by focusing onvariability and risk both during and after training.
A few papers study the effect of architecture design,weight initialization, and optimization techniques on DRL’s variance and instability [29, 9, 23].
While family of MDPs have not been used before for performance evaluations as per our knowledge,similarities can be seen when assessing the generalization capabilities of DRL agents [8, 7, 15]. Additionally, the family of MDPs for the same high level problem has been modeled as Contextual
MDPs (cMDP) [8], Hidden Parameter MDPs [13] and Epistemic POMDPs [17]. Another relateddirection is domain randomization in Sim2Real transfer [28, 40], which samples stochastic trainingenvironments in simulation in the hope of covering the true real-world test distribution.
In contrary, weare subject to a budget constraint, and allow training a different model for each different environment.
Conclusion and Future Work
In this work, we identify an important yet overlooked issue of task underspeciﬁcation in DRL—thereliance of reporting outcomes on somewhat arbitrarily selected point MDPs.
We detail the pitfalls ofpoint-MDP based evaluations in evaluating algorithmic generalization of DRL methods within a taskand what negative outcomes would arise due to this practice.
We provide series of recommendationsto alleviate this issue.
An exciting avenues of future work are explore if similar pitfalls occur whenevaluating algorithmic generalization of DRL methods across tasks and design efﬁcient yet effectivemethods that can produce good approximate performance estimates.
We intend for our ﬁndings toraise awareness in issues of task underspeciﬁcation that impact the empirical rigor of DRL, and aimto help move the needle toward a more disciplined science overall.
Broader Impact
The Institute for Ethical AI & Machine Learning had developed a series of eight principles toguide the responsible development of machine learning systems.
Practical accuracy is one of theprinciples, which emphasizes that accuracy and cost metric functions are aligned to the domainspeciﬁc applications .
This article contributes to bolstering the practical accuracy of RL, whenemployed for a complex downstream decision (e.g. whether to adopt an RL method for a societalsystem), by highlighting some limitations of current standard RL research practices and proposing toexplicitly consider a family of MDPs that constitutes the complex decision.
On the other hand, evenwith more empirically rigorous ML practices, there are still subjective aspects of the task distribution(analogous to family of MDPs in RL), so it remains important to not over-index on RL-based PE.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section 8.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See Table 4 and
Section C in Appendix.
Code and data used are separately submitted as supplementary materials.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] Provided the details in the respective sections where they arediscussed.
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] Attribution-NonCommercial-
ShareAlike [Table] Generic (CC BY-NC-SA 2.0)
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]