Results As shown in Figure 3, | d util ( f ) | increases along λ , especially when log( λ ) ≥ − 5 and | d util ( f ) | is positively correlated with R ( f ) . In other words, the stronger the regularization, the larger the imbalance in utilization between modalities we obverse. It conﬁrms the second conjecture in §3.2, which states that the stronger regularization we apply, the greedier the multi-modal learning process becomes. We see  | d speed | follows the same trend as | d util | . Again, it supports our choice of using the conditional learning speed to predict the conditional utilization rate.

Results As shown in Figure 3, | d util ( f ) | increases along λ , especially when log( λ ) ≥ − 5 . We also see that | d util ( f ) | is positively correlated with R ( f ) . In other words, the stronger the regularization is, the larger the imbalance in utilization between modalities we observe. We see that | d speed | follows the same trend as | d util | . Again, it supports our choice of using the conditional learning speed to predict the conditional utilization rate.

Instruction: Split first sentence in two and delete the third sentence

Instruction: Exclude redundant expression.


The classiﬁcation accuracies on the ﬁve benchmarks are shown in Table 1. MLG, as a kernel method, performs well on PROTEINS. However, it suffers from a long run time and takes more than 1 day on two larger datasets, as observed in INFOGRAPH. Our method achieves the best results in 4 out of 5 datasets compared with both kernel and unsupervised models, e.g., it achieves 1.5% improvement over previous state-of-the-art on Reddit-Binary, and 1.8% improvement on IMDB-Binary, which shows the superiority of our methods.

The classiﬁcation accuracies on the ﬁve benchmarks are shown in Table 4. MLG, as a kernel method, performs well on PROTEINS. However, it suffers from a long run time and takes more than 1 day on two larger datasets, as observed in INFOGRAPH. Our method achieves the best results in 4 out of 5 datasets compared with both kernel and unsupervised models, e.g., it achieves 1.5% improvement over previous state-of-the-art on Reddit-Binary, and 1.8% improvement on IMDB-Binary, which shows the superiority of our methods. When compared with supervised graph classiﬁcation models, ours beats the best supervised classiﬁcation model GIN on IMDB-BIN, on-par with GIN on PROTEINS, IMDB-MULTI, and only loses on REDDIT.


K rather than T . Furthermore, for ﬁnite horizon MDPs, the number of steps is equal to T = KH . In this case, the result in 6.1 can avoid the extra factor of B, c min and other logarithmic term. Theorem 6.1 applies to the general cost function with ρ set to 0. Note that the regret upper bound depends on the total number of time steps T , which is random. To replace the T -dependence by the


First, it is the tightest topological property of state spaces that has been studied in the literature of model checking and planning, as far as we know. Secondly, although the worst-case complexity of computing the diameter for a factored transition system, and succinct digraphs more generally, is Π P 2 -hard (Hemaspaandra et al. 2010), there are practical methods that can compositionally compute upper bounds on the diameter (Baumgartner, Kuehlmann, and Abraham 2002; Rintanen and Gretton 2013; Abdulaziz, Gretton, and

First, it is the tightest topological property of state spaces that has been studied. Secondly, although the worst-case complexity of computing the diameter for a succinct graph is Π P 2 -hard (Hemaspaandra et al. 2010), there are practical methods that can compositionally compute upper bounds on the diameter (Baumgartner, Kuehlmann, and Abraham 2002; Rintanen and Gretton 2013; Abdulaziz, Gretton, and

Instruction: Remove unnecessary details.

Instruction: Concise by removing unnecessary details.


Vision transformer. Transformer [42, 5] has recently achieved great success  in  computer vision [6,1 , 21, 18, 58, 50, 44]. Swin-transformer [21] restricts self-attention  to non-overlapping local windowswhile allowing cross-window connection to improve efficiency. SSA [29] divides  attention headsinto multiple groups to aggregate image features with different granularities. Guo et al. [9] and Zhao et al. [55] make the first step towards introducing the transformer for point cloud analysis. Recently, many approaches [45, 54, 25, 10, 22] apply local self-attention  on voxels to learn richerfeature representation. Our work extends the window-based attention on 3D voxelsby introducingscale-aware attention learning equipped with novel sampling strategies for the queries and the keys toimprove both accuracy and efficiency.

Vision transformer. Inspired by the great success of Transformer [35, 4] in NLP, some worksemploy it in the field of computer vision [5, 1, 18, 16, 50, 42, 36]. Swin-transformer [18] increasesefficiency by limiting self-attention computation to non-overlapping local windows while allowing forcross-window connection. SSA [24] divides the attention head into multiple groups and aggregatesimage features with different granularity, respectively, which has achieved excellent performance. In the field of point cloud, Guo et al. [8] and Zhao et al. [47] introduce transformer paradigm forpoint cloud classification and segmentation. Recently, many methods [37, 46, 22, 9, 19] applylocal self-attention mechanism on voxels to learn richer feature representations. Our model extendswindow attention to 3D voxels and flexibly combines different window sizes to capture multi-scaleand multi-granularity features while shunting them to different attention heads in SSA.

Instruction: Rewrite this paragraph for improved readability and clarity.


In MCMC, one chooses a transition kernel that leaves the target distribution invariant and constructs a Markov Chain by applying the kernel repeatedly. The MCMC method relies only on the ergodicity assumption. Other than that it is general, if enough computation is performed, the Markov Chain generates correct samples from any target distribution, no matter how complex the distribution is.

In MCMC, one chooses a transition kernel that leaves the target distribution invariant and constructs a Markov Chain by applying the kernel repeatedly. The MCMC method relies only on the ergodicity assumption, other than that it is general. If enough computation is performed, the Markov chain generates correct samples from any target distribution, no matter how complex the distribution is.

Instruction: Rephrase the paragraph

Instruction: Balance sentences length.


Choose a [ t ] = w.p. m ] = w.p 1 − m t then end end end are multiple types of arms and train a separate NeurWIN for each type. During testing, the controller calculates the index of each arm based on the arm’s NeurWIN and schedules the M arms with the highest indices.

For such scenarios, we consider that there are multiple types of arms and train a separate NeurWIN for each type. During testing, the controller calculates the index of each arm based on the arm’s state and schedules the M arms with the highest indices.


To incorporate the gradient of the target distribution into the sampler, we take inspiration from the HMC algorithm. Basic HMC starts with drawing a random initial momentum v 0 , followed by several steps of leapfrog integration. In the following, we denote the momentum variable after n updates by v n and position by x n . In a leapfrog step, the integrator ﬁrst updates v with a half step of the gradient: v n (cid:48) = v n − 1 − (cid:15) 2 ∂ x U ( x n − 1 ) , followed by a full step of x update: x n = x n − 1 + (cid:15)v n (cid:48) , and another half step of v update: v n = v n (cid:48) − (cid:15)
 ∂ x U ( x n ) . After several steps, the  update of x can be written as: x n = x 0 + (cid:80) ni =
 v i (cid:48) , which has the form x (cid:48) = x + z with z = (cid:80) ni v i (cid:48) = − nv 0 − n(cid:15) 2 (cid:2) ∂ x U ( x 0 ) (cid:3) − (cid:15) (cid:2)(cid:80) ni =1 ( n − i ) ∂ x U ( x i ) (cid:3) . Note that, although z does not have a tractable density, the equation suggests that a tractable model of z that use gradient information effectively should subtract the gradient of the target distribution evaluated at  intermediate point of x from a sample of a base distribution v 0 .

The gradient of the target distribution enters our model in those afﬁne transformations. To motivate the particular form we choose, we take a closer look at the HMC algorithm. Basic HMC starts with drawing a random initial momentum v 0 , followed by several steps of leapfrog integration. Let x n be the momentum variable after n updates by v n and position. In a leapfrog step, the integrator ﬁrst updates v with a half step of the gradient: v n (cid:48) = v n − 1 − (cid:15) 2 ∂ x U ( x n − 1 ) , followed by a full step of x update: x n = x n − 1 + (cid:15)v n (cid:48) , and another half step of v update: v n = v n (cid:48) − (cid:15)
 ∂ x U ( x n ) . After several steps, the overall update of x can be written as: x n = x 0 + (cid:80) ni =
 v i (cid:48) , which has the form x (cid:48) = x + z with z = (cid:80) ni v i (cid:48) = − nv 0 − n(cid:15) 2 (cid:2) ∂ x U ( x 0 ) (cid:3) − (cid:15) (cid:2)(cid:80) ni =1 ( n − i ) ∂ x U ( x i ) (cid:3) . The equation for generating z through afﬁne transformations, describes how the gradient of the target distribution, evaluated at some intermediate point of x , should be included.


We show a visualization of the  occupancy grids in Figure 9 (right). We visualize the occupancy grids by converting them to heightmaps. This is achieved by multiplying each voxel’s occupancy value by its height coordinate in the grid, and then taking a max along the grid’s height axis. The visualizations show that the occupancy module learns to ﬁll the “holes” of the partial view, effectively imagining the complete 3D scene.

We show a visualization of the estimated occupancy volumes in Figure 9-right. We visualize the occupancy volumes by converting them to heightmaps. This is achieved by multiplying each voxel’s occupancy value by its height coordinate in the grid, and then taking a max along the grid’s height axis. The visualizations show that the occupancy module learns to ﬁll the “holes” of the partial view.

Instruction: Remove second part of last sentence and Replace "grids" by "volumes"

Instruction: Delete unnecessary details. Make the text more formal.


Pointing operation to the edge target takes advantage of the cursor stopping at the edge of the screen to complete the pointing without precise control. However, pushing-edge (pushing the cursor to the edge of the screen) behavior increases the distance traveled by the mouse, thereby increasing the movement time. Yamanaka [25] defined PE (Path Efficiency) to calculate the efficiency of the cursor movements (Eq. 7).

A pointing operation for an edge target exploits the fact that the cur- sor stops at the edge of the screen to complete the pointing without precise control. However, pushing-edge behavior, i.e., pushing the cursor to the edge of the screen, increases the distance traveled by the mouse, and this increases the movement time. Yamanaka [28] defined PE (Path Efficiency) to calculate the efficiency of the cursor movements (Eq. 7).

Instruction: Change some words in this paragraph for the better

Instruction: Improve the linking between ideas to make the paragraph more precise and readable.


The second component focuses on modeling the global temporal patterns in the dataset through identifying a small set of temporal global basis functions . The basis time-series, when combined in different ways, can express the individual dynamics of each time series. In our model, the basis time-series are encoded in a trained seq-2-seq model (Sutskever et al., 2014) model in a functional form. Each time series is then associated with a learned embedding vector that speciﬁes the weights for decomposition along these basis functions. Predicting a time series into the future using this model then just involves extrapolating the global basis functions and combining them using its weight vector, without explicitly using the past values of that time series. The coherence constraints therefore only impose constraints on the embedding vectors of each time series, which can be easily modeled by a hierarchical regularization function. We call this component a basis decomposition model . In  Section A.2, we also provide theoretical justiﬁcation for how such hierarchical regularization using basis decomposition results in improved prediction accuracy.

The second component focuses on modeling the global temporal patterns in the dataset through identifying a small set of temporal global basis functions . The basis time-series, when combined in different ways, can express the individual dynamics of each time series. In our model, the basis time-series are encoded in a trained seq-2-seq model (Sutskever et al., 2014) model in a functional form. Each time series is then associated with a learned embedding vector that speciﬁes the weights for decomposition along these basis functions. Predicting a time series into the future using this model then just involves extrapolating the global basis functions and combining them using its weight vector, without explicitly using the past values of that time series. The coherence constraints therefore only impose constraints on the embedding vectors of each time series, which can be easily modeled by a hierarchical regularization function. We call this component a basis decomposition model . As we will see, this part of the model is only approximately coherent unless the embedding constraints hold exactly. In particular, in this paper, we focus on improving model accuracy rather than preserving exact coherency. In Section A.2, we also provide theoretical justiﬁcation for how such hierarchical regularization using basis decomposition results in improved prediction accuracy.


To this end, we define a kind of ( σ, δ ) -measure to mathematically quantify the data augmentation, and then provide an upper bound of the downstream classification error rate based on the measure. It reveals that the generalization ability of contrastive self-supervised learning is related to three key factors: alignment of positive samples, divergence of class centers, and concentration of augmented data. The first two factors are properties of learned representations, while the third one is determined by pre-defined data augmentation. With the above theoretical findings, we then study two canonical contrastive losses, InfoNCE and cross-correlation, to see how they satisfy the first two factors. Furthermore, we conduct various experiments to study the third factor, and observe that the downstream performance is highly correlated to the concentration of augmented data.

To this end, we define a kind of ( σ, δ ) -measure to mathematically quantify the data augmentation, and then provide an upper bound of the downstream classification error rate based on the measure. It reveals that the generalization ability of contrastive self-supervised learning is related to three key factors: alignment of positive samples, divergence of class centers, and concentration of augmented data. The first two factors are properties of learned representations, while the third one is determined by pre-defined data augmentation. We further investigate two canonical contrastive losses, InfoNCE and cross-correlation, to show how they provably achieve the first two factors. Moreover, we conduct experiments to study the third factor, and observe a strong correlation between downstream performance and the concentration of augmented data.

Instruction: Use accurate words.

Instruction: Make the second half of this paragraph more precise and direct.


We now introduce the theorem, which offers a guideline on the neural network architecture for operator learning. It suggests that if the entire architecture can be replaced with a fully connected neural network, large complexity should be required for training. It also verifies that the lower bound for a universal activation function is a sharp bound on the number of parameters. We note here an assumption, which is a sufficient condition for proving the theorem.

We now introduce the theorem, which offers a guideline on the neural network architecture for operator learning. It suggests that if the entire architecture can be replaced with a fully connected neural network, large complexity should be required for approximating the target function. It also verifies that the lower bound for a universal activation function is a sharp bound on the number of parameters. First, we give an assumption to obtain the theorem.

Instruction: Remove redundant details. Use more precise words.

Instruction: Make it more precise when necessary.


LogicRiskNet - a differentiable parametric ptSTL risk monitor. We describe how we may construct a ptSTL risk monitor based on a learned stochastic risk measure that can be applied to aprobabilistic description of human behaviors. We extend the deﬁnition of robustness degree (introduced in Section 2) to encompass belief states. In order to calculate the risk of a trajectory of beliefstates w.r.t a ptSTL formula φ , we need to modify the deﬁnition of the robustness. Recall that thebasic element of φ is a predicate of the form p ( x ) < c , deﬁne α ( x ) = c − p ( x ) as the predicatefunction (this is same as the robustness deﬁnition of the predicate). Since x is stochastic, rather thandeterministic, we can evaluate ptSTL formulas instead using a risk measure ρ : X → R , representing expectation, mean-variance, etc. {+
+} We can then apply this risk measure to the robustness deﬁnitionyielding α ρ ( x ) = ρ ( α ( x )) , x ∈ X (This treatment is similar to [12]).

LogicRiskNet - a differentiable parametric ptSTL risk monitor. We describe how we may construct a ptSTL risk monitor based on a learned stochastic risk measure that can be applied to aprobabilistic description of human behaviors. We extend the deﬁnition of robustness degree (introduced in Section 2) to encompass belief states. In order to calculate the risk of a trajectory of beliefstates w.r.t a ptSTL formula φ , we need to modify the deﬁnition of the robustness. Recall that thebasic element of φ is a predicate of the form p ( x ) < c , deﬁne α ( x ) = c − p ( x ) as the predicate (thisis same as the robustness degree for the predicate). Since x is stochastic, rather than deterministic,we can evaluate ptSTL formulas instead using a risk measure ρ : X → R , representing expectation,mean-variance, value-at-risk, etc. {+
+} In this work, we assume the expectation risk measure, but canequally well replace this with others. We can then apply this risk measure to the robustness deﬁnitionyielding α ρ ( x ) = ρ ( α ( x )) , x ∈ X (This treatment is similar to [13]).


According to the greedy learner hypothesis, we can make multi-modal learning less greedy by controlling the speed at which a multi-modal DNN learns to rely on each modality. To this end, we derive conditional learning speed to measure the speed at which the DNN learns from one modality. It serves as an efﬁcient proxy to the conditional utilization rate of the corresponding modality. We then propose the balanced multi-modal learning algorithm, which controls model’s conditional learning speed between modalities in order to prevent it from being greedy.

We aim to make multi-modal learning less greedy by controlling the speed at which a multi-modal DNN learns to rely on each modality. To this end, we deﬁne conditional learning speed to measure the speed at which the DNN learns from one modality. It serves as an efﬁcient proxy to the conditional utilization rate of the corresponding modality, as shown empirically in §5.2 and §5.3. We then propose the balanced multi-modal learning algorithm, which controls the difference in conditional learning speed between modalities that the model exhibits during training.


On the contrary, random convolution can induce a growing difﬁculty by increasing the number of factors on a single background. Therefore, the generalization rapidly decreases after augmentationinterrupted when training with a single background because the learning direction toward generalization about various backgrounds is not helpful to train. On the other hand, the training can have helpwhen their difﬁculty is solved by augmentation, such as Figure 2(b) and Figure 2(c). Thus, in deep RL, neural networks maintain the regularization when augmentation helps the training.

On the contrary, random convolution can induce a growing difficulty by increasing the number of factors on a single background. Therefore, the generalization rapidly decreases after augmentation is interrupted during training with a single background because the learning direction toward generalization about various backgrounds is not helpful to train. In contrast, the training can help when their difficulty is solved by augmentation (Figure 2(b), 2(c)). Thus, in deep RL, neural networks maintain the regularization when augmentation helps the training.

Instruction: Add missing spaces.

Instruction: Improve the english in the paragraph, make it slightly more formal.


Dynamic p. during training g ( (cid:101) w t ) (on pruned model) dynamically adapting mask m t + adaptive every few iterations + can recover from premature pruning from a theoretical point of view, and to provide further insights and interpretation. We do not require tuning of additional hyperparameters, and no retraining of the sparse model is needed (though can further improve performance).

Dynamic p. during training g ( (cid:101) w t ) (on pruned model) dynamically adapting mask m t + adaptive every few iterations + can recover from premature pruning tuning of additional hyperparameters, and no retraining of the sparse model is needed (though can further improve performance).


For example, Cohen et al. ; Rezende & Racani`ere (2021) approximate the OT map to define normalizing flows on Riemannian manifolds, Hamfeldt & Turnquist (2021a;b); Cui et al. (2019) derive algorithms to approximate the OT map on the sphere, Alvarez-Melis et al. HoyosIdrobo (2020) learn the transport map on hyperbolic spaces. However, the computational bottleneck to compute the Wasserstein distance on such spaces remains, and, as underlined in the conclusion of (Nadjahi, 2021), defining SW distances on manifolds would be of much interest. Notably, Rustamov & Majumdar (2020) proposed a variant of SW, based on the spectral decomposition of the LaplaceBeltrami operator, which generalizes to manifolds given the availability of the eigenvalues and eigenfunctions. However, it is not directly related to the original SW on Euclidean spaces.

For example, Cohen et al. Idrobo (2020) learn the transport map on hyperbolic spaces. However, the computational bottleneck to compute the Wasserstein distance on such spaces remains, and, as underlined in the conclusion of (Nadjahi, 2021), defining SW distances on manifolds would be of much interest. Notably, Rustamov & Majumdar (2020) proposed a variant of SW, based on the spectral decomposition of the LaplaceBeltrami operator, which generalizes to manifolds given the availability of the eigenvalues and eigenfunctions. However, it is not directly related to the original SW on Euclidean spaces.


We had to exclude 10 runs of DSPN due to signiﬁcantly worse results making their average uninformative. The following excluded DSPN runs were all using 40 iterations:

We had to exclude 10 runs of DSPN due to signiﬁcantly worse results making their average uninformative. We did not observe any stability issues with iDSPN, so we did not have to exclude any iDSPN runs. The following excluded DSPN runs were all using 40 iterations:


Figure 1. VEM uses expectile V -learning (EVL) to learn V -functions while avoiding extrapolation error in the action space. EVL uses an expectile operator that interpolates between Bellman expectation operator and optimality operator to balance behavior cloning and optimal value learning. Further, VEM integrates memory-based planning to improve the advantage estimation and accelerate the convergence of EVL. Finally, generalized advantage-weighted learning is used for policy learning with enhanced advantage estimation. A formal description for the VEM algorithm is shown in Algorithm 1 in Appendix A.1.

Figure 1. VEM uses expectile V -learning (EVL) to learn V -functions while avoiding extrapolation error in the action space conﬁnes value learning within the dataset to reduce extrapolation error. EVL uses an expectile operator that interpolates between Bellman expectation operator and optimality operator to balance behavior cloning and optimal value learning. Further, VEM integrates memory-based planning to improve the advantage estimation and accelerate the convergence of EVL. Finally, generalized advantage-weighted learning is used for policy learning with enhanced advantage estimation. A formal description for the VEM algorithm is shown in Algorithm 1 in Appendix A.1.


Traditional approaches to predicting the effect of mutation on protein binding can be roughly divided into two classes: biophysical methods and statistical methods. Biophysical methods focus on modeling inter-atomic interactions, e.g. hydrogen bonding, electrostatic forces, etc., with mechanical and statistical energy functions. These methods use their underlying energy function to sample conformations of the mutated protein complex and predict the change in binding free energy upon mutations (Schymkowitz et al., 2005; Park et al., 2016; Alford et al., 2017; Steinbrecher et al., 2017). Statistical methods are based on feature engineering. Descriptors that summarize the geometric, physical, evolutionary, and motif properties of proteinsare used to fit the statistical model that predicts the effect of mutations on binding free energy (Geng et al., 2019a; Zhang et al., 2020). Traditional methodsgenerally face the trade-off between speed and accuracy due to the time-consuming sampling process. Sophisticated energy functions and feature engineering underlying these methods depend heavily on human knowledge, thus limiting their pace to improve with the fast-growing of available protein structures.

Traditional approaches to predicting the effect of mutation on protein binding can be roughly divided into two classes: biophysical and statistical methods. Biophysical methods utilize energy functions to model inter-atomic interactions. These methods sample conformations of the mutated protein complex and predict changes in binding free energy (Schymkowitz et al., 2005; Park et al., 2016; Alford et al., 2017; Steinbrecher et al., 2017). Statistical methods rely on feature engineering, which uses descriptors summarizing geometric, physical, evolutionary, and motif properties of proteins to predict mutational effects (Geng et al., 2019a; Zhang et al., 2020). Traditional methods face the trade-off between speed and accuracy. Their performance depends heavily on human expertise, which limits their pace to improve with the fast-growing of available protein structures.

Instruction: Summarize this:

Instruction: Make this paragraph shorter.


LSTM or RNN networks for this task Fragkiadaki et al. ; Jain et al. ; Bütepage et al. (2017); Martinez et al. and these works produce reasonably realistic output at a number of tasks such as sitting, talking, smoking, etc. However, these existing methods also have a critical drawback: the motion becomes unrealistic within a couple of seconds and is unable to recover.

LSTM or RNN networks for this task (Fragkiadaki et al., 2015; Jain et al., 2016; Bütepage et al., 2017; Martinez et al., 2017), and these works produce reasonably realistic output at a number of tasks such as sitting, talking, smoking, etc. However, these existing methods also have a critical drawback: the motion becomes unrealistic within a couple of seconds and is unable to recover.


Electronic calendars have become instrumental in the manage- ment of daily activities [16–18]. They are used to coordinate interactions among individual schedules of family or team members and can convey meaning and values behind the priorities of schedul- ing [19]. Calendars have been used to visualize temporal trends that include everyday activities such as energy use in work places, fitness tracking, and work routines [20–22]. In healthcare, unit-of-use packaging that incorporates a simple day or date feature have been used in efforts to help manage prescriptions and improve adherence by prompting patients to maintain the prescribed dosing schedule [23]. While such prescription managers come as stand-alone tools, we look at the possibility of integrating prescription management into the main calendar already used by the patient. Such integration comes with challenges. The first challenge is how to render the pre- scription entries so that the user can differentiate between a normal calendar entry and one that is part of a prescription. The second chal- lenge is how to ensure that the patient rescheduling a prescription entry does so within the  safety envelop of the prescription.

Electronic calendars have become instrumental in the manage- ment of daily activities [16–18]. They are used to coordinate interactions among individual schedules of family or team members and can convey meaning and values behind the priorities of scheduling [19]. Calendars have been used to visualize temporal trends that include everyday activities such as energy use in work places, fitness tracking, and work routines [20–22]. We are interested in exploring the possibility of integrating pre- scription management into electronic calendars that are already used by many patients [23,24]. Such integration comes with challenges. The first challenge is how to render the prescription entries so that the user can differentiate between a normal calendar entry and one that is part of a prescription. The second challenge is how to ensure that patients who reschedule prescriptions do so within the specified safety constraints.

Instruction: Rewrite the latter half of this paragraph to make it more concise.

Instruction: Merge the two sentences in the middle  about integrating prescription management in a new shorter sentence. Improve the english in the last sentence.


Ermon, 2016), require a potentially large number of online samples during training, resulting in poor sample efficiency. Moreover, algorithms such as GAIL follow a similar training paradigm as in the Generative Adversarial Networks (GANs) (Goodfellow et al., 2014), which formulate the problem of IRL as a minimax optimization problem to learn a discriminator that implicitly minimizes an f-divergence. It was found however that AIL methods  can be very difficult to optimize, requiring careful tuning of hyperparameters (Orsini et al., 2021). Another popular approach is to perform distribution matching between the policy and the expert demonstrations as popularized by algorithms such as ValueDICE (Kostrikov et al., 2022b).

Ermon, 2016), require a potentially large number of online samples during training, resulting in poor sample efficiency. Moreover, algorithms, such as GAIL, follow a training paradigm that is similar to Generative Adversarial Networks (GANs) (Goodfellow et al., 2014), by formulating the problem of IRL as a minimax optimization problem to learn a discriminator that implicitly minimizes an fdivergence. It was found, however, that Adversarial imitation learning (AIL) methods such as GAIL can be very difficult to optimize, requiring careful tuning of hyperparameters (Orsini et al., 2021). Another popular approach is to perform distribution matching between the policy and the expert demonstrations (Englert et al., 2013; Kostrikov et al., 2022b).


Humans excel at recognizing objects such as handwritten digits from modified or perturbed inputs. Even if presented with digits which are translated, corrupted, or inverted, we can usually correctly label them without the need of re-learning them from scratch. This may be due to the fact that human intelligence utilizes mechanisms (such as translation) that are generic and generalize across object classes. These mechanisms are modular, re-usable and broadly applicable , and the problem of learning such mechanisms from data is a fundamental question for the study of transfer.

Humans are able to recognize objects such as handwritten digits based on distorted inputs. When presented with digits which are translated, corrupted, or inverted, we can usually correctly label them without the need of re-learning them from scratch. The same applies for new objects, essentially after having seen them once. This may be due to the fact that human intelligence utilizes mechanisms (such as translation) that are generic and generalize across object classes. These mechanisms are modular, re-usable and broadly applicable , and the problem of learning them from data is a fundamental question for the study of transfer.


Limitations and Future Work EAGER assumes the QA system was pre-trained using a preexisting set of example trajectories. Next steps will consist in investigating how to remove this limitation, e.g. by implementing autotelic strategies based on QG/QA learned online. Besides, in this work we tested our method on BabyAI, a 2D environment with synthetic language. In the future, we would like to consider a more complex language, generating more complex questions than the one obtained by masking, and testing our method on more realistic environments with true human instructions, as in the ALFRED dataset [32].

Limitations and Future Work EAGER assumes the QA system was pre-trained using a preexisting set of example trajectories. Next steps will consist in investigating how to remove this limitation, e.g. by implementing autotelic strategies based on QG/QA learned online. Besides, in this work we tested our method on BabyAI, a 2D environment with synthetic language. In the future, we would like to consider a more complex language, generating more complex questions than the one obtained by masking, and testing our method on more realistic environments with true human instructions, as in the ALFRED dataset [37]. Acknowledgments and Disclosure of Funding This work benefited from the use of the Jean Zay supercomputer associated with the Genci grant A0091011996, as well as from the ANR DeepCuriosity AI chair project.


In the case when a user has no control about the incoming gradient ∆ I ω = − d (cid:96)/ d y , the update ∆ I ωin ker P can be much larger in magnitude than ∆ I ω 1 in Im P . In theory,  this is not  very problematic, as updates in ker P do not affect the optimization problem. However, in practice, these irrelevant updates can lead to explosion of the cost vector as well as problems with adaptive optimizers, which will take into account the  irrelevant updates to adjust the learning rate and hence slow down the convergence.

In the case when a user has no control about the incoming gradient ∆ I ω = − d (cid:96)/ d y , the update ∆ I ωin ker P can be much larger in magnitude than ∆ I ω 1 in Im P . In theory, if updates were applied directly in cost space, this would not be very problematic, as updates in ker P do not affect the optimization problem. However, in practice, the gradient is further backpropagated to update the weights in the components before the solver. Spurious irrelevant components in the gradient can therefore easily overshadow the relevant part of the update, which is especially problematic as the gradient is computed from stochastic mini-batch samples.


(Chen et al., 2018) devised GCBD method to learn and generate noise in the given noisy images using W-GAN (Arjovsky et al., 2017) and utilized the unpaired clean images to build a supervised training set. Our GAN2GAN is related to (Chen et al., 2018), but we signiﬁcantly improve their noise learning step and do not use the clean data at all. Table 1 summarizes and compares the settings among the above mentioned recent baselines. We clearly see that only our GAN2GAN and N2V do not utilize any “sidekicks” that other methods take advantage of.

Chen et al. (2018) devised GCBD method to learn and generate noise in the given noisy images using W-GAN Arjovsky et al. and utilized the unpaired clean images to build a supervised training set. Our GAN2GAN is related to Chen et al. , but we signiﬁcantly improve their noise learning step and do not use the clean data at all. Table 1 summarizes and compares the settings among the above mentioned recent baselines. We clearly see that only our GAN2GAN and N2V do not utilize any “sidekicks” that other methods take advantage of.

Instruction: Integrates quotations into the text.


Transition reparametrized actions tion reparamtrization does not have to rely on any hierarchical structures in the ofﬂine data, and can therefore utilize highly suboptimal datasets (e.g., with random actions).

Transition reparametrized actions imitation without the need to reduce control frequency. Unlike learning temporal abstractions, action reparamtrization does not have to rely on any hierarchical structures in the ofﬂine data, and can therefore utilize highly suboptimal datasets (e.g., with random actions).


The proof can be divided into three parts. Firstly, we come up with a neural network approximation ρ NNn of ρ n of which size is O ( w · ϵ − m/rn ) within an error ϵ n . Next, construct a neural network approximation of Φ using the lemma 3. Finally, the inner product  is replaced with a neural network as in the inequality 10.

The proof can be divided into three parts. Firstly, we come up with a neural network approximation ρ NNn of ρ n of which size is O ( w · ϵ − m/rn ) within an error ϵ n . Next, construct a neural network approximation of Φ using the Lemma 3. Finally, the inner product π p n ( β n , τ n ) is replaced with a neural network as in (10) of Lemma 2.


Action Utility In order to use the relational action representations with RL, we follow the insight of utility network  π u from Jain et al. It takes the relational action representations, state, and  action  summary as input for each  action and outputs a utility score reﬂecting how useful the action  is for the current state and in relation to the other available actions. The utility score can be used as  Q-value directly if the training algorithm is DQN, whereas it can be used as a logit fed into a softmax function to form a probability distribution over the available actions.

Action Utility : To use the relational action representations with RL, we follow the utility network architecture π u from Jain et al. It takes the relational action representation, the state, and the action set summary as input for each available action in parallel. It outputs a utility score π u ( c Ra , s, ¯ c R ) for how useful an action a is for the current state and in relation to the other available actions. The utility scores can be used as a Q-value directly for value-based RL or as a logit fed into a softmax function to form a probability distribution over the available actions for policy-based RL.


Linear networks without activation functions are important subject, and there are a number of theoretical works on the implicit regularization in over-parameterized neural networks mainly focusing on linear models (Ji & Telgarsky, 2018; Gidel et al., 2019; Arora et al., 2019a). In contrast, whole properties of over-parameterized DNNs that may result from nonlinearity of activation functions cannot be captured by the NTK analysis. This is because DNN models optimized by gradient descent are approximated by linear models, speciﬁcally, a linear combination of corresponding NTKs. However, some mechanisms of the implicit regularization can depend on nonlinearity . This leads to the next question:

Linear networks without activation functions are important subject, and there are a number of theoretical works on the implicit regularization in over-parameterized neural networks mainly focusing on linear models (Ji & Telgarsky, 2018; Gidel et al., 2019; Arora et al., 2019a). In contrast, whole properties of over-parameterized DNNs that may result from nonlinearity of activation functions cannot be captured by the approximated linear models, speciﬁcally, the kernel regression predictor using the NTK. However, some mechanisms of the implicit regularization can depend on nonlinearity . This leads to the next question:


Table 1 lists the results of model fitting using all (120) data points. Along with comparing the adjusted R 2 data, we showed the Akaike information criterion ( AIC ) values because of the different number of constants included in the model  [2]. A model with  higher  R 2 and lower AIC was defined as the better model. If the difference between the AIC s was higher than 2, the difference was worth considering, and if is the difference was higher than 10, the difference was consid- ered significant. In experiment 1, we set the starting position of the trials. We excluded Eq. 5 and Eq. 6 from the comparison because the values of B in these models could not be obtained correctly.

Table 1 lists the results of model fitting using all 120 data points. We showed the Akaike information criterion ( AIC ) values because of the different number of constants included in the model along with the adjusted R 2 data [2]. A model with a higher adj. R 2 and lower AIC was defined as the better model. When the difference between the AIC s was higher than 2, the difference was worth considering; when it was higher than 10, it was considered significant. In Experiment 1, we set the starting position of the trials, and we excluded Eqs.and 6 from the comparison because the values of B in these models could not be obtained correctly.

Instruction: Restructured some sentences in this paragraph and merge the last two sentences

Instruction: Improve the linking between phrases.


Pointing (point targets such as buttons or icons) should be fast and accurate. The two main factors the affect the movement time are the target size and the distance from the initial position of the cursor to the target [11,19]. The movement time increases as the distance increases and the target size decreases. Furthermore, placing distrac- tors (which do not hide the cursor) on the path to the target increases the movement time [6,23]. By placing a notch, the user could miss the cursor position  inside the notch or  lose sight of the cursor, which may increase movement time considering the user avoids the notch or moves the mouse cursor carefully near the notch. In experiment 1,  longer movement timeswere recorded because the cursor was hidden by the notch, when pointing a target at the top edge from another target, which was also at the top edge.

Pointing, i e., using the cursor to point at targets such as buttons or icons should be fast and accurate. Two factors that affect the movement time are target size and distance from the initial position of the cursor to the target [11,19]. The movement time increases with an increase in distance and a decrease in target size. Further, placing distractors (which do not hide the cursor) on the path to the target increases the movement time [6,25]. The notch can cause a user to miss the cursor position when it is inside the notch or to lose sight of the cursor, which can increase movement time. Avoiding the notch or moving the cursor carefully around the notch can increase the movement time. We performed three experiments to evaluate the effect of the notch on the movement of the mouse cursor. In Experiment 1, we recorded longer movement times because the cursor was hidden by the notch when moving between targets at the top edge of the display.

Instruction: Improve English in this paragraph. Explain more about the experiments

Instruction: Add a sentence to introduce the experiment. Improve the paragraph for better readability.


Medication entries should have a marker that communicates the time that their reminder will be triggered. The marker should not communicatea time range (as we used in all three designs), but a point in time, such as a minute, when the reminder is triggered. The calendar should have daily summaries. These summaries should be used to give an overview of the list of medications to be administered every day. They should only contain the name of the medication. Their design should be such that the user can decide whether to add or remove them from the display. When dealing with conflicts, emphasis should be placed on theposition of the conflicting entries and not the connectors. If present, lines that connect conflicting entries should have less emphasis. On-calendar conflict representation should not be used as the main indication of an error after a rescheduling activity. The user shouldinstead be notified of the impending conflict beforehand. Participants preferred that normal, dismissible error messages be displayed and show the full information regarding the conflicts being introduced by the action. When rescheduling medication entries, cells that are either safe or unsafe should be communicated to the user by highlighting them using a color that implies either success or warning (e.g., green for the former and yellow for the latter). Although some participants felt that the design should not allow them to schedule an entry in the space that is likely to cause a conflict, there might be situations where this possibility would be desirable. The user should, in this case, be guided on possible moves that will resolve the conflict. This can be done by shading or using an outline for all the cells to which an entry may be rescheduled to resolve the conflict, and letting users configure the amount of warnings and error messages they want to receive.

Medication entries should have markers that communicate the times that their reminders will be triggered. The markers should not communicate time ranges but points in time when reminders are triggered. The calendar should have daily summaries of the list of medications to be administered each day. These summaries should only contain the name of the medication and users should be able to show or hide them. Medication conflicts should be emphasized on the conflicting entries rather than on the connectors. The user should be notified of a newly created conflict upon rescheduling an entry, preferably via dismissible error messages that describe the conflict. When rescheduling medication entries, cells that are either safe or unsafe should be highlighted to the user to guide their action. Although some participants felt that the design should not allow them to schedule an entry in the space that is likely to cause a conflict, there might be situations where this possibility is unavoidable. The user should, in this case, be guided on possible moves that will resolve the conflict. This can be done by shading or using an outline for all the cells to which an entry may be rescheduled to resolve the conflict, and letting users configure the amount of warnings and error messages they want to receive.

Instruction: Heavily remove details from this paragraph to make it more concise.

Instruction: Please condense my paragraph related to medication conflicts.


In practice, we stack multiple bijectives to enable more complex transform. The derivative of the composite can be computed efficiently by applying the chain rule. At inference time,  the inverse mapping f − 1 ( y ) can be computed efficiently (Rezende et al., 2020). To find the solution of f − 1 ( y ) , the first step is to locate the unique bin that contains y . Assuming y belongs to the k -th bin, finding its corresponding x amounts to finding the root of the quadratic equation f k ( x | x k,k +1 , y k,k +1 , δ k,k +1 ) = y in  [ x k , x k +1 ] , whose solution is closed-form.

In practice, we stack multiple bijectives to enable more complex transformation. The derivative of the composite can be computed efficiently using the chain rule. At inference time, we can efficiently compute the inverse mapping f − 1 ( y ) (Rezende et al., 2020): To find the solution of f − 1 ( y ) , the first step is to locate the unique bin that contains y . Assuming y belongs to the k -th bin, finding its corresponding x amounts to finding the root of the quadratic equation f k ( x | x k,k +1 , y k,k +1 , δ k,k +1 ) = y in the interval [ x k , x k +1 ] , for which a closed-form solution exists.

Instruction: Please, review this paragraph, modify only if necessary

Instruction: Improve the English of this paragraph.


Optimal transport defines a powerful geometry to measure the distribution discrepancy. Monge  first formulated optimal transport asa problem of finding an optimal mapping between two measures. However, this formulation cannot guarantee the existence and uniqueness of solutions. More applicable is Kantorovich’s formulation, which can be seen as a generalized Monge problem.

Optimal transport (OT) instantiates distribution discrepancy as the minimum transport cost, which provides a grip for quantifying the treatment selection bias in Figure 1(a). Monge (1781) first formulated OT as finding an optimal mapping between two distributions. However, this formulation cannot guarantee the existence and uniqueness of solutions. Kantorovich (2006) proposed a more applicable formulation in Definition 2.3, which can be seen as a generalization of Monge problem.


Scenes based targets namely traversable surfaces and object openability. On the test set, our (Cache) SIR consistently outperforms multiple baselines (Fig 5) including: Auto Encoder on AI2-THOR images, a Navigation agent within AI2-THOR, and Classiﬁer – a CNN trained to classify objects in

Predictions with synthetic images - We use SIRs to predict geometry and appearance-based targets namely depth, surface normals, object class, object depth and object normals, as well as affordancebased targets namely traversable surfaces and object openability. On the test set, our (Cache) SIR consistently outperforms multiple baselines (Fig 5) including: Auto Encoder on AI2-THOR images, a Navigation agent within AI2-THOR, and Classiﬁer – a CNN trained to classify objects in


We make this assumption for the following reasons. First of all, our ofﬂine learning guarantee (Theorem 3.2) provides simultaneously comparison to all the policies, which is stronger than competing with optimal policy only (whereas relaxed assumption sufﬁces, e.g. sup  x (cid:62) < ∞ (Uehara & Sun, 2021)). As a consequence, the behavior distribution µ must be able to explore each feature dimension for the result to be valid. Second, even if Assumption 2.2 does not hold, we can always restrict our algorithmic design to the effective subspan of Σ ph , which causes the alternative notion of κ := min h ∈ [ H ] { κ h : s.t. κ h > 0 } . In this scenario, learning the optimal policy cannot be guaranteed as a constant suboptimality gap needs to be suffered due to the lack of coverage and this is called assumption-free RL in Yin & Wang (2021b). Lastly, previous works analyzing the linear MDPs impose very similar assumptions, e.g. Xie et al. (2021a) Theorem 3.2 where Σ − 1 D exists and Min et al. (2021) for the OPE problem.

We make this assumption for the following reasons. First of all, our ofﬂine learning guarantee (Theorem 3.2) provides simultaneously comparison to all the policies, which is stronger than competing with optimal policy only (whereas relaxed assumption sufﬁces, e.g. sup x ∈ R d x Σ π(cid:63) x (cid:62) x Σ µ x (cid:62) < ∞ (Uehara and Sun, 2021)). As a consequence, the behavior distribution µ must be able to explore each feature dimension for the result to be valid. Second, even if Assumption 2.2 does not hold, we can always restrict our algorithmic design to the effective subspan of Σ ph , which causes the alternative notion of κ := min h ∈ [ H ] { κ h : s.t. κ h = smallest positive eigenvalue at time h } (see Appendix G.1 for detailed discussions). In this scenario, learning the optimal policy cannot be guaranteed as a constant suboptimality gap needs to be suffered due to the lack of coverage and this is called assumption-free RL in Yin and Wang (2021b). Lastly, previous works analyzing the linear MDPs impose very similar assumptions, e.g. Xie et al. (2021a) Theorem 3.2 where Σ − 1 D exists and Min et al. (2021) for the OPE problem.


In this section we show that neural networks progressively lose their ability to quickly ﬁt new targets when trained on sequential prediction tasks (i.e. settings in which the agent must solve a regression problem that iteratively changes over the course of training) including but not limited to those found in value-based RL. We ﬁnd that capacity loss is particularly pronounced in sparse prediction tasks, where many of the target values the agent seeks to predict are zero. To study the effect of extreme capacity loss on performance in greater depth, we present a special case of the target-ﬁtting capacity measure which is efﬁcient to compute and has the intuitive interpretation of measuring the ability of the representation to linearly disentangle states. We ﬁnd evidence that agents which have greater capacity according to this metric tend to achieve better performance in challenging environments from the Atari suite where agents fail to match human performance, and that those suffering from representation collapse according to this metric fail to make any learning progress at all.

In this section we demonstrate conditions under which neural networks progressively lose their ability to quickly ﬁt new targets when trained on sequences of prediction tasks including but not limited to those found in value-based RL. We ﬁnd that capacity loss is particularly pronounced in sparse prediction tasks, where many of the target values the agent seeks to predict are zero. To study the effect of extreme capacity loss on performance in greater depth, we present a special case of the target-ﬁtting capacity measure which is efﬁcient to compute and has the intuitive interpretation of measuring the ability of the representation to linearly disentangle states. We ﬁnd evidence that agents which have greater capacity according to this metric tend to achieve better performance in challenging environments from the Atari suite where agents fail to match human performance, and that those suffering from representation collapse according to this metric fail to make any learning progress at all.

Instruction: Rewrite the first sentence. Remove the example to make it shorter.

Instruction: Revise the first sentence in a more academic style. Remove unnecessary details.


Image segmentation is a largely researched field where neural networks find vast applications in many facets of technology. Some of the most popular approaches to train segmentation networks employ loss functions optimizing pixel-overlap, an objective that is insufficient for many segmentation tasks. In recent years, their limitations fueled a growing interest in topology-aware methods, which aim to recover the correct topology of the segmented structures. However, so far, none of the existing approaches achieve a spatially correct matching between the topological features (persistence barcodes) of label (ground truth) and prediction (output of the neural network).

Image segmentation is a largely researched field where neural networks find vast applications in many facets of technology. Some of the most popular approaches to train segmentation networks employ loss functions optimizing pixel-overlap, an objective that is insufficient for many segmentation tasks. In recent years, their limitations fueled a growing interest in topology-aware methods, which aim to recover the correct topology of the segmented structures. However, so far, none of the existing approaches achieve a spatially correct matching between the topological features of ground truth and prediction.


Image super-resolution (SR) is a fundamental computer vision task, which aims to recover a high-resolution (HR) image from its low-resolution (LR) counterpart. In general, image SR is ill-posed as a many-to-one mapping problem. To alleviate this problem, plenty of deep convolutional neural networks (CNNs) (Dong et al., 2014; 2016; Kim et al., 2016b; Tai et al., 2017b) have been investigated to achieve accurate mapping from LR image to its HR target.

Image super-resolution (SR), a classic task in computer vision, aims to recover a high-resolution (HR) image based on its low-resolution (LR) counterpart. Essentially, image SR is ill-posed as a many-to-one mapping problem. To tackle this problem, plenty of deep convolutional neural networks (CNNs) (Dong et al., 2014; 2016; Kim et al., 2016b; Zhang et al., 2018c; 2020; 2021) have been investigated to learn the accurate mapping from LR image to its HR target.

Instruction: Replace the citation to (Tai et al., 2017b) with a citation to (Zhang et al., 2018c; 2020; 2021). Improve the english of this paragraph.

Instruction: Make the language of this paragraph a bit more simple.


We gather the non-empty voxels within the query window {+size (e.g. s
 = (2 , 2 , 2) )+} and apply Chessboard Sampling (CBS) to sample the queries. {+key window sizes (e.g. s
 =+} For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and {+s
 = (4 , 4 , 4) ), respectively. Then, we apply CBS to obtain sampled queries, while employ FPS to+} get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale. Keys from windows of different sizes are assigned to differenthead groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

In MsSVT block, we gather the voxels with the query window {+size (e.g. s
 = (2 , 2 , 2) )+} and the {+key window sizes (e.g. s
 =+} (2 , 2 , 2) and {+s
 = (4 , 4 , 4) ), respectively. Then, we apply CBS to obtain sampled queries, while employ FPS to+} get sampled keys and values of same number (e.g. N K = 3 ) among all key windows. Finally, we feed the same query and the different keys into different attention head groups, respectively.


The rotamer density estimator (RDE) is a generative model for  sidechain structures. It can be used to predict sidechain conformations by sampling from the estimated distribution. We use the RDE to sample sidechain torsional angles (rotamers) for structures with  sidechains removed in our test split of PDB-REDO. For each amino acid, 10 rotamers are sampled independently, and the one with the highest probability is selected as the final prediction. We compare RDE with two baseline methods Rosetta (fixbb) (Leman et al., 2020) and SCWRL4 (Krivov et al., 2009). Table 4 shows that  RDE outperforms the baselines on all four torsional angles in terms of angular errors. Detailed per-amino-acid accuracy is presented in Table 11 in the appendix.

RDE is a generative model for protein sidechain structures, which can predict sidechain conformations by sampling from the estimated distribution. We use RDE to sample sidechain torsional angles (rotamers) for structures with 10% sidechains removed in our test split of PDB-REDO. For each residue, 10 rotamers are sampled independently, and the one with the highest probability is selected as the final prediction. We compare RDE with two baseline methods Rosetta (fixbb) (Leman et al., 2020) and SCWRL4 (Krivov et al., 2009). Our results shown in Table 4 demonstrate that the RDE outperforms the baselines on all four torsional angles in terms of angular errors. For a detailed per-amino-acid accuracy, please refer to Table 9 in the appendix.

Instruction: Make this paragraph more fluid.

Instruction: Make the beginning of the paragraph more concise. Make the end of the paragraph more fitting to the academic style.


Besides AT and RND, diverse defenses have also been proposed, and it would be interesting to see their results. DENT [19] optimizes the model in test-time, trying to learn the AE distribution. PNI [51] injects noise during training, making the learned weights less sensitive to input perturbations. TRS [60] ensembles three models with low attack transferability between each other. They are originally developed for gradient-based attacks, but also provide some protection against SQAs. As displayed in Table 3, however, they are not comparable to AAA-linear in real cases regarding the accuracy, calibration, and defense performance. Here, we also test a strong SQA QueryNet, which uses three architecture-alterable models to steal the DNN. Due to its utilization of large-scale testing samples, QueryNet greatly hurts DNNs, but AAA is still the defense that protects the model best.

Besides AT and RND, diverse defenses have also been proposed. DENT [19] optimizes the model in test-time. PNI [54] injects noise during training. TRS [64] ensembles three models with low attack transferability. They are developed for gradient-based attacks, but also provide protection against SQAs. However, seeing Table 3, they are not comparable to AAA-linear in real cases regarding the accuracy, calibration, and defense performance. Here, we also test a strong SQA QueryNet, which uses three architecture-alterable models to steal the DNN. Due to its utilization of large-scale testing samples, QueryNet greatly hurts DNNs, but AAA is still the defense that protects the model best.

Instruction: Remove unnecessary details.

Instruction: Delete unnecessary details, mostly in the two first sentences.


Lightweight Network. First, we revise EDSR baseline (i.e.16 residual blocks) (Lim et al., 2017)  by removing the ﬁnal Conv layer to save parameters. Same as IMDN (Hui et al., 2019), the reconstruction was done within the pixel-shufﬂe layer (Shi et al., 2016). We set the channel number in  revised EDSR baseline as 256 and then prune it to 45. For × 2, we reduce the number of parameters from 19.5M to 609K and name the compressed model as SRPN-L.

Lightweight Networks. First, we adapt the EDSR baseline (Lim et al., 2017) with 16 residual blocks by removing the ﬁnal convolutional layer to save parameters. The reconstruction upscaling is realized by the pixel-shufﬂe layer (Shi et al., 2016) following common practice. We set the channel number in the revised EDSR baseline as 256 and then prune it to 45. For × 2 scale, we reduce the number of parameters from 19.5M to 609K and name the compressed model as SRPN-Lite.

Instruction: Please, remove the clarifications that do are not necessary for the development of the idea:

Instruction: Remove Hui et al. citation and improve writing of the paragraph


We evaluate the performance on seven benchmarks on different  tasks, including text classification, natural language inference, similarity, and paraphrase detection. For NLI, we experiment with the SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018) benchmarks. For text classification, we evaluate on two sentiment analysis datasets, namely IMDB (Maas et al., 2011) and Yelp (YELP) (Zhang et al., 2015). We additionally evaluate on three low-resource datasets in the GLUE benchmark (Wang et al., 2019). These include paraphrase detection using MRPC (Dolan and Brockett, 2005), semantic textual similarity using STS-B (Cer et al., 2017), and textual entailment using RTE (Dagan et al., 2006). We evaluate on the standard validation and test splits. Since the test sets are not available for MNLI, we tune on the matched dev set and evaluate on the mismatched dev set (MNLI-M) or vice versa (see Appendix A for datasets statistics and more details on the experimental setups).

Datasets We evaluate the performance on seven different benchmarks for multiple tasks, in particular text classification, natural language inference, similarity, and paraphrase detection. For NLI, we experiment with two well-known NLI benchmarks, namely SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018). For text classification, we evaluate on two sentiment analysis datasets, namely IMDB (Maas et al., 2011) and Yelp2013 (YELP) (Zhang et al., 2015). We additionally evaluate on three low-resource datasets in the GLUE benchmark (Wang et al., 2019): 3 paraphrase detection using MRPC (Dolan & Brockett, 2005), semantic textual similarity using STS-B (Cer et al., 2017), and textual entailment using RTE (Dagan et al., 2006). For the GLUE benchmark, SNLI, and Yelp, we evaluate on the standard validation and test splits. For MNLI, since the test sets are not available, we tune on the matched dev set and evaluate on the mismatched dev set (MNLI-M) or vice versa. See Appendix A for datasets statistics and Appendix B for hyper-parameters of all methods.


We used  different apparatus for experiments 1 and 2, which did not have a significant effect on the conclusions of this study. We used a desktop PC (Intel Core i9-12900KF, GeForce RTX 3070 Ti, 32GB RAM, Windows 10 Home). The display was manufactured by AOPEN (model 25XV2QFbmiiprx; 24.5” diagonal, 1920 × 1080 pixels) and its refresh rate was set at 360 Hz. We used an optical  mouse, Logitech Gaming Mouse (G300s; 1600 DPI). The mouse- cursor speed via the OS setting was set to the middle of the slider in the control-display and ” Enhance pointer precision ” setting was turned on to match the participant’s usual settings. The experimental system was implemented with Hot Soup Processor 3.6 and used in  full-screen mode.

We used a different apparatus for both experiments; however this did not have a significant effect on the conclusions of this study. We used a desktop PC (Intel Core i9-12900KF, GeForce RTX 3070 Ti, 32 GB RAM, Windows 10 Home). The display was manufactured by AOPEN (model 25XV2QFbmiiprx; 24.5” diagonal, 1920 × 1080 pixels) and its refresh rate was set at 360 Hz. We used an optical mouse (Logitech gaming mouse, G300s; 1600 DPI, and the mouse- cursor speed based on the OS setting was set to the middle of the slider in the control display and the “ Enhance pointer precision ” setting was turned on to match the usual settings of the participant.). The experimental system was implemented with Hot Soup Processor 3.6 and used in the full-screen mode.

Instruction: Improve the English in this paragraph by choosing better words

Instruction: Improve the linking between phrases.


Recently, many methods to induce sparsity in neural networks have shown that it is possible to train models with an overwhelming fraction of the weights being
 Molchanov et al. ; Gale et al. ; Frankle and Carbin (2018); Louizos et al. ; Evci et al. ; Zhu and Gupta (2017). Many of these methods gradually decrease the number of weights in the network through training by using some combination of each weight’s gradient and magnitude. Fine grained sparsity is hard to accelerate on modern hardware, although there have been some recent results demonstrating that speedups are possible Elsen et al. Vecchi et al. (2019) considered inducing sparsity in quaternion networks. Primitives that increase computational density of fundamental interactions would increase the performance of sparse methods as demonstrated on the GPU by Mueller-Roemer et al. in scientiﬁc computing.

Recently, many methods to induce sparsity in neural networks have shown that it is possible to train models with an overwhelming fraction of the weights being
 (Molchanov et al., 2017; Gale et al., 2019; Frankle and Carbin, 2019; Louizos et al., 2018; Evci et al., 2019; Zhu and Gupta, 2017). Many of these methods gradually decrease the number of weights in the network through training by using some combination of each weight’s gradient and magnitude. Fine grained sparsity is hard to accelerate on modern hardware, although there have been some recent results demonstrating that speedups are possible (Elsen et al., 2020). (Vecchi et al., 2019) considered inducing sparsity in quaternion networks. Primitives that increase computational density of fundamental interactions would increase the performance of sparse methods as demonstrated on the GPU by (Mueller-Roemer et al., 2019) in scientiﬁc computing.

Instruction: Review this paragraph, make it easier to read.


Backdoor-based model watermarking relied on an assumption that the trigger matches hidden backdoors contained in the suspicious model. However, the assumption may not hold since the backdoor may be changed during the stealing process. In this section, we verify this limitation.

Backdoor-based model watermarking relied on an assumption that the trigger matches hidden backdoors contained in the suspicious model. However, the assumption may not hold. In this section, we verify this limitation.

Instruction: Remove information on why the assumption might not hold.

Instruction: Make the second sentence much shorter, only keep the main idea.


The explanation is that, for ranking-based loss, the incoming gradient does not point toward achievable solutions, as discussed in Sec. 3.2. For more details see Suppl. B.3. We conclude that for the retrieval experiment, Identity does not match the performance of BB. Presumably, this is because the crude approximation of the permutahedron by a sphere ignores too much structural information.

The explanation is that, for ranking-based loss, the incoming gradient does not point toward achievable solutions, as discussed in Sec. 3.2. For more details, including additional evaluation of other applicable projections, see Suppl. B.3. We conclude that for the retrieval experiment, Identity does not match the performance of BB. Presumably, this is because the crude approximation of the permutahedron by a sphere ignores too much structural information.


Fig. 3a shows that the ﬁrst two components take about 90% of the explained variance. Fig. 3b shows that these components include only τ 1 , avg degree, and modularity. The fact that n is not used means that the size of the graph as well as the density are not important for choosing the best measure. The fact that τ 2 is not used means that the difference in sizes of clusters is not important, too.

Fig. 3a shows that the ﬁrst two components take about 90% of the explained variance. Fig. 3b shows that these components include only τ 1 , avg degree, and modularity. The fact that n is not used means that the size of the graph as well as the density are not of primary importance for choosing the best measure. So is not τ 2 measuring the diversity of cluster sizes.


Therefore,  we seek to predict the change in binding free energy via estimating the change in conformational flexibility . Specifically, our method consists of three main parts. The first part is  a conditional generative model built upon normalizing flows on torus for estimating the density of amino acid sidechain conformations (rotamers) given the structural environment of the amino acid, named Rotamer Density Estimator (RDE). We choose normalizing flows because it allows exact likelihood computation, which is indispensable for the second part of the method — entropy estimation. We estimate the entropy of the rotamer distribution predicted by the model and use it as the measurement of conformational flexibility. Finally, we use the entropy of protein-protein interfaces at different states to predict the change in binding free energy ( ∆∆ G ). In addition to directly using the entropy to predict ∆∆ G , we also use simple neural networks to extract ∆∆ G from the unsupervised representations produced by the RDE.

Therefore, by comparing the entropy losses of wild-type and mutated protein complexes, we can estimate the effect of mutations on binding affinity. Based on this principle, we introduce a novel approach to predict the impact of amino acid mutations on proteinprotein interaction. The core of our method is the Rotamer Density Estimator (RDE), a conditional generative model that estimates the density of amino acid sidechain conformations (rotamers). We use the entropy of the estimated density as a metric of conformational flexibility. By subtracting the entropy of the separated proteins from the entropy of the complex, we obtain an estimation of binding affinity. Finally, we can assess the effect of mutations by comparing the estimated binding affinities of wild-type and mutant protein complexes. In addition to directly comparing entropy, we also employ neural networks to predict ∆∆ G from the representations learned by RDE.


Follow-up studies [12] on Snorkel have also shown that Snorkel users tended to select rules by looking at individual labeled instances and seeking to create rules reﬂecting the patterns they observe in the data. This study further demonstrated that label efﬁciency can be improved by showing users currently unlabeled instances and instances with high label conﬂict, i.e. seeking labels in highly valuable areas of the training data. It also demonstrates that even subject matter experts tend to relyon the data to create their rules, rather than creating them from independent knowledge.

Follow-up studies [12] on Snorkel have also shown that Snorkel users tended to select rules by looking at individual labeled instances and seeking to create rules reﬂecting the patterns they observe in the data. This study further demonstrated that label efﬁciency can be improved by showing users currently unlabeled instances and instances with high label conﬂict, i.e. seeking labels in highly valuable areas of the training data. ReGAL provides the possibility to extend this by automatically proposing rules suited to conﬂicted or unlabeled data slices, thus reducing the manual effort involved.


The only set prediction model we are aware of that can be exclusively multiset-equivariant is DSPN (Zhang et al., 2019; Huang et al., 2020). DESP (Zhang et al., 2021) is exclusively multiset-equivariant but not a standard set predictor. It also uses the  Jacobian of sorting, but has the slightly different goal of diverse sampling by learning set energies without a traditional set loss, so we do not compare against them in the experiments.

The only set prediction model we are aware of that can be exclusively multiset-equivariant is DSPN (Zhang et al., 2019; Huang et al., 2020). Note that Zhang et al. (2021a) also make use of the exclusively multiset-equivariant Jacobian of sorting, but instead focus on learning multimodal densities over sets, which is tangential to our work.

Instruction: Make this paragraph more concise by using more direct formulations

Instruction: Make the paragraph shorter but don't touch at the first sentence.


In this section we discuss how SafeNet can be instantiated in practice. There are two aspects the data owners need to agree upon before instantiating SafeNet: i) The MPC framework used for secure training and prediction phase and ii) the parameters in Theorem 6 to achieve poisoning robustness. The MPC framework is agreed upon by choosing the total number of outsourced servers N participating in the MPC, the number of corrupted servers T and the nature of the adversary (semihonest or malicious in the SOC paradigm). The owners then agree upon a filtering threshold ϕ and the number of poisoned owners t that can be tolerated. Once these parameters are chosen the maximum allowed error probability of the local models trained by the honest owners based on Lemma 5 and

In this section we discuss how SafeNet can be instantiated in practice. There are two aspects the data owners need to agree upon before instantiating SafeNet: i) The MPC framework used for secure training and prediction phase and ii) the parameters in Theorem 6 to achieve poisoning robustness. The owners agree upon the number of outsourced servers N participating in the MPC, the number of corrupted servers T along with the role of the adversary (semi-honest or malicious) in the MPC and consequently choose an appropriate training framework that satisfies this criteria. The owners then agree upon a filtering threshold ϕ and the number of poisoned owners t that can be tolerated. Once these parameters are chosen the maximum allowed error probability of the local models trained by the honest owners based on Lemma 5 and

Instruction: Rewrite the middle sentence of this paragraph to make it clearer.

Instruction: Rewrite the long sentence in the middle sentence to improve clarity.


TERGM models estimated within the markov chain assumption are typically incapable of generating and reproducing realistic dynamics observed in real-world online social networks. We hypothesized that increasing the model’s capacity to describe triadic network properties would reduce the error between the model and empirical observations. We propose TTERGM here to sequentially predict network probabilities by integrating the dynamics between influencers and followers (Code isavailable at https://github.com/alvin68633466/TTERGM-Social-Theory-Driven-network-simulation). TTERGM was run on a computer with 12900K CPU, 1080TI and 128GB RAM. Figure 2 shows the framework of TTERGM that has five major components - data collection module, network processing module, feature extraction module, pattern analysis module, and a generative network module. The data collection module is discussed in Section 3.3.

TERGM models estimated within the markov chain assumption are typically incapable of generating and reproducing realistic dynamics observed in real-world online social networks. We hypothesized that increasing the model’s capacity to describe triadic network properties would reduce the error between the model and empirical observations. We propose TTERGM here to sequentially predict network probabilities by integrating the dynamics between influencers and followers. TTERGM was run on a computer with 12900K CPU, 1080TI and 128GB RAM. Figure 2 shows the framework of TTERGM that has five major components - data collection module, network processing module, feature extraction module, pattern analysis module, and a generative network module.

Instruction: Remove the information about the code. And remove the last sentence.

Instruction: Remove the mentions to the code and to other sections.


Low-level image similarity. We propagatethe labels within visually coherent regions,as visual similarity often goes with semantic similarity. To generate an over-segmentation, we follow Hwang et al. by using the HED contour detector (Xie & Tu, 2015) (pre-trained on BSDS dataset (Arbelaez et al., 2010)) and the procedure in gPb-owt-ucm (Arbelaez et al., 2010). Such bottom-up segmentation techniques consider both local and global appearance afﬁnity without semantic information. Some of the segments could contain pixels from different categories. For learning pixel embedding at pixel i , we deﬁne  positive and negative segments w.r.t. i as i ’s own segment and all the other segments, denoted as V + and V − , respectively. Such a formulation helps learn embeddings that respect low-level visual cues. In our implementation, we only consider segments in the same image as  i ’s image as positive and negative samples. We follow SegSort Hwang et al. (2019) to align the contour-based over-segmentations with segmentations generated by K-Means clustering  in SegSort.

Low-level image similarity. To propagate labels within visually coherent regions, we generate a low-level over-segmentation. Following SegSort Hwang et al. (2019), we use the HED contour detector (Xie & Tu, 2015) (pre-trained on BSDS500 dataset (Arbelaez et al., 2010)) and gPb-owtucm (Arbelaez et al., 2010) to generate a segmentation without semantic information. We deﬁne i ’s positive and negative segments as i ’s own segment and all the other segments, denoted as V + and V − respectively. We only consider segments in the same image as pixel i ’s. We align the contour-based over-segmentations with segmentations generated by K-Means clustering as in SegSort.

Instruction: Make this paragraph considerably more concise. Remove any unnecessary details that are not essential for the main point of the paragraph.

Instruction: This paragraph is too long, make it almost 50% shorter but keep the important informations.


Neural Network Predictor The hidden representation h i of each amino acid for parameterizing the normalizing flows contains sufficient information about the rotamer probability density. To extract binding information from the representations in a more flexible way, we also try neural networks. Specifically, we use MLPs to transform the representation h i and apply max-pooling to obtain a global representation of the structure. Then, we subtract the representation of the wild-type structure from the representation of the mutant  and feed it to another MLP to predict ∆∆ G . To enforce anti-symmetry, we swap the wild-type and the mutant to predict − ∆∆ G  and take (∆∆ G − ( − ∆∆ G )) / 2 as the final prediction. The network is trained with the MSE loss. During training, gradients are not back-propagated through h i so the rotamer density estimator is frozen, as our goal is to  exploit the unsupervised representations learned by the RDE.

Neural Network Predictor Each residue’s hidden representation h i used to parameterize the normalizing flows contains sufficient information about the rotamer distribution. To extract binding information from these representations in a more flexible way, we employed neural networks. Specifically, we utilized a network that shares the same architecture as the encoder to transform the representation h i and applied max-pooling to obtain a global structure representation. We then subtracted the representation of the wild-type structure from the mutant representation and fed it into another MLP to predict ∆∆ G . To enforce anti-symmetry, we swapped the wild-type and mutant to predict − ∆∆ G , and computed (∆∆ G − ( − ∆∆ G )) / 2 as the final prediction. The network was trained using the MSE loss. During training, we freeze the RDE weights and do not back-propagate gradients through h i to fully exploit the unsupervised representations learned by RDE.


Remark 3 . The major novelty of obtaining these theoretical results relies on the developed variant of the augmented Lagrangian function and subsequently derived recursion of the successive dual variables, which quantify well the consensus errors resulting from both UL and LL optimization processes in terms of primal variables. Note that this is distinct from the existing theoretical analysis of stochastic algorithms [38, 40, 41], even including stochastic primal-dual algorithms [42, 45], and also the classical deterministic primal-dual algorithms [46] for decentralized optimization.

Remark 3 . The major novelty of obtaining these theoretical results relies on the developed variant of the augmented Lagrangian function and subsequently derived recursion of the successive dual variables, which quantify well the consensus errors resulting from both UL and LL optimization processes in terms of primal variables. Note that this is distinct from the existing theoretical analysis of stochastic algorithms, such as distributed SGD [38, 39], gradient tracking [40, 41], stochastic primal-dual algorithms [42, 45], etc.


Update criteria As mentioned before, Algorithm 1 runs in epochs indexed by j , and one epoch ends when either of the two update criteria is triggered (Line 9). The ﬁrst updating criterion is satisﬁed once the determinant of Σ t is doubled compared to the determinant at the end of the previous epoch. This is called lazy policy update that has been used in the linear bandits and RL literature (Abbasi-Yadkori et al., 2011; Zhou et al., 2021b), which reﬂects the diminishing return of learning the underlying transition. Moreover, this update criterion reduces the computational cost as the total number of epochs would be bounded by O (log T ) . The doubling visitation criterion used in tabular

Update criteria As mentioned before, Algorithm 1 runs in epochs indexed by j , and one epoch ends when either of the two update criteria is triggered (Line 9). The ﬁrst updating criterion is satisﬁed once the determinant of Σ t is doubled compared to the determinant at the end of the previous epoch. This is called lazy policy update that has been used in the linear bandits and RL literature (Abbasi-Yadkori et al., 2011; Zhou et al., 2021b), which reﬂects the diminishing return of learning the underlying transition. One intuition behind the determinant doubling criterion is that the determinant can be viewed as a surrogate measure of the exploration in the feature space. Thus one only updates the policy when enough further exploration has been made. Moreover, this update criterion reduces the computational cost as the total number of epochs would be bounded by O (log T ) . Here T denotes the total number of steps through all K episodes. The doubling visitation criterion used in tabular


We used a general cursor in allof our experiments. However, pointing-facilitation technique have been proposed to improve the pointing performance. For example,when using Bubble Cursor [13] or Ninja Cursors [18], the notch effect for the movement time may be reduced. However, we did not consider them in this study because of the somewhat tricky behavior of these techniques. In this study, we only considered the movement time and error rate for evaluation. However, placing the notch can increase psychological stress for users. For example, if the notch is changed to an area where the cursor cannot enter, the user may feel uncomfortable about the cursor unnecessarily getting caught in the notch. Therefore, the limitation of this study is that we only investigated the effect of the notch for movement time and error rate.

Bubble Clusters [26] and Attribute Gates [24]). We are interested in whether Eq. 8 can be applied to more than just predicting movement time in the scenario where the notch is placed. We used a general cursor in all our experiments. However, pointing-facilitation technique have been proposed to improve the pointing performance. For example, the notch effect for the movement time may be reduced when using Bubble Cursor [13] or Ninja Cursors [18]. However, we did not consider them in this study because of the somewhat tricky behavior of these techniques. We instructed the participants to avoid any clutching action during the trial; however, in actual use of the PC, the user may perform clutching actions. For example, if the cursor is hidden by a notch during a clutching action, the effect of the notch may be increased. Restricting the clutch action may have restricted the user’s operation strategy, which is a limitation of this study. In this study, we only considered the movement time and error rate for evaluation. However, placing the notch can increase the psy- chological stress experienced by users. For example, the user may feel uncomfortable about the cursor unnecessarily getting caught in the notch if the notch is changed to an area where the cursor cannot enter. Therefore, we only investigated the effect of the notch for movement time and error rate, which is another limitation.


The user model takes as input the user information(i.e. , a concatenation of user attributes and a sequence of the user interactions) and a set of item embeddings in the list. The user information is passed on to a single layer gated recurrent network(GRU (Cho et al., 2014)) followed by a 2-layer MLP to extract the compact representation of the state. Also, the set of item embeddings is processed by the same architecture of the GRU network(this and the one before are not shared and initialized differently) as the list-embedding. Finally, a 2-layer MLP takes as input the concatenation of those two embeddings(i.e. , state-embedding and list-embedding) and provides the scores of items in the list followed by the sigmoid function to transform to the individual click likelihood.

The user model takes as input the user information(i.e. , a concatenation of user attributes and a sequence of the user interactions) and a set of item embeddings in the list. The user information is passed on to a single layer gated recurrent network(GRU (Cho et al., 2014)) followed by a 2-layer MLP to extract the compact representation of the state. The same GRU network architecture (with different weights) processes the set of item embeddings into a list-embedding. Finally, a 2-layer MLP takes as input the concatenation of those two embeddings(i.e. , state-embedding and list-embedding) and provides the scores of items in the list followed by the sigmoid function to transform to the individual click likelihood.

Instruction: Make the third sentence shorter and easier to understand

Instruction: Simplify the convoluted sentences to make the paragraph more concise.


Temporal logic (TL)-guided policy learning is an area that we take have taken inspiration from. In this area, TL is often used to specify the ego agent’s desired high-level behavior and used to generaterewards. The authors of [17, 18, 19] provide surveys of recent work on the use of TL in RL. The exploration problem still exists in these methods. The authors of [20, 21] learns ﬁnite state automata from demonstration and use them to guide planning with the value iteration network which avoids exploration. It can sometimes be tedious to manually design a TL formula that yields satisfying behaviors. Work has been done to make components of the formula learnable from data. In [22]learns linear temporal logic (LTL) formulas from demonstrations. Given the close relationship between TL and automaton[23], the authors of [24] proposes a method that learns reward machines (an automata-like reward presentation) from demonstrations. A commonality to these methods is that the LTL that they use operate on propositions (binary variables with values true or false). Therefore,either high-level demonstrations (proposition traces) or a mapping from low-level states to propositions is required. This also limits these methods to operate on discreteand ﬁnite state and actionspaces. Our method works with STL which operates on continuous signals, therefore, we are able tolearn from continuous demonstration trajectories which is more readily available in robotic systems. The LogicRiskNet that we propose is also learnable from demonstrations.

Temporal logic (TL)-guided policy learning is an area that we take have taken inspiration from. In this area, TL is often used to specify the ego agent’s desired high-level behavior and used togenerate rewards. The authors of [18, 19, 20] provide surveys of recent work on the use of TL in RL. The exploration problem still exists in these methods. To address these challenges, the authorsof [21, 22] learn ﬁnite state automata from demonstration and use them to guide planning with the value iteration network which avoids exploration. It can sometimes be tedious to manually design a TL formula that yields satisfying behaviors. Work has been done to make components of the formula learnable from data. In [23], the authors propose learning linear temporal logic (LTL) formulas from demonstrations. Given the close relationship between TL and automaton[24], theauthors of [25] propose a method that learns reward machines (an automata-like reward presentation) from demonstrations. A shortcoming of these methods is that the LTL that they use operates onpropositions (binary variables with values true or false). Unlike our STL-based approach, theseapproaches require discrete state and action spaces and require the demonstrations themselves tohave the same discrete representations.

Instruction: Make this paragraph easier to read, remove unnecessary details if needed

Instruction: Summarize the last third of this paragraph in one sentence. Smooth out the writing.


We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig. Noted that weset both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ )  to compress the 3Dvoxels into a 2D feature map, where the query is the average  voxel features within the pillar window.

As shown in Fig 2, we build our 3D backbone by stacking Mixed-scale Sparse Voxel Transformer(MsSVT) blocks. It is worth noting that both the query and key window size in the last block of MsSVT are set to (1 , 1 , ∞ ) so as to compressing the 3D voxels into 2D feature map, where the queryis the average of all the voxel features within the pillar window.

Instruction: Improve the English of this paragraph.


This work studied the distillation of NN-based deep reinforcement learning agents into symbolic policies for performance-oriented congestion control in TCP. Our branched symbolic frameworkenjoys better simplicity and efﬁciency while exhibiting comparable and often improved performancesover their black-box teacher counterparts on both simulation and rigorous emulation testbeds. Ourresults point towards a fresh direction to make congestion control extremely light-weight and simpler,via a symbolic design. Our future work aims for more integrated neural-symbolic solutions and fastermodel-free online training/ﬁne-tuning for performance-oriented congestion control. Exploring thefairness between our learned CC and legacy CC is also an interesting next step. Besides, we also aimto apply symbolic distillation to a wider range of systems and networking problems.

This work studies the distillation of NN-based deep reinforcement learning agents into symbolic policies for performance-oriented congestion control in TCP. Our branched symbolic framework has better simplicity and efficiency while exhibiting comparable and often improved performance over their black-box teacher counterparts on both simulation and emulation environments. Our results point towards a fresh direction to make congestion control extremely light-weight, via a symbolic design. Our future work aims at more integrated neurosymbolic solutions and faster model-free online training/fine-tuning for performance-oriented congestion control. Exploring the fairness of neurosymbolic congestion control is also an interesting next step. Besides, we also aim to apply symbolic distillation to a wider range of systems and networking problems.

Instruction: Rewrite the second-to-last sentence to make it more general and shorten some formulations.


Let { A n , b n } be the dynamics parameters associated with node n . Even though only the  discrete states are associated with the leaf nodes, we will introduce dynamics at the internal nodes as well. These internal dynamics serve as a link between the leaf node dynamics via a hierarchical prior,

Let { A n , b n } be the dynamics parameters associated with node n . Although the locally linear dynamics of a discrete state are speciﬁed by the leaf nodes, we introduce dynamics at the internal nodes as well. These internal dynamics serve as a link between the leaf node dynamics via a hierarchical prior,


Table 1 summarizes the results, comparing the cost-sensitive robust error between the baseline model trained for overall robustness and a model trained using our cost-sensitive robust optimization. The proposed cost-sensitive robust defense model is trained with (cid:15) = 0 . 2 based on loss function (3.1) and the corresponding cost matrix C . The regularization parameter α is tuned via cross validation (see Appendix A in the supplementary materials for details). We report the selected best α , classiﬁcation error and cost-sensitive robust error on the testing dataset.

Table 1 summarizes the results, comparing the cost-sensitive robust error between the baseline model trained for overall robustness and a model trained using our cost-sensitive robust optimization. The proposed cost-sensitive robust defense model is trained with (cid:15) = 0 . 2 based on loss function (3.2) and the corresponding cost matrix C (see Appendix B.2 for comparison results with different (cid:15) ). The regularization parameter α is tuned via cross validation (see Appendix A in the supplementary materials for details). We report the selected best α , classiﬁcation error and cost-sensitive robust error on the testing dataset.


Cole & Warwicker, 2002; Kastritis & Bonvin, 2013). Intuitively speaking, amino acids on the interface  become less flexible (lower entropy) after they get contact with other proteins due to the geometrical and physical restraints imposed by the binding partner (Figure 1). Higher binding affinity implies better complementarity between the two parts, thus higher rigidity.

Cole & Warwicker, 2002; Kastritis & Bonvin, 2013). When two proteins bind, the residues located at the interface tend to become less flexible (i.e. having lower entropy) due to the physical and geometric constraints imposed by the binding partner (Figure 1). A higher amount of entropy loss corresponds to a stronger binding affinity.


Trigger Feature Hypothesis We hypothesize that the trigger features are sparsely encoded in only afew channels, while clean image features need to be encoded across many channels for the effectiveclassification . This is a key difference from normal data features that are presumably distributed moreevenly across channels, which indicates that these two types of features might behave differently incertain situations, leading to our main technical contribution. More illustrations of trigger featuresare provided in App. A3.

Trigger Feature Hypothesis We hypothesize that the trigger features are sparsely encoded in only afew channels, while clean image features need to be encoded across many channels . This is a keydifference from normal data features that are presumably distributed more evenly across channels,which indicates that these two types of features might behave differently in certain situations, leadingto our main technical contribution. Illustrations of triggers are provided in App. A3.

Instruction: Make this paragraph more concise.

Instruction: Make this paragraph a bit shorter.