wanyu/IteraTeR_full_sent huggingface dataset:
Original:  Our algorithm runs in time polynomial in the smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator. Empirically, GANs trained using our algorithm consistently learn a greater number of modes than gradient descent-ascent (GDA), optimistic mirror descent (OMD), andunrolled GANs when applied to a synthetic Gaussian mixture dataset. Moreover, they perform significantly better on CIFAR-10 than OMD and GDA when comparing the mean and standard deviation of the Inception Score respectively .
Revised:  Our algorithm runs in time polynomial in the smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that arise in GAN training. Our algorithm converges on these test functions and, when used to train GANs on synthetic and real-world datasets, trains stably and avoids mode collapse .
Edit intent: clarity
Original: Machine learning becomes increasingly important to tune or even synthesize the behavior of safety-critical components in highly non-trivial environments, where the inability to understand learned components in general, and neural nets in particular, poses serious obstacles to their adoption.
Revised: Machine learning becomes increasingly important to control the behavior of safety-critical components in highly non-trivial environments, where the inability to understand learned components in general, and neural nets in particular, poses serious obstacles to their adoption.
Edit intent: clarity
Original: Machine learning becomes increasingly important to tune or even synthesize the behavior of safety-critical components in highly non-trivial environments, where the inability to understand learned components in general, and neural nets in particular, poses serious obstacles to their adoption.
Revised: Machine learning becomes increasingly important to tune or even synthesize the behavior of safety and financially critical components in sophisticated environments, where the inability to understand learned components in general, and neural nets in particular, poses serious obstacles to their adoption.
Edit intent: clarity
Original: Self-attention architectures, which are rapidly pushing the frontier in natural language processing, demonstrate a surprising depth-inefficient behavior: Empirical signals indicate that increasing the internal representation (network width) is just as useful as increasing the number of self-attention layers (network depth).
Revised: Self-attention architectures, which are rapidly pushing the frontier in natural language processing, demonstrate a surprising depth-inefficient behavior: previous works indicate that increasing the internal representation (network width) is just as useful as increasing the number of self-attention layers (network depth).
Edit intent: clarity
Original:  In this paper, we theoretically study the interplay between depth and width in self-attention , and shed light on the root of the above phenomenon. We invalidate the seemingly plausible hypothesis by which widening is as effective as deepening for self-attention , and show that in fact stacking self-attention layers is so effective that it quickly saturates a capacity of the network width.
Revised:  In this paper, we theoretically study the interplay between depth and width in self-attention . We conduct systematic empirical ablations on networks of depths 6 to 48 that clearly reveal the theoretically predicted behaviors, and provide explicit quantitative suggestions regarding the optimal depth-to-width allocation for a given self-attention , and show that in fact stacking self-attention layers is so effective that it quickly saturates a capacity of the network width.
Edit intent: meaning-changed
Original:  We invalidate the seemingly plausible hypothesis by which widening is as effective as deepening for self-attention , and show that in fact stacking self-attention layers is so effective that it quickly saturates a capacity of the network width.
Revised:  We invalidate the seemingly plausible hypothesis by which widening is as effective as deepening for self-attention network size. The race towards beyond 1-Trillion parameter language models renders informed guidelines for increasing self-attention layers is so effective that it quickly saturates a capacity of the network width.
Edit intent: meaning-changed
Original:  We invalidate the seemingly plausible hypothesis by which widening is as effective as deepening for self-attention , and show that in fact stacking self-attention layers is so effective that it quickly saturates a capacity of the network width. Specifically, we pinpoint a "depth threshold" that is logarithmic in d_x, the network width: L_{\textrm{th self-attention operation, while for depths over the threshold we show that depth-inefficiency kicks in . Our predictions strongly accord with extensive empirical ablations in Kaplan et al. (2020), accounting for the different behaviors in the two depth-(in)efficiency regimes.
Revised:  We invalidate the seemingly plausible hypothesis by which widening is as effective as deepening for self-attention , and show that in fact stacking self-attention depth and width in tandem an essential ingredient. Our guidelines elucidate the depth-to-width trade-off in self-attention operation, while for depths over the threshold we show that depth-inefficiency kicks in . Our predictions strongly accord with extensive empirical ablations in Kaplan et al. (2020), accounting for the different behaviors in the two depth-(in)efficiency regimes.
Edit intent: clarity
Original:  Specifically, we pinpoint a "depth threshold" that is logarithmic in d_x, the network width: L_{\textrm{th self-attention operation, while for depths over the threshold we show that depth-inefficiency kicks in . Our predictions strongly accord with extensive empirical ablations in Kaplan et al. (2020), accounting for the different behaviors in the two depth-(in)efficiency regimes. By identifying network width as a limiting factor, our analysis indicates that solutions for dramatically increasing the width can facilitate the next leap in self-attention expressivity .
Revised:  Specifically, we pinpoint a "depth threshold" that is logarithmic in d_x, the network width: L_{\textrm{th self-attention networks of sizes up to the scale of GPT3 (which is too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter self-attention expressivity .
Edit intent: meaning-changed
Original:  Our guidelines elucidate the depth-to-width trade-off in self-attention networks of sizes up to the scale of GPT3 (which is too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter self-attention network.
Revised:  Our guidelines elucidate the depth-to-width trade-off in self-attention networks of sizes up to the scale of GPT3 (which we project to be too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter self-attention network.
Edit intent: clarity
Original:  Our guidelines elucidate the depth-to-width trade-off in self-attention networks of sizes up to the scale of GPT3 (which is too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter self-attention network.
Revised:  Our guidelines elucidate the depth-to-width trade-off in self-attention networks of sizes up to the scale of GPT3 (which is too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter network.
Edit intent: clarity
Original:  Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 4,002 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial, individual estimates , i. e., the estimation context.
Revised:  Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 2,885 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial, individual estimates , i. e., the estimation context.
Edit intent: meaning-changed
Original:  Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 4,002 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial, individual estimates , i. e., the estimation context. Specifically, we propose a feature of the estimation context that measures the suitability of the crowd to benefit from influence centralization and show its significant predictive powers empirically.
Revised:  Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 4,002 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial, individual estimates .
Edit intent: coherence
Original:  By adopting a framework that integrates both the structure of the social influence and the estimation context , we bring previously conflicting results under one theoretical framework and clarify the effects of social influence on the wisdom of crowds.
Revised:  By adopting a framework that integrates both the structure of the social influence and the distribution of the initial estimates , we bring previously conflicting results under one theoretical framework and clarify the effects of social influence on the wisdom of crowds.
Edit intent: clarity
Original:  In this work we propose to solve these problems using inexact derivative-free optimization algorithms which never require to solve the lower-level problem exactly. We provide global convergence and worst-case complexity analysis of our approach, and test our proposed framework on ROF-denoising and learning MRI sampling patterns.
Revised:  In this work we propose to solve these problems using inexact derivative-free optimization algorithms which never require to solve the lower-level problem solutions, but instead assume access to approximate solutions with controllable accuracy, which is achievable in practice. We prove global convergence and worst-case complexity analysis of our approach, and test our proposed framework on ROF-denoising and learning MRI sampling patterns.
Edit intent: meaning-changed
Original: Over the last decades, an impressive amount of non-linearities have been proposed to reconcile reduced-form macroeconomic models with the data. Many of them boil down to have linear regression coefficients evolving through time: threshold/switching/smooth-transition regression; structural breaks and random walk time-varying parameters. While all of these schemes are reasonably plausible in isolation, I argue that those are much more in agreement with the data if they are combined. To this end, I propose Macroeconomic Random Forests, which adapts the canonical Machine Learning (ML) algorithm to the problem of flexibly modeling evolving parameters in a linear macro equation.
Revised: I develop Macroeconomic Random Forest (MRF), an algorithm adapting the canonical Machine Learning (ML) algorithm to the problem of flexibly modeling evolving parameters in a linear macro equation.
Edit intent: clarity
Original:  The approach exhibits clear forecasting gains over a wide range of alternativesand successfully predicts the drastic 2008  rise in unemployment .
Revised:  Its main output, Generalized Time-Varying Parameters (GTVPs), is a versatile device nesting many popular nonlinearities (threshold/switching, smooth transition, structural breaks/change) and allowing for sophisticated new ones. The approach delivers clear forecasting gains over a wide range of alternativesand successfully predicts the drastic 2008 rise in unemployment .
Edit intent: meaning-changed
Original:  The approach exhibits clear forecasting gains over a wide range of alternativesand successfully predicts the drastic 2008  rise in unemployment . The obtained generalized time-varying parameters (GTVPs) are shown to behave differently compared to random walk coefficients by adapting nicely to the problem at hand, whether it is regime-switching behavior or long-run structural change. By dividing the typical ML interpretation burden into looking at each TVP separately, I find that the resulting forecasts are, in fact, quite interpretable. An application to the US Phillips curve reveals it is probably not flattening the way you think .
Revised:  The approach exhibits clear forecasting gains over a wide range of alternativesand successfully predicts the drastic 2008 rise in unemployment , and performs well for inflation. Unlike most ML-based methods, MRF is directly interpretable -- via its GTVPs. For instance, the successful unemployment forecast is due to the influence of forward-looking variables (e.g., term spreads, housing starts) nearly doubling before every recession. Interestingly, the Phillips curve has indeed flattened, and its might is highly cyclical .
Edit intent: meaning-changed
Original: We present three provably accurate, polynomial time, approximation algorithms for the Sparse Principal Component Analysis (SPCA ) problem, without imposing any restrictive assumptions on the input covariance matrix.
Revised: Principal component analysis (PCA) is a widely used dimension reduction technique in machine learning and multivariate statistics. To improve the interpretability of PCA, various approaches to obtain sparse principal direction loadings have been proposed, which are termed Sparse Principal Component Analysis (SPCA). In this paper, we present thresholding as a provably accurate, polynomial time, approximation algorithms for the Sparse Principal Component Analysis (SPCA ) problem, without imposing any restrictive assumptions on the input covariance matrix.
Edit intent: meaning-changed
Original:  The first algorithm is based on randomized matrix multiplication; the second algorithm is based on a novel deterministic thresholding scheme; and the third algorithm is based on a semidefinite programming relaxation of SPCA.
Revised:  The first algorithm is faster than current state-of-the-art; and performs well in practice. On the negative side, our (novel) theoretical bounds do not accurately predict the strong practical performance of this approach. The second algorithm solves a well-known semidefinite programming relaxation of SPCA.
Edit intent: meaning-changed
Original:  and the third algorithm is based on a semidefinite programming relaxation of SPCA. All algorithms come with provable guarantees and run in low-degree polynomial time. Our empirical evaluations confirm our theoretical findings .
Revised:  and the third algorithm is based on a semidefinite programming relaxation and then uses a novel, two step, deterministic thresholding scheme to compute a sparse principal vector. It works very well in practice and, remarkably, this solid practical performance is accurately predicted by our theoretical bounds, which bridge the theory-practice gap better than current state-of-the-art .
Edit intent: meaning-changed
Original:  We develop the annotation task based on a pilot annotation exercise on 50 NLP-ML scholarly articles presenting contributions to the five information extraction tasks 1. machine translation, 2. named entity recognition, 3. question answering, 4. relation classification, and 5. text classification.
Revised:  We develop the annotation task based on a pilot annotation exercise on 50 NLP-ML scholarly articles presenting contributions to five information extraction tasks 1. machine translation, 2. named entity recognition, 3. question answering, 4. relation classification, and 5. text classification.
Edit intent: fluency
Original:  The overarching goal of our endeavor is  : 1) to find a systematic set of patterns of subject-predicate-object statements for the semantic structuring of scholarly contributions that are more or less generically applicable for NLP-ML research articles;
Revised:  The overarching goal of our endeavor is four-fold : 1) to find a systematic set of patterns of subject-predicate-object statements for the semantic structuring of scholarly contributions that are more or less generically applicable for NLP-ML research articles;
Edit intent: meaning-changed
Original:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Revised:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Edit intent: fluency
Original:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Revised:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifiers for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Edit intent: fluency
Original:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Revised:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis and study several different realizations. Among them, we propose a systematic new approach to combining multiple predictions based on a dedicated neural network and develop mathematical analysis of it along with state-of-the-art experimental results. We also propose a heuristic-hybrid technique for combining models and back it up with experimental results on a representative benchmark dataset and comparisons to other methods1show the advantages of the new network architecture .
Edit intent: meaning-changed
Original:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Revised:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods to show the advantages of the new network architecture .
Edit intent: fluency
Original:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture .
Revised:  We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis . Experimental resultson representative benchmark datasets and comparisons to other methods1show the advantages of the new approaches .
Edit intent: clarity
Original:  We develop a method to study coherency and completeness conditions for linear dynamic forward-looking rational expectations models under an occasionally binding constraint. In the context of the simple New Keynesian model with a zero lower bound, this method shows that the coherency and completeness condition generally violates the Taylor principle. Rational expectations require time-varying and correlated support restrictions on the distribution of the structural shocks.
Revised:  We show that models with occasionally binding constraints are not generically coherent. Coherency requires restrictions on the parameters or on the support of the distribution of the structural shocks.
Edit intent: clarity
Original:  Rational expectations require time-varying and correlated support restrictions on the distribution of the structural shocks. With appropriate restrictions , a very large number of equilibria can be supported .
Revised:  Rational expectations require time-varying and correlated support restrictions on the distribution of the shocks. These restrictions are rarely explicitly acknowledged or imposed in the literature. Models whose coherency is based on support restrictions are generically incomplete, admitting a very large number of equilibria can be supported .
Edit intent: meaning-changed
Original: The Negative Pedal Curve (NPC) of the Ellipse with respect to a boundary point M is a 3-cusp deltoid which is the affine image of the Steiner Curve .
Revised: The Negative Pedal Curve (NPC) of the Ellipse with respect to a boundary point M is a 3-cusp closed-curve which is the affine image of the Steiner Curve .
Edit intent: meaning-changed
Original: The Negative Pedal Curve (NPC) of the Ellipse with respect to a boundary point M is a 3-cusp deltoid which is the affine image of the Steiner Curve .
Revised: The Negative Pedal Curve (NPC) of the Ellipse with respect to a boundary point M is a 3-cusp deltoid which is the affine image of the Steiner Deltoid .
Edit intent: fluency
Original:  what should the robot do when the human correction cannot be explained by the featuresit already has access to? Recent progress in deep Inverse Reinforcement Learning (IRL) suggests that the robot could fall back on demonstrations: ask the human for demonstrations of the task , and recover a reward defined over not just the known features, but also the raw state space.
Revised:  what should the robot do when the human correction cannot be explained by these features, recent work in deep Inverse Reinforcement Learning (IRL) suggests that the robot could fall back on demonstrations: ask the human for demonstrations of the task , and recover a reward defined over not just the known features, but also the raw state space.
Edit intent: clarity