ali5341/scitldr-chat-format huggingface dataset:
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Theoretical properties of the global optimizer of two-layer Neural Network

Paper content:
In this paper, we study the problem of optimizing a two-layer artificial neural network that best fits a training dataset.
We look at this problem in the setting where the number of parameters is greater than the number of sampled points.
We show that for a wide class of differentiable activation functions (this class involves most nonlinear functions and excludes piecewise linear functions), we have that arbitrary first-order optimal solutions satisfy global optimality provided the hidden layer is non-singular.
We essentially show that these non-singular hidden layer matrix satisfy a ``"good" property for these big class of activation functions.
Techniques involved in proving this result inspire us to look at a new algorithmic, where in between two gradient step of hidden layer, we add a stochastic gradient descent (SGD) step of the output layer.
In this new algorithmic framework, we extend our earlier result and show that for all finite iterations the hidden layer satisfies the``good" property mentioned earlier therefore partially explaining success of noisy gradient methods and addressing the issue of data independency of our earlier result.
Both of these results are easily extended to hidden layers given by a flat matrix from that of a square matrix.
Results are applicable even if network has more than one hidden layer provided all inner hidden layers are arbitrary, satisfy non-singularity, all activations are from the given class of differentiable functions and optimization is only with respect to the outermost hidden layer.
Separately, we also study the smoothness properties of the objective function and show that it is actually Lipschitz smooth, i.e., its gradients do not change sharply.
We use smoothness properties to guarantee asymptotic convergence of $O(1/\text{number of iterations})$ to a first-order optimal solution.

assistant: text: This paper talks about theoretical properties of first-order optimal point of two layer neural network in over-parametrized case
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: DONUT: CTC-based Query-by-Example Keyword Spotting

Paper content:
Keyword spotting—or wakeword detection—is an essential feature for hands-free operation of modern voice-controlled devices.
With such devices becoming ubiquitous, users might want to choose a personalized custom wakeword.
In this work, we present DONUT, a CTC-based algorithm for online query-by-example keyword spotting that enables custom wakeword detection.
The algorithm works by recording a small number of training examples from the user, generating a set of label sequence hypotheses from these training examples, and detecting the wakeword by aggregating the scores of all the hypotheses given a new audio recording.
Our method combines the generalization and interpretability of CTC-based keyword spotting with the user-adaptation and convenience of a conventional query-by-example system.
DONUT has low computational requirements and is well-suited for both learning and inference on embedded systems without requiring private user data to be uploaded to the cloud.
In this paper, we proposed DONUT, an efficient algorithm for online query-by-example keyword spotting using CTC.
The algorithm learns a list of hypothetical label sequences from the user's speech during enrollment and uses these hypotheses to score audios at test time.
We showed that the model is interpretable, and thus easy to inspect, debug, and tweak, yet at the same time has high accuracy.Because training a wakeword model amounts to a simple beam search, it is possible to train a model on the user's device without uploading a user's private voice data to the cloud.Our technique is in principle applicable to any domain in which a user would like to teach a system to recognize a sequence of events, such as a melody (a sequence of musical notes) or a gesture (a sequence of hand movements).
It would be interesting to see how well the proposed technique transfers to these other domains.

assistant: text: We propose an interpretable model for detecting user-chosen wakewords that learns from the user's examples.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Make SVM great again with Siamese kernel for  few-shot learning

Paper content:
While deep neural networks have shown outstanding results in a wide range of applications,
learning from a very limited number of examples is still a challenging
task.
Despite the difficulties of the few-shot learning, metric-learning techniques
showed the potential of the neural networks for this task.
While these methods
perform well, they don’t provide satisfactory results.
In this work, the idea of
metric-learning is extended with Support Vector Machines (SVM) working mechanism,
which is well known for generalization capabilities on a small dataset.
Furthermore, this paper presents an end-to-end learning framework for training
adaptive kernel SVMs, which eliminates the problem of choosing a correct kernel
and good features for SVMs.
Next, the one-shot learning problem is redefined
for audio signals.
Then the model was tested on vision task (using Omniglot
dataset) and speech task (using TIMIT dataset) as well.
Actually, the algorithm
using Omniglot dataset improved accuracy from 98.1% to 98.5% on the one-shot
classification task and from 98.9% to 99.3% on the few-shot classification task.

assistant: text: The proposed method is an end-to-end neural SVM, which is optimized for few-shot learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Training Domain Specific Models for Energy-Efficient Object Detection

Paper content:
We propose an end-to-end framework for training domain specific models (DSMs) to obtain both high accuracy and computational efficiency for object detection tasks.
DSMs are trained with distillation and focus on achieving high accuracy at a limited domain (e.g. fixed view of an intersection).
We argue that DSMs can capture essential features well even with a small model size, enabling higher accuracy and efficiency than traditional techniques.
In addition, we improve the training efficiency by reducing the dataset size by culling easy to classify images from the training set.
For the limited domain, we observed that compact DSMs significantly surpass the accuracy of COCO trained models of the same size.
By training on a compact dataset, we show that with an accuracy drop of only 3.6%, the training time can be reduced by 93%.

assistant: text: High object-detection accuracy can be obtained by training domain specific compact models and the training can be very short.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A quantifiable testing of global translational invariance in Convolutional and Capsule Networks

Paper content:
We design simple and quantifiable testing of global translation-invariance in deep learning models trained on the MNIST dataset.
Experiments on convolutional and capsules neural networks show that both models have poor performance in dealing with global translation-invariance; however, the performance improved by using data augmentation.
Although the capsule network is better on the MNIST testing dataset, the convolutional neural network generally has better performance on the translation-invariance.

assistant: text: Testing of global translational invariance in Convolutional and Capsule Networks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Rendering Model: Joint Generation and Prediction for Semi-Supervised Learning

Paper content:
Unsupervised and semi-supervised learning are important problems that are especially challenging with complex data like natural images.
Progress on these problems would accelerate if we had access to appropriate generative models under which to pose the associated inference tasks.
Inspired by the success of Convolutional Neural Networks (CNNs) for supervised prediction in images, we design the Neural Rendering Model (NRM), a new hierarchical probabilistic generative model whose inference calculations correspond to those in a CNN.
The NRM introduces a small set of latent variables at each level of the model and enforces dependencies among all the latent variables via a conjugate prior distribution.
The conjugate prior yields a new regularizer for learning based on the paths rendered in the generative model for training CNNs–the Rendering Path Normalization (RPN).
We demonstrate that this regularizer improves generalization both in theory and in practice.
Likelihood estimation in the NRM yields the new Max-Min cross entropy training loss, which suggests a new deep network architecture–the Max- Min network–which exceeds or matches the state-of-art for semi-supervised and supervised learning on SVHN, CIFAR10, and CIFAR100.

assistant: text: We develop a new deep generative model for semi-supervised learning and propose a new Max-Min cross-entropy for training CNNs.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Local Editing of Cross-Surface Mappings with Iterative Least Squares Conformal Maps

Paper content:
In this paper, we propose a novel approach to improve a given surface mapping through local refinement.
The approach
receives an established mapping between two surfaces and follows four phases:
(i) inspection of the mapping and creation of a sparse
set of landmarks in mismatching regions;
(ii) segmentation with a low-distortion region-growing process based on flattening the
segmented parts;
(iii) optimization of the deformation of segmented parts to align the landmarks in the planar parameterization domain;
and
(iv) aggregation of the mappings from segments to update the surface mapping.
In addition, we propose a new method to deform the
mesh in order to meet constraints (in our case, the landmark alignment of phase
(iii)).
We incrementally adjust the cotangent weights for
the constraints and apply the deformation in a fashion that guarantees that the deformed mesh will be free of flipped faces and will have
low conformal distortion.
Our new deformation approach, Iterative Least Squares Conformal Mapping (ILSCM), outperforms other
low-distortion deformation methods.
The approach is general, and we tested it by improving the mappings from different existing surface
mapping methods.
We also tested its effectiveness by editing the mappings for a variety of 3D objects.

assistant: text: We propose a novel approach to improve a given cross-surface mapping through local refinement with a new iterative method to deform the mesh in order to meet user constraints.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Masked Translation Model

Paper content:
We introduce the masked translation model (MTM) which combines encoding and decoding of sequences within the same model component.
The MTM is based on the idea of masked language modeling and supports both autoregressive and non-autoregressive decoding strategies by simply changing the order of masking.
In experiments on the WMT 2016 Romanian-English task, the MTM shows strong constant-time translation performance, beating all related approaches with comparable complexity.
We also extensively compare various decoding strategies supported by the MTM, as well as several length modeling techniques and training settings.

assistant: text: We use a transformer encoder to do translation by training it in the style of a masked translation model.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Plug and Play Language Models: A Simple Approach to Controlled Text Generation

Paper content:
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities.
However, controlling attributes of the generated language (e.g. switching topic or sentiment) is difficult without modifying the model architecture or fine-tuning on attribute-specific data and entailing the significant cost of retraining.
We propose a simple alternative: the Plug and Play Language Model (PPLM) for controllable language generation, which combines a pretrained LM with one or more simple attribute classifiers that guide text generation without any further training of the LM.
In the canonical scenario we present, the attribute models are simple classifiers consisting of a user-specified bag of words or a single learned layer with 100,000 times fewer parameters than the LM.
Sampling entails a forward and backward pass in which gradients from the attribute model push the LM's hidden activations and thus guide the generation.
Model samples demonstrate control over a range of topics and sentiment styles, and extensive automated and human annotated evaluations show attribute alignment and fluency.
PPLMs are flexible in that any combination of differentiable attribute models may be used to steer text generation, which will allow for diverse and creative applications beyond the examples given in this paper.

assistant: text: We control the topic and sentiment of text generation (almost) without any training.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness

Paper content:
The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations.
Nonetheless, min-max optimization beyond the purpose of AT has not been rigorously explored in the research of adversarial attack and defense.
In particular, given a set of risk sources (domains), minimizing the maximal loss induced from the domain set can be reformulated as a general min-max problem that is different from AT.
Examples of this general formulation include attacking model ensembles, devising universal perturbation under multiple inputs or data transformations, and generalized AT over different types of attack models.
We show that these problems can be solved under a unified and theoretically principled min-max optimization framework.
We also show that the self-adjusted domain weights learned from our method provides a means to explain the difficulty level of attack and defense over multiple domains.
Extensive experiments show that our approach leads to substantial performance improvement over the conventional averaging strategy.

assistant: text: A unified min-max optimization framework for adversarial attack and defense
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Teaching GAN to generate per-pixel annotation

Paper content:
We propose a method for joint image and per-pixel annotation synthesis with GAN.
We demonstrate that GAN has good high-level representation of target data that can be easily projected to semantic segmentation masks.
This method can be used to create a training dataset for teaching separate semantic segmentation network.
Our experiments show that such segmentation network successfully generalizes on real data.
Additionally, the method outperforms supervised training when the number of training samples is small, and works on variety of different scenes and classes.
The source code of the proposed method will be publicly available.

assistant: text: GAN-based method for joint image and per-pixel annotation synthesis
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Depth separation and weight-width trade-offs for sigmoidal neural networks

Paper content:
Some recent work has shown separation between the expressive power of depth-2 and depth-3 neural networks.
These separation results are shown by constructing functions and input distributions, so that the function is well-approximable by a depth-3 neural network of polynomial size but it cannot be well-approximated under the chosen input distribution by any depth-2 neural network of polynomial size.
These results are not robust and require carefully chosen functions as well as input distributions.
We show a similar separation between the expressive power of depth-2 and depth-3 sigmoidal neural networks over a large class of input distributions, as long as the weights are polynomially bounded.
While doing so, we also show that depth-2 sigmoidal neural networks with small width and small weights can be well-approximated by low-degree multivariate polynomials.

assistant: text: depth-2-vs-3 separation for sigmoidal neural networks over general distributions
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Few-Shot Intent Inference via Meta-Inverse Reinforcement Learning

Paper content:
A significant challenge for the practical application of reinforcement learning toreal world problems is the need to specify an oracle reward function that correctly defines a task.
Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior.
While appealing, it can be impractically expensive to collect datasets of demonstrations that cover the variation common in the real world (e.g. opening any type of door).
Thus in practice, IRL must commonly be performed with only a limited set of demonstrations where it can be exceedingly difficult to unambiguously recover a reward function.
In this work, we exploit the insight that demonstrations from other tasks can be used to constrain the set of possible reward functions by learning a "prior" that is specifically optimized for the ability to infer expressive reward functions from limited numbers of demonstrations.
We demonstrate that our method can efficiently recover rewards from images for novel tasks and provide intuition as to how our approach is analogous to learning a prior.

assistant: text: The applicability of inverse reinforcement learning is often hampered by the expense of collecting expert demonstrations; this paper seeks to broaden its applicability by incorporating prior task information through meta-learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

Paper content:
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level.
We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods.
The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses.
As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution.
Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.
In this paper, we proposed a method for response characterization for LSTM networks to predict cell-contributions to the overall decision of a learned network on both the cell and network-level resolution.
We further verified and validated our predictions by performing an ablation analysis to identify cell's which contribution heavily to the network's output decision with our simple response characterization method.
The resulting method establishes a novel building block for interpreting LSTM networks.
The LSTM network's dynamic-space is broad and cannot be fully captured by fundamental input sequences.
However, our methodology demonstrates that practical sub-regions of dynamics are reachable by response metrics which we use to build a systematic testbench for LSTM interpretability.
We have open-sourced our algorithm to encourage other researchers to further explore dynamics of LSTM cells and interpret the kinetics of their sequential models.In the future, we aim to extend our approach to even more data modalities and analyze the training phase of LSTMs to interpret the learning of the converged dynamics presented in this work.7
Acknowledgment

assistant: text: Introducing the response charactrization method for interpreting cell dynamics in learned long short-term memory (LSTM) networks.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Parametrized Hierarchical Procedures for Neural Programming

Paper content:
Neural programs are highly accurate and structured policies that perform algorithmic tasks by controlling the behavior of a computation mechanism.
Despite the potential to increase the interpretability and the compositionality of the behavior of artificial agents, it remains difficult to learn from demonstrations neural networks that represent computer programs.
The main challenges that set algorithmic domains apart from other imitation learning domains are the need for high accuracy, the involvement of specific structures of data, and the extremely limited observability.
To address these challenges, we propose to model programs as Parametrized Hierarchical Procedures (PHPs).
A PHP is a sequence of conditional operations, using a program counter along with the observation to select between taking an elementary action, invoking another PHP as a sub-procedure, and returning to the caller.
We develop an algorithm for training PHPs from a set of supervisor demonstrations, only some of which are annotated with the internal call structure, and apply it to efficient level-wise training of multi-level PHPs.
We show in two benchmarks, NanoCraft and long-hand addition, that PHPs can learn neural programs more accurately from smaller amounts of both annotated and unannotated demonstrations.

assistant: text: We introduce the PHP model for hierarchical representation of neural programs, and an algorithm for learning PHPs from a mixture of strong and weak supervision.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

Paper content:
We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset.
SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem.
To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme.
The target therefore gets to choose the source samples which are most informative for its own classification task.
Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting.
SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

assistant: text: Learning with limited training data by exploiting "helpful" instances from a rich data source.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A Deep Dive into Count-Min Sketch for Extreme Classification in Logarithmic Memory

Paper content:
Extreme Classification Methods have become of paramount importance, particularly for Information Retrieval (IR) problems, owing to the development of smart algorithms that are scalable to industry challenges.
One of the prime class of models that aim to solve the memory and speed challenge of extreme multi-label learning is Group Testing.
Multi-label Group Testing (MLGT) methods construct label groups by grouping original labels either randomly or based on some similarity and then train smaller classifiers to first predict the groups and then recover the original label vectors.
Recently, a novel approach called MACH (Merged Average Classifiers via Hashing) was proposed which projects the huge label vectors to a small and manageable count-min sketch (CMS) matrix and then learns to predict this matrix to recover the original prediction probabilities.
Thereby, the model memory scales O(logK) for K classes.
MACH is a simple algorithm which works exceptionally well in practice.
Despite this simplicity of MACH, there is a big gap between the theoretical understanding of the trade-offs with MACH.
In this paper we fill this gap.
Leveraging the theory of count-min sketch we provide precise quantification of the memory-identifiablity tradeoffs.
We extend the theory to the case of multi-label classification, where the dependencies make the estimators hard to calculate in closed forms.
To mitigate this issue, we propose novel quadratic approximation using the Inclusion-Exclusion Principle.
Our estimator has significantly lower reconstruction error than the typical CMS estimator across various values of number of classes K, label sparsity and compression ratio.

assistant: text: How to estimate original probability vector for millions of classes from count-min sketch measurements -  a theoretical and practical setup.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Deep Gradient Boosting -- Layer-wise Input Normalization of Neural Networks

Paper content:
Stochastic gradient descent (SGD) has been the dominant optimization method for training deep neural networks due to its many desirable properties.
One of the more remarkable and least understood quality of SGD is that it generalizes relatively well
on unseen data even when the neural network has millions of parameters.
We hypothesize that in certain cases it is desirable to relax its intrinsic generalization properties and introduce an extension of SGD called deep gradient boosting (DGB).
The key idea of DGB is that back-propagated gradients inferred using the chain rule can be viewed as pseudo-residual targets of a gradient boosting problem.
Thus at each layer of a neural network the weight update is calculated by solving the corresponding boosting problem using a linear base learner.
The resulting weight update formula can also be viewed as a normalization procedure of the data that arrives at each layer during the forward pass.
When implemented as a separate input normalization layer (INN) the new architecture shows improved performance on image recognition tasks when compared to the same architecture without normalization layers.
As opposed to batch normalization (BN), INN has no learnable parameters however it matches its performance on CIFAR10 and ImageNet classification tasks.

assistant: text: What can we learn about training neural networks if we treat each layer as a gradient boosting problem?
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Minimizing FLOPs to Learn Efficient Sparse Representations

Paper content:
Deep representation learning has become one of the most widely adopted approaches for visual search, recommendation, and identification.
Retrieval of such  representations from a large database is however computationally challenging.
Approximate methods based on learning compact representations, have been widely explored for this problem, such as locality sensitive hashing, product quantization, and PCA.
In this work, in contrast to learning compact representations, we propose to learn high dimensional and sparse representations that have similar representational capacity as dense embeddings while being more efficient due to sparse matrix multiplication operations which can be much faster than dense multiplication.
Following the key insight that the number of operations decreases quadratically with the sparsity of embeddings provided the non-zero entries are distributed uniformly across dimensions, we propose a novel approach to learn such distributed sparse embeddings via the use of a carefully constructed regularization function that directly minimizes a continuous relaxation of the number of floating-point operations (FLOPs) incurred during retrieval.
Our experiments show that our approach is competitive to the other baselines and yields a similar or better speed-vs-accuracy tradeoff on practical datasets.

assistant: text: We propose an approach to learn sparse high dimensional representations that are fast to search, by incorporating a surrogate of the number of operations directly into the loss function.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: signSGD via Zeroth-Order Oracle

Paper content:
In this paper, we design and analyze a new zeroth-order (ZO) stochastic optimization algorithm, ZO-signSGD, which enjoys dual advantages of gradient-free operations and signSGD.
The latter requires only the sign information of  gradient estimates but is able to achieve a comparable  or even better convergence speed than SGD-type algorithms.
Our study  shows that ZO signSGD requires $\sqrt{d}$ times more iterations than signSGD, leading to a convergence rate of  $O(\sqrt{d}/\sqrt{T})$ under mild conditions, where $d$ is the number of optimization variables, and $T$ is the number of iterations.
In addition, we analyze the effects of different types of gradient estimators on the convergence of ZO-signSGD, and propose two variants of ZO-signSGD that  at least  achieve $O(\sqrt{d}/\sqrt{T})$ convergence rate.
On the application side we explore the connection between ZO-signSGD and  black-box adversarial attacks in robust deep learning.
Our empirical evaluations on image classification datasets MNIST and CIFAR-10 demonstrate the superior performance of ZO-signSGD on the generation of   adversarial examples from black-box neural networks.

assistant: text: We design and analyze a new zeroth-order stochastic optimization algorithm, ZO-signSGD, and demonstrate its connection and application to black-box adversarial attacks in robust deep learning
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: MUSE: Multi-Scale Attention Model for Sequence to Sequence Learning

Paper content:
Transformers have achieved state-of-the-art results on a variety of natural language processing tasks.
Despite good performance, Transformers are still weak in long sentence modeling where the global attention map is too dispersed to capture valuable information.
In such case, the local/token features that are also significant to sequence modeling are omitted to some extent.
To address this problem, we propose a Multi-scale attention model (MUSE) by concatenating attention networks with convolutional networks and position-wise feed-forward networks to explicitly capture local and token features.
Considering the parameter size and computation efficiency, we re-use the feed-forward layer in the original Transformer and adopt a lightweight dynamic convolution as implementation.
Experimental results show that the proposed model achieves substantial performance improvements over Transformer, especially on long sentences, and pushes the state-of-the-art from 35.6 to 36.2 on IWSLT 2014  German to English translation task,  from 30.6 to 31.3 on  IWSLT 2015 English to Vietnamese translation task.
We also reach the state-of-art performance on  WMT 2014 English to French translation dataset, with a BLEU score of 43.2.

assistant: text: This paper propose a new model  which combines multi scale information for sequence to sequence learning.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Evaluation and Comparison of Usability of Four Mobile Breathing Training Visualizations

Paper content:
Breathing exercises are an accessible way to manage stress and many mental illness symptoms.
Traditionally, learning breathing exercises involved in-person guidance or audio recordings.
The shift to mobile devices has led to a new way of learning and engaging in breathing exercises as seen in the rise of multiple mobile applications with different breathing representations.
However, limited work has been done to investigate the effectiveness of these visual representations in supporting breathing pace as measured by synchronization.
We utilized a within-subjects study to evaluate four common breathing visuals to understand which is most effective in providing breathing exercise guidance.
Through controlled lab studies and interviews, we identified two representations with clear advantages over the others.
In addition, we found that auditory guidance was not preferred by all users.
We identify potential usability issues with the representations and suggest design guidelines for future development of app-supported breathing training.

assistant: text: We utilized a within-subjects study to evaluate four paced breathing visuals common in mobile apps to understand which is most effective in providing breathing exercise guidance.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: PocketFlow: An Automated Framework for Compressing and Accelerating Deep Neural Networks

Paper content:
Deep neural networks are widely used in various domains, but the prohibitive computational complexity prevents their deployment on mobile devices.
Numerous model compression algorithms have been proposed, however, it is often difficult and time-consuming to choose proper hyper-parameters to obtain an efficient compressed model.
In this paper, we propose an automated framework for model compression and acceleration, namely PocketFlow.
This is an easy-to-use toolkit that integrates a series of model compression algorithms and embeds a hyper-parameter optimization module to automatically search for the optimal combination of hyper-parameters.
Furthermore, the compressed model can be converted into the TensorFlow Lite format and easily deployed on mobile devices to speed-up the inference.
PocketFlow is now open-source and publicly available at https://github.com/Tencent/PocketFlow.

assistant: text: We propose PocketFlow, an automated framework for model compression and acceleration, to facilitate deep learning models' deployment on mobile devices.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Gradient Descent Happens in a Tiny Subspace

Paper content:
We show that in a variety of large-scale deep learning scenarios the gradient dynamically converges to a very small subspace after a short period of training.
The subspace is spanned by a few top eigenvectors of the Hessian (equal to the number of classes in the dataset), and is mostly preserved over long periods of training.
A simple argument then suggests that gradient descent may happen mostly in this subspace.
We give an example of this effect in a solvable model of classification, and we comment on possible implications for optimization and learning.

assistant: text: For classification problems with k classes, we show that the gradient tends to live in a tiny, slowly-evolving subspace spanned by the eigenvectors corresponding to the k-largest eigenvalues of the Hessian.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning The Difference That Makes A Difference With Counterfactually-Augmented Data

Paper content:
Despite alarm over the reliance of machine learning systems on so-called spurious patterns in training data, the term lacks coherent meaning in standard statistical frameworks.
However, the language of causality offers clarity: spurious associations are those due to a common cause (confounding) vs direct or indirect effects.
In this paper, we focus on NLP, introducing methods and resources for training models insensitive to spurious patterns.
Given documents and their initial labels, we task humans with revise each document to accord with a counterfactual target label, asking that the revised documents be internally coherent while avoiding any gratuitous changes.
Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa.
Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain.
While classifiers trained on either original or manipulated data alone  are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are insensitive to this signal.
We will publicly release both datasets.

assistant: text: Humans in the loop revise documents to accord with counterfactual labels, resulting resource helps to reduce reliance on spurious associations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning data-derived privacy preserving representations from information metrics

Paper content:
It is clear that users should own and control their data and privacy.
Utility providers are also becoming more interested in guaranteeing data privacy.
Therefore, users and providers can and should collaborate in privacy protecting challenges, and this paper addresses this new paradigm.
We propose a framework where the user controls what characteristics of the data they want to share (utility) and what they want to keep private (secret), without necessarily asking the utility provider to change its existing machine learning algorithms.
We first analyze the space of privacy-preserving representations and derive natural information-theoretic bounds on the utility-privacy trade-off when disclosing a sanitized version of the data X. We present explicit learning architectures to learn privacy-preserving representations that approach this bound in a data-driven fashion.
We describe important use-case scenarios where the utility providers are willing to collaborate with the sanitization process.
We study space-preserving transformations where the utility provider can use the same algorithm on original and sanitized data, a critical and novel attribute to help service providers accommodate varying privacy requirements with a single set of utility algorithms.
We illustrate this framework through  the implementation of three use cases; subject-within-subject, where we tackle the problem of having a face identity detector that works only on a consenting subset of users, an important application, for example, for mobile devices activated by face recognition; gender-and-subject, where we preserve facial verification while hiding the gender attribute for users who choose to do so; and emotion-and-gender, where we hide independent variables, as is the case of hiding gender while preserving emotion detection.

assistant: text: Learning privacy-preserving transformations from data. A collaborative approach
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Networks with Structural Resistance to Adversarial Attacks

Paper content:
In adversarial attacks to machine-learning classifiers, small perturbations are added to input that is correctly classified.
The perturbations yield adversarial examples, which are virtually indistinguishable from the unperturbed input, and yet are misclassified.
In standard neural networks used for deep learning, attackers can craft adversarial examples from most input to cause a misclassification of their choice.
We introduce a new type of network units, called RBFI units, whose non-linear structure makes them inherently resistant to adversarial attacks.
On permutation-invariant MNIST, in absence of adversarial attacks, networks using RBFI units match the performance of networks using sigmoid units, and are slightly below the accuracy of networks with ReLU units.
When subjected to adversarial attacks based on projected gradient descent or fast gradient-sign methods, networks with RBFI units retain accuracies above 75%, while ReLU or Sigmoid see their accuracies reduced to below 1%.
Further, RBFI networks trained on regular input either exceed or closely match the accuracy of sigmoid and ReLU network trained with the help of adversarial examples.
The non-linear structure of RBFI units makes them difficult to train using standard gradient descent.
We show that RBFI networks of RBFI units can be efficiently trained to high accuracies using pseudogradients, computed using functions especially crafted to facilitate learning instead of their true derivatives.

assistant: text: We introduce a type of neural network that is structurally resistant to adversarial attacks, even when trained on unaugmented training sets.  The resistance is due to the stability of network units wrt input perturbations.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Learning to play slot cars and Atari 2600 games in just minutes

Paper content:
Machine learning algorithms for controlling devices will need to learn quickly, with few trials.
Such a goal can be attained with concepts borrowed from continental philosophy and formalized using tools from the mathematical theory of categories.
Illustrations of this approach are presented on a cyberphysical system: the slot car game, and also on Atari 2600 games.

assistant: text: Continental-philosophy-inspired approach to learn with few data.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Conservative Uncertainty Estimation By Fitting  Prior Networks

Paper content:
Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks.
In this paper, we theoretically justify a scheme for estimating uncertainties, based on sampling from a prior distribution.
Crucially, the uncertainty estimates are shown to be conservative in the sense that they never underestimate a posterior uncertainty obtained by a hypothetical Bayesian algorithm.
We also show concentration, implying that the uncertainty estimates converge to zero as we get more data.
Uncertainty estimates obtained from random priors can be adapted to any deep network architecture and trained using standard supervised learning pipelines.
We provide experimental evaluation of random priors on calibration and out-of-distribution detection on typical computer vision tasks, demonstrating that they outperform deep ensembles in practice.

assistant: text: We provide theoretical support to uncertainty estimates for deep learning obtained fitting random priors.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Multi-hop Question Answering via Reasoning Chains

Paper content:
Multi-hop question answering requires models to gather information from different parts of a text to answer a question.
Most current approaches learn to address this task in an end-to-end way with neural networks, without maintaining an explicit representation of the reasoning process.
We propose a method to extract a discrete reasoning chain over the text, which consists of a series of sentences leading to the answer.
We then feed the extracted chains to a BERT-based QA model to do final answer prediction.
Critically, we do not rely on gold annotated chains or ``supporting facts:'' at training time, we derive pseudogold reasoning chains using heuristics based on named entity recognition and coreference resolution.
Nor do we rely on these annotations at test time, as our model learns to extract chains from raw text alone.
We test our approach on two recently proposed large multi-hop question answering datasets: WikiHop and HotpotQA, and achieve state-of-art performance on WikiHop and strong performance on HotpotQA.
Our analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way.
Furthermore, human evaluation shows that our extracted chains allow humans to give answers with high confidence, indicating that these are a strong intermediate abstraction for this task.

assistant: text: We improve answering of questions that require multi-hop reasoning extracting an intermediate chain of sentences.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Kernel RNN Learning (KeRNL)

Paper content:
We describe Kernel RNN Learning (KeRNL), a reduced-rank, temporal eligibility trace-based approximation to backpropagation through time (BPTT) for training recurrent neural networks (RNNs) that gives competitive performance to BPTT on long time-dependence tasks.
The approximation replaces a rank-4 gradient learning tensor, which describes how past hidden unit activations affect the current state, by a simple reduced-rank product of a sensitivity weight and a temporal eligibility trace.
In this structured approximation motivated by node perturbation, the sensitivity weights and eligibility kernel time scales are themselves learned by applying perturbations.
The rule represents another step toward biologically plausible or neurally inspired ML, with lower complexity in terms of relaxed architectural requirements (no symmetric return weights), a smaller memory demand (no unfolding and storage of states over time), and a shorter feedback time.

assistant: text: A biologically plausible learning rule for training recurrent neural networks
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Multi-hop question answering requires models to gather information from different parts of a text to answer a question.
Most current approaches learn to address this task in an end-to-end way with neural networks, without maintaining an explicit representation of the reasoning process.
We propose a method to extract a discrete reasoning chain over the text, which consists of a series of sentences leading to the answer.
We then feed the extracted chains to a BERT-based QA model to do final answer prediction.
Critically, we do not rely on gold annotated chains or ``supporting facts:'' at training time, we derive pseudogold reasoning chains using heuristics based on named entity recognition and coreference resolution.
Nor do we rely on these annotations at test time, as our model learns to extract chains from raw text alone.
We test our approach on two recently proposed large multi-hop question answering datasets: WikiHop and HotpotQA, and achieve state-of-art performance on WikiHop and strong performance on HotpotQA.
Our analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way.
Furthermore, human evaluation shows that our extracted chains allow humans to give answers with high confidence, indicating that these are a strong intermediate abstraction for this task.

assistant: text: We improve answering of questions that require multi-hop reasoning extracting an intermediate chain of sentences.
user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: 

Paper content:
Autoregressive recurrent neural decoders that generate sequences of tokens one-by-one and left-to-right are the workhorse of modern machine translation.
In this work, we propose a new decoder architecture that can generate natural language sequences in an arbitrary order.
Along with generating tokens from a given vocabulary, our model additionally learns to select the optimal position for each produced token.
The proposed decoder architecture is fully compatible with the seq2seq framework and can be used as a drop-in replacement of any classical decoder.
We demonstrate the performance of our new decoder on the IWSLT machine translation task as well as inspect and interpret the learned decoding patterns by analyzing how the model selects new positions for each subsequent token.

assistant: text: new out-of-order decoder for neural machine translation