Miracle-dz/UniFAR-FTUs huggingface dataset:
Deep learning models are not robust against noise in clinical text

Artificial Intelligence (AI) systems are attracting increasing interest in the medical domain due to their ability to learn complicated tasks that require human intelligence and expert knowledge. AI systems that utilize high-performance Natural Language Processing (NLP) models have achieved state-of-the-art results on a wide variety of clinical text processing benchmarks. They have even outperformed human accuracy on some tasks. However, performance evaluation of such AI systems have been limited to accuracy measures on curated and clean benchmark datasets that may not properly reflect how robustly these systems can operate in real-world situations. In order to address this challenge, we introduce and implement a wide variety of perturbation methods that simulate different types of noise and variability in clinical text data. While noisy samples produced by these perturbation methods can often be understood by humans, they may cause AI systems to make erroneous decisions. Conducting extensive experiments on several clinical text processing tasks, we evaluated the robustness of high-performance NLP models against various types of character-level and word-level noise. The results revealed that the NLP models performance degrades when the input contains small amounts of noise. This study is a significant step towards exposing vulnerabilities of AI models utilized in clinical text processing systems. The proposed perturbation methods can be used in performance evaluation tests to assess how robustly clinical NLP models can operate on noisy data, in real-world settings.
Rethinking the Aligned and Misaligned Features in One-stage Object Detection

One-stage object detectors rely on a point feature to predict the detection results. However, the point feature often lacks the information of the whole object, thereby leading to a misalignment between the object and the point feature. Meanwhile, the classiﬁcation and regression tasks are sensitive to different object regions, but their features are spatially aligned. Both of these two problems hinder the detection performance. In order to solve these two problems, we propose a simple and plug-in operator that can generate aligned and dis-entangled features for each task, respectively, without breaking the fully convolutional manner. By predicting two task-aware point sets that are located in each sensitive region, the proposed operator can align the point feature with the object and disentangle the two tasks from the spatial dimension. We also reveal an interesting ﬁnding of the opposite effect of the long-range skip connection for classiﬁcation and regression. On the basis of the O bject-A ligned and T ask-disentangled operator (OAT), we propose OAT-Net, which explicitly exploits point-set features for accurate detection results. Extensive experiments on the MS-COCO dataset show that OAT can consistently boost different state-of-the-art one-stage detectors by ∼ 2 AP. Notably, OAT-Net with Res2Net-101-DCN backbone achieves 53.7 AP on the COCO test-dev .
CeMux: Maximizing the Accuracy of Stochastic Mux Adders and an Application to Filter Design

Stochastic computing (SC) is a low-cost computational paradigm that has promising applications in digital filter design, image processing, and neural networks. Fundamental to these applications is the weighted addition operation, which is most often implemented by a multiplexer (mux) tree. Mux-based adders have very low area but typically require long bitstreams to reach practical accuracy thresholds when the number of summands is large. In this work, we first identify the main contributors to mux adder error. We then demonstrate with analysis and experiment that two new techniques, precise sampling and full correlation, can target and mitigate these error sources. Implementing these techniques in hardware leads to the design of CeMux (Correlation-enhanced Multiplexer), a stochastic mux adder that is significantly more accurate and uses much less area than traditional weighted adders. We compare CeMux to other SC and hybrid designs for an electrocardiogram filtering case study that employs a large digital filter. One major result is that CeMux is shown to be accurate even for large input sizes. CeMux's higher accuracy leads to a latency reduction of 4× to 16× over other designs. Furthermore, CeMux uses about 35% less area than existing designs, and we demonstrate that a small amount of accuracy can be traded for a further 50% reduction in area. Finally, we compare CeMux to a conventional binary design and we show that CeMux can achieve a 50% to 73% area reduction for similar power and latency as the conventional design but at a slightly higher level of error.
A Partition Filter Network for Joint Entity and Relation Extraction

In joint entity and relation extraction, existing work either sequentially encode task-specific features, leading to an imbalance in inter-task feature interaction where features extracted later have no direct contact with those that come first. Or they encode entity features and relation features in a parallel manner, meaning that feature representation learning for each task is largely independent of each other except for input sharing. We propose a partition filter network to model two-way interaction between tasks properly, where feature encoding is decomposed into two steps: partition and filter. In our encoder, we leverage two gates: entity and relation gate, to segment neurons into two task partitions and one shared partition. The shared partition represents inter-task information valuable to both tasks and is evenly shared across two tasks to ensure proper two-way interaction. The task partitions represent intra-task information and are formed through concerted efforts of both gates, making sure that encoding of task-specific features is dependent upon each other. Experiment results on six public datasets show that our model performs significantly better than previous approaches. In addition, contrary to what previous work has claimed, our auxiliary experiments suggest that relation prediction is contributory to named entity prediction in a non-negligible way. The source code can be found at https://github.com/Coopercoppers/PFN.
The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers

Recently, many datasets have been proposed to test the systematic generalization ability of neural networks. The companion baseline Transformers, typically trained with default hyper-parameters from standard tasks, are shown to fail dramatically. Here we demonstrate that by revisiting model configurations as basic as scaling of embeddings, early stopping, relative positional embedding, and Universal Transformer variants, we can drastically improve the performance of Transformers on systematic generalization. We report improvements on five popular datasets: SCAN, CFQ, PCFG, COGS, and Mathematics dataset. Our models improve accuracy from 50% to 85% on the PCFG productivity split, and from 35% to 81% on COGS. On SCAN, relative positional embedding largely mitigates the EOS decision problem (Newman et al., 2020), yielding 100% accuracy on the length split with a cutoff at 26. Importantly, performance differences between these models are typically invisible on the IID data split. This calls for proper generalization validation sets for developing neural networks that generalize systematically. We publicly release the code to reproduce our results.
Dynamical instability of collapsed dark matter halos

A self-interacting dark matter halo can experience gravothermal collapse, resulting in a central core with an ultrahigh density. It can further contract and collapse into a black hole, a mechanism proposed to explain the origin of supermassive black holes. We study dynamical instability of the core in general relativity. We use a truncated Maxwell-Boltzmann distribution to model the dark matter distribution and solve the Tolman-Oppenheimer-Volkoff equation. For given model parameters, we obtain a series of equilibrium configurations and examine their dynamical instability based on considerations of total energy, binding energy, fractional binding energy, and adiabatic index. Our numerical results indicate that the core can collapse into a black hole when the fractional binding energy reaches 0.035 with a central gravitational redshift of 0.5. We further show for the instability to occur in the classical regime, the boundary temperature of the core should be at least 10% of the mass of dark matter particles; for a 109 M⊙ seed black hole, the particle mass needs to be larger than a few keV. These results can be used to constrain different collapse models, in particular, those with dissipative dark matter interactions. https://github.com/michaelwxfeng/truncated-Maxwell-Boltzmann.
A central limit theorem for m-dependent variables

Central limit theorems for m-dependent variables under various conditions have a long history. Pioneering results, for a fixed m, were given by Hoeffding and Robbins [10] and Diananda [5] (for an m-dependent sequence), and Orey [12] (more generally, and also for a triangular array). The results were extended to the case of increasing m = m(n), see for example Bergström [1], Berk [2], Romano and Wolf [16]. The purpose of the present paper is to give a simple and general central limit theorem which includes several previous results, but to our knowledge has not been stated before in this form. We state first the case of a fixed m, where we only have to assume the usual Lindeberg condition. For notation, see Section 2.
A New Generation of Neutrino Cross Section Experiments: Challenges and Opportunities

Our knowledge of neutrino cross sections at the GeV scale, instrumental to test CP symmetry violation in the leptonic sector, has grown substantially in the last two decades. Still, their precision and understanding are far from the standard needed in contemporary neutrino physics. Nowadays, the knowledge of the neutrino cross section at O(10%) causes the main systematic uncertainty in oscillation experiments and jeopardizes their physics reach. In this paper, we envision the opportunities for a new generation of cross section experiments to be run in parallel with DUNE and HyperKamiokande. We identify the most prominent physics goals by looking at the theory and experimental limitations of the previous generation of experiments. We highlight the priorities in the theoretical understanding of GeV cross sections and the experimental challenges of this new generation of facilities.
Revisiting event study designs: robust and efficient estimation

We develop a framework for difference-in-differences designs with staggered treatment adoption and heterogeneous causal effects. We show that conventional regression-based estimators fail to provide unbiased estimates of relevant estimands absent strong restrictions on treatment-effect homogeneity. We then derive the efficient estimator addressing this challenge, which takes an intuitive"imputation"form when treatment-effect heterogeneity is unrestricted. We characterize the asymptotic behavior of the estimator, propose tools for inference, and develop tests for identifying assumptions. Our method applies with time-varying controls, in triple-difference designs, and with certain non-binary treatments. We show the practical relevance of our results in a simulation study and an application. Studying the consumption response to tax rebates in the United States, we find that the notional marginal propensity to consume is between 8 and 11 percent in the first quarter - about half as large as benchmark estimates used to calibrate macroeconomic models - and predominantly occurs in the first month after the rebate.
Exponential clustering of bipartite quantum entanglement at arbitrary temperatures

Many inexplicable phenomena in low-temperature many-body physics are a result of macroscopic quantum eﬀects. Such macroscopic quantumness is often evaluated via long-range entanglement, that is, entanglement in the macroscopic length scale. Long-range entanglement is employed to characterize novel quantum phases and serves as a critical resource for quantum computation. How-ever, the conditions under which long-range entanglement is stable, even at room temperatures, remain unclear. In this regard, this study demonstrated the unstable nature of bi-partite long-range entanglement at arbitrary temperatures, which exponentially decays with distance. The proposed theorem is a no-go theorem pertaining to the existence of long-range entanglement. The obtained results were consistent with existing observations, indicating that long-range entanglement at non-zero temperatures can exist in topologically ordered phases, where tripartite correlations are dominant. The derivation in this study introduced a quantum correlation deﬁned by the convex roof of the standard correlation function. Further, an exponential clustering theorem for generic quantum many-body systems under such a quantum correlation at arbitrary temperatures was established, which yielded the primary result by relating quantum correlation with quantum entanglement. Moreover, a simple application of analytical techniques was demonstrated by deriving a general limit on the Wigner-Yanase-Dyson skew and quantum Fisher information; this is expected to attract signiﬁcant attention in the ﬁeld of quantum metrology. Notably, this study reveals the novel general aspects of low-temperature quantum physics and clariﬁes the characterization of long-range entanglement.
A Morphological Classification of 18,190 Molecular Clouds Identified in 12CO Data from the MWISP Survey

We attempt to visually classify the morphologies of 18,190 molecular clouds, which are identified in the 12CO(1–0) spectral line data over ∼450 deg2 of the second Galactic quadrant from the Milky Way Imaging Scroll Painting project. Using the velocity-integrated intensity maps of the 12CO(1–0) emission, molecular clouds are first divided into unresolved and resolved ones. The resolved clouds are further classified as nonfilaments or filaments. Among the 18,190 molecular clouds, ∼25% are unresolved, ∼64% are nonfilaments, and ∼11% are filaments. In the terms of the integrated flux of 12CO(1–0) spectra of all 18,190 molecular clouds, ∼90% are from filaments, ∼9% are from nonfilaments, and the remaining ∼1% are from unresolved sources. Although nonfilaments are dominant in the number of the discrete molecular clouds, filaments are the main contributor of 12CO emission flux. We also present the number distributions of the physical parameters of the molecular clouds in our catalog, including their angular sizes, velocity spans, peak intensities of 12CO(1–0) emission, and 12CO(1–0) total fluxes. We find that there is a systematic difference between the angular sizes of the nonfilaments and filaments, with the filaments tending to have larger angular scales. The H2 column densities of them are not significantly different. We also discuss the observational effects, such as those induced by the finite spatial resolution, beam dilution, and line-of-sight projection, on the morphological classification of molecular clouds in our sample.
Monitoring Fast Superconducting Qubit Dynamics Using a Neural Network

Weak measurements of a superconducting qubit produce noisy voltage signals that are weakly correlated with the qubit state. To recover individual quantum trajectories from these noisy signals, traditional methods require slow qubit dynamics and substantial prior information in the form of calibration experiments. Monitoring rapid qubit dynamics, e.g. during quantum gates, requires more complicated methods with increased demand for prior information. Here, we experimentally demonstrate an alternative method for accurately tracking rapidly driven superconducting qubit trajectories that uses a Long Short-Term Memory (LSTM) artiﬁcial neural network with minimal prior information. Despite few training assumptions, the LSTM produces trajectories that include qubit-readout resonator correlations due to a ﬁnite detection bandwidth. In addition to revealing rotated measurement eigenstates and a reduced measurement rate in agreement with theory for a ﬁxed drive, the trained LSTM also correctly reconstructs evolution for an unknown drive with rapid modulation. Our work enables new applications of weak measurements with faster or initially unknown qubit dynamics, such as the diagnosis of coherent errors in quantum gates.
Prior Signal Editing for Graph Filter Posterior Fairness Constraints

Graph filters are an emerging paradigm that systematizes information propagation in graphs as transformation of prior node values, called graph signals, to posterior scores. In this work, we study the problem of mitigating disparate impact, i.e. posterior score differences between a protected set of sensitive nodes and the rest, while minimally editing scores to preserve recommendation quality. To this end, we develop a scheme that respects propagation mechanisms by editing graph signal priors according to their posteriors and node sensitivity, where a small number of editing parameters can be tuned to constrain or eliminate disparate impact. We also theoretically explain that coarse prior editing can locally optimize posteriors objectives thanks to graph filter robustness. We experiment on a diverse collection of 12 graphs with varying number of nodes, where our approach performs equally well or better than previous ones in minimizing disparate impact and preserving posterior AUC under fairness constraints.
Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers

In this work, we consider the problem of sequence-to-sequence alignment for signals containing outliers. Assuming the absence of outliers, the standard Dynamic Time Warping (DTW) algorithm efficiently computes the optimal alignment between two (generally) variable-length sequences. While DTW is robust to temporal shifts and dilations of the signal, it fails to align sequences in a meaningful way in the presence of outliers that can be arbitrarily interspersed in the sequences. To address this problem, we introduce Drop-DTW, a novel algorithm that aligns the common signal between the sequences while automatically dropping the outlier elements from the matching. The entire procedure is implemented as a single dynamic program that is efficient and fully differentiable. In our experiments, we show that Drop-DTW is a robust similarity measure for sequence retrieval and demonstrate its effectiveness as a training loss on diverse applications. With Drop-DTW, we address temporal step localization on instructional videos, representation learning from noisy videos, and cross-modal representation learning for audio-visual retrieval and localization. In all applications, we take a weakly- or unsupervised approach and demonstrate state-of-the-art results under these settings.
ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation

Co-occurrent visual pattern makes aggregating contextual information a common paradigm to enhance the pixel representation for semantic image segmentation. The existing approaches focus on modeling the context from the perspective of the whole image, i.e., aggregating the image-level contextual information. Despite impressive, these methods weaken the significance of the pixel representations of the same category, i.e., the semantic-level contextual information. To address this, this paper proposes to augment the pixel representations by aggregating the image-level and semantic-level contextual information, respectively. First, an image-level context module is designed to capture the contextual information for each pixel in the whole image. Second, we aggregate the representations of the same category for each pixel where the category regions are learned under the supervision of the ground-truth segmentation. Third, we compute the similarities between each pixel representation and the image-level contextual information, the semantic-level contextual information, respectively. At last, a pixel representation is augmented by weighted aggregating both the image-level contextual information and the semantic-level contextual information with the similarities as the weights. Integrating the image-level and semantic-level context allows this paper to report state-of-the-art accuracy on four benchmarks, i.e., ADE20K, LIP, COCOStuff and Cityscapes 1.
Translation Error Detection as Rationale Extraction

Recent Quality Estimation (QE) models based on multilingual pre-trained representations have achieved very competitive results in predicting the overall quality of translated sentences. However, detecting specifically which translated words are incorrect is a more challenging task, especially when dealing with limited amounts of training data. We hypothesize that, not unlike humans, successful QE models rely on translation errors to predict overall sentence quality. By exploring a set of feature attribution methods that assign relevance scores to the inputs to explain model predictions, we study the behaviour of state-of-the-art sentence-level QE models and show that explanations (i.e. rationales) extracted from these models can indeed be used to detect translation errors. We therefore (i) introduce a novel semi-supervised method for word-level QE; and (ii) propose to use the QE task as a new benchmark for evaluating the plausibility of feature attribution, i.e. how interpretable model explanations are to humans.
Continual learning under domain transfer with sparse synaptic bursting

Existing machines are functionally specific tools that were made for easy prediction and control. Tomorrow's machines may be closer to biological systems in their mutability, resilience, and autonomy. But first they must be capable of learning and retaining new information without being exposed to it arbitrarily often. Past efforts to engineer such systems have sought to build or regulate artificial neural networks using disjoint sets of weights that are uniquely sensitive to specific tasks or inputs. This has not yet enabled continual learning over long sequences of previously unseen data without corrupting existing knowledge: a problem known as catastrophic forgetting. In this paper, we introduce a system that can learn sequentially over previously unseen datasets (ImageNet, CIFAR-100) with little forgetting over time. This is done by controlling the activity of weights in a convolutional neural network on the basis of inputs using top-down regulation generated by a second feed-forward neural network. We find that our method learns continually under domain transfer with sparse bursts of activity in weights that are recycled across tasks, rather than by maintaining task-specific modules. Sparse synaptic bursting is found to balance activity and suppression such that new functions can be learned without corrupting extant knowledge, thus mirroring the balance of order and disorder in systems at the edge of chaos. This behavior emerges during a prior pre-training (or 'meta-learning') phase in which regulated synapses are selectively disinhibited, or grown, from an initial state of uniform suppression through prediction error minimization.
Intersections of various spectra of operator matrices

This paper is concerned with general n × n upper triangular operator matrices with given diagonal entries. We characterize perturbations of the left (right) essential spectrum, the essential spectrum, as well as the left (right) the Weyl spectrum of such operators, thus generalizing and improving some results of [3], [4], [6], [16], [17], [25] from n = 2 to case of n ≥ 2. We show that related results hold in arbitrary Banach spaces without assuming separability, thus extending and improving recent results from [22], [23]. For the lack of adjointness and orthogonality in Banach spaces, we use an adequate alternative: the concept of inner generalized inverse.