Miracle-dz/UniFAR-FTUs huggingface dataset:
Fine-grained Sentiment Analysis with Faithful Attention

While the general task of textual sentiment classification has been widely studied, much less research looks specifically at sentiment between a specified source and target. To tackle this problem, we experimented with a state-of-the-art relation extraction model. Surprisingly, we found that despite reasonable performance, the model's attention was often systematically misaligned with the words that contribute to sentiment. Thus, we directly trained the model's attention with human rationales and improved our model performance by a robust 4~8 points on all tasks we defined on our data sets. We also present a rigorous analysis of the model's attention, both trained and untrained, using novel and intuitive metrics. Our results show that untrained attention does not provide faithful explanations; however, trained attention with concisely annotated human rationales not only increases performance, but also brings faithful explanations. Encouragingly, a small amount of annotated human rationales suffice to correct the attention in our task.
Style Transfer for Texts: to Err is Human, but Error Margins Matter

This paper shows that standard assessment methodology for style transfer has several significant problems. First, the standard metrics for style accuracy and semantics preservation vary significantly on different re-runs. Therefore one has to report error margins for the obtained results. Second, starting with certain values of bilingual evaluation understudy (BLEU) between input and output and accuracy of the sentiment transfer the optimization of these two standard metrics diverge from the intuitive goal of the style transfer task. Finally, due to the nature of the task itself, there is a specific dependence between these two metrics that could be easily manipulated. Under these circumstances, we suggest taking BLEU between input and human-written reformulations into consideration for benchmarks. We also propose three new architectures that outperform state of the art in terms of this metric.
A King’s Ransom for Encryption: Ransomware Classification using Augmented One-Shot Learning and Bayesian Approximation

Newly emerging variants of ransomware pose an ever-growing threat to computer systems governing every aspect of modern life through the handling and analysis of big data. While various recent security-based approaches have focused on ransomware detection at the network or system level, easy-to-use post-infection ransomware classification for the lay user has not been attempted before. In this paper, we investigate the possibility of classifying the ransomware a system is infected with simply based on a screenshot of the splash screen or the ransom note captured using a consumer camera commonly found in any modern mobile device. To train and evaluate our system, we create a sample dataset of the splash screens of 50 well-known ransomware variants. In our dataset, only a single training image is available per ransomware. Instead of creating a large training dataset of ransomware screenshots, we simulate screenshot capture conditions via carefully-designed data augmentation techniques, enabling simple and efficient one-shot learning. Moreover, using model uncertainty obtained via Bayesian approximation, we ensure special input cases such as unrelated non-ransomware images and previously-unseen ransomware variants are correctly identified for special handling and not mis-classified. Extensive experimental evaluation demonstrates the efficacy of our work, with accuracy levels of up to 93.6% for ransomware classification.
Revisiting Heterogeneous Defect Prediction: How Far Are We?

Until now, researchers have proposed several novel heterogeneous defect prediction HDP methods with promising performance. To the best of our knowledge, whether HDP methods can perform significantly better than unsupervised methods has not yet been thoroughly investigated. In this article, we perform a replication study to have a holistic look in this issue. In particular, we compare state-of-the-art five HDP methods with five unsupervised methods. Final results surprisingly show that these HDP methods do not perform significantly better than some of unsupervised methods (especially the simple unsupervised methods proposed by Zhou et al.) in terms of two non-effort-aware performance measures and four effort-aware performance measures. Then, we perform diversity analysis on defective modules via McNemar's test and find the prediction diversity is more obvious when the comparison is performed between the HDP methods and the unsupervised methods than the comparisons only between the HDP methods or between the unsupervised methods. This shows the HDP methods and the unsupervised methods are complementary to each other in identifying defective models to some extent. Finally, we investigate the feasibility of five HDP methods by considering two satisfactory criteria recommended by previous CPDP studies and find the satisfactory ratio of these HDP methods is still pessimistic. The above empirical results implicate there is still a long way for heterogeneous defect prediction to go. More effective HDP methods need to be designed and the unsupervised methods should be considered as baselines.
Demystifying Learning Rate Polices for High Accuracy Training of Deep Neural Networks

Learning Rate (LR) is an important hyperparameter to tune for effective training of deep neural networks (DNNs). Even for the baseline of a constant learning rate, it is non-trivial to choose a good constant value for training a DNN. Dynamic learning rates involve multi-step tuning of LR values at various stages of the training process and offer high accuracy and fast convergence. However, they are much harder to tune. In this paper, we present a comprehensive study of 13 learning rate functions and their associated LR policies by examining their range parameters, step parameters, and value update parameters. We propose a set of metrics for evaluating and selecting LR policies, including the classification confidence, variance, cost, and robustness, and implement them in LRBench, an LR benchmarking system. LRBench can assist end-users and DNN developers to select good LR policies and avoid bad LR policies for training their DNNs. We tested LRBench on Caffe, an open source deep learning framework, to showcase the tuning optimization of LR policies. Evaluated through extensive experiments, we attempt to demystify the tuning of LR policies by identifying good LR policies with effective LR value ranges and step sizes for LR update schedules.
Bell correlations from previously entangled states of light

Based on the Bell theorem and the resulting interpretation of the violation of Bell inequalities by experimental data, it has been believed that a theoretical computation of Bell correlations requires explicit use of the entanglement formalism. Physically this implies a permanent superposition of light waves in spite of the physical separation that occurs in the course of their propagation to spatially separated detectors. Nevertheless, in the present model, Bell correlations are calculated using explicitly separated waves with photon occurrence probabilities given by wave intensities based on nonlinear optics and quantum electrodynamics. The non-locality implied in the conventional calculation is thereby eliminated, but interference effects between photon excited waves and vacuum waves are assumed, as specified to be necessary in the design of Bell-experiment sources.
Enhancing frustrated double ionization with no electronic correlation in triatomic molecules using counter-rotating two-color circular laser fields

We demonstrate significant enhancement of frustrated double ionization (FDI) in the two-electron triatomic molecule D$_{3}^{+}$ when driven by counter-rotating two-color circular (CRTC) laser fields. We employ a three-dimensional semiclassical model that fully accounts for electron and nuclear motion in strong fields. For different pairs of wavelengths, we compute the probabilities of the FDI pathways as a function of the ratio of the two field-strengths. We identify a pathway of frustrated double ionization that is not present in strongly-driven molecules with linear fields. In this pathway the first ionization step is "frustrated" and electronic correlation is essentially absent. This pathway is responsible for enhancing frustrated double ionization with CRTC fields. We also employ a simple model that predicts many of the main features of the probabilities of the FDI pathways as a function of the ratio of the two field-strengths.
Exploring the Mass Segregation Effect of X-Ray Sources in Globular Clusters. II. The Case of Terzan 5

Using archival Chandra observations with a total effective exposure of 734 ks, we derive an updated catalog of point sources in the massive globular cluster (GC) Terzan 5. Our catalog covers an area of 58.1 arcmin2 (R ≤ 4.′3) with 489 X-ray sources, and more than 75% of these sources are first detected in this cluster. We find significant dips in the radial distribution profiles of X-ray sources in Terzan 5, with the projected distance and width of the distribution dips for bright (LX ≳ 9.5 × 1030 erg s−1) X-ray sources larger than those of the faint (LX ≲ 9.5 × 1030 erg s−1) sources. By fitting the radial distribution of the X-ray sources with a “generalized King model,” we estimated an average mass of 1.48 ± 0.11 and 1.27 ± 0.13 M⊙ for the bright and faint X-ray sources, respectively. These results are in agreement with that observed in 47 Tuc, which may suggest a universal mass segregation effect for X-ray sources in GCs. Compared with 47 Tuc, we show that the two-body relaxation timescale of Terzan 5 is much smaller, but its dynamical age is significantly younger than 47 Tuc. These features suggest that the evolution of Terzan 5 is not purely driven by two-body relaxation, and the tidal stripping effect also plays an important role in accelerating the dynamical evolution of this cluster.
A new asymmetric $\epsilon$-insensitive pinball loss function based support vector quantile regression model

In this paper, we propose a novel asymmetric $\epsilon$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $\epsilon$-insensitive zone approach in it but, they fail to extend the $\epsilon$-insensitive approach for quantile estimation in true sense. The proposed asymmetric $\epsilon$-insensitive pinball loss function can make an asymmetric $\epsilon$- insensitive zone of fixed width around the data and divide it using $\tau$ value for the estimation of the $\tau$th quantile. The use of the proposed asymmetric $\epsilon$-insensitive pinball loss function in Support Vector Quantile Regression (SVQR) model improves its prediction ability significantly. It also brings the sparsity back in SVQR model. Further, the numerical results obtained by several experiments carried on artificial and real world datasets empirically show the efficacy of the proposed `$\epsilon$-Support Vector Quantile Regression' ($\epsilon$-SVQR) model over other existing SVQR models.
The Strengths and Behavioral Quirks of Java Bytecode Decompilers

During compilation from Java source code to bytecode, some information is irreversibly lost. In other words, compilation and decompilation of Java code is not symmetric. Consequently, the decompilation process, which aims at producing source code from bytecode, must establish some strategies to reconstruct the information that has been lost. Modern Java decompilers tend to use distinct strategies to achieve proper decompilation. In this work, we hypothesize that the diverse ways in which bytecode can be decompiled has a direct impact on the quality of the source code produced by decompilers. We study the effectiveness of eight Java decompilers with respect to three quality indicators: syntactic correctness, syntactic distortion and semantic equivalence modulo inputs. This study relies on a benchmark set of 14 real-world open-source software projects to be decompiled (2041 classes in total). Our results show that no single modern decompiler is able to correctly handle the variety of bytecode structures coming from real-world programs. Even the highest ranking decompiler in this study produces syntactically correct output for 84% of classes of our dataset and semantically equivalent code output for 78% of classes.
On the Capacity of Secure Distributed Matrix Multiplication

The problem of secure distributed batch matrix multiplication (SDBMM) studies the communication efficiency of retrieving a sequence of desired matrix products ${\bf AB}$ $=$ $({\bf A}_1{\bf B}_1,$ ${\bf A}_2{\bf B}_2,$ $\cdots,$ ${\bf A}_S{\bf B}_S)$ from $N$ distributed servers where the constituent matrices ${\bf A}=({\bf A}_1, {\bf A}_2, \cdots, {\bf A}_S)$ and ${\bf B}=({\bf B}_1, {\bf B}_2,\cdots,{\bf B}_S)$ are stored in $X$-secure coded form, i.e., any group of up to $X$ colluding servers learn nothing about ${\bf A, B}$. It is assumed that ${\bf A}_s\in\mathbb{F}_q^{L\times K}, {\bf B}_s\in\mathbb{F}_q^{K\times M}, s\in\{1,2,\cdots, S\}$ are uniformly and independently distributed and $\mathbb{F}_q$ is a large finite field. The rate of an SDBMM scheme is defined as the ratio of the number of bits of desired information that is retrieved, to the total number of bits downloaded on average. The supremum of achievable rates is called the capacity of SDBMM. In this work we explore the capacity of SDBMM, as well as several of its variants, e.g., where the user may already have either ${\bf A}$ or ${\bf B}$ available as side-information, and/or where the security constraint for either ${\bf A}$ or ${\bf B}$ may be relaxed. We obtain converse bounds, as well as achievable schemes for various cases of SDBMM, depending on the $L, K, M, N, X$ parameters, and identify parameter regimes where these bounds match. A remarkable aspect of our upper bounds is a connection between SDBMM and a form of private information retrieval (PIR) problem, known as multi-message $X$-secure $T$-private information retrieval (MM-XSTPIR). Notable features of our achievable schemes include the use of cross-subspace alignment and a transformation argument that converts a scalar multiplication problem into a scalar addition problem, allowing a surprisingly efficient solution.
SDIT: Scalable and Diverse Cross-domain Image Translation

Recently, image-to-image translation research has witnessed remarkable progress. Although current approaches successfully generate diverse outputs or perform scalable image transfer, these properties have not been combined into a single method. To address this limitation, we propose SDIT: Scalable and Diverse image-to-image translation. These properties are combined into a single generator. The diversity is determined by a latent variable which is randomly sampled from a normal distribution. The scalability is obtained by conditioning the network on the domain attributes. Additionally, we also exploit an attention mechanism that permits the generator to focus on the domain-specific attribute. We empirically demonstrate the performance of the proposed method on face mapping and other datasets beyond faces.
Adaptative Inference Cost With Convolutional Neural Mixture Models

Despite the outstanding performance of convolutional neural networks (CNNs) for many vision tasks, the required computational cost during inference is problematic when resources are limited. In this context, we propose Convolutional Neural Mixture Models (CNMMs), a probabilistic model embedding a large number of CNNs that can be jointly trained and evaluated in an efficient manner. Within the proposed framework, we present different mechanisms to prune subsets of CNNs from the mixture, allowing to easily adapt the computational cost required for inference. Image classification and semantic segmentation experiments show that our method achieve excellent accuracy-compute trade-offs. Moreover, unlike most of previous approaches, a single CNMM provides a large range of operating points along this trade-off, without any re-training.
Message Passing Attention Networks for Document Understanding

Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: this https URL .
The Transference Architecture for Automatic Post-Editing

In automatic post-editing (APE) it makes sense to condition post-editing (pe) decisions on both the source (src) and the machine translated text (mt) as input. This has led to multi-encoder based neural APE approaches. A research challenge now is the search for architectures that best support the capture, preparation and provision of src and mt information and its integration with pe decisions. In this paper we present an efficient multi-encoder based APE model, called transference. Unlike previous approaches, it (i) uses a transformer encoder block for src, (ii) followed by a decoder block, but without masking for self-attention on mt, which effectively acts as second encoder combining src –> mt, and (iii) feeds this representation into a final decoder block generating pe. Our model outperforms the best performing systems by 1 BLEU point on the WMT 2016, 2017, and 2018 English–German APE shared tasks (PBSMT and NMT). Furthermore, the results of our model on the WMT 2019 APE task using NMT data shows a comparable performance to the state-of-the-art system. The inference time of our model is similar to the vanilla transformer-based NMT system although our model deals with two separate encoders. We further investigate the importance of our newly introduced second encoder and find that a too small amount of layers does hurt the performance, while reducing the number of layers of the decoder does not matter much.
Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models

The state-of-the-art pre-trained language representation models, such as Bidirectional Encoder Representations from Transformers (BERT), rarely incorporate commonsense knowledge or other knowledge explicitly. We propose a pre-training approach for incorporating commonsense knowledge into language representation models. We construct a commonsense-related multi-choice question answering dataset for pre-training a neural language representation model. The dataset is created automatically by our proposed "align, mask, and select" (AMS) method. We also investigate different pre-training tasks. Experimental results demonstrate that pre-training models using the proposed approach followed by fine-tuning achieve significant improvements over previous state-of-the-art models on two commonsense-related benchmarks, including CommonsenseQA and Winograd Schema Challenge. We also observe that fine-tuned models after the proposed pre-training approach maintain comparable performance on other NLP tasks, such as sentence classification and natural language inference tasks, compared to the original BERT models. These results verify that the proposed approach, while significantly improving commonsense-related NLP tasks, does not degrade the general language representation capabilities.
The Wolf–Rayet Content of the Galaxies of the Local Group and Beyond

Wolf–Rayet stars (WRs) represent the end of a massive star’s life as it is about to turn into a supernova. Obtaining complete samples of such stars across a large range of metallicities poses observational challenges, but presents us with an exacting way to test current stellar evolutionary theories. A technique we have developed and refined involves interference filter imaging combined with image subtraction and crowded-field photometry. This helps us address one of the most controversial topics in current massive star research: the relative importance of binarity in the evolution of massive stars and formation of WRs. Here, we discuss the current state of the field, including how the observed WR populations match with the predictions of both single and binary star evolutionary models. We end with what we believe are the most important next steps in WR research.
Impact of Planetary Mass Uncertainties on Exoplanet Atmospheric Retrievals

In current models used to interpret exoplanet atmospheric observations, the planetary mass is treated as a prior and is measured/estimated independently with external methods, such as radial velocity or transit timing variation techniques. This approach is necessary as available spectroscopic data do not have sufficient wavelength coverage and/or signal-to-noise to infer the planetary mass. We examine here whether the planetary mass can be directly retrieved from transit spectra as observed by future space observatories, which will provide higher quality spectra. More in general, we quantify the impact of mass uncertainties on spectral retrieval analyses for a host of atmospheric scenarios. Our approach is both analytical and numerical: we first use simple approximations to extract analytically the influence of each atmospheric/planetary parameter to the wavelength-dependent transit depth. We then adopt a fully Bayesian retrieval model to quantify the propagation of the mass uncertainty onto other atmospheric parameters. We found that for clear-sky, gaseous atmospheres the posterior distributions are the same when the mass is known or retrieved. The retrieved mass is very accurate, with a precision of more than 10%, provided the wavelength coverage and signal-to-noise ratio are adequate. When opaque clouds are included in the simulations, the uncertainties in the retrieved mass increase, especially for high altitude clouds. However, atmospheric parameters such as the temperature and trace-gas abundances are unaffected by the knowledge of the mass. Secondary atmospheres, expected to be present in many super-Earths, are more challenging due to the higher degree of freedom for the atmospheric main component, which is unknown. For broad wavelength range and adequate signal-to-noise observations, the mass can still be retrieved accurately and precisely if clouds are not present, and so are all the other atmospheric/planetary parameters. When clouds are added, we find that the mass uncertainties may impact substantially the retrieval of the mean molecular weight: an independent characterization of the mass would therefore be helpful to capture/confirm the main atmospheric constituent.