Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-Task Learning and Adapted Knowledge Models for Emotion-Cause Extraction

Detecting what emotions are expressed in text is a well-studied problem in natural language processing. However, research on finer grained emotion analysis such as what causes an emotion is still in its infancy. We present solutions that tackle both emotion recognition and emotion cause detection in a joint fashion. Considering that common-sense knowledge plays an important role in understanding implicitly expressed emotions and the reasons for those emotions, we propose novel methods that combine common-sense knowledge via adapted knowledge models with multi-task learning to perform joint emotion classification and emotion cause tagging. We show performance improvement on both tasks when including common-sense reasoning and a multitask framework. We provide a thorough analysis to gain insights into model performance.
BinarizedAttack: Structural Poisoning Attacks to Graph-based Anomaly Detection

Graph-based Anomaly Detection (GAD) is becoming prevalent due to the powerful representation abilities of graphs as well as recent advances in graph mining techniques. These GAD tools, however, expose a new attacking surface, ironically due to their unique advantage of being able to exploit the relations among data. That is, attackers now can manipulate those relations (i.e., the structure of the graph) to allow some target nodes to evade detection. In this paper, we exploit this vulnerability by designing a new type of targeted structural poisoning attacks to a representative regression-based GAD system termed OddBall. Specifically, we formulate the attack against OddBall as a bi-level optimization problem, where the key technical challenge is to efficiently solve the problem in a discrete domain. We propose a novel attack method termed BinarizedAttack based on gradient descent. Comparing to prior arts, BinarizedAttack can better use the gradient information, making it particularly suitable for solving combinatorial optimization problems. Furthermore, we investigate the attack transferability of BinarizedAttack by employing it to attack other representation-learning-based GAD systems. Our comprehensive experiments demonstrate that BinarizedAttack is very effective in enabling target nodes to evade graph-based anomaly detection tools with limited attacker's budget, and in the black-box transfer attack setting, BinarizedAttack is also tested effective and in particular, can significantly change the node embeddings learned by the GAD systems. Our research thus opens the door to studying a new type of attack against security analytic tools that rely on graph data.
Does gamification affect flow experience? A systematic literature review

In recent years, studies in different areas have used gamification to improve users' flow experience. However, due to the high variety of the conducted studies and the lack of secondary studies (e.g., systematic literature reviews) in this field, it is difficult to get the state-of-the-art of this research domain. To address this problem, we conducted a systematic literature review to identify i) which gamification design methods have been used in the studies about gamification and Flow Theory, ii) which gamification elements have been used in these studies, iii) which methods have been used to evaluate the users' flow experience in gamified settings, and iv) how gamification affects users' flow experience. The main results show that there is growing interest to this field, as the number of publications is increasing. The most significant interest is in the area of gamification in education. However, there is no unanimity regarding the preferred method of the study or the effects of gamification on users' experience. Our results highlight the importance of conducting new experimental studies investigating how gamification affects the users' flow experience in different gamified settings, applications and domains.
Polarimetric signature of ocean as detected by near-infrared Earthshine observations

Context. The discovery of an extrasolar planet with an ocean has crucial importance in the search for life beyond Earth. The polari- metric detection of specularly reﬂected light from a smooth liquid surface is anticipated theoretically, though the polarimetric signature of Earth’s oceans has not yet been conclusively detected in disk-integrated planetary light. Aims. We aim to detect and measure the polarimetric signature of the Earth’s oceans. Methods. We conducted near-infrared polarimetry for lunar Earthshine and collected data on 32 nights with a variety of ocean frac- tions in the Earthshine-contributing region. Results. A clear positive correlation was revealed between the polarization degree and ocean fraction. We found hourly variations in polarization in accordance with rotational transition of the ocean fraction. The ratios of the variation to the typical polarization degree were as large as ∼ 0.2–1.4. Conclusions. Our observations provide plausible evidence of the polarimetric signature attributed to Earth’s oceans. Near-infrared polarimetry may be considered a prospective technique in the search for exoplanetary oceans.
LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking

Entity linking (EL) is the task of disambiguating mentions appearing in text by linking them to entities in a knowledge graph, a crucial task for text understanding, question answering or conversational systems. In the special case of short-text EL, which poses additional challenges due to limited context, prior approaches have reached good performance by employing heuristics-based methods or purely neural approaches. Here, we take a different, neuro-symbolic approach that combines the advantages of using interpretable rules based on first-order logic with the performance of neural learning. Even though constrained to use rules, we show that we reach competitive or better performance with SoTA black-box neural approaches. Furthermore, our framework has the benefits of extensibility and transferability. We show that we can easily blend existing rule templates given by a human expert, with multiple types of features (priors, BERT encodings, box embeddings, etc), and even with scores resulting from previous EL methods, thus improving on such methods. As an example of improvement, on the LC-QuAD-1.0 dataset, we show more than 3% increase in F1 score relative to previous SoTA. Finally, we show that the inductive bias offered by using logic results in a set of learned rules that transfers from one dataset to another, sometimes without finetuning, while still having high accuracy.
Extraction of a second power‐law tail of the density distribution in simulated clouds

The emergence and development of a power-law tail (PLT) at the high-density end of the observed column-density distribution is thought to be indicative for advanced evolution of star-forming molecular clouds. As shown from many numerical simulations, it corresponds to a morphologically analogous evolution of the mass-density distribution (\rhopdf). The latter may display also a second, shallower PLT at the stage of collapse of the first formed protostellar cores. It is difficult to estimate the parameters of a possible second PLT due to resolution constraints. To address the issue, we extend the method for the extraction of single PLTs from arbitrary density distributions suggested by Veltchev et al.(2019) to detect a second PLT. The technique is elaborated through tests on an analytic \rhopdf{} and applied to a set of hydrodynamical high-resolution simulations of isothermal self-gravitating clouds. In all but one case two PLTs were detected -- the first slope is always steeper and the second one is typically $\partial \ln V /\ln \rho \sim -1$. These results are in a good agreement with numerical and theoretical works and do suggest that the technique extracts correctly double PLTs from smooth PDFs.
Cyclic Shuffle Compatibility

Consider a permutation π to be any finite list of distinct positive integers. A statistic is a function St whose domain is all permutations. Let π σ be the set of shuffles of two disjoint permutations π and σ. We say that St is shuffle compatible if the distribution of St over π σ depends only on St(π), St(σ), and the lengths of π and σ. This notion is implicit in Stanley’s work on P -partitions and was first explicitly studied by Gessel and Zhuang. One of the places where shuffles are useful is in describing the product in the algebra of quasisymmetric functions. Recently Adin, Gessel, Reiner, and Roichman defined an algebra of cyclic quasisymmetric functions where a cyclic version of shuffling comes into play. The purpose of this paper is to define and study cyclic shuffle compatibility. In particular, we show how one can lift shuffle compatibility results for (linear) permutations to cyclic ones. We then apply this result to cyclic descents and cyclic peaks. We also discuss the problem of finding a cyclic analogue of the major index.
Causal Bias Quantification for Continuous Treatment

In this work we develop a novel characterization of marginal causal effect and causal bias in the continuous treatment setting. We show they can be expressed as an expectation with respect to a conditional probability distribution, which can be estimated via standard statistical and probabilistic methods. All terms in the expectations can be computed via automatic differentiation, also for highly nonlinear models. We further develop a new complete criterion for identifiability of causal effects via covariate adjustment, showing the bias equals zero if the criterion is met. We study the effectiveness of our framework in three different scenarios: linear models under confounding, overcontrol and endogenous selection bias; a nonlinear model where full identifiability cannot be achieved because of missing data; a simulated medical study of statins and atherosclerotic cardiovascular disease.
EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2021: Team M3EM Technical Report

In this report, we describe the technical details of our submission to the 2021 EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition. Leveraging multiple modalities has been proved to benefit the Unsupervised Domain Adaptation (UDA) task. In this work, we present Multi-Modal Mutual Enhancement Module (M3EM), a deep module for jointly considering information from multiple modalities to find the most transferable representations across domains. We achieve this by implementing two sub-modules for enhancing each modality using the context of other modalities. The first sub-module exchanges information across modalities through the semantic space, while the second sub-module finds the most transferable spatial region based on the consensus of all modalities.
A Lopsided Outer Solar System?

Axisymmetric disks of eccentric orbits in near-Keplerian potentials are unstable and undergo exponential growth in inclination. Recently, Zderic et al. showed that an idealized disk then saturates to a lopsided mode. Here we show, using N-body simulations, that this apsidal clustering also occurs in a primordial Scattered Disk in the outer solar system, which includes the orbit-averaged gravitational influence of the giant planets. We explain the dynamics using Lynden-Bell's mechanism for bar formation in galaxies. We also show surface density and line-of-sight velocity plots at different times during the instability, highlighting the formation of concentric circles and spiral arms in velocity space.
The Dimpled Manifold Model of Adversarial Examples in Machine Learning

The extreme fragility of deep neural networks, when presented with tiny perturbations in their inputs, was independently discovered by several research groups in 2013. However, despite enormous effort, these adversarial examples remained a counterintuitive phenomenon with no simple testable explanation. In this paper, we introduce a new conceptual framework for how the decision boundary between classes evolves during training, which we call the {\em Dimpled Manifold Model}. In particular, we demonstrate that training is divided into two distinct phases. The first phase is a (typically fast) clinging process in which the initially randomly oriented decision boundary gets very close to the low dimensional image manifold, which contains all the training examples. Next, there is a (typically slow) dimpling phase which creates shallow bulges in the decision boundary that move it to the correct side of the training examples. This framework provides a simple explanation for why adversarial examples exist, why their perturbations have such tiny norms, and why they look like random noise rather than like the target class. This explanation is also used to show that a network that was adversarially trained with incorrectly labeled images might still correctly classify most test images, and to show that the main effect of adversarial training is just to deepen the generated dimples in the decision boundary. Finally, we discuss and demonstrate the very different properties of on-manifold and off-manifold adversarial perturbations. We describe the results of numerous experiments which strongly support this new model, using both low dimensional synthetic datasets and high dimensional natural datasets.
Introductory Economics: Gender, Majors, and Future Performance

By investigating the exam scores of introductory economics in a business school in Taiwan between 2008 and 2019, we find three sets of results: First, we find no significant difference between genders in the exam scores. Second, students’ majors are significantly associated with their exam scores, which likely reflects their academic ability measured at college admission. Third, the exam scores are strong predictors of students’ future academic performance.
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

Vision Transformers (ViT) have been shown to attain highly competitive performance for a wide range of vision applications, such as image classification, object detection and semantic image segmentation. In comparison to convolutional neural networks, the Vision Transformer's weaker inductive bias is generally found to cause an increased reliance on model regularization or data augmentation ("AugReg"for short) when training on smaller training datasets. We conduct a systematic empirical study in order to better understand the interplay between the amount of training data, AugReg, model size and compute budget. As one result of this study we find that the combination of increased compute and AugReg can yield models with the same performance as models trained on an order of magnitude more training data: we train ViT models of various sizes on the public ImageNet-21k dataset which either match or outperform their counterparts trained on the larger, but not publicly available JFT-300M dataset.
The 9-Connected Excluded Minors for the Class of Quasi-graphic Matroids

The class of quasi-graphic matroids, recently introduced by Geelen, Gerards, and Whittle, is minor closed and contains both the class of lifted-graphic matroids and the class of frame matroids, each of which generalises the class of graphic matroids. In this paper, we prove that the matroids U 3 , 7 and U 4 , 7 are the only 9-connected excluded minors for the class of quasi-graphic matroids.
On Invariance Penalties for Risk Minimization

The Invariant Risk Minimization (IRM) principle was first proposed by Arjovsky et al. [2019] to address the domain generalization problem by leveraging data heterogeneity from differing experimental conditions. Specifically, IRM seeks to find a data representation under which an optimal classifier remains invariant across all domains. Despite the conceptual appeal of IRM, the effectiveness of the originally proposed invariance penalty has recently been brought into question. In particular, there exists counterexamples for which that invariance penalty can be arbitrarily small for non-invariant data representations. We propose an alternative invariance penalty by revisiting the Gramian matrix of the data representation. We discuss the role of its eigenvalues in the relationship between the risk and the invariance penalty, and demonstrate that it is ill-conditioned for said counterexamples. The proposed approach is guaranteed to recover an invariant representation for linear settings under mild non-degeneracy conditions. Its effectiveness is substantiated by experiments on DomainBed and InvarianceUnitTest, two extensive test beds for domain generalization.
Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting

Instance-level contrastive learning techniques, which rely on data augmentation and a contrastive loss function, have found great success in the domain of visual representation learning. They are not suitable for exploiting the rich dynamical structure of video however, as operations are done on many augmented instances. In this paper we propose "Video Cross-Stream Prototypical Contrasting", a novel method which predicts consistent prototype assignments from both RGB and optical flow views, operating on sets of samples. Specifically, we alternate the optimization process; while optimizing one of the streams, all views are mapped to one set of stream prototype vectors. Each of the assignments is predicted with all views except the one matching the prediction, pushing representations closer to their assigned prototypes. As a result, more efficient video embeddings with ingrained motion information are learned, without the explicit need for optical flow computation during inference. We obtain state-of-the-art results on nearest-neighbour video retrieval and action recognition, outperforming previous best by +3.2% on UCF101 using the S3D backbone (90.5% Top-1 acc), and by +7.2% on UCF101 and +15.1% on HMDB51 using the R(2+1)D backbone.1
Efficient Self-supervised Vision Transformers for Representation Learning

This paper investigates two techniques for developing efficient self-supervised vision transformers (EsViT) for visual representation learning. First, we show through a comprehensive empirical study that multi-stage architectures with sparse self-attentions can significantly reduce modeling complexity but with a cost of losing the ability to capture fine-grained correspondences between image regions. Second, we propose a new pre-training task of region matching which allows the model to capture fine-grained region dependencies and as a result significantly improves the quality of the learned vision representations. Our results show that combining the two techniques, EsViT achieves 81.3% top-1 on the ImageNet linear probe evaluation, outperforming prior arts with around an order magnitude of higher throughput. When transferring to downstream linear classification tasks, EsViT outperforms its supervised counterpart on 17 out of 18 datasets. The code and models are publicly available: https://github.com/microsoft/esvit
Interaction-driven dynamical quantum phase transitions in a strongly correlated bosonic system

We study dynamical quantum phase transitions (DQPTs) in the extended Bose-Hubbard model after a sudden quench of the nearest-neighbor interaction strength. Using the time-dependent density matrix renormalization group, we demonstrate that interaction-driven DQPTs can appear after quenches between two topologically trivial insulating phases -- a phenomenon that has so far only been studied between gapped and gapless phases. These DQPTs occur when the interaction strength crosses a certain threshold value that does not coincide with the equilibrium phase boundaries, which is in contrast to quenches that involve a change of topology. In order to elucidate the nonequilibrium excitations during the time evolution, we define a new set of string and parity order parameters. We find a close connection between DQPTs and these newly defined order parameters for both types of quenches. In the interaction-driven case, the order parameter exhibits a singularity at the time of the DQPT only when the quench parameter is close to the threshold value. Finally, the timescales of DQPTs are scrutinized and different kinds of power laws are revealed for the topological and interaction-driven cases.
Energy Prioritized Caching for Cellular D2D Networks

Multimedia content transmission heavily taxes network resources and puts a significant burden on wireless systems in terms of capacity and energy consumption. In this context, device-to-device (D2D) paradigm has a utilitarian value to alleviate the network burden by utilizing short-range transmissions with less energy cost. For the realization of proper D2D networking, caching in devices is essential. Additionally, caching needs to operate efficiently with the aim of realizing network-wide energy improvements. To this end, we study multimedia caching in cellular D2D networks and propose an energy prioritized D2D caching (EPDC) algorithm in this work. We also investigate the optimal caching policy in that system setting. The impact of device capacity and D2D transmission range on energy consumption is studied by focusing on different operation modes such as D2D and base station transmissions. According to the simulation results, EPDC algorithm has substantial energy-efficiency gains over the commonly utilized Least Recently Used (LRU) algorithm and content-based caching algorithms PDC and SXO. For larger D2D transmission ranges, this improvement becomes more evident.
PAC Prediction Sets Under Covariate Shift

An important challenge facing modern machine learning is how to rigorously quantify the uncertainty of model predictions. Conveying uncertainty is especially important when there are changes to the underlying data distribution that might invalidate the predictive model. Yet, most existing uncertainty quantification algorithms break down in the presence of such shifts. We propose a novel approach that addresses this challenge by constructing \emph{probably approximately correct (PAC)} prediction sets in the presence of covariate shift. Our approach focuses on the setting where there is a covariate shift from the source distribution (where we have labeled training examples) to the target distribution (for which we want to quantify uncertainty). Our algorithm assumes given importance weights that encode how the probabilities of the training examples change under the covariate shift. In practice, importance weights typically need to be estimated; thus, we extend our algorithm to the setting where we are given confidence intervals for the importance weights. We demonstrate the effectiveness of our approach on covariate shifts based on DomainNet and ImageNet. Our algorithm satisfies the PAC constraint, and gives prediction sets with the smallest average normalized size among approaches that always satisfy the PAC constraint.
On NI and NJ skew PBW extensions

Abstract We establish necessary or sufficient conditions to guarantee that skew Poincaré–Birkhoff–Witt extensions are NI or NJ rings. Our results extend those corresponding for skew polynomial rings and establish similar properties for other families of noncommutative rings such as universal enveloping algebras and examples of differential operators.