Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Memorization Properties of Contrastive Learning

Memorization studies of deep neural networks (DNNs) help to understand what patterns and how do DNNs learn, and motivate improvements to DNN training approaches. In this work, we investigate the memorization properties of SimCLR, a widely used contrastive self-supervised learning approach, and compare them to the memorization of supervised learning and random labels training. We find that both training objects and augmentations may have different complexity in the sense of how SimCLR learns them. Moreover, we show that SimCLR is similar to random labels training in terms of the distribution of training objects complexity.
Evidential Deep Learning for Open Set Action Recognition

In a real-world scenario, human actions are typically out of the distribution from training data, which requires a model to both recognize the known actions and reject the unknown. Different from image data, video actions are more challenging to be recognized in an open-set setting due to the uncertain temporal dynamics and static bias of human actions. In this paper, we propose a Deep Evidential Action Recognition (DEAR) method to recognize actions in an open testing set. Specifically, we formulate the action recognition problem from the evidential deep learning (EDL) perspective and propose a novel model calibration method to regularize the EDL training. Besides, to mitigate the static bias of video representation, we propose a plug-and-play module to debias the learned representation through contrastive learning. Experimental results show that our DEAR method achieves consistent performance gain on multiple mainstream action recognition models and benchmarks. Code and pre-trained models are available at https://www.rit. edu/actionlab/dear.
Soft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfer

Multilingual pre-trained contextual embedding models (Devlin et al., 2019) have achieved impressive performance on zero-shot cross-lingual transfer tasks. Finding the most effective fine-tuning strategy to fine-tune these models on high-resource languages so that it transfers well to the zero-shot languages is a non-trivial task. In this paper, we propose a novel meta-optimizer to soft-select which layers of the pre-trained model to freeze during fine-tuning. We train the meta-optimizer by simulating the zero-shot transfer scenario. Results on cross-lingual natural language inference show that our approach improves over the simple fine-tuning baseline and X-MAML (Nooralahzadeh et al., 2020).
Human Perception of Audio Deepfakes

The recent emergence of deepfakes has brought manipulated and generated content to the forefront of machine learning research. Automatic detection of deepfakes has seen many new machine learning techniques. Human detection capabilities, however, are far less explored. In this paper, we present results from comparing the abilities of humans and machines for detecting audio deepfakes used to imitate someone's voice. For this, we use a web-based application framework formulated as a game. Participants were asked to distinguish between real and fake audio samples. In our experiment, 410 unique users competed against a state-of-the-art AI deepfake detection algorithm for 13229 total of rounds of the game. We find that humans and deepfake detection algorithms share similar strengths and weaknesses, both struggling to detect certain types of attacks. This is in contrast to the superhuman performance of AI in many application areas such as object detection or face recognition. Concerning human success factors, we find that IT professionals have no advantage over non-professionals but native speakers have an advantage over non-native speakers. Additionally, we find that older participants tend to be more susceptible than younger ones. These insights may be helpful when designing future cybersecurity training for humans as well as developing better detection algorithms.
A Statistical Model of Word Rank Evolution

The availability of large linguistic data sets enables data-driven approaches to study linguistic change. The Google Books corpus unigram frequency data set is used to investigate the word rank dynamics in eight languages. We observed the rank changes of the unigrams from 1900 to 2008 and compared it to a Wright-Fisher inspired model that we developed for our analysis. The model simulates a neutral evolutionary process with the restriction of having no disappearing and added words. This work explains the mathematical framework of the model written as a Markov Chain with multinomial transition probabilities to show how frequencies of words change in time. From our observations in the data and our model, word rank stability shows two types of characteristics: (1) the increase/decrease in ranks are monotonic, or (2) the rank stays the same. Based on our model, high-ranked words tend to be more stable while low-ranked words tend to be more volatile. Some words change in ranks in two ways: (a) by an accumulation of small increasing/decreasing rank changes in time and (b) by shocks of increase/decrease in ranks. Most words in all of the languages we have looked at are rank stable, but not as stable as a neutral model would predict. The stopwords and Swadesh words are observed to be rank stable across eight languages indicating linguistic conformity in established languages. These signatures suggest unigram frequencies in all languages have changed in a manner inconsistent with a purely neutral evolutionary process.
An all-sky sample of intermediate- to high-mass OBA-type eclipsing binaries observed by TESS

Context. Intermediate- to high-mass stars are the least numerous types of stars, and they are less well understood than their more numerous low-mass counterparts in terms of their internal physical processes. Modelling the photometric variability of a large sample of main-sequence intermediate- to high-mass stars in eclipsing binary systems will help to improve the models for such stars. Aims. Our goal is to compose a homogeneously compiled sample of main-sequence intermediate- to high-mass OBA-type dwarfs in eclipsing binary systems from TESS photometry. We search for binaries with and without pulsations and determine their approximate ephemerides. Methods. Our selection starts from a catalogue of dwarfs with colours corresponding to those of OBA-type dwarfs in the TESS Input Catalog. We develop a new automated method aimed at detecting eclipsing binaries in the presence of a strong pulsational and/or rotational signal relative to the eclipse depths and apply it to publicly available 30-min cadence TESS light curves. Results. Using targets with TESS magnitudes below 15 and cuts in the 2MASS magnitude bands of J − H < 0.045 and J − K < 0.06 as the most stringent criteria, we arrive at a total of 189 981 intermediate- to high-mass candidates, 91 193 of which have light curves from at least one of two data reduction pipelines. The eclipsing binary detection and subsequent manual check for false positives resulted in 3155 unique OBA-type eclipsing binary candidates. Conclusions. Our sample of eclipsing binary stars in the intermediate- to high-mass regime allows for future binary (and asteroseismic) modelling with the aim to better understand the internal physical processes in this hot part of the main sequence.
Enumeration of Polyominoes & Polycubes Composed of Magnetic Cubes

This paper examines a family of designs for magnetic cubes and counts how many configurations are possible for each design as a function of the number of modules. Magnetic modular cubes are cubes with magnets arranged on their faces. The magnets are positioned so that each face has either magnetic south or north pole outward. Moreover, we require that the net magnetic moment of the cube passes through the center of opposing faces. These magnetic arrangements enable coupling when cube faces with opposite polarity are brought in close proximity and enable moving the cubes by controlling the orientation of a global magnetic field. This paper investigates the 2D and 3D shapes that can be constructed by magnetic modular cubes, and describes all possible magnet arrangements that obey these rules. We select ten magnetic arrangements and assign a "color" to each of them for ease of visualization and reference. We provide a method to enumerate the number of unique polyominoes and polycubes that can be constructed from a given set of colored cubes. We use this method to enumerate all arrangements for up to 20 modules in 2D and 16 modules in 3D. We provide a motion planner for 2D assembly and through simulations compare which arrangements require fewer movements to generate and which arrangements are more common. Hardware demonstrations explore the self-assembly and disassembly of these modules in 2D and 3D.
Historic and physical wandering domains for wild blender-horseshoes

We present diffeomorphisms of wild blender-horseshoes which belong to Cr (1⩽r<∞) closures of two types of diffeomorphisms, one of which has a historic contracting wandering domain, and the other has a non-trivial Dirac physical measure supported by saddle periodic orbit. It is a non-trivial extension of Colli–Vargas’ model (Colli and Vargas 2001 Ergod. Theory Dyn. Syst. 21 1657–81) to the higher dimensional dynamics with the use of wild blender-horseshoes.
Extension of Additive Valuations to General Valuations on the Existence of EFX

Envy freeness is one of the most widely studied notions in fair division. Because envy-free allocations do not always exist when items are indivisible, several relaxations have been considered. Among them, possibly the most compelling notion is envy freeness up to any item (EFX). Informally speaking, EFX requires that no agent i envies another agent j after the removal of any item in j’s bundle. The existence of EFX allocations is a major open problem. We study the existence of EFX allocations when agents have general valuations. For general valuations, it is known that an EFX allocation always exists (i) when n = 2 or (ii) when all agents have identical valuations, where n is the number of agents. It is also known that an EFX allocation always exists when one can leave at most n − 1 items unallocated. We develop new techniques and extend some results of additive valuations to general valuations on the existence of EFX allocations. We show that an EFX allocation always exists (i) when all agents have one of two general valuations or (ii) when the number of items is at most n + 3. We also show that an EFX allocation always exists when one can leave at most n − 2 items unallocated. In addition to the positive results, we construct an instance with n = 3, in which an existing approach does not work. Funding: This work was partially supported by Kyoto University and Toyota Motor Corporation [Joint Project “Advanced Mathematical Science for Mobility Society”].
Encoding Impact of Network Modification on Controllability via Edge Centrality Matrix

This article develops tools to quantify the importance of agent interactions and its impact on global performance metrics for networks with scalar states modeled as linear time-invariant systems. We consider Gramian-based performance metrics and propose a novel notion of edge centrality that encodes the first-order variation in the metric with respect to the modification of the corresponding edge weight, including for those edges not present in the network. The proposed edge centrality matrix (ECM) is additive over the set of inputs, i.e., it captures the specific contribution to each edge’s centrality of the presence of any given actuator. We provide a full characterization of the ECM structure for the class of directed stem-bud networks, showing that nonzero entries are only possible at specific sub/super-diagonals determined by the network size and the length of its bud. We also provide bounds on the value of the trace, trace inverse, and log-det of the Gramian before and after single-edge modifications, and on the edge-modification weight to ensure the modified network retains stability. Simulations show the utility of the proposed edge centrality notion and validate our results.
Characterizing Social Imaginaries and Self-Disclosures of Dissonance in Online Conspiracy Discussion Communities

Online discussion platforms provide a forum to strengthen and propagate belief in misinformed conspiracy theories. Yet, they also offer avenues for conspiracy theorists to express their doubts and experiences of cognitive dissonance. Such expressions of dissonance may shed light on who abandons misguided beliefs and under what circumstances. This paper characterizes self-disclosures of dissonance about QAnon-a conspiracy theory initiated by a mysterious leader "Q" and popularized by their followers ? anons"-in conspiratorial subreddits. To understand what dissonance and disbelief mean within conspiracy communities, we first characterize their social imaginaries-a broad understanding of how people collectively imagine their social existence. Focusing on 2K posts from two image boards, 4chan and 8chan, and 1.2 M comments and posts from 12 subreddits dedicated to QAnon, we adopt a mixed-methods approach to uncover the symbolic language representing the movement,expectations,practices,heroes and foes of the QAnon community. We use these social imaginaries to create a computational framework for distinguishing belief and dissonance from general discussion about QAnon, surfacing in the 1.2M comments. We investigate the dissonant comments to characterize the dissonance expressed along QAnon social imaginaries. Further, analyzing user engagement with QAnon conspiracy subreddits, we find that self-disclosures of dissonance correlate with a significant decrease in user contributions and ultimately with their departure from the community. Our work offers a systematic framework for uncovering the dimensions and coded language related to QAnon social imaginaries and can serve as a toolbox for studying other conspiracy theories across different platforms. We also contribute a computational framework for identifying dissonance self-disclosures and measuring the changes in user engagement surrounding dissonance. Our work provide insights into designing dissonance based interventions that can potentially dissuade conspiracists from engaging in online conspiracy discussion communities.
A Tandem Framework Balancing Privacy and Security for Voice User Interfaces

Speech synthesis, voice cloning, and voice conversion techniques present severe privacy and security threats to users of voice user interfaces (VUIs). These techniques transform one or more elements of a speech signal, e.g., identity and emotion, while preserving linguistic information. Adversaries may use advanced transformation tools to trigger a spoofing attack using fraudulent biometrics for a legitimate speaker. Conversely, such techniques have been used to generate privacy-transformed speech by suppressing personally identifiable attributes in the voice signals, achieving anonymization. Prior works have studied the security and privacy vectors in parallel, and thus it raises alarm that if a benign user can achieve privacy by a transformation, it also means that a malicious user can break security by bypassing the anti-spoofing mechanism. In this paper, we take a step towards balancing two seemingly conflicting requirements: security and privacy. It remains unclear what the vulnerabilities in one domain imply for the other, and what dynamic interactions exist between them. A better understanding of these aspects is crucial for assessing and mitigating vulnerabilities inherent with VUIs and building effective defenses. In this paper,(i) we investigate the applicability of the current voice anonymization methods by deploying a tandem framework that jointly combines anti-spoofing and authentication models, and evaluate the performance of these methods;(ii) examining analytical and empirical evidence, we reveal a duality between the two mechanisms as they offer different ways to achieve the same objective, and we show that leveraging one vector significantly amplifies the effectiveness of the other;(iii) we demonstrate that to effectively defend from potential attacks against VUIs, it is necessary to investigate the attacks from multiple complementary perspectives(security and privacy).
Performance landscape of resource-constrained platforms targeting DNNs

Over the recent years, a significant number of complex, deep neural networks have been developed for a variety of applications including speech and face recognition, computer vision in the areas of health-care, automatic translation, image classification, etc. Moreover, there is an increasing demand in deploying these networks in resource-constrained edge devices. As the computational demands of these models keep increasing, pushing to their limits the targeted devices, the constant development of new hardware systems tailored to those workloads has been observed. Since programmability of these diverse and complex platforms -- compounded by the rapid development of new DNN models -- is a major challenge, platform vendors have developed Machine Learning tailored SDKs to maximize the platform's performance. This work investigates the performance achieved on a number of modern commodity embedded platforms coupled with the vendors' provided software support when state-of-the-art DNN models from image classification, object detection and image segmentation are targeted. The work quantifies the relative latency gains of the particular embedded platforms and provides insights on the relationship between the required minimum batch size for achieving maximum throughput, concluding that modern embedded systems reach their maximum performance even for modest batch sizes when a modern state of the art DNN model is targeted. Overall, the presented results provide a guide for the expected performance for a number of state-of-the-art DNNs on popular embedded platforms across the image classification, detection and segmentation domains.
Engineering quantum states and electronic landscapes through surface molecular nanoarchitectures

Surfaces are at the frontier of every known solid. They provide versatile supports for functional nanostructures and mediate essential physicochemical processes. Being intimately related with 2D materials, interfaces and atomically thin films often feature distinct electronic states with respect to the bulk, which are key for many relevant properties, such as catalytic activity, interfacial charge-transfer, or crystal growth mechanisms. Of particular interest is reducing the surface electrons' dimensionality and spread with atomic precision, to induce novel quantum properties via lateral scattering and confinement. Both atomic manipulation and supramolecular principles provide access to custom-designed molecular superlattices, which tailor the surface electronic landscape and influence fundamental chemical and physical properties at the nanoscale. Herein, we review the confinement of surface state electrons focusing on their interaction with molecule-based scaffolds created by molecular manipulation and self-assembly protocols under ultrahigh vacuum conditions. Starting from the quasi-free 2D electron gas present at the (111)-terminated surface planes of noble metals, we illustrate the enhanced molecule-based structural complexity and versatility compared to simple atoms. We survey low-dimensional confining structures in the form of artificial lattices, molecular nanogratings or quantum dot arrays, which are constructed upon appropriate choice of their building constituents. Whenever the realized (metal-)organic networks exhibit long-range order, modified surface band structures with characteristic features emerge, revealing intriguing physical properties, such as discretization, quantum coupling or energy and effective mass renormalization. Such collective electronic states can be additionally modified by positioning guest species at the voids of open nanoarchitectures [. ].
Variational Characterization and Identification of Reaction Coordinates in Stochastic Systems

Reaction coordinates are indicators of hidden, low-dimensional mechanisms that govern the long-term behavior of high-dimensional stochastic systems. We present a novel, very general characterization of these coordinates and provide conditions for their existence. We show that these conditions are fulfilled for slow-fast systems, metastable systems, and other systems with known good reaction coordinates. Further, we formulate these conditions as a variational principle, i.e., define a loss function whose minimizers are optimal reaction coordinates. Remarkably, the numerical effort required to evaluate the loss function scales only with the complexity of the underlying, low-dimensional mechanism, and not with that of the full system. In summary, we provide the theoretical foundation for an efficient computation of reaction coordinates via modern machine learning techniques.
Electro‐optic switching with liquid crystal graphene

Graphene oxide (GO) particles in aqueous dispersions can form liquid crystal (LC) phases at extremely low concentrations due to the extremely high aspect ratio of the flakes and noticeably, they possess an extremely large Kerr coefficient attractive for low power consumption electro‐optic devices. Reduced graphene does not easily form LC phases in water due to its hydrophobic nature but here we show that stable dispersions of reduced graphene oxide can be realized with surfactants and that they exhibit birefringence upon shearing as well as under application of electric fields. The performance of the system is largely superior to GO LC possessing longer time stability and drastically improved electro‐optic properties with an induced birefringence twice as large at the same field strength thanks to the almost recovery of graphene properties upon reduction. (© 2016 WILEY‐VCH Verlag GmbH &Co. KGaA, Weinheim)
TLA: Twitter Linguistic Analysis

Linguistics has been instrumental in developing a deeper understanding of human nature. Words are indispensable to bequeath the thoughts, emotions, and purpose of any human interaction, and critically analyzing these words can elucidate the social and psychological behavior and characteristics of these social animals. Social media has become a platform for human interaction on a large scale and thus gives us scope for collecting and using that data for our study. However, this entire process of collecting, labeling, and analyzing this data iteratively makes the entire procedure cumbersome. To make this entire process easier and structured, we would like to introduce TLA(Twitter Linguistic Analysis). In this paper, we describe TLA and provide a basic understanding of the framework and discuss the process of collecting, labeling, and analyzing data from Twitter for a corpus of languages while providing detailed labeled datasets for all the languages and the models are trained on these datasets. The analysis provided by TLA will also go a long way in understanding the sentiments of different linguistic communities and come up with new and innovative solutions for their problems based on the analysis.