Miracle-dz/UniFAR-FTUs huggingface dataset:
HumanAL: calibrating human matching beyond a single task

This work offers a novel view on the use of human input as labels, acknowledging that humans may err. We build a novel behavioral profile for human annotators which is used as a feature representation of the provided input. We show that by utilizing black-box machine learning, we can take into account human behavior and calibrate their input to improve the labeling quality. To support our claims and provide a proof-of-concept, we experiment with three different matching tasks, namely, schema matching, entity matching and text matching. Our empirical evaluation suggests that the method can improve the quality of gathered labels in multiple settings including cross-domain (across different matching tasks).
Variance Reduction based Partial Trajectory Reuse to Accelerate Policy Gradient Optimization

Built on our previous study on green simulation assisted policy gradient (GS-PG) focusing on trajectory-based reuse, in this paper, we consider infinite-horizon Markov Decision Processes and create a new importance sampling based policy gradient optimization approach to support dynamic decision making. The existing GS-PG method was designed to learn from complete episodes or process trajectories, which limits its applicability to low-data situations and flexible online process control. To overcome this limitation, the proposed approach can selectively reuse the most related partial trajectories, i.e., the reuse unit is based on per-step or per-decision historical observations. In specific, we create a mixture likelihood ratio (MLR) based policy gradient optimization that can leverage the information from historical state-action transitions generated under different behavioral policies. The proposed variance reduction experience replay (VRER) approach can intelligently select and reuse most relevant transition observations, improve the policy gradient estimation, and accelerate the learning of optimal policy. Our empirical study demonstrates that it can improve optimization convergence and enhance the performance of state-of-the-art policy optimization approaches such as actor-critic method and proximal policy optimizations.
Collective Relevance Labeling for Passage Retrieval

Deep learning for Information Retrieval (IR) requires a large amount of high-quality query-document relevance labels, but such labels are inherently sparse. Label smoothing redistributes some observed probability mass over unobserved instances, often uniformly, uninformed of the true distribution. In contrast, we propose knowledge distillation for informed labeling, without incurring high computation overheads at evaluation time. Our contribution is designing a simple but efficient teacher model which utilizes collective knowledge, to outperform state-of-the-arts distilled from a more complex teacher model. Specifically, we train up to \times8 faster than the state-of-the-art teacher, while distilling the rankings better. Our code is publicly available at https://github.com/jihyukkim-nlp/CollectiveKD.
An Analysis of the Power Imbalance on the Uplink of Power-Domain NOMA

This paper analyzes the power imbalance factor on the uplink of a 2-user Power-domain NOMA system and reveals that the minimum value of the average error probability is achieved when the user signals are perfectly balanced in terms of power as in Multi-User MIMO with power control. The analytic result is obtained by analyzing the pairwise error probability and exploiting a symmetry property of the error events. This result is supported by computer simulations using the QPSK and 16QAM signal formats and uncorrelated Rayleigh fading channels. This finding leads to the questioning of the basic philosophy of Power-domain NOMA and suggests that the best strategy for uncorrelated channels is to perfectly balance the average signal powers received from the users and to use a maximum likelihood receiver for their detection.
Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder

Dense retrievers encode queries and documents and map them in an embedding space using pre-trained language models. These embeddings need to be high-dimensional to fit training signals and guarantee the retrieval effectiveness of dense retrievers. However, these high-dimensional embeddings lead to larger index storage and higher retrieval latency. To reduce the embedding dimensions of dense retrieval, this paper proposes a Conditional Autoencoder (ConAE) to compress the high-dimensional embeddings to maintain the same embedding distribution and better recover the ranking features. Our experiments show that ConAE is effective in compressing embeddings by achieving comparable ranking performance with its teacher model and making the retrieval system more efficient. Our further analyses show that ConAE can alleviate the redundancy of the embeddings of dense retrieval with only one linear layer. All codes of this work are available at https://github.com/NEUIR/ConAE.
The Unreliability of Explanations in Few-Shot In-Context Learning

How can prompting a large language model like GPT-3 with explanations improve in-context learning? We focus speciﬁcally on two NLP tasks that involve reasoning over text, namely question answering and natural language inference. Including explanations in the prompt and having the model generate them does not consistently improve performance in the settings we study, contrary to recent results on symbolic reasoning tasks (Nye et al., 2021; Wei et al., 2022). Despite careful prompting, explanations generated by GPT-3 may not even be factually grounded in the input, even on simple tasks with straightforward extractive explanations. However, these ﬂawed explanations can still be useful as a way to verify GPT-3’s predictions post-hoc. Through analysis in three settings, we show that explanations judged as good by humans—those that are logically consistent with the input and the prediction—usually indicate more accurate predictions. Following these observations, we present a framework for calibrating model predictions based on the reliability of the explanations. Our framework trains calibrators using automatically extracted scores that approximately assess the reliability of explanations, which helps improve performance across three different datasets
Influence of winding configuration on the performance of surface-mounted PM machines

This paper presents a detailed performance comparison of Surface-Mounted PM machines (SMPMMs) equipped with distributed and concentrated windings. In order to reveal the merits and demerits of these kind of winding configurations three SMPMMs have been designed with identical air-gap length, stator and rotor outer and inner dimensions, number of poles and stack length. This comparative investigation is performed by using finite element analysis. Key parameters and performance characteristics, such as leakage-inductance, back EMF, average torque, torque quality, machine losses, and efficiency are compared.
QLEVR: A Diagnostic Dataset for Quantificational Language and Elementary Visual Reasoning

Synthetic datasets have successfully been used to probe visual question-answering datasets for their reasoning abilities. CLEVR (johnson2017clevr), for example, tests a range of visual reasoning abilities. The questions in CLEVR focus on comparisons of shapes, colors, and sizes, numerical reasoning, and existence claims. This paper introduces a minimally biased, diagnostic visual question-answering dataset, QLEVR, that goes beyond existential and numerical quantification and focus on more complex quantifiers and their combinations, e.g., asking whether there are more than two red balls that are smaller than at least three blue balls in an image. We describe how the dataset was created and present a first evaluation of state-of-the-art visual question-answering models, showing that QLEVR presents a formidable challenge to our current models. Code and Dataset are available at https://github.com/zechenli03/QLEVR
Watching the watchers: bias and vulnerability in remote proctoring software

Educators are rapidly switching to remote proctoring and examination software for their testing needs, both due to the COVID-19 pandemic and the expanding virtualization of the education sector. State boards are increasingly utilizing these software for high stakes legal and medical licensing exams. Three key concerns arise with the use of these complex software: exam integrity, exam procedural fairness, and exam-taker security and privacy. We conduct the first technical analysis of each of these concerns through a case study of four primary proctoring suites used in U.S. law school and state attorney licensing exams. We reverse engineer these proctoring suites and find that despite promises of high-security, all their anti-cheating measures can be trivially bypassed and can pose significant user security risks. We evaluate current facial recognition classifiers alongside the classifier used by Examplify, the legal exam proctoring suite with the largest market share, to ascertain their accuracy and determine whether faces with certain skin tones are more readily flagged for cheating. Finally, we offer recommendations to improve the integrity and fairness of the remotely proctored exam experience.
Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning

Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modelling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds and in a relatively complex environment from the nuScenes dataset. Our experimental results demonstrate the ability of spatiotemporal networks to understand scene dynamics without the need for HD-Maps and explicit modeling dynamic objects. We publicly release our occupancy grid dataset based on nuScenes to support further research.
Searching for Broadband Pulsed Beacons from 1883 Stars Using Neural Networks

The search for extraterrestrial intelligence at radio frequencies has largely been focused on continuous-wave narrowband signals. We demonstrate that broadband pulsed beacons are energetically efficient compared to narrowband beacons over longer operational timescales. Here, we report the first extensive survey searching for such broadband pulsed beacons toward 1883 stars as a part of the Breakthrough Listen’s search for advanced intelligent life. We conducted 233 hr of deep observations across 4–8 GHz using the Robert C. Byrd Green Bank Telescope and searched for three different classes of signals with artificial (or negative) dispersion. We report a detailed search—leveraging a convolutional neural network classifier on high-performance GPUs—deployed for the very first time in a large-scale search for signals from extraterrestrial intelligence. Due to the absence of any signal-of-interest from our survey, we place a constraint on the existence of broadband pulsed beacons in our solar neighborhood: ≲1 in 1000 stars have transmitter power densities ≳105 W Hz−1 repeating ≤500 s at these frequencies.
Variational quantum algorithm for unconstrained black box binary optimization: Application to feature selection

We introduce a variational quantum algorithm to solve unconstrained black box binary optimization problems, i.e., problems in which the objective function is given as black box. This is in contrast to the typical setting of quantum algorithms for optimization where a classical objective function is provided as a given Quadratic Unconstrained Binary Optimization problem and mapped to a sum of Pauli operators. Furthermore, we provide theoretical justification for our method based on convergence guarantees of quantum imaginary time evolution.To investigate the performance of our algorithm and its potential advantages, we tackle a challenging real-world optimization problem: feature selection. This refers to the problem of selecting a subset of relevant features to use for constructing a predictive model such as fraud detection. Optimal feature selection---when formulated in terms of a generic loss function---offers little structure on which to build classical heuristics, thus resulting primarily in ‘greedy methods’. This leaves room for (near-term) quantum algorithms to be competitive to classical state-of-the-art approaches. We apply our quantum-optimization-based feature selection algorithm, termed VarQFS, to build a predictive model for a credit risk data set with 20 and 59 input features (qubits) and train the model using quantum hardware and tensor-network-based numerical simulations, respectively. We show that the quantum method produces competitive and in certain aspects even better performance compared to traditional feature selection techniques used in today's industry.
Detection of Propaganda Techniques in Visuo-Lingual Metaphor in Memes

The exponential rise of social media networks has allowed the production, distribu-tion, and consumption of data at a phenomenal rate. Moreover, the social media revolution has brought a unique phenomenon to social media platforms called Internet memes. Internet memes are one of the most popular contents used on social media, and they can be in the form of images with a witty, catchy, or satirical text description. In this paper, we are dealing with propaganda that is often seen in Internet memes in recent times. Propaganda is communication, which frequently includes psychological and rhetorical techniques to manipulate or inﬂuence an audience to act or respond as the propagandist wants. To detect propaganda in Internet memes, we propose a multimodal deep learning fusion system that fuses the text and image feature representations and outperforms individual models based solely on either text or image modalities.
FisheyeDistill: Self-Supervised Monocular Depth Estimation with Ordinal Distillation for Fisheye Cameras

In this paper, we deal with the problem of monocular depth estimation for ﬁsheye cameras in a self-supervised manner. A known issue of self-supervised depth estimation is that it suffers in low-light/over-exposure conditions and in large homogeneous regions. To tackle this issue, we propose a novel ordinal distillation loss that distills the ordinal information from a large teacher model. Such a teacher model, since having been trained on a large amount of diverse data, can capture the depth ordering information well, but lacks in preserving accurate scene geometry. Combined with self-supervised losses, we show that our model can not only generate reasonable depth maps in challenging environments but also better recover the scene geometry. We further leverage the ﬁsheye cameras of an AR-Glasses device to collect an indoor dataset to facilitate evaluation.
Exploring the structural and optoelectronic properties of natural insulating phlogopite in van der Waals heterostructures

Naturally occurring van der Waals crystals have brought unprecedented interest to nanomaterial researchers in recent years. So far, more than 1800 layered materials (LMs) have been identified but only a few insulating and naturally occurring LMs were deeply investigated. Phyllosilicate minerals, which are a class of natural and abundant LMs, have been recently considered as a low-cost source of insulating nanomaterials. Within this family an almost barely explored material emerges: phlogopite (KMg3(AlSi3)O10(OH)2). Here we carry out a high throughput characterization of this LM by employing several experimental techniques, corroborating the major findings with first-principles calculations. We show that monolayers (1L) and few-layers of this material are air and temperature stable, as well as easily obtained by the standard mechanical exfoliation technique, have an atomically flat surface, and lower bandgap than its bulk counterpart, an unusual trend in LMs. We also systematically study the basic properties of ultrathin phlogopite and demonstrate that natural phlogopite presents iron impurities in its crystal lattice, which decreases its bandgap from about 7 eV to 3.6 eV. Finally, we combine phlogopite crystals with 1L-WS2 in ultrathin van der Waals heterostructures and present a photoluminescence study, revealing a significant enhancement on the 1L-WS2 optical quality (i.e. higher recombination efficiency through neutral excitons) similarly to that obtained on 1L-WS2/hexagonal boron nitride heterostructures. Our proof-of-concept study shows that phlogopite should be regarded as a good and promising candidate for LM-based applications as a low-cost layered nanomaterial.
Generalized Reconfigurable Intelligent Surfaces: From Transmitting and Reflecting Modes to Single-, Group-, and Fully-Connected Architectures

—Reconﬁgurable intelligent surfaces (RISs) are envisioned as a promising technology for future wireless communications. With various hardware realizations, RISs can work under different modes (reﬂective/transmissive/hybrid) or have different architectures (single-/group-/fully-connected). However, most existing research focused on either reﬂective RISs or single-connected hybrid RISs while there is lack of a comprehensive study for RISs unifying different modes/architectures. In this paper, we solve this issue by analyzing and proposing a general RIS-aided communication model which uniﬁes the reﬂective/transmissive/hybrid modes and single-/group-/fully-connected architectures. With the proposed model, we consider jointly designing the transmit precoder and RIS beamformer to maximize the sum-rate for RIS-aided systems. Leveraging fractional programming theory, the original sum-rate maximization problem is equivalently transformed into a multi-block optimization, which can be solved by block coordinate descent methods. We also provide simulation results to compare the performance of RISs with different modes/architectures. Compared with single-connected hybrid RISs, fully-and group-connected hybrid RISs can increase the sum-rate by around 75% and 37% for Rayleigh fading channels.
Low-Dimensional Invariant Embeddings for Universal Geometric Learning

This paper studies separating invariants: mappings on D -dimensional domains which are invariant to an appropriate group action and which separate orbits. The motivation for this study comes from the usefulness of separating invariants in proving universality of equivariant neural network architectures. We observe that in several cases the cardinality of separating invariants proposed in the machine learning literature is much larger than the dimension D . As a result, the theoretical universal constructions based on these separating invariants are unrealistically large. Our goal in this paper is to resolve this issue. We show that when a continuous family of semi-algebraic separating invariants is available, separation can be obtained by randomly selecting $$2D+1 $$ 2 D + 1 of these invariants. We apply this methodology to obtain an efficient scheme for computing separating invariants for several classical group actions which have been studied in the invariant learning literature. Examples include matrix multiplication actions on point clouds by permutations, rotations, and various other linear groups. Often the requirement of invariant separation is relaxed and only generic separation is required. In this case, we show that only $$D+1$$ D + 1 invariants are required. More importantly, generic invariants are often significantly easier to compute, as we illustrate by discussing generic and full separation for weighted graphs. Finally we outline an approach for proving that separating invariants can be constructed also when the random parameters have finite precision.
Finite-time bounds on the probabilistic violation of the second law of thermodynamics

Jarzynski’s equality sets a strong bound on the probability of violating the second law of thermodynamics by extracting work beyond the free energy difference. We derive finite-time refinements to this bound for driven systems in contact with a thermal Markovian environment, which can be expressed in terms of the geometric notion of thermodynamic length. We show that finite-time protocols converge to Jarzynski’s bound at a rate slower than 1/\sqrt{\tau}1/τ, where \tauτ is the total time of the work-extraction protocol. Our result highlights a new application of minimal dissipation processes and demonstrates a connection between thermodynamic geometry and the higher order statistical properties of work.
Boundary condition analysis of first and second order topological insulators

We analytically study boundary conditions of the Dirac fermion models on a lattice, which describe the first and second order topological insulators. We obtain the dispersion relations of the edge and hinge states by solving these boundary conditions, and clarify that the Hamiltonian symmetry may provide a constraint on the boundary condition. We also demonstrate the edge-hinge analog of the bulk-edge correspondence, in which the nontrivial topology of the gapped edge state ensures gaplessness of the hinge state.
Let's go to the Alien Zoo: Introducing an experimental framework to study usability of counterfactual explanations for machine learning

Introduction To foster usefulness and accountability of machine learning (ML), it is essential to explain a model's decisions in addition to evaluating its performance. Accordingly, the field of explainable artificial intelligence (XAI) has resurfaced as a topic of active research, offering approaches to address the “how” and “why” of automated decision-making. Within this domain, counterfactual explanations (CFEs) have gained considerable traction as a psychologically grounded approach to generate post-hoc explanations. To do so, CFEs highlight what changes to a model's input would have changed its prediction in a particular way. However, despite the introduction of numerous CFE approaches, their usability has yet to be thoroughly validated at the human level. Methods To advance the field of XAI, we introduce the Alien Zoo, an engaging, web-based and game-inspired experimental framework. The Alien Zoo provides the means to evaluate usability of CFEs for gaining new knowledge from an automated system, targeting novice users in a domain-general context. As a proof of concept, we demonstrate the practical efficacy and feasibility of this approach in a user study. Results Our results suggest the efficacy of the Alien Zoo framework for empirically investigating aspects of counterfactual explanations in a game-type scenario and a low-knowledge domain. The proof of concept study reveals that users benefit from receiving CFEs compared to no explanation, both in terms of objective performance in the proposed iterative learning task, and subjective usability. Discussion With this work, we aim to equip research groups and practitioners with the means to easily run controlled and well-powered user studies to complement their otherwise often more technology-oriented work. Thus, in the interest of reproducible research, we provide the entire code, together with the underlying models and user data: https://github.com/ukuhl/IntroAlienZoo.