Reviewerly/exHarmony huggingface dataset:
Gotcha: Real-Time Video Deepfake Detection via Challenge-Response

With the rise of AI-enabled Real-Time Deepfakes (RTDFs), the integrity of online video interactions has become a growing concern. RTDFs have now made it feasible to replace an imposter's face with their victim in live video interactions. Such advancement in deepfakes also coaxes detection to rise to the same standard. However, existing deepfake detection techniques are asynchronous and hence ill-suited for RTDFs. To bridge this gap, we propose a challenge-response approach that establishes authenticity in live settings. We focus on talking-head style video interaction and present a taxonomy of challenges that specifically target inherent limitations of RTDF generation pipelines. We evaluate representative examples from the taxonomy by collecting a unique dataset comprising eight challenges, which consistently and visibly degrades the quality of state-of-the-art deepfake generators. These results are corroborated both by humans and a new automated scoring function, leading to 88.6% and 80.1% AUC, respectively. The findings underscore the promising potential of challenge-response systems for explainable and scalable real-time deepfake detection in practical scenarios. We provide access to data and code at https://github.com/mittalgovind/GOTCHA-Deepfakes
Explore Contextual Information for 3D Scene Graph Generation

3D scene graph generation (SGG) has been of high interest in computer vision. Although the accuracy of 3D SGG on coarse classification and single relation label has been gradually improved, the performance of existing works is still far from being perfect for fine-grained and multi-label situations. In this paper, we propose a framework fully exploring contextual information for the 3D SGG task, which attempts to satisfy the requirements of fine-grained entity class, multiple relation labels, and high accuracy simultaneously. Our proposed approach is composed of a Graph Feature Extraction module and a Graph Contextual Reasoning module, achieving appropriate information-redundancy feature extraction, structured organization, and hierarchical inferring. Our approach achieves superior or competitive performance over previous methods on the 3DSSG dataset, especially on the relationship prediction sub-task.
Visual Prompting for Adversarial Robustness

In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time. Compared to conventional adversarial defenses, VP allows us to design universal (i.e., data-agnostic) input prompting templates, which have plug-and-play capabilities at testing time to achieve desired model performance without introducing much computation overhead. Although VP has been successfully applied to improving model generalization, it remains elusive whether and how it can be used to defend against adversarial attacks. We investigate this problem and show that the vanilla VP approach is not effective in adversarial defense since a universal input prompt lacks the capacity for robust learning against sample-specific adversarial perturbations. To circumvent it, we propose a new VP method, termed Class-wise Adversarial Visual Prompting (C-AVP), to generate class-wise visual prompts so as to not only leverage the strengths of ensemble prompts but also optimize their interrelations to improve model robustness. Our experiments show that C-AVP outperforms the conventional VP method, with 2.1X standard accuracy gain and 2X robust accuracy gain. Compared to classical test-time defenses, C-AVP also yields a 42X inference time speedup.
On-Demand Entanglement of Molecules in a Reconfigurable Optical Tweezer Array

Entanglement is crucial to many quantum applications including quantum information processing, simulation of quantum many-body systems, and quantum-enhanced sensing. Molecules, because of their rich internal structure and interactions, have been proposed as a promising platform for quantum science. Deterministic entanglement of individually controlled molecules has nevertheless been a long-standing experimental challenge. Here we demonstrate, for the first time, on-demand entanglement of individually prepared molecules. Using the electric dipolar interaction between pairs of molecules prepared using a reconfigurable optical tweezer array, we realize an entangling two-qubit gate, and use it to deterministically create Bell pairs. Our results demonstrate the key building blocks needed for quantum information processing, simulation of quantum spin models, and quantum-enhanced sensing. They also open up new possibilities such as using trapped molecules for quantum-enhanced fundamental physics tests and exploring collisions and chemical reactions with entangled matter.
SQuId: Measuring Speech Naturalness in Many Languages

Much of text-to-speech research relies on human evaluation, which incurs heavy costs and slows down the development process. The problem is particularly acute in heavily multilingual applications, where recruiting and polling judges can take weeks. We introduce SQuId (Speech Quality Identification), a multilingual naturalness prediction model trained on over a million ratings and tested in 65 locales-the largest effort of this type to date. The main insight is that training one model on many locales consistently outperforms mono-locale baselines. We present our task, the model, and show that it outperforms a competitive baseline based on w2v-BERT and VoiceMOS by 50.0%. We then demonstrate the effectiveness of cross-locale transfer during fine-tuning and highlight its effect on zero-shot locales, i.e., locales for which there is no fine-tuning data. Through a series of analyses, we highlight the role of non-linguistic effects such as sound artifacts in cross-locale transfer. Finally, we present the effect of our design decision, e.g., model size, pre-training diversity, and language rebalancing with several ablation experiments.
Context Generation Improves Open Domain Question Answering

Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge. However, they do not fully exploit the parameterized knowledge. To address this issue, we propose a two-stage, closed-book QA framework which employs a coarse-to-fine approach to extract relevant knowledge and answer a question. Our approach first generates a related context for a given question by prompting a pretrained LM. We then prompt the same LM for answer prediction using the generated context and the question. Additionally, to eliminate failure caused by context uncertainty, we marginalize over generated contexts. Experimental results on three QA benchmarks show that our method significantly outperforms previous closed-book QA methods (e.g. exact matching 68.6% vs. 55.3%), and is on par with open-book methods that exploit external knowledge sources (e.g. 68.6% vs. 68.0%). Our method is able to better exploit the stored knowledge in pretrained LMs without adding extra learnable parameters or needing finetuning, and paves the way for hybrid models that integrate pretrained LMs with external knowledge.
Smooth Trajectory Collision Avoidance through Deep Reinforcement Learning

Collision avoidance is a crucial task in vision-guided autonomous navigation. Solutions based on deep reinforcement learning (DRL) has become increasingly popular. In this work, we proposed several novel agent state and reward function designs to tackle two critical issues in DRL-based navigation solutions: 1) smoothness of the trained flight trajectories; and 2) model generalization to handle unseen environments. Formulated under a DRL framework, our model relies on margin reward and smoothness constraints to ensure UAVs fly smoothly while greatly reducing the chance of collision. The proposed smoothness reward minimizes a combination of first-order and second-order derivatives of flight trajectories, which can also drive the points to be evenly distributed, leading to stable flight speed. To enhance the agent's capability of handling new unseen environments, two practical setups are proposed to improve the invariance of both the state and reward function when deploying in different scenes. Experiments demonstrate the effectiveness of our overall design and individual components.
Improving information retention in large scale online continual learning

Given a stream of data sampled from non-stationary distributions, online continual learning (OCL) aims to adapt efficiently to new data while retaining existing knowledge. The typical approach to address information retention (the ability to retain previous knowledge) is keeping a replay buffer of a fixed size and computing gradients using a mixture of new data and the replay buffer. Surprisingly, the recent work (Cai et al., 2021) suggests that information retention remains a problem in large scale OCL even when the replay buffer is unlimited, i.e., the gradients are computed using all past data. This paper focuses on this peculiarity to understand and address information retention. To pinpoint the source of this problem, we theoretically show that, given limited computation budgets at each time step, even without strict storage limit, naively applying SGD with constant or constantly decreasing learning rates fails to optimize information retention in the long term. We propose using a moving average family of methods to improve optimization for non-stationary objectives. Specifically, we design an adaptive moving average (AMA) optimizer and a moving-average-based learning rate schedule (MALR). We demonstrate the effectiveness of AMA+MALR on large-scale benchmarks, including Continual Localization (CLOC), Google Landmarks, and ImageNet. Code will be released upon publication.
Interactive Language: Talking to Robots in Real Time

We present a framework for building interactive, real-time, natural language-instructable robots in the real world, and we open source related assets (dataset, environment, benchmark, and policies). Trained with behavioral cloning on a dataset of hundreds of thousands of language-annotated trajectories, a produced policy can proficiently execute an order of magnitude more commands than previous works: specifically we estimate a 93.5% success rate on a set of 87,000 unique natural language strings specifying raw end-to-end visuo-linguo-motor skills in the real world. We find that the same policy is capable of being guided by a human via real-time language to address a wide range of precise long-horizon rearrangement goals, e.g. "make a smiley face out of blocks". The dataset we release comprises nearly 600,000 language-labeled trajectories, an order of magnitude larger than prior available datasets. We hope the demonstrated results and associated assets enable further advancement of helpful, capable, natural-language-interactable robots. See videos at https://interactive-language.github.io.
How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization

Despite the clear performance benefits of data augmentations, little is known about why they are so effective. In this paper, we disentangle several key mechanisms through which data augmentations operate. Establishing an exchange rate between augmented and additional real data, we find that in out-of-distribution testing scenarios, augmentations which yield samples that are diverse, but inconsistent with the data distribution can be even more valuable than additional training data. Moreover, we find that data augmentations which encourage invariances can be more valuable than invariance alone, especially on small and medium sized training sets. Following this observation, we show that augmentations induce additional stochasticity during training, effectively flattening the loss landscape.
Are Sample-Efficient NLP Models More Robust?

Recent results in image classification and extractive question answering have observed that pre-trained models trained on less in-distribution data have better out-of-distribution performance. However, it is unclear how broadly these trends hold. We conduct a large empirical study across three tasks, three broadly-applicable modeling interventions (increasing model size, using a different adaptation method, and pre-training on more data), and 14 diverse datasets to investigate the relationship between sample efficiency (amount of data needed to reach a given ID accuracy) and robustness (how models fare on OOD evaluation). We find that higher sample efficiency is only correlated with better average OOD robustness on some modeling interventions and tasks, but not others. On individual datasets, models with lower sample efficiency can even be more robust. These results suggest that general-purpose methods for improving sample efficiency are unlikely to yield universal OOD robustness improvements, since such improvements are highly dataset- and task-dependent. Even in an era of large, multi-purpose pretrained models, task-specific decisions may often be necessary for OOD generalization.
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers a stable performance gain to several state-of-the-art rehearsal CL methods across multiple datasets, both in the presence and absence of pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available at https://github.com/aimagelab/LiDER
Quantum Depth in the Random Oracle Model

We give a comprehensive characterization of the computational power of shallow quantum circuits combined with classical computation. Specifically, for classes of search problems, we show that the following statements hold, relative to a random oracle: (a) $\mathsf{BPP}^{\mathsf{QNC}^{\mathsf{BPP}}} \neq \mathsf{BQP}$. This refutes Jozsa's conjecture [QIP 05] in the random oracle model. As a result, this gives the first instantiatable separation between the classes by replacing the oracle with a cryptographic hash function, yielding a resolution to one of Aaronson's ten semi-grand challenges in quantum computing. (b) $\mathsf{BPP}^{\mathsf{QNC}} \nsubseteq \mathsf{QNC}^{\mathsf{BPP}}$ and $\mathsf{QNC}^{\mathsf{BPP}} \nsubseteq \mathsf{BPP}^{\mathsf{QNC}}$. This shows that there is a subtle interplay between classical computation and shallow quantum computation. In fact, for the second separation, we establish that, for some problems, the ability to perform adaptive measurements in a single shallow quantum circuit, is more useful than the ability to perform polynomially many shallow quantum circuits without adaptive measurements. (c) There exists a 2-message proof of quantum depth protocol. Such a protocol allows a classical verifier to efficiently certify that a prover must be performing a computation of some minimum quantum depth. Our proof of quantum depth can be instantiated using the recent proof of quantumness construction by Yamakawa and Zhandry [STOC 22].
The Role of Parents in the Development of Numerical Literacy in Early Childhood

This study aimed to examine the role of parents in developing numeracy literacy skills in early childhood. This study uses a quantitative research design. The population in this study amounted to 100 respondents with research subjects, namely parents who have school-age children. The sampling technique used is the total sampling technique. The data analysis technique in this study used descriptive statistical techniques. The results showed that the role of parents in the development of numeracy literacy in early childhood was good, with a percentage of 79.68%. These results are supported by the suitability of the results on each question item which includes parents' efforts to improve literacy skills and numeracy literacy skills in early childhood. In this case, it can be interpreted that the role of the family is very important in increasing literacy. The development of numeracy literacy skills in children is the family's responsibility, especially parents as the first and foremost educators for children. Therefore, literacy, as the ability to identify information needs, seek, retrieve, process, and communicate information, must be improved.
Quantum Annealing Sampling with a Bias Field

The presence of a bias field, encoding some information about the target state, can enhance the performance of quantum optimization methods. Here we investigate the effect of such a bias field on the outcome of quantum annealing sampling, at the example of the exact cover problem. The sampling is carried out on a D-Wave machine, and different bias configurations are benchmarked against the unbiased sampling procedure. It is found that the biased annealing algorithm works particularly well for larger problem sizes, where the Hamming distance between bias and target configuration becomes less important. This work motivates future research efforts for finding good bias configurations, either on the quantum machine itself, or in a hybrid fashion via classical algorithms.
LG-MLFormer: Local and Global MLP for Image Captioning

Abstract Self-attention-based image captioning model exists visual features’ spatial information loss problem, introducing relative position encoding can solve the problem to some extent. However, it will bring additional parameters and greater computational complexity. To solve the above problem, we propose a novel local-global MLFormer (LG-MLFormer) with specifically designed encoder module Local-global Multi-Layer Perceptron (LG-MLP). The LG-MLP can capture the latent correlations between different images and its linear stacking calculation mode can reduce computational complexity. It consists of two independent local MLP (LM) modules and a cross-domain global MLP (CDGM) module. The LM specially designs the mapping dimension between linear layers to realize the self-compensation of visual features’ spatial information without introducing relative position encoding. The CDGM module aggregates cross-domain potential correlations between grid-based features and region-based features to realize the complementary advantages of these global and local semantic associations. Experiments on the Karpathy test split and the online test server reveal that our approach provides superior or comparable performance to the state-of-the-art (SOTA). Trained models and code for reproducing the experiments are publicly available at: https://github.com/wxx1921/LGMLFormer-local-and-global-mlp-for-image-captioning.
Treasure Hunt as a Method of Learning Mathematics

Future jobs will require students to handle interdisciplinary knowledge that goes beyond computing techniques when it comes to mathematics education. The traditional approach to education insufficiently develops the needed mathematical skills and mathematical literacy and there is a need to devise new methods of teaching and learning. We introduce game-based learning in mathematics education through carefully designed activities aiming at interdisciplinarity and increasing students’ motivation, mathematical and digital literacy, physical activity, level of knowledge, etc. We propose the treasure hunt cantered around cryptography as an example activity that satisfies the described criteria. Parts of the clues leading to the treasure are hidden using different ciphers, following the historical development of cryptography. The treasure hunt is time-limited to one hour and, to successfully complete the search, students need to learn, adapt, and apply different methods of encryption and decryption. To complete the hunt in time, students must work as a team and effectively divide the tasks among themselves. Students have at their disposal all instructions and manuals in written form, and they are compelled to read with comprehension. We conducted the activity with several groups of high school students with no previous knowledge and skills in the field of cryptography. Students were asked to take a pre-test and post-test survey about their experience and familiarity with cryptography concepts. Most teams completed the search and achieved all learning outcomes. In the paper, we will describe the activity in detail, analyse the test results and give additional examples of game-based activities in mathematics education.
Using mathematics game-based intervention on children with special educational needs: Preliminary findings

Many video games incorporate positive learning principles, stimulating students' cognitive functioning and promoting problem-solving and spatial abilities. The high levels of engagement and involvement that some students can achieve with video games are notable. Hence, video games for children with a long history of school failure, such as children with special educational needs (SEN). Moreover, it may give educators immediate and ongoing assessment of students' progress. When complemented with human tutoring, video games as a game-based intervention may improve mathematics performance since instruction is more effective when adapted to students' learning needs and pace.&#x0D; As part of the research project "GBl4deaf – Game-based Learning for Deaf Students (PTDC/COM-CSS/32022/2017), the video game "Space adventure: Defend the planet!" was designed to stimulate arithmetic competencies in deaf and hearing children. The player must use elementary arithmetical and spatial concepts to rebuild an abandoned space station. Each challenge has three difficulty levels. In the game challenge used in this study, the player must add or remove particles to collect resources. The current research focuses on two questions: The study follows two research questions: Q1: Did the students make any progress in mathematics achievement after playing the video game?; Q2: Is the gameplay of Space adventure: Defend the planet! an engaging experience for players? A pre-and post-game mathematical test was applied to measure mathematics achievement and an observational grid to gather information about arithmetical procedures. Ten fourth- to ninth-graders participated in the study - four girls and six boys, aged between 9 and 16, three deaf and seven hearing students with different special needs (dyscalculia, cognitive deficits, autism spectrum disorder, deafness and Asperger syndrome). Due to the COVID-19 pandemic lockdown, children played the video game using Zoom video conference software in 8-12 sessions (50 minutes, two a three times/week). The results show 4% to 19% of mathematics progression after children played the video game and indicate that they maintain the use of counting-based procedures throughout the game sessions. For instance, they kept counting both addends starting from 1 or counting by 1, 2, 5 or 10 using a number line. The current data suggest that the videogame "Space adventure: Defend the planet!" allows educators to gather immediate information about students' difficulties and progression.