Reviewerly/exHarmony huggingface dataset:
Deterministic Sparse Pattern Matching via the Baur-Strassen Theorem

How fast can you test whether a constellation of stars appears in the night sky? This question can be modeled as the computational problem of testing whether a set of points $P$ can be moved into (or close to) another set $Q$ under some prescribed group of transformations. Consider, as a simple representative, the following problem: Given two sets of at most $n$ integers $P,Q\subseteq[N]$, determine whether there is some shift $s$ such that $P$ shifted by $s$ is a subset of $Q$, i.e., $P+s=\{p+s:p\in P\}\subseteq Q$. This problem, to which we refer as the Constellation problem, can be solved in near-linear time $O(n\log n)$ by a Monte Carlo randomized algorithm [Cardoze, Schulman; FOCS'98] and time $O(n\log^2 N)$ by a Las Vegas randomized algorithm [Cole, Hariharan; STOC'02]. Moreover, there is a deterministic algorithm running in time $n\cdot2^{O(\sqrt{\log n\log\log N})}$ [Chan, Lewenstein; STOC'15]. An interesting question left open by these previous works is whether Constellation is in deterministic near-linear time (i.e., with only polylogarithmic overhead). We answer this question positively by giving an $n\cdot(\log N)^{O(1)}$-time deterministic algorithm for the Constellation problem. Our algorithm extends to various more complex Point Pattern Matching problems in higher dimensions, under translations and rigid motions, and possibly with mismatches, and also to a near-linear-time derandomization of the Sparse Wildcard Matching problem on strings. We find it particularly interesting how we obtain our deterministic algorithm. All previous algorithms are based on the same baseline idea, using additive hashing and the Fast Fourier Transform. In contrast, our algorithms are based on new ideas, involving a surprising blend of combinatorial and algebraic techniques. At the heart lies an innovative application of the Baur-Strassen theorem from algebraic complexity theory.
BUT CHiME-7 system description

This paper describes the joint effort of Brno University of Technology (BUT), AGH University of Krakow and University of Buenos Aires on the development of Automatic Speech Recognition systems for the CHiME-7 Challenge. We train and evaluate various end-to-end models with several toolkits. We heavily relied on Guided Source Separation (GSS) to convert multi-channel audio to single channel. The ASR is leveraging speech representations from models pre-trained by self-supervised learning, and we do a fusion of several ASR systems. In addition, we modified external data from the LibriSpeech corpus to become a close domain and added it to the training. Our efforts were focused on the far-field acoustic robustness sub-track of Task 1 - Distant Automatic Speech Recognition (DASR), our systems use oracle segmentation.
Grounded and Well-rounded: A Methodological Approach to the Study of Cross-modal and Cross-lingual Grounding

Grounding has been argued to be a crucial component towards the development of more complete and truly semantically competent artificial intelligence systems. Literature has divided into two camps: While some argue that grounding allows for qualitatively different generalizations, others believe it can be compensated by mono-modal data quantity. Limited empirical evidence has emerged for or against either position, which we argue is due to the methodological challenges that come with studying grounding and its effects on NLP systems. In this paper, we establish a methodological framework for studying what the effects are - if any - of providing models with richer input sources than text-only. The crux of it lies in the construction of comparable samples of populations of models trained on different input modalities, so that we can tease apart the qualitative effects of different input sources from quantifiable model performances. Experiments using this framework reveal qualitative differences in model behavior between cross-modally grounded, cross-lingually grounded, and ungrounded models, which we measure both at a global dataset level as well as for specific word representations, depending on how concrete their semantics is.
Rather a Nurse than a Physician -- Contrastive Explanations under Investigation

Contrastive explanations, where one decision is explained in contrast to another, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative. This claim has never been empirically validated. We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals). We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings. A cross-comparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans. Moreover, model-based explanations computed in both settings align equally well with human rationales. Thus, we empirically find that humans do not necessarily explain in a contrastive manner.9 pages, long paper at ACL 2022 proceedings.
Recasting Continual Learning as Sequence Modeling

In this work, we aim to establish a strong connection between two significant bodies of machine learning research: continual learning and sequence modeling. That is, we propose to formulate continual learning as a sequence modeling problem, allowing advanced sequence models to be utilized for continual learning. Under this formulation, the continual learning process becomes the forward pass of a sequence model. By adopting the meta-continual learning (MCL) framework, we can train the sequence model at the meta-level, on multiple continual learning episodes. As a specific example of our new formulation, we demonstrate the application of Transformers and their efficient variants as MCL methods. Our experiments on seven benchmarks, covering both classification and regression, show that sequence models can be an attractive solution for general MCL.
Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization

We consider decentralized gradient-free optimization of minimizing Lipschitz continuous functions that satisfy neither smoothness nor convexity assumption. We propose two novel gradient-free algorithms, the Decentralized Gradient-Free Method (DGFM) and its variant, the Decentralized Gradient-Free Method$^+$ (DGFM$^{+}$). Based on the techniques of randomized smoothing and gradient tracking, DGFM requires the computation of the zeroth-order oracle of a single sample in each iteration, making it less demanding in terms of computational resources for individual computing nodes. Theoretically, DGFM achieves a complexity of $\mathcal O(d^{3/2}\delta^{-1}\varepsilon ^{-4})$ for obtaining an $(\delta,\varepsilon)$-Goldstein stationary point. DGFM$^{+}$, an advanced version of DGFM, incorporates variance reduction to further improve the convergence behavior. It samples a mini-batch at each iteration and periodically draws a larger batch of data, which improves the complexity to $\mathcal O(d^{3/2}\delta^{-1} \varepsilon^{-3})$. Moreover, experimental results underscore the empirical advantages of our proposed algorithms when applied to real-world datasets.
Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scaling of Texts with Large Language Models

Existing text scaling methods often require a large corpus, struggle with short texts, or require labeled data. We develop a text scaling method that leverages the pattern recognition capabilities of generative large language models (LLMs). Specifically, we propose concept-guided chain-of-thought (CGCoT), which uses prompts designed to summarize ideas and identify target parties in texts to generate concept-specific breakdowns, in many ways similar to guidance for human coder content analysis. CGCoT effectively shifts pairwise text comparisons from a reasoning problem to a pattern recognition problem. We then pairwise compare concept-specific breakdowns using an LLM. We use the results of these pairwise comparisons to estimate a scale using the Bradley-Terry model. We use this approach to scale affective speech on Twitter. Our measures correlate more strongly with human judgments than alternative approaches like Wordfish. Besides a small set of pilot data to develop the CGCoT prompts, our measures require no additional labeled data and produce binary predictions comparable to a RoBERTa-Large model fine-tuned on thousands of human-labeled tweets. We demonstrate how combining substantive knowledge with LLMs can create state-of-the-art measures of abstract concepts.
Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learning

In inverse reinforcement learning (IRL), the central objective is to infer underlying reward functions from observed expert behaviors in a way that not only explains the given data but also generalizes to unseen scenarios. This ensures robustness against reward ambiguity where multiple reward functions can equally explain the same expert behaviors. While significant efforts have been made in addressing this issue, current methods often face challenges with high-dimensional problems and lack a geometric foundation. This paper harnesses the optimal transport (OT) theory to provide a fresh perspective on these challenges. By utilizing the Wasserstein distance from OT, we establish a geometric framework that allows for quantifying reward ambiguity and identifying a central representation or centroid of reward functions. These insights pave the way for robust IRL methodologies anchored in geometric interpretations, offering a structured approach to tackle reward ambiguity in high-dimensional settings.
LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation

The convergence of embodied agents and large language models (LLMs) has brought significant advancements to embodied instruction following. Particularly, the strong reasoning capabilities of LLMs make it possible for robots to perform long-horizon tasks without expensive annotated demonstrations. However, public benchmarks for testing the long-horizon reasoning capabilities of language-conditioned robots in various scenarios are still missing. To fill this gap, this work focuses on the tabletop manipulation task and releases a simulation benchmark, \textit{LoHoRavens}, which covers various long-horizon reasoning aspects spanning color, size, space, arithmetics and reference. Furthermore, there is a key modality bridging problem for long-horizon manipulation tasks with LLMs: how to incorporate the observation feedback during robot execution for the LLM's closed-loop planning, which is however less studied by prior work. We investigate two methods of bridging the modality gap: caption generation and learnable interface for incorporating explicit and implicit observation feedback to the LLM, respectively. These methods serve as the two baselines for our proposed benchmark. Experiments show that both methods struggle to solve some tasks, indicating long-horizon manipulation tasks are still challenging for current popular models. We expect the proposed public benchmark and baselines can help the community develop better models for long-horizon tabletop manipulation tasks.
Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs

Knowledge graph entity typing (KGET) aims at inferring plausible types of entities in knowledge graphs. Existing approaches to KGET focus on how to better encode the knowledge provided by the neighbors and types of an entity into its representation. However, they ignore the semantic knowledge provided by the way in which types can be clustered together. In this paper, we propose a novel method called Multi-view Contrastive Learning for knowledge graph Entity Typing (MCLET), which effectively encodes the coarse-grained knowledge provided by clusters into entity and type embeddings. MCLET is composed of three modules: i) Multi-view Generation and Encoder module, which encodes structured information from entity-type, entity-cluster and cluster-type views; ii) Cross-view Contrastive Learning module, which encourages different views to collaboratively improve view-specific representations of entities and types; iii) Entity Typing Prediction module, which integrates multi-head attention and a Mixture-of-Experts strategy to infer missing entity types. Extensive experiments show the strong performance of MCLET compared to the state-of-the-art
A Note on Improving Variational Estimation for Multidimensional Item Response Theory

Survey instruments and assessments are frequently used in many domains of social science. When the constructs that these assessments try to measure become multifaceted, multidimensional item response theory (MIRT) provides a unified framework and convenient statistical tool for item analysis, calibration, and scoring. However, the computational challenge of estimating MIRT models prohibits its wide use because many of the extant methods can hardly provide results in a realistic time frame when the number of dimensions, sample size, and test length are large. Instead, variational estimation methods, such as Gaussian Variational Expectation Maximization (GVEM) algorithm, have been recently proposed to solve the estimation challenge by providing a fast and accurate solution. However, results have shown that variational estimation methods may produce some bias on discrimination parameters during confirmatory model estimation, and this note proposes an importance weighted version of GVEM (i.e., IW-GVEM) to correct for such bias under MIRT models. We also use the adaptive moment estimation method to update the learning rate for gradient descent automatically. Our simulations show that IW-GVEM can effectively correct bias with modest increase of computation time, compared with GVEM. The proposed method may also shed light on improving the variational estimation for other psychometrics models.
Hierarchical Superpixel Relation Graph Combined with Convolutional Sparse Coding for Self-Supervised Hyperspectral Image Denoising

Self-supervised methods have recently been widely used for hyperspectral image (HSI) denoising. As only a single noisy HSI required to be restored is used for learning, effectively exploring the spatial-spectral joint information for HSI representation becomes to be a more critical problem. In this paper, we propose a self-supervised denoiser based on a blind-spot network and a new feature extraction method that combines convolutional neural network (CNN) and graph neural network (GNN). The features from the convolutional backbone network are fed into the GNN as graph nodes. With hierarchical superpixel segmentation of HSI, the feature nodes with contextual and hierarchical relationships are feature-enhanced by the powerful feature interaction capability of GNN. Experimental results show that our method is competitive compared with state-of-the-art methods.
Data Stream Unsupervised Partitioning Based on Optimized Fuzzy C-Means

Data stream partitioning is an important technique in data mining to analyze data streams in real-time. In this context, lots of data stream partitioning methods have been proposed. In the state of the art, most existing methods need to specify the number of classes before partitioning and/or introduce user-defined parameters for which the parameter values may differ for different data sets. In practice, it is difficult to determine the number of classes and the parameter values. Therefore, we propose in this paper an unsupervised and non-parametric method based on the Optimized Fuzzy C-Means algorithm. It has 2 steps. First is to partition a series of data chunks and then partition the intermediate classes formed before. The performance of the proposed algorithm is evaluated and compared with the recent state-of-the-art methods on hyperspectral image data sets.
Blind Hyperspectral Image Restoration

A new blind restoration method for hyperspectral images is proposed in this paper. This method eliminates prior known information and avoids empirical tuning of the regularization parameters. It involves two major steps: the first step is an unsupervised partitioning using affinity propagation to blindly select spectral components and instead of estimating the PSF using all spectral components, we use only the exemplar component of each correlated group. The second step involves restoration of the hyperspectral image using the appropriate estimated PSF for each group. A multi-scale pyramidal model is used to estimate the PSF and the latent image on each selected spectral component. Hyperspectral images are used to assess the proposed method. Three objective criteria are adopted the mean of PSNR, SSIM, and L1-norm of the estimation error.
RGB-Infrared Paired-Images Generation Based on Feature Disentangle and Cross-Modality Reconstruction

Infrared image acquisition is extremely challenging due to the huge spend of manpower and material resources. This paper presents a novel approach for generating paired RGB-infrared images by leveraging feature disentanglement and cross-modality reconstruction. The proposed model effectively extracts unique style features and shared content features from input RGB or infrared images and integrates these features to generate synthetic images corresponding to their respective modalities. This method aims to bridge the gap between different imaging modalities. Experimental evaluations were conducted on the ThermalWorld, LLVIP and SYSU-MM01 datasets, demonstrating the superiority of our method over recent image generation networks in terms of generated infrared image quality. The effectiveness of the model is further evidenced by its superior performance in a pedestrian re-identification task.
Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Stream Data

Online nonparametric estimators are gaining popularity due to their efficient computation and competitive generalization abilities. An important example includes variants of stochastic gradient descent. These algorithms often take one sample point at a time and instantly update the parameter estimate of interest. In this work we consider model selection and hyperparameter tuning for such online algorithms. We propose a weighted rolling-validation procedure, an online variant of leave-one-out cross-validation, that costs minimal extra computation for many typical stochastic gradient descent estimators. Similar to batch cross-validation, it can boost base estimators to achieve a better, adaptive convergence rate. Our theoretical analysis is straightforward, relying mainly on some general statistical stability assumptions. The simulation study underscores the significance of diverging weights in rolling validation in practice and demonstrates its sensitivity even when there is only a slim difference between candidate estimators.
Simulating 2D lattice gauge theories on a qudit quantum computer

Particle physics underpins our understanding of the world at a fundamental level by describing the interplay of matter and forces through gauge theories. Yet, despite their unmatched success, the intrinsic quantum mechanical nature of gauge theories makes important problem classes notoriously difficult to address with classical computational techniques. A promising way to overcome these roadblocks is offered by quantum computers, which are based on the same laws that make the classical computations so difficult. Here, we present a quantum computation of the properties of the basic building block of two-dimensional lattice quantum electrodynamics, involving both gauge fields and matter. This computation is made possible by the use of a trapped-ion qudit quantum processor, where quantum information is encoded in $d$ different states per ion, rather than in two states as in qubits. Qudits are ideally suited for describing gauge fields, which are naturally high-dimensional, leading to a dramatic reduction in the quantum register size and circuit complexity. Using a variational quantum eigensolver, we find the ground state of the model and observe the interplay between virtual pair creation and quantized magnetic field effects. The qudit approach further allows us to seamlessly observe the effect of different gauge field truncations by controlling the qudit dimension. Our results open the door for hardware-efficient quantum simulations with qudits in near-term quantum devices.
An Interactive VR Game Featuring the Story of Migrant Worker Poet in Southwest China

With its unique form of expression and powerful interactivity, virtual reality (VR) art has brought new possibilities for artistic creation and audience experience. It broadens the boundaries of artistic expression, promotes interdisciplinary cooperation, raises the audience’s attention to social issues, and promotes the popularization of aesthetic education. The study developed a visual narrative based on the text of the poetry collection as a virtual experience game to discuss the social issues of the relationship between labor and social networks in the context of today’s technological innovation. Experience highly empathetic artistic storytelling through VR games and how to connect the virtual and physical worlds through textual expression and visual effects. The study realizes the VR linear narrative experience with poems as clues to promote narrative VR interactive experience.
Enhanced Transformer Architecture for Natural Language Processing

Transformer is a state-of-the-art model in the field of natural language processing (NLP). Current NLP models primarily increase the number of transformers to improve processing performance. However, this technique requires a lot of training resources such as computing capacity. In this paper, a novel structure of Transformer is proposed. It is featured by full layer normalization, weighted residual connection, positional encoding exploiting reinforcement learning, and zero masked self-attention. The proposed Transformer model, which is called Enhanced Transformer, is validated by the bilingual evaluation understudy (BLEU) score obtained with the Multi30k translation dataset. As a result, the Enhanced Transformer achieves 202.96% higher BLEU score as compared to the original transformer with the translation dataset.
Intelligent Software Tooling for Improving Software Development

Software has eaten the world with many of the necessities and quality of life services people use requiring software. Therefore, tools that improve the software development experience can have a significant impact on the world such as generating code and test cases, detecting bugs, question and answering, etc., The success of Deep Learning (DL) over the past decade has shown huge advancements in automation across many domains, including Software Development processes. One of the main reasons behind this success is the availability of large datasets such as open-source code available through GitHub or image datasets of mobile Graphical User Interfaces (GUIs) with RICO and ReDRAW to be trained on. Therefore, the central research question my dissertation explores is: In what ways can the software development process be improved through leveraging DL techniques on the vast amounts of unstructured software engineering artifacts?