Reviewerly/exHarmony huggingface dataset:
Assessing Distractors in Multiple-Choice Tests

Multiple-choice tests are a common approach for assessing candidates' comprehension skills. Standard multiple-choice reading comprehension exams require candidates to select the correct answer option from a discrete set based on a question in relation to a contextual passage. For appropriate assessment, the distractor answer options must by definition be incorrect but plausible and diverse. However, generating good quality distractors satisfying these criteria is a challenging task for content creators. We propose automated assessment metrics for the quality of distractors in multiple-choice reading comprehension tests. Specifically, we define quality in terms of the incorrectness, plausibility and diversity of the distractor options. We assess incorrectness using the classification ability of a binary multiple-choice reading comprehension system. Plausibility is assessed by considering the distractor confidence - the probability mass associated with the distractor options for a standard multi-class multiple-choice reading comprehension system. Diversity is assessed by pairwise comparison of an embedding-based equivalence metric between the distractors of a question. To further validate the plausibility metric we compare against candidate distributions over multiple-choice questions and agreement with a ChatGPT model's interpretation of distractor plausibility and diversity.
LuminanceL1Loss: A loss function which measures percieved brightness and colour differences

We introduce LuminanceL1Loss, a novel loss function designed to enhance the performance of image restoration tasks. We demonstrate its superiority over MSE when applied to the Retinexformer, BUIFD and DnCNN architectures. Our proposed LuminanceL1Loss leverages a unique approach by transforming images into grayscale and subsequently computing the MSE loss for both grayscale and color channels. Experimental results demonstrate that this innovative loss function consistently outperforms traditional methods, showcasing its potential in image denoising and other related tasks in image reconstruction. It demonstrates gains up to 4.7dB. The results presented in this study highlight the efficacy of LuminanceL1Loss for various image restoration tasks.
Goodness-of-Fit Tests for Linear Non-Gaussian Structural Equation Models

The field of causal discovery develops model selection methods to infer cause-effect relations among a set of random variables. For this purpose, different modelling assumptions have been proposed to render cause-effect relations identifiable. One prominent assumption is that the joint distribution of the observed variables follows a linear non-Gaussian structural equation model. In this paper, we develop novel goodness-of-fit tests that assess the validity of this assumption in the basic setting without latent confounders as well as in extension to linear models that incorporate latent confounders. Our approach involves testing algebraic relations among second and higher moments that hold as a consequence of the linearity of the structural equations. Specifically, we show that the linearity implies rank constraints on matrices and tensors derived from moments. For a practical implementation of our tests, we consider a multiplier bootstrap method that uses incomplete U-statistics to estimate subdeterminants, as well as asymptotic approximations to the null distribution of singular values. The methods are illustrated, in particular, for the T\"ubingen collection of benchmark data sets on cause-effect pairs.
Hybrid Focal and Full-Range Attention Based Graph Transformers

The paradigm of Transformers using the self-attention mechanism has manifested its advantage in learning graph-structured data. Yet, Graph Transformers are capable of modeling full range dependencies but are often deficient in extracting information from locality. A common practice is to utilize Message Passing Neural Networks (MPNNs) as an auxiliary to capture local information, which however are still inadequate for comprehending substructures. In this paper, we present a purely attention-based architecture, namely Focal and Full-Range Graph Transformer (FFGT), which can mitigate the loss of local information in learning global correlations. The core component of FFGT is a new mechanism of compound attention, which combines the conventional full-range attention with K-hop focal attention on ego-nets to aggregate both global and local information. Beyond the scope of canonical Transformers, the FFGT has the merit of being more substructure-aware. Our approach enhances the performance of existing Graph Transformers on various open datasets, while achieves compatible SOTA performance on several Long-Range Graph Benchmark (LRGB) datasets even with a vanilla transformer. We further examine influential factors on the optimal focal length of attention via introducing a novel synthetic dataset based on SBM-PATTERN.
Object-Centric Learning with Slot Mixture Module

Object-centric architectures usually apply a differentiable module to the entire feature map to decompose it into sets of entity representations called slots. Some of these methods structurally resemble clustering algorithms, where the cluster's center in latent space serves as a slot representation. Slot Attention is an example of such a method, acting as a learnable analog of the soft k-means algorithm. Our work employs a learnable clustering method based on the Gaussian Mixture Model. Unlike other approaches, we represent slots not only as centers of clusters but also incorporate information about the distance between clusters and assigned vectors, leading to more expressive slot representations. Our experiments demonstrate that using this approach instead of Slot Attention improves performance in object-centric scenarios, achieving state-of-the-art results in the set property prediction task.
Byzantine-Tolerant Methods for Distributed Variational Inequalities

Robustness to Byzantine attacks is a necessity for various distributed training scenarios. When the training reduces to the process of solving a minimization problem, Byzantine robustness is relatively well-understood. However, other problem formulations, such as min-max problems or, more generally, variational inequalities, arise in many modern machine learning and, in particular, distributed learning tasks. These problems significantly differ from the standard minimization ones and, therefore, require separate consideration. Nevertheless, only one work (Adibi et al., 2022) addresses this important question in the context of Byzantine robustness. Our work makes a further step in this direction by providing several (provably) Byzantine-robust methods for distributed variational inequality, thoroughly studying their theoretical convergence, removing the limitations of the previous work, and providing numerical comparisons supporting the theoretical findings.
Negotiation Strategies in Ubiquitous Human-Computer Interaction: A Novel Storyboards Scale &amp; Field Study

In today's connected society, self-tracking technologies (STTs), such as wearables and mobile fitness apps, empower humans to improve their health and well-being through ubiquitous physical activity monitoring, with several personal and societal benefits. Despite the advances in such technologies' hardware, low user engagement and decreased effectiveness limitations demand more informed and theoretically-founded Human-Computer Interaction designs. To address these challenges, we build upon the previously unexplored Leisure Constraints Negotiation Model and the Transtheoretical Model to systematically define and assess the effectiveness of STTs' features that acknowledge users' contextual constraints and establish human-negotiated STTs narratives. Specifically, we introduce and validate a human-centric scale, StoryWear, which exploits and explores eleven dimensions of negotiation strategies that humans utilize to overcome constraints regarding exercise participation, captured through an inclusive storyboards format. Based on our preliminary studies, StoryWear shows high reliability, rendering it suitable for future work in ubiquitous computing. Our results indicate that negotiation strategies vary in perceived effectiveness and have higher appeal for existing STTs' users, with self-motivation, commitment, and understanding of the negative impact of non-exercise placed at the top. Finally, we give actionable guidelines for real-world implementation and a commentary on the future of personalized training.
Using large language models to study human memory for meaningful narratives

One of the most impressive achievements of the AI revolution is the development of large language models that can generate meaningful text and respond to instructions in plain English with no additional training necessary. Here we show that language models can be used as a scientific instrument for studying human memory for meaningful material. We developed a pipeline for designing large scale memory experiments and analyzing the obtained results. We performed online memory experiments with a large number of participants and collected recognition and recall data for narratives of different lengths. We found that both recall and recognition performance scale linearly with narrative length. Furthermore, in order to investigate the role of narrative comprehension in memory, we repeated these experiments using scrambled versions of the presented stories. We found that even though recall performance declined significantly, recognition remained largely unaffected. Interestingly, recalls in this condition seem to follow the original narrative order rather than the scrambled presentation, pointing to a contextual reconstruction of the story in memory.
Towards Open-world Cross-Domain Sequential Recommendation: A Model-Agnostic Contrastive Denoising Approach

Cross-domain sequential recommendation (CDSR) aims to address the data sparsity problems that exist in traditional sequential recommendation (SR) systems. The existing approaches aim to design a specific cross-domain unit that can transfer and propagate information across multiple domains by relying on overlapping users with abundant behaviors. However, in real-world recommender systems, CDSR scenarios usually consist of a majority of long-tailed users with sparse behaviors and cold-start users who only exist in one domain. This leads to a drop in the performance of existing CDSR methods in the real-world industry platform. Therefore, improving the consistency and effectiveness of models in open-world CDSR scenarios is crucial for constructing CDSR models (\textit{1st} CH). Recently, some SR approaches have utilized auxiliary behaviors to complement the information for long-tailed users. However, these multi-behavior SR methods cannot deliver promising performance in CDSR, as they overlook the semantic gap between target and auxiliary behaviors, as well as user interest deviation across domains (\textit{2nd} CH).
FAIR Knowledge Graphs with Semantic Units: a Prototype

Knowledge graphs and ontologies are becoming increasingly important in the context of making data and metadata findable, accessible, interoperable, and reusable (FAIR). We introduce the concept of Semantic Units for organizing Knowledge Graphs into identifiable and semantically meaningful subgraphs. Each Semantic Unit is represented in the graph by its own resource that instantiates a Semantic Unit class. Different types of Semantic Units are distinguished, and together they can organize a Knowledge Graph into different levels of representational granularity with partially overlapping, partially enclosed subgraphs that users of Knowledge Graphs can refer to for making statements about statements. The use of Semantic Units in Knowledge Graphs supports making them FAIR and increases the human-reader-actionability of their data and metadata by increasing the graph's cognitive interoperability by increasing its explorability for a human reader. We introduce a minimal prototype web application for a user-driven FAIR Knowledge Graph that is based on Semantic Units.
Simultaneous Discovery of Quantum Error Correction Codes and Encoders with a Noise-Aware Reinforcement Learning Agent

In the ongoing race towards experimental implementations of quantum error correction (QEC), finding ways to automatically discover codes and encoding strategies tailored to the qubit hardware platform is emerging as a critical problem. Reinforcement learning (RL) has been identified as a promising approach, but so far it has been severely restricted in terms of scalability. In this work, we significantly expand the power of RL approaches to QEC code discovery. Explicitly, we train an RL agent that automatically discovers both QEC codes and their encoding circuits for a given gate set, qubit connectivity and error model, from scratch. This is enabled by a reward based on the Knill-Laflamme conditions and a vectorized Clifford simulator, allowing us to scale our results to 20 physical qubits and distance 5 codes. Moreover, we introduce the concept of a noise-aware meta-agent, which learns to produce encoding strategies simultaneously for a range of noise models, thus leveraging transfer of insights between different situations. Our approach opens the door towards hardware-adapted accelerated discovery of QEC approaches across the full spectrum of quantum hardware platforms of interest.
Decentralized Personalized Online Federated Learning

Vanilla federated learning does not support learning in an online environment, learning a personalized model on each client, and learning in a decentralized setting. There are existing methods extending federated learning in each of the three aspects. However, some important applications on enterprise edge servers (e.g. online item recommendation at global scale) involve the three aspects at the same time. Therefore, we propose a new learning setting \textit{Decentralized Personalized Online Federated Learning} that considers all the three aspects at the same time. In this new setting for learning, the first technical challenge is how to aggregate the shared model parameters from neighboring clients to obtain a personalized local model with good performance on each client. We propose to directly learn an aggregation by optimizing the performance of the local model with respect to the aggregation weights. This not only improves personalization of each local model but also helps the local model adapting to potential data shift by intelligently incorporating the right amount of information from its neighbors. The second challenge is how to select the neighbors for each client. We propose a peer selection method based on the learned aggregation weights enabling each client to select the most helpful neighbors and reduce communication cost at the same time. We verify the effectiveness and robustness of our proposed method on three real-world item recommendation datasets and one air quality prediction dataset.
Towards Few-Annotation Learning in Computer Vision: Application to Image Classification and Object Detection tasks

In this thesis, we develop theoretical, algorithmic and experimental contributions for Machine Learning with limited labels, and more specifically for the tasks of Image Classification and Object Detection in Computer Vision. In a first contribution, we are interested in bridging the gap between theory and practice for popular Meta-Learning algorithms used in Few-Shot Classification. We make connections to Multi-Task Representation Learning, which benefits from solid theoretical foundations, to verify the best conditions for a more efficient meta-learning. Then, to leverage unlabeled data when training object detectors based on the Transformer architecture, we propose both an unsupervised pretraining and a semi-supervised learning method in two other separate contributions. For pretraining, we improve Contrastive Learning for object detectors by introducing the localization information. Finally, our semi-supervised method is the first tailored to transformer-based detectors.
Parrondo's effect in continuous-time quantum walks

We present the first manifestation of a Parrondo's effect in a continuous-time quantum walk (CTQW). In our protocol we consider a CTQW in the presence of time-dependent transition defect. Our results show that the alternation between defects, that individually are detrimental to the wavepacket spreading, can paradoxically enhance overall wavepacket propagation. Our findings pave the way for the exploration of unconventional mechanisms that can potentially harness the adverse effects of defects to enhance quantum transport.
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

Large Language Models (LLMs) with billions of parameters are prime targets for network pruning, removing some model weights without hurting performance. Prior approaches such as magnitude pruning, SparseGPT, and Wanda, either concentrated solely on weights or integrated weights with activations for sparsity. However, they overlooked the informative gradients derived from pretrained LLMs. In this paper, we present a novel sparsity-centric pruning method for pretrained LLMs, termed Gradient-based Language Model Pruner (GBLM-Pruner). GBLM-Pruner leverages the first-order term of the Taylor expansion, operating in a training-free manner by harnessing properly normalized gradients from a few calibration samples to determine the pruning metric, and substantially outperforms competitive counterparts like SparseGPT and Wanda in multiple benchmarks. Intriguingly, by incorporating gradients, unstructured pruning with our method tends to reveal some structural patterns, which mirrors the geometric interdependence inherent in the LLMs' parameter structure. Additionally, GBLM-Pruner functions without any subsequent retraining or weight updates to maintain its simplicity as other counterparts. Extensive evaluations on LLaMA-1 and LLaMA-2 across various benchmarks show that GBLM-Pruner surpasses magnitude pruning, Wanda and SparseGPT by significant margins. We further extend our approach on Vision Transformer. Our code and models are available at https://github.com/VILA-Lab/GBLM-Pruner.
Air-to-Air Simulated Drone Dataset for AI-powered problems

This paper introduces the multi-view Air-to-Air Simulated Drone Dataset (A2A-SDD), a comprehensive simulated drone dataset captured using AirSim <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">©</sup> . The dataset encompasses diverse scenarios where one or two drones are pursued by one to three monitoring drones. It includes five types of drones, such as DJI models and a generic quadrotor model, recorded in various weather conditions and environments. Both loaded and unloaded drones are represented, and the dataset provides extensive annotations, including object detection and XYZ co-ordinates. The dataset offers potential applications in training deep learning-based models for counter-UAV measures such as localization and payload detection in single- and multi-view cases. Furthermore, preliminary experiments demonstrate the promising performance of trained networks on practical data, affirming the dataset’s value in addressing real-world drone challenges using optical sensors. The synthetic dataset is publicly available on GitHub (https://github.com/CARG-uOttawa/Multiview-Air-to-Air-simulated-drone-dataset).
Cryptanalysis of the «Vershyna» digital signature algorithm

The CRYSTALS-Dilithium digital signature algorithm, which was selected as the prototype of the new «Vershyna» digital signature algorithm, is analyzed in this paper. The characteristics of the National Digital Signature Standard Project and the construction of the «Vershyna» algorithm are also presented. During the analysis of the project, the predicted number of iterations that the algorithm must perform to create the correct signature was calculated. In addition, basic theoretical information about the structure of Fiat-Shamir with aborts and its security in quantum and classical models oracle models is also provided. We obtain our own results on the resistance of the «Vershyna» algorithm to the attack without the use of a message in classical and quantum oracle models. The resistance of the «Vershyna» algorithm to a key recovery attack is based on the assumption of the hardness of the MLWE~problem, and the resistance to existential signature forgery is based on the assumption of the hardness of the MSIS~problem. In this work, the expected level of hardness of SIS~and LWE~problems is calculated, to which there are reductions from MSIS~and MLWE~problems.
Scope Attributes and Systemic Effect in Estimation Practices for Software Projects

Estimation in software projects allows organizations to predict scope, needed resources, and schedule leading to better performance. Inaccurate estimates may cause misalignment – amongst stakeholders and between the situation and design of the project – reducing performance and confidence in the development process. This paper focuses on software estimation, in particular attributes of scope and their usefulness to estimation. Scope is the tangible outcomes of project tasks. Attributes of scope include software functionality, dependence, and newness. A study of estimation practices began with semi-structured interviews conducted with leading software industry organizations. The interviewed practitioners characterized key steps involved in estimation and evaluated their organization’s use of scope attributes. In addition, interview questions probed the consideration of scope topology and systemic effect. Findings from these interviews suggest that subjective assessment remains the most common method for estimation. Additionally, release level estimation processes are reported as informal and intuition-based, lacking analysis of systemic characteristics of scope especially dependence. Dependencies are often missing or insufficiently considered during estimation yet considered only by some at later stages of development. Findings from the interviews also suggest that estimation relies on priorities of perceived value rather than systemic criticality and project target feasibility. The next stage of this research proposes an expanded survey released to a wider practitioner population. Additionally, the findings suggest a need for longitudinal empirical studies to determine how knowledge of dependencies and analysis of systemic effect are imbued in organizations and become useful in estimation processes.
New Nighttime Dehazing Algorithm Based On Dark Channel Prior Enhancement

Abstract Due to various sources of interference in nighttime haze scenes,the resulting dehazed images are generally dim and dull,with diffuse illumination sources and a poor signal-to-noise ratio when compared with daytime haze scenes.In this paper,we propose a new method for nighttime dehazing using dark channel prior(DCP) enhancement.Firstly,to mitigate the effects of highlight sources and pseudo-light source regions on ambient illumination estimation,a hybrid processing method is introduced combines side window box filtering and fast edge preserving filtering to pre-process nighttime haze images.The ambient illumination estimate of the image is derived based on the DCP theory.Secondly,the adaptive light source matrix mechanism is used to fuse and enhance the transmit-tance of light source and non-light source regions to further improve the initial transmittance problem of the image.After processing,the transmittance map is compensated by gamma correction for the light source.Finally,we replace the atmospheric scattering model to obtain a better dehazed image.The nighttime dehazing method was compared with other image dehazing methods,and significant improvements were found in both subjective and objective evaluations.The obtained images are more in line with the visual characteristics of human eyes,and the evaluation indexes such as PSNR,SSIM and NIQE have been improved.