Reviewerly/exHarmony huggingface dataset:
Reconstruction Probing

We propose reconstruction probing, a new analysis method for contextualized representations based on reconstruction probabilities in masked language models (MLMs). This method relies on comparing the reconstruction probabilities of tokens in a given sequence when conditioned on the representation of a single token that has been fully contextualized and when conditioned on only the decontextualized lexical prior of the model. This comparison can be understood as quantifying the contribution of contextualization towards reconstruction -- the difference in the reconstruction probabilities can only be attributed to the representational change of the single token induced by contextualization. We apply this analysis to three MLMs and find that contextualization boosts reconstructability of tokens that are close to the token being reconstructed in terms of linear and syntactic distance. Furthermore, we extend our analysis to finer-grained decomposition of contextualized representations, and we find that these boosts are largely attributable to static and positional embeddings at the input layer.
NADBenchmarks -- a compilation of Benchmark Datasets for Machine Learning Tasks related to Natural Disasters

Climate change has increased the intensity, frequency, and duration of extreme weather events and natural disasters across the world. While the increased data on natural disasters improves the scope of machine learning (ML) in this field, progress is relatively slow. One bottleneck is the lack of benchmark datasets that would allow ML researchers to quantify their progress against a standard metric. The objective of this short paper is to explore the state of benchmark datasets for ML tasks related to natural disasters, categorizing them according to the disaster management cycle. We compile a list of existing benchmark datasets introduced in the past five years. We propose a web platform - NADBenchmarks - where researchers can search for benchmark datasets for natural disasters, and we develop a preliminary version of such a platform using our compiled list. This paper is intended to aid researchers in finding benchmark datasets to train their ML models on, and provide general directions for topics where they can contribute new benchmark datasets.
Multi-hop Evidence Retrieval for Cross-document Relation Extraction

Relation Extraction (RE) has been extended to cross-document scenarios because many relations are not simply described in a single document. This inevitably brings the challenge of efficient open-space evidence retrieval to support the inference of cross-document relations, along with the challenge of multi-hop reasoning on top of entities and evidence scattered in an open set of documents. To combat these challenges, we propose MR.COD (Multi-hop evidence retrieval for Cross-document relation extraction), which is a multi-hop evidence retrieval method based on evidence path mining and ranking. We explore multiple variants of retrievers to show evidence retrieval is essential in cross-document RE. We also propose a contextual dense retriever for this setting. Experiments on CodRED show that evidence retrieval with MR.COD effectively acquires crossdocument evidence and boosts end-to-end RE performance in both closed and open settings.
A stochastic method to compute the $L^2$ localisation landscape

The $L^2$ localisation landscape of L. Herviou and J. H. Bardarson is a generalisation of the localisation landscape of M. Filoche and S. Mayboroda. We propose a stochastic method to compute the $L^2$ localisation landscape that enables the calculation of landscapes using sparse matrix methods. We also propose an energy filtering of the $L^2$ landscape which can be used to focus on eigenstates with energies in any chosen range of the energy spectrum. We demonstrate the utility of these suggestions by applying the $L^2$ landscape to Anderson's model of localisation in one and two dimensions, and also to localisation in a model of the quantum Hall effect.
Probing The Unitarity of Quantum Evolution Through Periodic Driving

As quantum computers and simulators begin to produce results that cannot be verified classically, it becomes imperative to develop a variety of tools to detect and diagnose experimental errors on these devices. While state or process tomography is a natural way to characterize sources of experimental error, the intense measurement requirements make these strategies infeasible in all but the smallest of quantum systems. In this work, we formulate signatures of unitary evolution based on specific properties of periodically driven quantum systems. The absence of these signatures indicates a break either in the unitarity or periodicity condition on the evolution. We experimentally detect incoherent error on a trapped-ion quantum computer using these signatures. Our method is based on repeated measurements of a single observable, making this a low-cost evaluation of error with measurement requirements that scales according to the character of the dynamics, rather than the system size.
Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond

This paper presents a comprehensive survey of low-light image and video enhancement, addressing two primary challenges in the field. The first challenge is the prevalence of mixed over-/under-exposed images, which are not adequately addressed by existing methods. In response, this work introduces two enhanced variants of the SICE dataset: SICE_Grad and SICE_Mix, designed to better represent these complexities. The second challenge is the scarcity of suitable low-light video datasets for training and testing. To address this, the paper introduces the Night Wenzhou dataset, a large-scale, high-resolution video collection that features challenging fast-moving aerial scenes and streetscapes with varied illuminations and degradation. This study also conducts an extensive analysis of key techniques and performs comparative experiments using the proposed and current benchmark datasets. The survey concludes by highlighting emerging applications, discussing unresolved challenges, and suggesting future research directions within the LLIE community. The datasets are available at https://github.com/ShenZheng2000/LLIE_Survey.
Resource Utilization Monitoring for Raw Data Query Processing

Scientific experiments, simulations, and modern applications generate large amounts of data. Data is stored in raw format to avoid the high loading time of traditional database management systems. Researchers have proposed many techniques to improve query execution time for raw data and reduce data loading time for traditional systems. The core of all the proposed techniques is efficient utilization of resources by processing only required data or reducing operations on data. The processed data caching in the main memory or disk can resolve this issue and avoid repeated processing of data. However, limitations of resources like main memory space, storage IO speeds, and additional storage space requirements on disk need to be considered to provide reliable and scalable solutions for cloud or in-house deployments. This paper presents improvements to the raw data query processing framework by integrating a resource monitoring module. The experiments were performed using a scientific dataset known Sloan Digital Sky Survey (SDSS). Analysis of monitored resources revealed that sampling queries had the lowest resource utilization. The PostgresRAW can answer simple 0-JOIN queries faster than PostgreSQL. While one or more JOIN complex queries need to be answered using PostgreSQL to reduce workload execution time (WET). The results section discusses resource requirements of simple, complex, and sampling type queries. The result analysis of query types and resource utilization patterns assisted in proposing Query Complexity Aware (QCA) and Resource Utilization Aware (RUA) data partitioning techniques for raw engines and DBMS to reduce cost or data to result time.
Requirements Engineering for Artificial Intelligence Systems: A Systematic Mapping Study

[Context] In traditional software systems, Requirements Engineering (RE) activities are well-established and researched. However, building Artificial Intelligence (AI) based software with limited or no insight into the system's inner workings poses significant new challenges to RE. Existing literature has focused on using AI to manage RE activities, with limited research on RE for AI (RE4AI). [Objective] This paper investigates current approaches for specifying requirements for AI systems, identifies available frameworks, methodologies, tools, and techniques used to model requirements, and finds existing challenges and limitations. [Method] We performed a systematic mapping study to find papers on current RE4AI approaches. We identified 43 primary studies and analysed the existing methodologies, models, tools, and techniques used to specify and model requirements in real-world scenarios. [Results] We found several challenges and limitations of existing RE4AI practices. The findings highlighted that current RE applications were not adequately adaptable for building AI systems and emphasised the need to provide new techniques and tools to support RE4AI. [Conclusion] Our results showed that most of the empirical studies on RE4AI focused on autonomous, self-driving vehicles and managing data requirements, and areas such as ethics, trust, and explainability need further research.
Training language models to summarize narratives improves brain alignment

Building systems that achieve a deeper understanding of language is one of the central goals of natural language processing (NLP). Towards this goal, recent works have begun to train language models on narrative datasets which require extracting the most critical information by integrating across long contexts. However, it is still an open question whether these models are learning a deeper understanding of the text, or if the models are simply learning a heuristic to complete the task. This work investigates this further by turning to the one language processing system that truly understands complex language: the human brain. We show that training language models for deeper narrative understanding results in richer representations that have improved alignment to human brain activity. We further find that the improvements in brain alignment are larger for character names than for other discourse features, which indicates that these models are learning important narrative elements. Taken together, these results suggest that this type of training can indeed lead to deeper language understanding. These findings have consequences both for cognitive neuroscience by revealing some of the significant factors behind brain-NLP alignment, and for NLP by highlighting that understanding of long-range context can be improved beyond language modeling.
Critic-Guided Decoding for Controlled Text Generation

Steering language generation towards objectives or away from undesired content has been a long-standing goal in utilizing language models (LM). Recent work has demonstrated reinforcement learning and weighted decoding as effective approaches to achieve a higher level of language control and quality with pros and cons. In this work, we propose a novel critic decoding method for controlled language generation (CriticControl) that combines the strengths of reinforcement learning and weighted decoding. Specifically, we adopt the actor-critic framework to train an LM-steering critic from non-differentiable reward models. And similar to weighted decoding, our method freezes the language model and manipulates the output token distribution using called critic, improving training efficiency and stability. Evaluation of our method on three controlled generation tasks, namely topic control, sentiment control, and detoxification, shows that our approach generates more coherent and well-controlled texts than previous methods. In addition, CriticControl demonstrates superior generalization ability in zero-shot settings. Human evaluation studies also corroborate our findings.
Algorithmic Shadow Spectroscopy

We present shadow spectroscopy as a simulator-agnostic quantum algorithm for estimating energy gaps using very few circuit repetitions (shots) and no extra resources (ancilla qubits) beyond performing time evolution and measurements. The approach builds on the fundamental feature that every observable property of a quantum system must evolve according to the same harmonic components: we can reveal them by post-processing classical shadows of time-evolved quantum states to extract a large number of time-periodic signals $N_o\propto 10^8$, whose frequencies correspond to Hamiltonian energy differences with Heisenberg-limited precision. We provide strong analytical guarantees that (a) quantum resources scale as $O(\log N_o)$, while the classical computational complexity is linear $O(N_o)$, (b) the signal-to-noise ratio increases with the number of analysed signals as $\propto \sqrt{N_o}$, and (c) peak frequencies are immune to reasonable levels of noise. Moreover, performing shadow spectroscopy to probe model spin systems and the excited state conical intersection of molecular CH$_2$ in simulation verifies that the approach is intuitively easy to use in practice, robust against gate noise, amiable to a new type of algorithmic-error mitigation technique, and uses orders of magnitude fewer number of shots than typical near-term quantum algorithms -- as low as 10 shots per timestep is sufficient. Finally, we measured a high-quality, experimental shadow spectrum of a spin chain on readily-available IBM quantum computers, achieving the same precision as in noise-free simulations without using any advanced error mitigation.
C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation

Temporal action segmentation tags action labels for every frame in an input untrimmed video containing multiple actions in a sequence. For the task of temporal action segmentation, we propose an encoder-decoder-style architecture named C2F-TCN featuring a "coarse-to-fine" ensemble of decoder outputs. The C2F-TCN framework is enhanced with a novel model agnostic temporal feature augmentation strategy formed by the computationally inexpensive strategy of the stochastic max-pooling of segments. It produces more accurate and well-calibrated supervised results on three benchmark action segmentation datasets. We show that the architecture is flexible for both supervised and representation learning. In line with this, we present a novel unsupervised way to learn frame-wise representation from C2F-TCN. Our unsupervised learning approach hinges on the clustering capabilities of the input features and the formation of multi-resolution features from the decoder's implicit structure. Further, we provide the first semi-supervised temporal action segmentation results by merging representation learning with conventional supervised learning. Our semi-supervised learning scheme, called ``Iterative-Contrastive-Classify (ICC)'', progressively improves in performance with more labeled data. The ICC semi-supervised learning in C2F-TCN, with 40% labeled videos, performs similar to fully supervised counterparts.
Nervus: A Comprehensive Deep Learning Classification, Regression, and Prognostication Tool for both Medical Image and Clinical Data Analysis

The goal of our research is to create a comprehensive and flexible library that is easy to use for medical imaging research, and capable of handling grayscale images, multiple inputs (both images and tabular data), and multi-label tasks. We have named it Nervus. Based on the PyTorch library, which is suitable for AI for research purposes, we created a four-part model to handle comprehensive inputs and outputs. Nervus consists of four parts. First is the dataloader, then the feature extractor, the feature mixer, and finally the classifier. The dataloader preprocesses the input data, the feature extractor extracts the features between the training data and ground truth labels, feature mixer mixes the features of the extractors, and the classifier classifies the input data from feature mixer based on the task. We have created Nervus, which is a comprehensive and flexible model library that is easy to use for medical imaging research which can handle grayscale images, multi-inputs and multi-label tasks. This will be helpful for researchers in the field of radiology.
NP4G : Network Programming for Generalization

Automatic programming has been actively studied for a long time by various approaches including genetic programming. In recent years, automatic programming using neural networks such as GPT-3 has been actively studied and is attracting a lot of attention. However, these methods are illogical inference based on experience by enormous learning, and their thinking process is unclear. Even using the method by logical inference with a clear thinking process, the system that automatically generates any programs has not yet been realized. Especially, the inductive inference generalized by logical inference from one example is an important issue that the artificial intelligence can acquire knowledge by itself. In this study, we propose NP4G: Network Programming for Generalization, which can automatically generate programs by inductive inference. Because the proposed method can realize "sequence", "selection", and "iteration" in programming and can satisfy the conditions of the structured program theorem, it is expected that NP4G is a method automatically acquire any programs by inductive inference. As an example, we automatically construct a bitwise NOT operation program from several training data by generalization using NP4G. Although NP4G only randomly selects and connects nodes, by adjusting the number of nodes and the number of phase of "Phased Learning", we show the bitwise NOT operation programs are acquired in a comparatively short time and at a rate of about 7 in 10 running. The source code of NP4G is available on GitHub as a public repository.
An AI-Powered VVPAT Counter for Elections in India

The Election Commission of India has introduced Voter Verified Paper Audit Trail since 2019. This mechanism has increased voter confidence at the time of casting the votes. However, physical verification of the VVPATs against the party level counts from the EVMs is done only in 5 (randomly selected) machines per constituency. The time required to conduct physical verification becomes a bottleneck in scaling this activity for 100% of machines in all constituencies. We proposed an automated counter powered by image processing and machine learning algorithms to speed up the process and address this issue.
Error-detectable bosonic entangling gates with a noisy ancilla

Bosonic quantum error correction has proven to be a successful approach for extending the coherence of quantum memories, but to execute deep quantum circuits, high-fidelity gates between encoded qubits are needed. To that end, we present a family of error-detectable two-qubit gates for a variety of bosonic encodings. From a new geometric framework based on a "Bloch sphere" of bosonic operators, we construct $ZZ_L(\theta)$ and $\text{eSWAP}(\theta)$ gates for the binomial, 4-legged cat, dual-rail and several other bosonic codes. The gate Hamiltonian is simple to engineer, requiring only a programmable beamsplitter between two bosonic qubits and an ancilla dispersively coupled to one qubit. This Hamiltonian can be realized in circuit QED hardware with ancilla transmons and microwave cavities. The proposed theoretical framework was developed for circuit QED but is generalizable to any platform that can effectively generate this Hamiltonian. Crucially, one can also detect first-order errors in the ancilla and the bosonic qubits during the gates. We show that this allows one to reach error-detected gate fidelities at the $10^{-4}$ level with today's hardware, limited only by second-order hardware errors.
Continual Learning Approaches for Anomaly Detection

Anomaly Detection is a relevant problem that arises in numerous real-world applications, especially when dealing with images. However, there has been little research for this task in the Continual Learning setting. In this work, we introduce a novel approach called SCALE (SCALing is Enough) to perform Compressed Replay in a framework for Anomaly Detection in Continual Learning setting. The proposed technique scales and compresses the original images using a Super Resolution model which, to the best of our knowledge, is studied for the first time in the Continual Learning setting. SCALE can achieve a high level of compression while maintaining a high level of image reconstruction quality. In conjunction with other Anomaly Detection approaches, it can achieve optimal results. To validate the proposed approach, we use a real-world dataset of images with pixel-based anomalies, with the scope to provide a reliable benchmark for Anomaly Detection in the context of Continual Learning, serving as a foundation for further advancements in the field.
Improving Narrative Relationship Embeddings by Training with Additional Inverse-Relationship Constraints

We consider the problem of embedding character-entity relationships from the reduced semantic space of narratives, proposing and evaluating the assumption that these relationships hold under a reflection operation. We analyze this assumption and compare the approach to a baseline state-of-the-art model with a unique evaluation that simulates efficacy on a downstream clustering task with human-created labels. Although our model creates clusters that achieve Silhouette scores of -.084, outperforming the baseline -.227, our analysis reveals that the models approach the task much differently and perform well on very different examples. We conclude that our assumption might be useful for specific types of data and should be evaluated on a wider range of tasks.
Some aspects of the changes in the task and results of the mathematics state maturity exam in 2012–2021

In this work, the aspects of the task of the state maturity exam in mathematics (structure and content of the program) and the change of students' achievements (difficulty of the tasks and differentiating factor) in 2012–2021 are analyzed. Between 2012 and 2021, the exam program changed once (in 2014; implemented since 2015). The changes in content were minor in all four topic areas. There were structural changes in the types of exam tasks: the number of open-ended questions increased, while the number of multiple-choice and short task questions decreased. Changes in both content and structure were reflected in some changes in outcomes of the exam. Only one statistical difference between the difficulty of the exam (2017 and 2020 exams, which were the easiest and most difficult in 2012–2021, respectively) has been observed.
A Study on Prediction Performance Measurement of Automated Machine Learning

In digital economics, where value creation using big data becomes important, the ability to analyze data using machine learning and deep learning technology is a key activity in corporate activities. Nevertheless, companies consider it difficult to introduce machine learning and artificial intelligence technologies because they need an understanding of the business as well as data and analysis algorithms. Accordingly, services such as automated machine learning have emerged for easy use of machine learning. In this study, the authors explored the automated machine learning service and compared the random forest and extreme gradient boosting analysis results using WiseProphet and Python. WiseProphet is used as a representative of automated machine learning solutions because it is a cloud-based service that anyone can easily access and can be used in various ways. It is contrasted with the model implemented by Python, which writes code with No coding. As a result of comparing the prediction performance, WiseProphet automatically outperformed the analysis result by parameter optimization.