OpenRaiser/Intern-Atlas huggingface dataset:
A Probabilistic Transformation of Distance-Based Outliers
Venue: arXiv cs.LG 2023
Year: 2023

The scores of distance-based outlier detection methods are difficult to interpret, making it challenging to determine a cut-off threshold between normal and outlier data points without additional context. We describe a generic transformation of distance-based outlier scores into interpretable, probabilistic estimates. The transformation is ranking-stable and increases the contrast between normal and outlier data points. Determining distance relationships between data points is necessary to identify the nearest-neighbor relationships in the data, yet, most of the computed distances are typically discarded. We show that the distances to other data points can be used to model distance probability distributions and, subsequently, use the distributions to turn distance-based outlier scores into outlier probabilities. Our experiments show that the probabilistic transformation does not impact detection performance over numerous tabular and image benchmark datasets but results in interpretable outlier scores with increased contrast between normal and outlier samples. Our work generalizes to a wide range of distance-based outlier detection methods, and because existing distance computations are used, it adds no significant computational overhead.
ANALYSE -- Learning to Attack Cyber-Physical Energy Systems With Intelligent Agents
Venue: arXiv cs.CR 2023
Year: 2023

The ongoing penetration of energy systems with information and communications technology (ICT) and the introduction of new markets increase the potential for malicious or profit-driven attacks that endanger system stability. To ensure security-of-supply, it is necessary to analyze such attacks and their underlying vulnerabilities, to develop countermeasures and improve system design. We propose ANALYSE, a machine-learning-based software suite to let learning agents autonomously find attacks in cyber-physical energy systems, consisting of the power system, ICT, and energy markets. ANALYSE is a modular, configurable, and self-documenting framework designed to find yet unknown attack types and to reproduce many known attack strategies in cyber-physical energy systems from the scientific literature.
Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancement
Venue: arXiv cs.CV 2023
Year: 2023

Recently, Users Generated Content (UGC) videos becomes ubiquitous in our daily lives. However, due to the limitations of photographic equipments and techniques, UGC videos often contain various degradations, in which one of the most visually unfavorable effects is the underexposure. Therefore, corresponding video enhancement algorithms such as Low-Light Video Enhancement (LLVE) have been proposed to deal with the specific degradation. However, different from video enhancement algorithms, almost all existing Video Quality Assessment (VQA) models are built generally rather than specifically, which measure the quality of a video from a comprehensive perspective. To the best of our knowledge, there is no VQA model specially designed for videos enhanced by LLVE algorithms. To this end, we first construct a Low-Light Video Enhancement Quality Assessment (LLVE-QA) dataset in which 254 original low-light videos are collected and then enhanced by leveraging 8 LLVE algorithms to obtain 2,060 videos in total. Moreover, we propose a quality assessment model specialized in LLVE, named Light-VQA. More concretely, since the brightness and noise have the most impact on low-light enhanced VQA, we handcraft corresponding features and integrate them with deep-learning-based semantic features as the overall spatial information. As for temporal information, in addition to deep-learning-based motion features, we also investigate the handcrafted brightness consistency among video frames, and the overall temporal information is their concatenation. Subsequently, spatial and temporal information is fused to obtain the quality-aware representation of a video. Extensive experimental results show that our Light-VQA achieves the best performance against the current State-Of-The-Art (SOTA) on LLVE-QA and public dataset. Dataset and Codes can be found at https://github.com/wenzhouyidu/Light-VQA.
DLUE: Benchmarking Document Language Understanding
Venue: arXiv cs.CL 2023
Year: 2023

Understanding documents is central to many real-world tasks but remains a challenging topic. Unfortunately, there is no well-established consensus on how to comprehensively evaluate document understanding abilities, which significantly hinders the fair comparison and measuring the progress of the field. To benchmark document understanding researches, this paper summarizes four representative abilities, i.e., document classification, document structural analysis, document information extraction, and document transcription. Under the new evaluation framework, we propose \textbf{Document Language Understanding Evaluation} -- \textbf{DLUE}, a new task suite which covers a wide-range of tasks in various forms, domains and document genres. We also systematically evaluate six well-established transformer models on DLUE, and find that due to the lengthy content, complicated underlying structure and dispersed knowledge, document understanding is still far from being solved, and currently there is no neural architecture that dominates all tasks, raising requirements for a universal document understanding architecture.
MetaSRL++: A Uniform Scheme for Modelling Deeper Semantics
Venue: arXiv cs.CL 2023
Year: 2023

Despite enormous progress in Natural Language Processing (NLP), our field is still lacking a common deep semantic representation scheme. As a result, the problem of meaning and understanding is typically sidestepped through more simple, approximative methods. This paper argues that in order to arrive at such a scheme, we also need a common modelling scheme. It therefore introduces MetaSRL++, a uniform, language- and modality-independent modelling scheme based on Semantic Graphs, as a step towards a common representation scheme; as well as a method for defining the concepts and entities that are used in these graphs. Our output is twofold. First, we illustrate MetaSRL++ through concrete examples. Secondly, we discuss how it relates to existing work in the field.
An Exploratory Study on the Evidence of Hackathons' Role in Solving OSS Newcomers' Challenges
Venue: arXiv cs.SE 2023
Year: 2023

Background: OSS projects face various challenges. One major challenge is to onboard and integrate newcomers to the project. Aim: We aim to understand and discuss the challenges newcomers face when joining an OSS project and present evidence on how hackathons can mitigate those challenges. Method: We conducted two searches on digital libraries to (1) explore challenges faced by newcomers to join OSS projects, and (2) collect evidence on how hackathons were used to address them. We defined four evidence categories (positive, inconclusive, and no evidence) to classify evidence how hackathons address challenges. In addition, we investigated whether a hackathon event was related to an OSS project or not. Result: We identified a range of newcomer challenges that were successfully addressed using hackathons. However, not all of the solutions we identified were applied in the context of OSS. Conclusion: There seems to be potential in using hackathons to overcome newcomers' challenges in OSS projects and allow them to integrate faster into the project.
Interactive and Incremental Learning of Spatial Object Relations from Human Demonstrations
Venue: arXiv cs.RO 2023
Year: 2023

Humans use semantic concepts such as spatial relations between objects to describe scenes and communicate tasks such as "Put the tea to the right of the cup" or "Move the plate between the fork and the spoon." Just as children, assistive robots must be able to learn the sub-symbolic meaning of such concepts from human demonstrations and instructions. We address the problem of incrementally learning geometric models of spatial relations from few demonstrations collected online during interaction with a human. Such models enable a robot to manipulate objects in order to fulfill desired spatial relations specified by verbal instructions. At the start, we assume the robot has no geometric model of spatial relations. Given a task as above, the robot requests the user to demonstrate the task once in order to create a model from a single demonstration, leveraging cylindrical probability distribution as generative representation of spatial relations. We show how this model can be updated incrementally with each new demonstration without access to past examples in a sample-efficient way using incremental maximum likelihood estimation, and demonstrate the approach on a real humanoid robot.
Generate Compilers from Hardware Models!
Venue: arXiv cs.PL 2023
Year: 2023

Compiler backends should be automatically generated from hardware design language (HDL) models of the hardware they target. Generating compiler components directly from HDL can provide stronger correctness guarantees, ease development effort, and encourage hardware exploration. Past work has already championed this idea; here we argue that advances in program synthesis make the approach more feasible. We present a concrete example by demonstrating how FPGA technology mappers can be automatically generated from SystemVerilog models of an FPGA's primitives using program synthesis.
Boosting Event Extraction with Denoised Structure-to-Text Augmentation
Venue: arXiv cs.CL 2023
Year: 2023

Event extraction aims to recognize pre-defined event triggers and arguments from texts, which suffer from the lack of high-quality annotations. In most NLP applications, involving a large scale of synthetic training data is a practical and effective approach to alleviate the problem of data scarcity. However, when applying to the task of event extraction, recent data augmentation methods often neglect the problem of grammatical incorrectness, structure misalignment, and semantic drifting, leading to unsatisfactory performances. In order to solve these problems, we propose a denoised structure-to-text augmentation framework for event extraction DAEE, which generates additional training data through the knowledge-based structure-to-text generation model and selects the effective subset from the generated data iteratively with a deep reinforcement learning agent. Experimental results on several datasets demonstrate that the proposed method generates more diverse text representations for event extraction and achieves comparable results with the state-of-the-art.
CDDO-HS:Child Drawing Development Optimization Harmony Search Algorithm
Venue: arXiv cs.NE 2023
Year: 2023

Child drawing development optimization (CDDO) is a recent example of a metaheuristic algorithm. The motive for inventing this method is children's learning behavior and cognitive development, with the golden ratio employed to optimize their artwork's aesthetic value. Unfortunately, CDDO suffers from low performance in the exploration phase, and the local best solution stagnates. Harmony search (HS) is a highly competitive algorithm relative to other prevalent metaheuristic algorithms, as its exploration phase performance on unimodal benchmark functions is outstanding. Thus, to avoid these issues, we present CDDOHS, a hybridization of both standards of CDDO and HS. The hybridized model proposed consists of two phases. Initially, the pattern size (PS) is relocated to the algorithm's core and the initial pattern size is set to 80 % of the total population size. Second, the standard harmony search (HS) is added to the pattern size (PS) for the exploration phase to enhance and update the solution after each iteration. Experiments are evaluated using two distinct standard benchmark functions, known as classical test functions, including 23 common functions and 10 CEC-C06 2019 functions. Additionally, the suggested CDDOHS is compared to CDDO, HS, and six other widely used algorithms. Using the Wilcoxon ranksum test, the results indicate that CDDOHS beats alternative algorithms.
Addressing computational challenges in physical system simulations with machine learning
Venue: arXiv cs.LG 2023
Year: 2023

In this paper, we present a machine learning-based data generator framework tailored to aid researchers who utilize simulations to examine various physical systems or processes. High computational costs and the resulting limited data often pose significant challenges to gaining insights into these systems or processes. Our approach involves a two-step process: initially, we train a supervised predictive model using a limited simulated dataset to predict simulation outcomes. Subsequently, a reinforcement learning agent is trained to generate accurate, simulation-like data by leveraging the supervised model. With this framework, researchers can generate more accurate data and know the outcomes without running high computational simulations, which enables them to explore the parameter space more efficiently and gain deeper insights into physical systems or processes. We demonstrate the effectiveness of the proposed framework by applying it to two case studies, one focusing on earthquake rupture physics and the other on new material development.
Bug or not Bug? Analysing the Reasons Behind Metamorphic Relation Violations
Venue: arXiv cs.SE 2023
Year: 2023

Metamorphic Testing (MT) is a testing technique that can effectively alleviate the oracle problem. MT uses Metamorphic Relations (MRs) to determine if a test case passes or fails. MRs specify how the outputs should vary in response to specific input changes when executing the System Under Test (SUT). If a particular MR is violated for at least one test input (and its change), there is a high probability that the SUT has a fault. On the other hand, if a particular MR is not violated, it does not guarantee that the SUT is fault free. However, deciding if the MR is being violated due to a bug or because the MR does not hold/fit for particular conditions generated by specific inputs remains a manual task and unexplored. In this paper, we develop a method for refining MRs to offer hints as to whether a violation results from a bug or arises from the MR not being matched to certain test data under specific circumstances. In our initial proof-of-concept, we derive the relevant information from rules using the Association Rule Mining (ARM) technique. In our initial proof-of-concept, we validate our method on a toy example and discuss the lessons learned from our experiments. Our proof-of-concept demonstrates that our method is applicable and that we can provide suggestions that help strengthen the test suite for regression testing purposes.
Wavelet-based Unsupervised Label-to-Image Translation
Venue: arXiv cs.CV 2023
Year: 2023

Semantic Image Synthesis (SIS) is a subclass of image-to-image translation where a semantic layout is used to generate a photorealistic image. State-of-the-art conditional Generative Adversarial Networks (GANs) need a huge amount of paired data to accomplish this task while generic unpaired image-to-image translation frameworks underperform in comparison, because they color-code semantic layouts and learn correspondences in appearance instead of semantic content. Starting from the assumption that a high quality generated image should be segmented back to its semantic layout, we propose a new Unsupervised paradigm for SIS (USIS) that makes use of a self-supervised segmentation loss and whole image wavelet based discrimination. Furthermore, in order to match the high-frequency distribution of real images, a novel generator architecture in the wavelet domain is proposed. We test our methodology on 3 challenging datasets and demonstrate its ability to bridge the performance gap between paired and unpaired models.
Prompt-Tuning Decision Transformer with Preference Ranking
Venue: arXiv cs.LG 2023
Year: 2023

Prompt-tuning has emerged as a promising method for adapting pre-trained models to downstream tasks or aligning with human preferences. Prompt learning is widely used in NLP but has limited applicability to RL due to the complex physical meaning and environment-specific information contained within RL prompts. These factors require supervised learning to imitate the demonstrations and may result in a loss of meaning after learning. Additionally, directly extending prompt-tuning approaches to RL is challenging because RL prompts guide agent behavior based on environmental modeling and analysis, rather than filling in missing information, making it unlikely that adjustments to the prompt format for downstream tasks, as in NLP, can yield significant improvements. In this work, we propose the Prompt-Tuning DT algorithm to address these challenges by using trajectory segments as prompts to guide RL agents in acquiring environmental information and optimizing prompts via black-box tuning to enhance their ability to contain more relevant information, thereby enabling agents to make better decisions. Our approach involves randomly sampling a Gaussian distribution to fine-tune the elements of the prompt trajectory and using preference ranking function to find the optimization direction, thereby providing more informative prompts and guiding the agent towards specific preferences in the target environment. Extensive experiments show that with only 0.03% of the parameters learned, Prompt-Tuning DT achieves comparable or even better performance than full-model fine-tuning in low-data scenarios. Our work contributes to the advancement of prompt-tuning approaches in RL, providing a promising direction for optimizing large RL agents for specific preference tasks.
Newad: A register map automation tool for Verilog
Venue: arXiv cs.AR 2023
Year: 2023

Large scale scientific instrumentation-and-control FPGA gateware designs have numerous run-time settable parameters. These can be used either for user-level control or by automated processes (e.g., calibration). The number of such parameters in a single design can reach on the order of 1000, and keeps evolving as the gateware and its functionality evolves. One must keep track of which module the registers belong to, where the registers need to be decoded, and how to express the properties (or even semantics) of the register to the next level of user or software. Note, the registers maybe embedded anywhere throughout the module hierarchy. Purely manual handling of these tasks by HDL developers is considered burdensome and error-prone at this scale. Typically these registers are writable via an on-chip bus, vaguely VME-like, that is controlled by an on-chip or off-chip CPU. There have been several attempts in the community to address this task at different levels. However, we have found no tool that is able to generate a register map, generate decoders and encoders with minimal overhead to the developer. So, here we present a tool that scours native HDL source files and looks for specific language-supported attributes and automatically generates a register map and bus decoders, respecting multiple clock domains, and presents a JSON file to the network that maps register names to addresses.
A Range-Null Space Decomposition Approach for Fast and Flexible Spectral Compressive Imaging
Venue: arXiv cs.CV 2023
Year: 2023

We present RND-SCI, a novel framework for compressive hyperspectral image (HSI) reconstruction. Our framework decomposes the reconstructed object into range-space and null-space components, where the range-space part ensures the solution conforms to the compression process, and the null-space term introduces a deep HSI prior to constraining the output to have satisfactory properties. RND-SCI is not only simple in design with strong interpretability but also can be easily adapted to various HSI reconstruction networks, improving the quality of HSIs with minimal computational overhead. RND-SCI significantly boosts the performance of HSI reconstruction networks in retraining, fine-tuning or plugging into a pre-trained off-the-shelf model. Based on the framework and SAUNet, we design an extremely fast HSI reconstruction network, RND-SAUNet, which achieves an astounding 91 frames per second while maintaining superior reconstruction accuracy compared to other less time-consuming methods. Code and models are available at https://github.com/hustvl/RND-SCI.
Application-Agnostic Language Modeling for On-Device ASR
Venue: arXiv cs.CL 2023
Year: 2023

On-device automatic speech recognition systems face several challenges compared to server-based systems. They have to meet stricter constraints in terms of speed, disk size and memory while maintaining the same accuracy. Often they have to serve several applications with different distributions at once, such as communicating with a virtual assistant and speech-to-text. The simplest solution to serve multiple applications is to build application-specific (language) models, but this leads to an increase in memory. Therefore, we explore different data- and architecture-driven language modeling approaches to build a single application-agnostic model. We propose two novel feed-forward architectures that find an optimal trade off between different on-device constraints. In comparison to the application-specific solution, one of our novel approaches reduces the disk size by half, while maintaining speed and accuracy of the original model.
OpenVR: Teleoperation for Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

Across the robotics field, quality demonstrations are an integral part of many control pipelines. However, collecting high-quality demonstration trajectories remains time-consuming and difficult, often resulting in the number of demonstrations being the performance bottleneck. To address this issue, we present a method of Virtual Reality (VR) Teleoperation that uses an Oculus VR headset to teleoperate a Franka Emika Panda robot. Although other VR teleoperation methods exist, our code is open source, designed for readily available consumer hardware, easy to modify, agnostic to experimental setup, and simple to use.
Exploring outlooks towards generative AI-based assistive technologies for people with Autism
Venue: arXiv cs.HC 2023
Year: 2023

The last few years have significantly increased global interest in generative artificial intelligence. Deepfakes, which are synthetically created videos, emerged as an application of generative artificial intelligence. Fake news and pornographic content have been the two most prevalent negative use cases of deepfakes in the digital ecosystem. Deepfakes have some advantageous applications that experts in the subject have thought of in the areas of filmmaking, teaching, etc. Research on the potential of deepfakes among people with disabilities is, however, scarce or nonexistent. This workshop paper explores the potential of deepfakes as an assistive technology. We examined Reddit conversations regarding Nvdia's new videoconferencing feature which allows participants to maintain eye contact during online meetings. Through manual web scraping and qualitative coding, we found 162 relevant comments discussing the relevance and appropriateness of the technology for people with Autism. The themes identified from the qualitative codes indicate a number of concerns for technology among the autistic community. We suggest that developing generative AI-based assistive solutions will have ramifications for human-computer interaction (HCI), and present open questions that should be investigated further in this space.