OpenRaiser/Intern-Atlas huggingface dataset:
Hierarchical Entity Alignment for Attribute-Rich Event-Driven Graphs
Venue: arXiv cs.SI 2022
Year: 2022

This paper addresses the problem of entity alignment in attribute-rich event-driven graphs. Unlike many other entity alignment problems, we are interested in aligning entities based on the similarity of their actions, i.e., entities that participate in similar events are more likely to be the same. We model the generative process of this problem as a Bayesian model and derive our proposed algorithm from the posterior predictive distribution. We apply our Hierarchical Entity AlignmenT (HEAT) algorithm to two datasets, one on publications and the other on financial transactions, derived from real data and provided to us by an external collaborator.
Security Evaluation of Block-based Image Encryption for Vision Transformer against Jigsaw Puzzle Solver Attack
Venue: arXiv cs.CR 2022
Year: 2022

The aim of this paper is to evaluate the security of a block-based image encryption for the vision transformer against jigsaw puzzle solver attacks. The vision transformer, a model for image classification based on the transformer architecture, is carried out by dividing an image into a grid of square patches. Some encryption schemes for the vision transformer have been proposed by applying block-based image encryption such as block scrambling and rotating to patches of the image. On the other hand, the security of encryption scheme for the vision transformer has never evaluated. In this paper, jigsaw puzzle solver attacks are utilized to evaluate the security of encrypted images by regarding the divided patches as pieces of a jigsaw puzzle. In experiments, an image is resized and divided into patches to apply block scrambling-based image encryption, and then the security of encrypted images for the vision transformer against jigsaw puzzle solver attacks is evaluated.
A Longitudinal Dataset of Twitter ISIS Users
Venue: arXiv cs.SI 2022
Year: 2022

We present a large longitudinal dataset of tweets from two sets of users that are suspected to be affiliated with ISIS. These sets of users are identified based on a prior study and a campaign aimed at shutting down ISIS Twitter accounts. These users have engaged with known ISIS accounts at least once during 2014-2015 and are still active as of 2021. Some of them have directly supported the ISIS users and their tweets by retweeting them, and some of the users that have quoted tweets of ISIS, have uncertain connections to ISIS seed accounts. This study and the dataset represent a unique approach to analyzing ISIS data. Although much research exists on ISIS online activities, few studies have focused on individual accounts. Our approach to validating accounts as well as developing a framework for differentiating accounts' functionality (e.g., propaganda versus operational planning) offers a foundation for future research. We perform some descriptive statistics and preliminary analyses on our collected data to provide deeper insight and highlight the significance and practicality of such analyses. We further discuss several cross-disciplinary potential use cases and research directions.
Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic
Venue: arXiv cs.RO 2022
Year: 2022

Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.
Choice of technology and evaluation of the production capabilities of a 3d printer robot for creating elements of experimental equipment for the production of biofuel components
Venue: arXiv cs.RO 2022
Year: 2022

Elements of experimental equipment for the production of biofuel components must meet high reliability and safety requirements. At the same time, in the course of research on the subject of creating equipment for the production of biofuels, a variable range of equipment is regularly proposed and should be checked. The manufacture of elements of such equipment by traditional methods is expensive and inefficient, time-consuming, which negatively affects the speed of scientific research. To this end, it is proposed to develop a robotic 3D printing complex that provides maximum flexibility in creating mock-ups and test samples of equipment for the production of biofuel components. The article discusses the experience of successfully creating equipment elements for the production of fuels using 3d printing. Next, the choice of a robotization scheme for a 3D printing installation is described and the choice of printing technology is substantiated. The article also presents the results of calculating the parameters of the 3v-printer robot and the results of calculating the similarity parameters for the implementation and evaluation of control algorithms. The results of a numerical experiment for calculating the strength characteristics of equipment elements manufactured using the selected 3d printing technology are presented.
Robot-printer for creating elements of technological equipment for the production of components of biofuel compositions
Venue: arXiv cs.RO 2022
Year: 2022

This study is devoted to the search for new scientific and technical solutions in the field of renewable energy sources, in particular biofuels. Biomass is the main fuel for green energy, accounting for two thirds of the energy produced from renewable sources. The further development of the industry depends on the improvement of the equipment and technologies used in it. On the example of a cleaning apparatus, a new technology for prototyping its parts using a robotic module is shown and tested. The use of plastics as parts of technological equipment is a modern trend and may be due to the low adhesion strength of various substances to the surface of these parts due to poor wettability and low values of the surface energy of these materials compared to metals.
Semantics-Aware Active Fault Detection in Status Updating Systems
Venue: arXiv cs.NI 2022
Year: 2022

With its growing number of deployed devices and applications, the Internet of Things (IoT) raises significant challenges for network maintenance procedures. In this work we address a problem of active fault detection in an IoT scenario, whereby a monitor can probe a remote device in order to acquire fresh information and facilitate fault detection. However, probing could have a significant impact on the system's energy and communication resources. To this end, we utilize Age of Information as a measure of the freshness of information at the monitor and adopt a semantics-aware communication approach between the monitor and the remote device. In semantics-aware communications, the processes of generating and transmitting information are treated jointly to consider the importance of information and the purpose of communication. We formulate the problem as a Partially Observable Markov Decision Process and show analytically that the optimal policy is of a threshold type. Finally, we use a computationally efficient stochastic approximation algorithm to approximate the optimal policy and present numerical results that exhibit the advantage of our approach compared to a conventional delay-based probing policy.
Eikonal Fields for Refractive Novel-View Synthesis
Venue: arXiv cs.GR 2022
Year: 2022

We tackle the problem of generating novel-view images from collections of 2D images showing refractive and reflective objects. Current solutions assume opaque or transparent light transport along straight paths following the emission-absorption model. Instead, we optimize for a field of 3D-varying Index of Refraction (IoR) and trace light through it that bends toward the spatial gradients of said IoR according to the laws of eikonal light transport.
Auto-Transfer: Learning to Route Transferrable Representations
Venue: arXiv cs.LG 2022
Year: 2022

Knowledge transfer between heterogeneous source and target networks and tasks has received a lot of attention in recent times as large amounts of quality labeled data can be difficult to obtain in many applications. Existing approaches typically constrain the target deep neural network (DNN) feature representations to be close to the source DNNs feature representations, which can be limiting. We, in this paper, propose a novel adversarial multi-armed bandit approach that automatically learns to route source representations to appropriate target representations following which they are combined in meaningful ways to produce accurate target models. We see upwards of 5\% accuracy improvements compared with the state-of-the-art knowledge transfer methods on four benchmark (target) image datasets CUB200, Stanford Dogs, MIT67, and Stanford40 where the source dataset is ImageNet. We qualitatively analyze the goodness of our transfer scheme by showing individual examples of the important features focused on by our target network at different layers compared with the (closest) competitors. We also observe that our improvement over other methods is higher for smaller target datasets making it an effective tool for small data applications that may benefit from transfer learning.
Too much information: why CDCL solvers need to forget learned clauses
Venue: arXiv cs.AI 2022
Year: 2022

Conflict-driven clause learning (CDCL) is a remarkably successful paradigm for solving the satisfiability problem of propositional logic. Instead of a simple depth-first backtracking approach, this kind of solver learns the reason behind occurring conflicts in the form of additional clauses. However, despite the enormous success of CDCL solvers, there is still only a limited understanding of what influences the performance of these solvers in what way. Considering different measures, this paper demonstrates, quite surprisingly, that clause learning (without being able to get rid of some clauses) can not only help the solver but can oftentimes deteriorate the solution process dramatically. By conducting extensive empirical analysis, we furthermore find that the runtime distributions of CDCL solvers are multimodal. This multimodality can be seen as a reason for the deterioration phenomenon described above. Simultaneously, it also gives an indication of why clause learning in combination with clause deletion is virtually the de facto standard of SAT solving, in spite of this phenomenon. As a final contribution, we show that Weibull mixture distributions can accurately describe the multimodal distributions. Thus, adding new clauses to a base instance has an inherent effect of making runtimes long-tailed. This insight provides an explanation as to why the technique of forgetting clauses is useful in CDCL solvers apart from the optimization of unit propagation speed.
MMSys'22 Grand Challenge on AI-based Video Production for Soccer
Venue: arXiv cs.CV 2022
Year: 2022

Soccer has a considerable market share of the global sports industry, and the interest in viewing videos from soccer games continues to grow. In this respect, it is important to provide game summaries and highlights of the main game events. However, annotating and producing events and summaries often require expensive equipment and a lot of tedious, cumbersome, manual labor. Therefore, automating the video production pipeline providing fast game highlights at a much lower cost is seen as the "holy grail". In this context, recent developments in Artificial Intelligence (AI) technology have shown great potential. Still, state-of-the-art approaches are far from being adequate for practical scenarios that have demanding real-time requirements, as well as strict performance criteria (where at least the detection of official events such as goals and cards must be 100% accurate). In addition, event detection should be thoroughly enhanced by annotation and classification, proper clipping, generating short descriptions, selecting appropriate thumbnails for highlight clips, and finally, combining the event highlights into an overall game summary, similar to what is commonly aired during sports news. Even though the event tagging operation has by far received the most attention, an end-to-end video production pipeline also includes various other operations which serve the overall purpose of automated soccer analysis. This challenge aims to assist the automation of such a production pipeline using AI. In particular, we focus on the enhancement operations that take place after an event has been detected, namely event clipping (Task 1), thumbnail selection (Task 2), and game summarization (Task 3). Challenge website: https://mmsys2022.ie/authors/grand-challenge.
Detecting Privacy Requirements from User Stories with NLP Transfer Learning Models
Venue: arXiv cs.SE 2022
Year: 2022

To provide privacy-aware software systems, it is crucial to consider privacy from the very beginning of the development. However, developers do not have the expertise and the knowledge required to embed the legal and social requirements for data protection into software systems. Objective: We present an approach to decrease privacy risks during agile software development by automatically detecting privacy-related information in the context of user story requirements, a prominent notation in agile Requirement Engineering (RE). Methods: The proposed approach combines Natural Language Processing (NLP) and linguistic resources with deep learning algorithms to identify privacy aspects into User Stories. NLP technologies are used to extract information regarding the semantic and syntactic structure of the text. This information is then processed by a pre-trained convolutional neural network, which paved the way for the implementation of a Transfer Learning technique. We evaluate the proposed approach by performing an empirical study with a dataset of 1680 user stories. Results: The experimental results show that deep learning algorithms allow to obtain better predictions than those achieved with conventional (shallow) machine learning methods. Moreover, the application of Transfer Learning allows to considerably improve the accuracy of the predictions, ca. 10%. Conclusions: Our study contributes to encourage software engineering researchers in considering the opportunities to automate privacy detection in the early phase of design, by also exploiting transfer learning models.
RoboKrill : a metachronal drag-based swimmer robot
Venue: arXiv cs.RO 2022
Year: 2022

Marine exploration is essential to understanding ocean processes and organisms. While the use of current unmanned underwater vehicles has enabled many discoveries, there are still plenty of limitations toward exploring complex environments. Bio-inspired robots are a promising solution for highly maneuverable underwater swimming at moderate speeds. Krill, especially, are efficient swimmers in the intermediate Reynolds number regime and can inform engineering solutions for ocean exploration. In this paper, we present the design, manufacture, and validation of a new krill-inspired, metachronal, drag-based robotic system. By combining active and passive actuation of the joints with 3D printed parts, our unique design recreates the swimming kinematics of Euphausia superba in a compact and reproducible robotic platform. The motion of the anterior and posterior appendage segments is achieved using servo motors and a multi-link mechanism, while the out-of-plane motion of the biramous distal segments is attained via fluid-structure interactions. Going forward, our platform will be leveraged to study metachronal, drag-based swimmers across taxa to identify unifying success mechanisms at different scales, facilitating the development of a new generation of underwater robots.
Identifying Suitable Tasks for Inductive Transfer Through the Analysis of Feature Attributions
Venue: arXiv cs.LG 2022
Year: 2022

Transfer learning approaches have shown to significantly improve performance on downstream tasks. However, it is common for prior works to only report where transfer learning was beneficial, ignoring the significant trial-and-error required to find effective settings for transfer. Indeed, not all task combinations lead to performance benefits, and brute-force searching rapidly becomes computationally infeasible. Hence the question arises, can we predict whether transfer between two tasks will be beneficial without actually performing the experiment? In this paper, we leverage explainability techniques to effectively predict whether task pairs will be complementary, through comparison of neural network activation between single-task models. In this way, we can avoid grid-searches over all task and hyperparameter combinations, dramatically reducing the time needed to find effective task pairs. Our results show that, through this approach, it is possible to reduce training time by up to 83.5% at a cost of only 0.034 reduction in positive-class F1 on the TREC-IS 2020-A dataset.
Spectral and Energy Efficiency of DCO-OFDM in Visible Light Communication Systems with Finite-Alphabet Inputs
Venue: arXiv cs.IT 2022
Year: 2022

The bound of the information transmission rate of direct current biased optical orthogonal frequency division multiplexing (DCO-OFDM) for visible light communication (VLC) with finite-alphabet inputs is yet unknown, where the corresponding spectral efficiency (SE) and energy efficiency (EE) stems out as the open research problems. In this paper, we derive the exact achievable rate of {the} DCO-OFDM system with finite-alphabet inputs for the first time. Furthermore, we investigate SE maximization problems of {the} DCO-OFDM system subject to both electrical and optical power constraints. By exploiting the relationship between the mutual information and the minimum mean-squared error, we propose a multi-level mercury-water-filling power allocation scheme to achieve the maximum SE. Moreover, the EE maximization problems of {the} DCO-OFDM system are studied, and the Dinkelbach-type power allocation scheme is developed for the maximum EE. Numerical results verify the effectiveness of the proposed theories and power allocation schemes.
A Survey on Retrieval-Augmented Text Generation
Venue: arXiv cs.CL 2022
Year: 2022

Recently, retrieval-augmented text generation attracted increasing attention of the computational linguistics community. Compared with conventional generation models, retrieval-augmented text generation has remarkable advantages and particularly has achieved state-of-the-art performance in many NLP tasks. This paper aims to conduct a survey about retrieval-augmented text generation. It firstly highlights the generic paradigm of retrieval-augmented generation, and then it reviews notable approaches according to different tasks including dialogue response generation, machine translation, and other generation tasks. Finally, it points out some important directions on top of recent methods to facilitate future research.
NeuRegenerate: A Framework for Visualizing Neurodegeneration
Venue: arXiv cs.LG 2022
Year: 2022

Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections is limited to general observations using population analysis. In this paper, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject, for specified age-timepoints.To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (cycleGAN) that translates features of neuronal structures in a region, across age-timepoints, for large brain microscopy volumes. We improve the reconstruction quality of neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss.Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. We show that neuReGANerator has a reconstruction accuracy of 94% in predicting neuronal structures. Finally, to visualize the predicted change in projections, NeuRegenerate offers two modes: (1) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, across the age timepoints, and (2) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes in neuronal fibers within the cholinergic system of the mouse brain between a young and old specimen.
An ASP approach for reasoning on neural networks under a finitely many-valued semantics for weighted conditional knowledge bases
Venue: arXiv cs.AI 2022
Year: 2022

Weighted knowledge bases for description logics with typicality have been recently considered under a "concept-wise" multipreference semantics (in both the two-valued and fuzzy case), as the basis of a logical semantics of MultiLayer Perceptrons (MLPs). In this paper we consider weighted conditional ALC knowledge bases with typicality in the finitely many-valued case, through three different semantic constructions. For the boolean fragment LC of ALC we exploit ASP and "asprin" for reasoning with the concept-wise multipreference entailment under a phi-coherent semantics, suitable to characterize the stationary states of MLPs. As a proof of concept, we experiment the proposed approach for checking properties of trained MLPs. The paper is under consideration for acceptance in TPLP.
Efficient Memory Partitioning in Software Defined Hardware
Venue: arXiv cs.AR 2022
Year: 2022

As programmers turn to software-defined hardware (SDH) to maintain a high level of productivity while programming hardware to run complex algorithms, heavy-lifting must be done by the compiler to automatically partition on-chip arrays. In this paper, we introduce an automatic memory partitioning system that can quickly compute more efficient partitioning schemes than prior systems. Our system employs a variety of resource-saving optimizations and an ML cost model to select the best partitioning scheme from an array of candidates. We compared our system against various state-of-the-art SDH compilers and FPGAs on a variety of benchmarks and found that our system generates solutions that, on average, consume 40.3% fewer logic resources, 78.3% fewer FFs, 54.9% fewer Block RAMs (BRAMs), and 100% fewer DSPs.