OpenRaiser/Intern-Atlas huggingface dataset:
Better Short than Greedy: Interpretable Models through Optimal Rule Boosting
Venue: arXiv cs.LG 2021
Year: 2021

Rule ensembles are designed to provide a useful trade-off between predictive accuracy and model interpretability. However, the myopic and random search components of current rule ensemble methods can compromise this goal: they often need more rules than necessary to reach a certain accuracy level or can even outright fail to accurately model a distribution that can actually be described well with a few rules. Here, we present a novel approach aiming to fit rule ensembles of maximal predictive power for a given ensemble size (and thus model comprehensibility). In particular, we present an efficient branch-and-bound algorithm that optimally solves the per-rule objective function of the popular second-order gradient boosting framework. Our main insight is that the boosting objective can be tightly bounded in linear time of the number of covered data points. Along with an additional novel pruning technique related to rule redundancy, this leads to a computationally feasible approach for boosting optimal rules that, as we demonstrate on a wide range of common benchmark problems, consistently outperforms the predictive performance of boosting greedy rules.
Content Selection Network for Document-grounded Retrieval-based Chatbots
Venue: arXiv cs.CL 2021
Year: 2021

Grounding human-machine conversation in a document is an effective way to improve the performance of retrieval-based chatbots. However, only a part of the document content may be relevant to help select the appropriate response at a round. It is thus crucial to select the part of document content relevant to the current conversation context. In this paper, we propose a document content selection network (CSN) to perform explicit selection of relevant document contents, and filter out the irrelevant parts. We show in experiments on two public document-grounded conversation datasets that CSN can effectively help select the relevant document contents to the conversation context, and it produces better results than the state-of-the-art approaches. Our code and datasets are available at https://github.com/DaoD/CSN.
Analysis of Information Flow Through U-Nets
Venue: arXiv cs.LG 2021
Year: 2021

Deep Neural Networks (DNNs) have become ubiquitous in medical image processing and analysis. Among them, U-Nets are very popular in various image segmentation tasks. Yet, little is known about how information flows through these networks and whether they are indeed properly designed for the tasks they are being proposed for. In this paper, we employ information-theoretic tools in order to gain insight into information flow through U-Nets. In particular, we show how mutual information between input/output and an intermediate layer can be a useful tool to understand information flow through various portions of a U-Net, assess its architectural efficiency, and even propose more efficient designs.
Malware Detection and Analysis: Challenges and Research Opportunities
Venue: arXiv cs.CR 2021
Year: 2021

Malwares are continuously growing in sophistication and numbers. Over the last decade, remarkable progress has been achieved in anti-malware mechanisms. However, several pressing issues (e.g., unknown malware samples detection) still need to be addressed adequately. This article first presents a concise overview of malware along with anti-malware and then summarizes various research challenges. This is a theoretical and perspective article that is hoped to complement earlier articles and works.
Successive-Cancellation Decoding of Binary Polar Codes Based on Symmetric Parametrization
Venue: arXiv cs.IT 2021
Year: 2021

This paper introduces algorithms for the successive-cancellation decoding and the successive-cancellation list decoding of binary polar source/channel codes. By using the symmetric parametrization of conditional probability, we reduce both space and time complexity compared to the original algorithm introduced by Tal and Vardy.
Allocating Opportunities in a Dynamic Model of Intergenerational Mobility
Venue: arXiv cs.CY 2021
Year: 2021

Opportunities such as higher education can promote intergenerational mobility, leading individuals to achieve levels of socioeconomic status above that of their parents. We develop a dynamic model for allocating such opportunities in a society that exhibits bottlenecks in mobility; the problem of optimal allocation reflects a trade-off between the benefits conferred by the opportunities in the current generation and the potential to elevate the socioeconomic status of recipients, shaping the composition of future generations in ways that can benefit further from the opportunities. We show how optimal allocations in our model arise as solutions to continuous optimization problems over multiple generations, and we find in general that these optimal solutions can favor recipients of low socioeconomic status over slightly higher-performing individuals of high socioeconomic status -- a form of socioeconomic affirmative action that the society in our model discovers in the pursuit of purely payoff-maximizing goals. We characterize how the structure of the model can lead to either temporary or persistent affirmative action, and we consider extensions of the model with more complex processes modulating the movement between different levels of socioeconomic status.
Fire Threat Detection From Videos with Q-Rough Sets
Venue: arXiv cs.CV 2021
Year: 2021

This article defines new methods for unsupervised fire region segmentation and fire threat detection from video stream. Fire in control serves a number of purposes to human civilization, but it could simultaneously be a threat once its spread becomes uncontrolled. There exists many methods on fire region segmentation and fire non-fire classification. But the approaches to determine the threat associated with fire is relatively scare, and no such unsupervised method has been formulated yet. Here we focus on developing an unsupervised method with which the threat of fire can be quantified and accordingly generate an alarm in automated surveillance systems in indoor as well as in outdoors. Fire region segmentation without any manual intervention/ labelled data set is a major challenge while formulating such a method. Here we have used rough approximations to approximate the fire region, and to manage the incompleteness of the knowledge base, due to absence of any prior information. Utility maximization of Q-learning has been used to minimize ambiguities in the rough approximations. The new set approximation method, thus developed here, is named as Q-rough set. It is used for fire region segmentation from video frames. The threat index of fire flame over the input video stream has been defined in sync with the relative growth in the fire segments on the recent frames. All theories and indices defined here have been experimentally validated with different types of fire videos, through demonstrations and comparisons, as superior to the state of the art.
Collaborative Teacher-Student Learning via Multiple Knowledge Transfer
Venue: arXiv cs.LG 2021
Year: 2021

Knowledge distillation (KD), as an efficient and effective model compression technique, has been receiving considerable attention in deep learning. The key to its success is to transfer knowledge from a large teacher network to a small student one. However, most of the existing knowledge distillation methods consider only one type of knowledge learned from either instance features or instance relations via a specific distillation strategy in teacher-student learning. There are few works that explore the idea of transferring different types of knowledge with different distillation strategies in a unified framework. Moreover, the frequently used offline distillation suffers from a limited learning capacity due to the fixed teacher-student architecture. In this paper we propose a collaborative teacher-student learning via multiple knowledge transfer (CTSL-MKT) that prompts both self-learning and collaborative learning. It allows multiple students learn knowledge from both individual instances and instance relations in a collaborative way. While learning from themselves with self-distillation, they can also guide each other via online distillation. The experiments and ablation studies on four image datasets demonstrate that the proposed CTSL-MKT significantly outperforms the state-of-the-art KD methods.
Differential Euler: Designing a Neural Network approximator to solve the Chaotic Three Body Problem
Venue: arXiv cs.LG 2021
Year: 2021

The three body problem is a special case of the n body problem where one takes the initial positions and velocities of three point masses and attempts to predict their motion over time according to Newtonian laws of motion and universal gravitation. Though analytical solutions have been found for special cases, the general problem remains unsolved; the solutions that do exist are impractical. Fortunately, for many applications, we may not need to solve the problem completely, i.e., predicting with reasonable accuracy for some time steps, may be sufficient. Recently, Breen et al attempted to approximately solve the three body problem using a simple neural network. Although their methods appear to achieve some success in reducing the computational overhead, their model is extremely restricted, applying to a specialized 2D case. The authors do not provide explanations for critical decisions taken in their experimental design, no details on their model or architecture, and nor do they publish their code. Moreover, the model does not generalize well to unseen cases. In this paper, we propose a detailed experimental setup to determine the feasibility of using neural networks to solve the three body problem up to a certain number of time steps. We establish a benchmark on the dataset size and set an accuracy threshold to measure the viability of our results for practical applications. Then, we build our models according to the listed class of NNs using a dataset generated from standard numerical integrators. We gradually increase the complexity of our data set to determine whether NNs can learn a representation of the chaotic three body problem well enough to replace numerical integrators in real life scenarios.
Adv-OLM: Generating Textual Adversaries via OLM
Venue: arXiv cs.CL 2021
Year: 2021

Deep learning models are susceptible to adversarial examples that have imperceptible perturbations in the original input, resulting in adversarial attacks against these models. Analysis of these attacks on the state of the art transformers in NLP can help improve the robustness of these models against such adversarial inputs. In this paper, we present Adv-OLM, a black-box attack method that adapts the idea of Occlusion and Language Models (OLM) to the current state of the art attack methods. OLM is used to rank words of a sentence, which are later substituted using word replacement strategies. We experimentally show that our approach outperforms other attack methods for several text classification tasks.
A Joint Diagonalization Based Efficient Approach to Underdetermined Blind Audio Source Separation Using the Multichannel Wiener Filter
Venue: arXiv cs.SD 2021
Year: 2021

This paper presents a computationally efficient approach to blind source separation (BSS) of audio signals, applicable even when there are more sources than microphones (i.e., the underdetermined case). When there are as many sources as microphones (i.e., the determined case), BSS can be performed computationally efficiently by independent component analysis (ICA). Unfortunately, however, ICA is basically inapplicable to the underdetermined case. Another BSS approach using the multichannel Wiener filter (MWF) is applicable even to this case, and encompasses full-rank spatial covariance analysis (FCA) and multichannel non-negative matrix factorization (MNMF). However, these methods require massive numbers of matrix inversions to design the MWF, and are thus computationally inefficient. To overcome this drawback, we exploit the well-known property of diagonal matrices that matrix inversion amounts to mere inversion of the diagonal elements and can thus be performed computationally efficiently. This makes it possible to drastically reduce the computational cost of the above matrix inversions based on a joint diagonalization (JD) idea, leading to computationally efficient BSS. Specifically, we restrict the N spatial covariance matrices (SCMs) of all N sources to a class of (exactly) jointly diagonalizable matrices. Based on this approach, we present FastFCA, a computationally efficient extension of FCA. We also present a unified framework for underdetermined and determined audio BSS, which highlights a theoretical connection between FastFCA and other methods. Moreover, we reveal that FastFCA can be regarded as a regularized version of approximate joint diagonalization (AJD).
Stress Testing of Meta-learning Approaches for Few-shot Learning
Venue: arXiv cs.LG 2021
Year: 2021

Meta-learning (ML) has emerged as a promising learning method under resource constraints such as few-shot learning. ML approaches typically propose a methodology to learn generalizable models. In this work-in-progress paper, we put the recent ML approaches to a stress test to discover their limitations. Precisely, we measure the performance of ML approaches for few-shot learning against increasing task complexity. Our results show a quick degradation in the performance of initialization strategies for ML (MAML, TAML, and MetaSGD), while surprisingly, approaches that use an optimization strategy (MetaLSTM) perform significantly better. We further demonstrate the effectiveness of an optimization strategy for ML (MetaLSTM++) trained in a MAML manner over a pure optimization strategy. Our experiments also show that the optimization strategies for ML achieve higher transferability from simple to complex tasks.
Positive Geometries for Barycentric Interpolation
Venue: arXiv cs.CG 2021
Year: 2021

We propose a novel theoretical framework for barycentric interpolation, using concepts recently developed in mathematical physics. Generalized barycentric coordinates are defined similarly to Shepard's method, using positive geometries - subsets which possess a rational function naturally associated to their boundaries. Positive geometries generalize certain properties of simplices and convex polytopes to a large variety of geometric objects. Our framework unifies several previous constructions, including the definition of Wachspress coordinates over polytopes in terms of adjoints and dual polytopes. We also discuss potential applications to interpolation in 3D line space, mean-value coordinates and splines.
Commutative Event Sourcing vs. Triple Graph Grammars
Venue: arXiv cs.SE 2021
Year: 2021

This paper proposes Commutative Event Sourcing as a simple and reliable mechanism for model synchronisation, bidirectional model to model transformations, incremental updates, and collaborative editing. Commutative Event Sourcing is a restricted form of a Triple Graph Grammar where the rules or editing commands are either overwriting or commutative. This restriction gets rid of a lot of Triple Graph Grammar complexity and it becomes possible to implement model synchronisation manually. Thus, you are not restricted to Java as your programming language and you do not need to use a proprietary library, framework, or tool. You do not even have to dig into graph grammar theory.
Image-to-Image Translation: Methods and Applications
Venue: arXiv cs.CV 2021
Year: 2021

Image-to-image translation (I2I) aims to transfer images from a source domain to a target domain while preserving the content representations. I2I has drawn increasing attention and made tremendous progress in recent years because of its wide range of applications in many computer vision and image processing problems, such as image synthesis, segmentation, style transfer, restoration, and pose estimation. In this paper, we provide an overview of the I2I works developed in recent years. We will analyze the key techniques of the existing I2I works and clarify the main progress the community has made. Additionally, we will elaborate on the effect of I2I on the research and industry community and point out remaining challenges in related fields.
The Capacity of the Amplitude-Constrained Vector Gaussian Channel
Venue: arXiv cs.IT 2021
Year: 2021

The capacity of multiple-input multiple-output additive white Gaussian noise channels is investigated under peak amplitude constraints on the norm of the input vector. New insights on the capacity-achieving input distribution are presented. Furthermore, it is provided an iterative algorithm to numerically evaluate both the information capacity and the optimal input distribution of such channel.
Geometric Moment Invariants to Motion Blur
Venue: arXiv cs.CG 2021
Year: 2021

In this paper, we focus on removing interference of motion blur by the derivation of motion blur invariants.Unlike earlier work, we don't restore any blurred image. Based on geometric moment and mathematical model of motion blur, we prove that geometric moments of blurred image and original image are linearly related. Depending on this property, we can analyse whether an existing moment-based feature is invariant to motion blur. Surprisingly, we find some geometric moment invariants are invariants to not only spatial transform but also motion blur. Meanwhile, we test invariance and robustness of these invariants using synthetic and real blur image datasets. And the results show these invariants outperform some widely used blur moment invariants and non-moment image features in image retrieval, classification and template matching.
Adversarial Machine Learning for Flooding Attacks on 5G Radio Access Network Slicing
Venue: arXiv cs.NI 2021
Year: 2021

Network slicing manages network resources as virtual resource blocks (RBs) for the 5G Radio Access Network (RAN). Each communication request comes with quality of experience (QoE) requirements such as throughput and latency/deadline, which can be met by assigning RBs, communication power, and processing power to the request. For a completed request, the achieved reward is measured by the weight (priority) of this request. Then, the reward is maximized over time by allocating resources, e.g., with reinforcement learning (RL). In this paper, we introduce a novel flooding attack on 5G network slicing, where an adversary generates fake network slicing requests to consume the 5G RAN resources that would be otherwise available to real requests. The adversary observes the spectrum and builds a surrogate model on the network slicing algorithm through RL that decides on how to craft fake requests to minimize the reward of real requests over time. We show that the portion of the reward achieved by real requests may be much less than the reward that would be achieved when there was no attack. We also show that this flooding attack is more effective than other benchmark attacks such as random fake requests and fake requests with the minimum resource requirement (lowest QoE requirement). Fake requests may be detected due to their fixed weight. As an attack enhancement, we present schemes to randomize weights of fake requests and show that it is still possible to reduce the reward of real requests while maintaining the balance on weight distributions.
Work-sensitive Dynamic Complexity of Formal Languages
Venue: arXiv cs.LO 2021
Year: 2021

Which amount of parallel resources is needed for updating a query result after changing an input? In this work we study the amount of work required for dynamically answering membership and range queries for formal languages in parallel constant time with polynomially many processors. As a prerequisite, we propose a framework for specifying dynamic, parallel, constant-time programs that require small amounts of work. This framework is based on the dynamic descriptive complexity framework by Patnaik and Immerman.
Soft Genetic Programming Binary Classifiers
Venue: arXiv cs.LG 2021
Year: 2021

The study of the classifier's design and it's usage is one of the most important machine learning areas. With the development of automatic machine learning methods, various approaches are used to build a robust classifier model. Due to some difficult implementation and customization complexity, genetic programming (GP) methods are not often used to construct classifiers. GP classifiers have several limitations and disadvantages. However, the concept of "soft" genetic programming (SGP) has been developed, which allows the logical operator tree to be more flexible and find dependencies in datasets, which gives promising results in most cases. This article discusses a method for constructing binary classifiers using the SGP technique. The test results are presented. Source code - https://github.com/survexman/sgp_classifier.
How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
Venue: arXiv cs.AI 2021
Year: 2021

There have been several research works proposing new Explainable AI (XAI) methods designed to generate model explanations having specific properties, or desiderata, such as fidelity, robustness, or human-interpretability. However, explanations are seldom evaluated based on their true practical impact on decision-making tasks. Without that assessment, explanations might be chosen that, in fact, hurt the overall performance of the combined system of ML model + end-users. This study aims to bridge this gap by proposing XAI Test, an application-grounded evaluation methodology tailored to isolate the impact of providing the end-user with different levels of information. We conducted an experiment following XAI Test to evaluate three popular post-hoc explanation methods -- LIME, SHAP, and TreeInterpreter -- on a real-world fraud detection task, with real data, a deployed ML model, and fraud analysts. During the experiment, we gradually increased the information provided to the fraud analysts in three stages: Data Only, i.e., just transaction data without access to model score nor explanations, Data + ML Model Score, and Data + ML Model Score + Explanations. Using strong statistical analysis, we show that, in general, these popular explainers have a worse impact than desired. Some of the conclusion highlights include: i) showing Data Only results in the highest decision accuracy and the slowest decision time among all variants tested, ii) all the explainers improve accuracy over the Data + ML Model Score variant but still result in lower accuracy when compared with Data Only; iii) LIME was the least preferred by users, probably due to its substantially lower variability of explanations from case to case.
The Internet of Things in Ports: Six Key Security and Governance Challenges for the UK (Policy Brief)
Venue: arXiv cs.CY 2021
Year: 2021

In January 2019, the UK Government published its Maritime 2050 on Navigating the Future strategy. In the strategy, the government highlighted the importance of digitalization (with well-designed regulatory support) to achieve its goal of ensuring that the UK plays a global leadership role in the maritime sector. Ports, the gateways for 95% of UK trade movements, were identified as key sites for investment in technological innovation. The government identified the potential of the Internet of Things (IoT), in conjunction with other information-sharing technologies, such as shared data platforms, and Artificial Intelligence applications (AI), to synchronize processes within the port ecosystem leading to improved efficiency, safety, and environmental benefits, including improved air quality and lower greenhouse gas emissions.