OpenRaiser/Intern-Atlas huggingface dataset:
The role of Blockchain in DDoS attacks mitigation: techniques, open challenges and future directions
Venue: arXiv cs.CR 2022
Year: 2022

With the proliferation of new technologies such as Internet of Things (IOT) and Software-Defined Networking(SDN) in the recent years, the distributed denial of service (DDoS)attack vector has broadened and opened new opportunities for more sophisticated DDoS attacks on the targeted victims. The new attack vector includes unsecured and vulnerable IoT devices connected to the internet, denial of service vulnerabilities like southbound channel saturation in the SDN architecture. Given the high-volume and pervasive nature of these attacks, it is beneficial for stakeholders to collaborate in detecting and mitigating the denial of service attacks in a timely manner. The blockchain technology is considered to improve the security aspects owing to the decentralized design, secured distributed storage and privacy. A thorough exploration and classification of blockchain techniques used for DDoS attack mitigation is not explored in the prior art. This paper reviews and categorizes the existed state-of-the-art DDoS mitigation solutions based on blockchain technology. The DDoS mitigation techniques are classified based on the solution deployment location i.e. network based, near attacker location, near victim location and hybrid solutions in the network architecture with emphasis on the IoT and SDN architectures. Additionally, based on our study, the research challenges and future directions to implement the blockchain based DDoS mitigation solutions are discussed. We believe that this paper could serve as a starting point and reference resource for future researchers working on denial of service attacks detection and mitigation using blockchain technology.
Bandit Sampling for Multiplex Networks
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks have gained prominence due to their excellent performance in many classification and prediction tasks. In particular, they are used for node classification and link prediction which have a wide range of applications in social networks, biomedical data sets, and financial transaction graphs. Most of the existing work focuses primarily on the monoplex setting where we have access to a network with only a single type of connection between entities. However, in the multiplex setting, where there are multiple types of connections, or \emph{layers}, between entities, performance on tasks such as link prediction has been shown to be stronger when information from other connection types is taken into account. We propose an algorithm for scalable learning on multiplex networks with a large number of layers. The efficiency of our method is enabled by an online learning algorithm that learns how to sample relevant neighboring layers so that only the layers with relevant information are aggregated during training. This sampling differs from prior work, such as MNE, which aggregates information across \emph{all} layers and consequently leads to computational intractability on large networks. Our approach also improves on the recent layer sampling method of \textsc{DeePlex} in that the unsampled layers do not need to be trained, enabling further increases in efficiency.We present experimental results on both synthetic and real-world scenarios that demonstrate the practical effectiveness of our proposed approach.
Invertible Tabular GANs: Killing Two Birds with OneStone for Tabular Data Synthesis
Venue: arXiv cs.LG 2022
Year: 2022

Tabular data synthesis has received wide attention in the literature. This is because available data is often limited, incomplete, or cannot be obtained easily, and data privacy is becoming increasingly important. In this work, we present a generalized GAN framework for tabular synthesis, which combines the adversarial training of GANs and the negative log-density regularization of invertible neural networks. The proposed framework can be used for two distinctive objectives. First, we can further improve the synthesis quality, by decreasing the negative log-density of real records in the process of adversarial training. On the other hand, by increasing the negative log-density of real records, realistic fake records can be synthesized in a way that they are not too much close to real records and reduce the chance of potential information leakage. We conduct experiments with real-world datasets for classification, regression, and privacy attacks. In general, the proposed method demonstrates the best synthesis quality (in terms of task-oriented evaluation metrics, e.g., F1) when decreasing the negative log-density during the adversarial training. If increasing the negative log-density, our experimental results show that the distance between real and fake records increases, enhancing robustness against privacy attacks.
Linear Time Kernel Matrix Approximation via Hyperspherical Harmonics
Venue: arXiv cs.LG 2022
Year: 2022

We propose a new technique for constructing low-rank approximations of matrices that arise in kernel methods for machine learning. Our approach pairs a novel automatically constructed analytic expansion of the underlying kernel function with a data-dependent compression step to further optimize the approximation. This procedure works in linear time and is applicable to any isotropic kernel. Moreover, our method accepts the desired error tolerance as input, in contrast to prevalent methods which accept the rank as input. Experimental results show our approach compares favorably to the commonly used Nystrom method with respect to both accuracy for a given rank and computational time for a given accuracy across a variety of kernels, dimensions, and datasets. Notably, in many of these problem settings our approach produces near-optimal low-rank approximations. We provide an efficient open-source implementation of our new technique to complement our theoretical developments and experimental results.
How to Understand Masked Autoencoders
Venue: arXiv cs.CV 2022
Year: 2022

"Masked Autoencoders (MAE) Are Scalable Vision Learners" revolutionizes the self-supervised learning method in that it not only achieves the state-of-the-art for image pre-training, but is also a milestone that bridges the gap between visual and linguistic masked autoencoding (BERT-style) pre-trainings. However, to our knowledge, to date there are no theoretical perspectives to explain the powerful expressivity of MAE. In this paper, we, for the first time, propose a unified theoretical framework that provides a mathematical understanding for MAE. Specifically, we explain the patch-based attention approaches of MAE using an integral kernel under a non-overlapping domain decomposition setting. To help the research community to further comprehend the main reasons of the great success of MAE, based on our framework, we pose five questions and answer them with mathematical rigor using insights from operator theory.
APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) enables training models at different sites and updating the weights from the training instead of transferring data to a central location and training as in classical machine learning. The FL capability is especially important to domains such as biomedicine and smart grid, where data may not be shared freely or stored at a central location because of policy challenges. Thanks to the capability of learning from decentralized datasets, FL is now a rapidly growing research field, and numerous FL frameworks have been developed. In this work, we introduce APPFL, the Argonne Privacy-Preserving Federated Learning framework. APPFL allows users to leverage implemented privacy-preserving algorithms, implement new algorithms, and simulate and deploy various FL algorithms with privacy-preserving techniques. The modular framework enables users to customize the components for algorithms, privacy, communication protocols, neural network models, and user data. We also present a new communication-efficient algorithm based on an inexact alternating direction method of multipliers. The algorithm requires significantly less communication between the server and the clients than does the current state of the art. We demonstrate the computational capabilities of APPFL, including differentially private FL on various test datasets and its scalability, by using multiple algorithms and datasets on different computing environments.
Trained Model in Supervised Deep Learning is a Conditional Risk Minimizer
Venue: arXiv cs.LG 2022
Year: 2022

We proved that a trained model in supervised deep learning minimizes the conditional risk for each input (Theorem 2.1). This property provided insights into the behavior of trained models and established a connection between supervised and unsupervised learning in some cases. In addition, when the labels are intractable but can be written as a conditional risk minimizer, we proved an equivalent form of the original supervised learning problem with accessible labels (Theorem 2.2). We demonstrated that many existing works, such as Noise2Score, Noise2Noise and score function estimation can be explained by our theorem. Moreover, we derived a property of classification problem with noisy labels using Theorem 2.1 and validated it using MNIST dataset. Furthermore, We proposed a method to estimate uncertainty in image super-resolution based on Theorem 2.2 and validated it using ImageNet dataset. Our code is available on github.
A Novel Image Descriptor with Aggregated Semantic Skeleton Representation for Long-term Visual Place Recognition
Venue: arXiv cs.CV 2022
Year: 2022

In a Simultaneous Localization and Mapping (SLAM) system, a loop-closure can eliminate accumulated errors, which is accomplished by Visual Place Recognition (VPR), a task that retrieves the current scene from a set of pre-stored sequential images through matching specific scene-descriptors. In urban scenes, the appearance variation caused by seasons and illumination has brought great challenges to the robustness of scene descriptors. Semantic segmentation images can not only deliver the shape information of objects but also their categories and spatial relations that will not be affected by the appearance variation of the scene. Innovated by the Vector of Locally Aggregated Descriptor (VLAD), in this paper, we propose a novel image descriptor with aggregated semantic skeleton representation (SSR), dubbed SSR-VLAD, for the VPR under drastic appearance-variation of environments. The SSR-VLAD of one image aggregates the semantic skeleton features of each category and encodes the spatial-temporal distribution information of the image semantic information. We conduct a series of experiments on three public datasets of challenging urban scenes. Compared with four state-of-the-art VPR methods- CoHOG, NetVLAD, LOST-X, and Region-VLAD, VPR by matching SSR-VLAD outperforms those methods and maintains competitive real-time performance at the same time.
What's Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification
Venue: arXiv cs.CV 2022
Year: 2022

Surface cracks are a very common indicator of potential structural faults. Their early detection and monitoring is an important factor in structural health monitoring. Left untreated, they can grow in size over time and require expensive repairs or maintenance. With recent advances in computer vision and deep learning algorithms, the automatic detection and segmentation of cracks for this monitoring process have become a major topic of interest. This review aims to give researchers an overview of the published work within the field of crack analysis algorithms that make use of deep learning. It outlines the various tasks that are solved through applying computer vision algorithms to surface cracks in a structural health monitoring setting and also provides in-depth reviews of recent fully, semi and unsupervised approaches that perform crack classification, detection, segmentation and quantification. Additionally, this review also highlights popular datasets used for cracks and the metrics that are used to evaluate the performance of those algorithms. Finally, potential research gaps are outlined and further research directions are provided.
Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methods
Venue: arXiv cs.AI 2022
Year: 2022

In the last decade neural network have made huge impact both in industry and research due to their ability to extract meaningful features from imprecise or complex data, and by achieving super human performance in several domains. However, due to the lack of transparency the use of these networks is hampered in the areas with safety critical areas. In safety-critical areas, this is necessary by law. Recently several methods have been proposed to uncover this black box by providing interpreation of predictions made by these models. The paper focuses on time series analysis and benchmark several state-of-the-art attribution methods which compute explanations for convolutional classifiers. The presented experiments involve gradient-based and perturbation-based attribution methods. A detailed analysis shows that perturbation-based approaches are superior concerning the Sensitivity and occlusion game. These methods tend to produce explanations with higher continuity. Contrarily, the gradient-based techniques are superb in runtime and Infidelity. In addition, a validation the dependence of the methods on the trained model, feasible application domains, and individual characteristics is attached. The findings accentuate that choosing the best-suited attribution method is strongly correlated with the desired use case. Neither category of attribution methods nor a single approach has shown outstanding performance across all aspects.
Impact of Parameter Sparsity on Stochastic Gradient MCMC Methods for Bayesian Deep Learning
Venue: arXiv cs.LG 2022
Year: 2022

Bayesian methods hold significant promise for improving the uncertainty quantification ability and robustness of deep neural network models. Recent research has seen the investigation of a number of approximate Bayesian inference methods for deep neural networks, building on both the variational Bayesian and Markov chain Monte Carlo (MCMC) frameworks. A fundamental issue with MCMC methods is that the improvements they enable are obtained at the expense of increased computation time and model storage costs. In this paper, we investigate the potential of sparse network structures to flexibly trade-off model storage costs and inference run time against predictive performance and uncertainty quantification ability. We use stochastic gradient MCMC methods as the core Bayesian inference method and consider a variety of approaches for selecting sparse network structures. Surprisingly, our results show that certain classes of randomly selected substructures can perform as well as substructures derived from state-of-the-art iterative pruning methods while drastically reducing model training times.
Private Information Delivery with Coded Storage
Venue: arXiv cs.IT 2022
Year: 2022

In private information delivery (PID) problem, there are $K$ messages stored across $N$ servers, each capable of storing $M$ messages and a user. Servers want to convey one of the $K$ messages to the user without revealing the identity (index) of the message conveyed. The capacity of PID problem is defined as maximum number of bits of the desired message that can be conveyed privately, per bit of total communication, to the user. For the restricted case of replicated systems, where coded messages or splitting one message into several servers is not allowed, the capacity of PID has been characterized by Hua Sun in "Private Information Delivery, IEEE Transactions on Information Theory, December 2020" in terms of $K, N$ and $M.$ In this paper, we study the problem of PID with coded storage at the servers. For a class of problems called {\it bi-regular PID} we characterize the capacity for $N=K/M$ and for $N>K/M$ we provide an achievable scheme. In both the cases the rates achieved are more than the rates achievable with the replicated systems.
Predictive Beamforming for Integrated Sensing and Communication in Vehicular Networks: A Deep Learning Approach
Venue: arXiv cs.IT 2022
Year: 2022

The implementation of integrated sensing and communication (ISAC) highly depends on the effective beamforming design exploiting accurate instantaneous channel state information (ICSI). However, channel tracking in ISAC requires large amount of training overhead and prohibitively large computational complexity. To address this problem, in this paper, we focus on ISAC-assisted vehicular networks and exploit a deep learning approach to implicitly learn the features of historical channels and directly predict the beamforming matrix for the next time slot to maximize the average achievable sum-rate of system, thus bypassing the need of explicit channel tracking for reducing the system signaling overhead. To this end, a general sum-rate maximization problem with Cramer-Rao lower bounds-based sensing constraints is first formulated for the considered ISAC system. Then, a historical channels-based convolutional long short-term memory network is designed for predictive beamforming that can exploit the spatial and temporal dependencies of communication channels to further improve the learning performance. Finally, simulation results show that the proposed method can satisfy the requirement of sensing performance, while its achievable sum-rate can approach the upper bound obtained by a genie-aided scheme with perfect ICSI available.
TimeLMs: Diachronic Language Models from Twitter
Venue: arXiv cs.CL 2022
Year: 2022

Despite its importance, the time variable has been largely neglected in the NLP and language model literature. In this paper, we present TimeLMs, a set of language models specialized on diachronic Twitter data. We show that a continual learning strategy contributes to enhancing Twitter-based language models' capacity to deal with future and out-of-distribution tweets, while making them competitive with standardized and more monolithic benchmarks. We also perform a number of qualitative analyses showing how they cope with trends and peaks in activity involving specific named entities or concept drift.
Group Fairness Is Not Derivable From Justice: a Mathematical Proof
Venue: arXiv cs.CY 2022
Year: 2022

We argue that an imperfect criminal law procedure cannot be group-fair, if 'group fairness' involves ensuring the same chances of acquittal or convictions to all innocent defendants independently of their morally arbitrary features. We show mathematically that only a perfect procedure (involving no mistake), a non-deterministic one, or a degenerate one (everyone or no one is convicted) can guarantee group fairness, in the general case. Following a recent proposal, we adopt a definition of group fairness, requiring that individuals who are equal in merit ought to have the same statistical chances of obtaining advantages and disadvantages, in a way that is statistically independent of any of their feature that does not count as merit. We explain by mathematical argument that the only imperfect procedures offering an a-priori guarantee of fairness in relation to all non-merit trait are lotteries or degenerate ones (i.e., everyone or no one is convicted). To provide a more intuitive point of view, we exploit an adjustment of the well-known ROC space, in order to represent all possible procedures in our model by a schematic diagram. The argument seems to be equally valid for all human procedures, provided they are imperfect. This clearly includes algorithmic decision-making, including decisions based on statistical predictions, since in practice all statistical models are error prone.
Learning MAX-SAT from Contextual Examples for Combinatorial Optimisation
Venue: arXiv cs.AI 2022
Year: 2022

Combinatorial optimisation problems are ubiquitous in artificial intelligence. Designing the underlying models, however, requires substantial expertise, which is a limiting factor in practice. The models typically consist of hard and soft constraints, or combine hard constraints with an objective function. We introduce a novel setting for learning combinatorial optimisation problems from contextual examples. These positive and negative examples show - in a particular context - whether the solutions are good enough or not. We develop our framework using the MAX-SAT formalism as it is simple yet powerful setting having these features. We study the learnability of MAX-SAT models. Our theoretical results show that high-quality MAX-SAT models can be learned from contextual examples in the realisable and agnostic settings, as long as the data satisfies an intuitive "representativeness" condition. We also contribute two implementations based on our theoretical results: one leverages ideas from syntax-guided synthesis while the other makes use of stochastic local search techniques. The two implementations are evaluated by recovering synthetic and benchmark models from contextual examples. The experimental results support our theoretical analysis, showing that MAX-SAT models can be learned from contextual examples. Among the two implementations, the stochastic local search learner scales much better than the syntax-guided implementation while providing comparable or better models.
Verification-Aided Deep Ensemble Selection
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks (DNNs) have become the technology of choice for realizing a variety of complex tasks. However, as highlighted by many recent studies, even an imperceptible perturbation to a correctly classified input can lead to misclassification by a DNN. This renders DNNs vulnerable to strategic input manipulations by attackers, and also oversensitive to environmental noise. To mitigate this phenomenon, practitioners apply joint classification by an *ensemble* of DNNs. By aggregating the classification outputs of different individual DNNs for the same input, ensemble-based classification reduces the risk of misclassifications due to the specific realization of the stochastic training process of any single DNN. However, the effectiveness of a DNN ensemble is highly dependent on its members *not simultaneously erring* on many different inputs. In this case study, we harness recent advances in DNN verification to devise a methodology for identifying ensemble compositions that are less prone to simultaneous errors, even when the input is adversarially perturbed -- resulting in more robustly-accurate ensemble-based classification. Our proposed framework uses a DNN verifier as a backend, and includes heuristics that help reduce the high complexity of directly verifying ensembles. More broadly, our work puts forth a novel universal objective for formal verification that can potentially improve the robustness of real-world, deep-learning-based systems across a variety of application domains.
Covert backscatter communication with directional MIMO
Venue: arXiv cs.IT 2022
Year: 2022

We study a backscatter communication protocol over a AWGN channel, where a transmitter illuminates a tag with a directional multi-antenna. The tag performs load modulation on the signal while hiding its physical presence from a warden. We show that, if the transmitter-to-tag channel is inaccessible to the warden, then $Θ(n)$ reliable and covert bits can be transmitted over $n$ channel usages. This overcomes the square-root law for covert communication. This paper provides the first evidence for practical implementation of covert backscatter communication, with potential applications in IoT security.