OpenRaiser/Intern-Atlas huggingface dataset:
BERT-LID: Leveraging BERT to Improve Spoken Language Identification
Venue: arXiv cs.CL 2022
Year: 2022

Language identification is the task of automatically determining the identity of a language conveyed by a spoken segment. It has a profound impact on the multilingual interoperability of an intelligent speech system. Despite language identification attaining high accuracy on medium or long utterances(>3s), the performance on short utterances (<=1s) is still far from satisfactory. We propose a BERT-based language identification system (BERT-LID) to improve language identification performance, especially on short-duration speech segments. We extend the original BERT model by taking the phonetic posteriorgrams (PPG) derived from the front-end phone recognizer as input. Then we deployed the optimal deep classifier followed by it for language identification. Our BERT-LID model can improve the baseline accuracy by about 6.5% on long-segment identification and 19.9% on short-segment identification, demonstrating our BERT-LID's effectiveness to language identification.
Read before Generate! Faithful Long Form Question Answering with Machine Reading
Venue: arXiv cs.CL 2022
Year: 2022

Long-form question answering (LFQA) aims to generate a paragraph-length answer for a given question. While current work on LFQA using large pre-trained model for generation are effective at producing fluent and somewhat relevant content, one primary challenge lies in how to generate a faithful answer that has less hallucinated content. We propose a new end-to-end framework that jointly models answer generation and machine reading. The key idea is to augment the generation model with fine-grained, answer-related salient information which can be viewed as an emphasis on faithful facts. State-of-the-art results on two LFQA datasets, ELI5 and MS MARCO, demonstrate the effectiveness of our method, in comparison with strong baselines on automatic and human evaluation metrics. A detailed analysis further proves the competency of our methods in generating fluent, relevant, and more faithful answers.
Blockchain-based RBAC Model with Separation of Duties constraint in Cloud Environment
Venue: arXiv cs.CR 2022
Year: 2022

In recent years, cloud computing has been developing rapidly and is widely used in various fields such as commerce and scientific research. However, security issues, including access control, are a very important problem in popularizing cloud computing and this has influenced its wide application of cloud computing. As one of the solutions to these problems, we have proposed a blockchain-based role-based access control model with the separation of duties constraints in a cloud environment. In the model, we used Hyperledger Fabric as a blockchain platform for storing the access control policies and provided several functions for effective role management. In addition, we presented an access control scheme for cloud storage data by combining the proposed model and the verification mechanism for the user's ownership of a role and analyzed the security properties of the scheme. Finally, we deployed Hyperledger Fabric test network, implemented an online test system that performs access control using the proposed scheme in the Ali cloud environment, and evaluated the model performance in this scenario.
Clustering Drives Cooperation on Reputation Networks, All Else Fixed
Venue: arXiv cs.SI 2022
Year: 2022

Reputation-based cooperation on social networks offers a causal mechanism between graph properties and social trust. Recent papers on the `structural microfoundations` of the society used this insight to show how demographic processes, such as falling fertility, urbanisation, and migration, can alter the logic of human societies. This paper demonstrates the underlying mechanism in a way that is accessible to scientists not specialising in networks. Additionally, the paper shows that, when the size and degree of the network is fixed (i.e., all graphs have the same number of agents, who all have the same number of connections), it is the clustering coefficient that drives differences in how cooperative social networks are.
Exploring Wilderness Characteristics Using Explainable Machine Learning in Satellite Imagery
Venue: arXiv cs.CV 2022
Year: 2022

Wilderness areas offer important ecological and social benefits and there are urgent reasons to discover where their positive characteristics and ecological functions are present and able to flourish. We apply a novel explainable machine learning technique to satellite images which show wild and anthropogenic areas in Fennoscandia. Occluding certain activations in an interpretable artificial neural network we complete a comprehensive sensitivity analysis regarding wild and anthropogenic characteristics. This enables us to predict detailed and high-resolution sensitivity maps highlighting these characteristics. Our artificial neural network provides an interpretable activation space increasing confidence in our method. Within the activation space, regions are semantically arranged. Our approach advances explainable machine learning for remote sensing, offers opportunities for comprehensive analyses of existing wilderness, and has practical relevance for conservation efforts.
RIS-Assisted Quasi-Static Broad Coverage for Wideband mmWave Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable intelligent surfaces (RISs) can establish favorable wireless environments to combat the severe attenuation and blockages in millimeter-wave (mmWave) bands. However, to achieve the optimal enhancement of performance, the instantaneous channel state information (CSI) needs to be estimated at the cost of a large overhead that scales with the number of RIS elements and the number of users. In this paper, we design a quasi-static broad coverage at the RIS with the reduced overhead based on the statistical CSI. We propose a design framework to synthesize the power pattern reflected by the RIS that meets the customized requirements of broad coverage. For the communication of broadcast channels, we generalize the broad coverage of the single transmit stream to the scenario of multiple streams. Moreover, we employ the quasi-static broad coverage for a multiuser orthogonal frequency division multiplexing access (OFDMA) system, and derive the analytical expression of the downlink rate, which is proved to increase logarithmically with the power gain reflected by the RIS. By taking into account the overhead of channel estimation, the proposed quasi-static broad coverage even outperforms the design method that optimizes the RIS phases using the instantaneous CSI. Numerical simulations are conducted to verify these observations.
The Impact of IT Projects Complexity on Cost Overruns and Schedule Delays
Venue: arXiv cs.SE 2022
Year: 2022

This study aims to assist Company leaders and management team in providing more accurate time and cost estimations for future software projects. This paper focuses on analyzing the relationship between project complexity and cost and time overrun. The sample data of around 50 projects are collected from the Changepoint database. The two research hypotheses were defined and tested using statistical techniques. The quantitative analysis includes descriptive analysis and regression modelling. Experimental results show a strong positive linear relationship between project complexity and cost and time overrun. Based on the study outcome, there is a need to focus on the planning phase of complex projects and allocate more time and budget to estimate the completion date and the initial budget of this type of project more accurately.
On genetic programming representations and fitness functions for interpretable dimensionality reduction
Venue: arXiv cs.NE 2022
Year: 2022

Dimensionality reduction (DR) is an important technique for data exploration and knowledge discovery. However, most of the main DR methods are either linear (e.g., PCA), do not provide an explicit mapping between the original data and its lower-dimensional representation (e.g., MDS, t-SNE, isomap), or produce mappings that cannot be easily interpreted (e.g., kernel PCA, neural-based autoencoder). Recently, genetic programming (GP) has been used to evolve interpretable DR mappings in the form of symbolic expressions. There exists a number of ways in which GP can be used to this end and no study exists that performs a comparison. In this paper, we fill this gap by comparing existing GP methods as well as devising new ones. We evaluate our methods on several benchmark datasets based on predictive accuracy and on how well the original features can be reconstructed using the lower-dimensional representation only. Finally, we qualitatively assess the resulting expressions and their complexity. We find that various GP methods can be competitive with state-of-the-art DR algorithms and that they have the potential to produce interpretable DR mappings.
An Adaptable and Agnostic Flow Scheduling Approach for Data Center Networks
Venue: arXiv cs.NI 2022
Year: 2022

Cloud applications have reshaped the model of services and infrastructure of the Internet. Search engines, social networks, content delivery and retail and e-commerce sites belong to this group of applications. An important element in the architecture of data centers where these applications run is the communication infrastructure, commonly known as data center networks (DCNs). A critical challenge DCNs have to address is the processing of the traffic of cloud applications, which due to its properties is essentially different to the traffic of other Internet applications. In order to improve the responsiveness and throughput of applications, DCNs should be able to prioritize short flows (a few KB) over long flows (several MB). However, given the time and space variations the traffic presents, the information about flow sizes is not available in advance in order to plan the flow scheduling. In this paper, we present an adaptable mechanism called Adaptable Workload-Agnostic Flow Scheduling (AWAFS). It is an adaptable approach that can adjust in an agnostic way the scheduling configuration of DCN forwarding devices. This agnostic adjustment contributes to reduce the Flow Completion Time (FCT) of those short flows, representing around 85% of the traffic handled by cloud applications. Our evaluation results based on simulation show that AWAFS can reduce the average FCT of short flows between 16.9% and 45.2% when compared to the best existing agnostic non-adaptable solution, without inducing starvation on long flows. Indeed, it can provide improvements as high as 39% for long flows. Additionally, AWAFS can improve the FCT for short flows in scenarios with high heterogeneity in the traffic present in the network, with a reduction up to 5% for the average FCT and 15% for the tail FCT.
Multi-Channel Man-in-the-Middle Attacks Against Protected Wi-Fi Networks: A State of the Art Review
Venue: arXiv cs.CR 2022
Year: 2022

Multi-Channel Man-in-the-Middle (MitM) attacks are special MitM attacks capable of manipulating encrypted Wi-Fi wireless frames between two legitimate endpoints. Since its inception in 2014, attackers have been targeting WPA Wi-Fi networks to perform different attacks, such as cipher downgrades, denial of service, key reinstallation Man-in-the-Middle (MitM) attacks (KRACK) in 2017, and recently FragAttacks in 2021, which widely impacted millions of Wi-Fi Multi-Channel MitM (MC-MitM) devices, especially IoT devices. To the best of our knowledge, there are no studies in the literature that KRACK holistically review the different types of Multi-Channel MitM enabled attacks and analyze their potential Internet of Things (IoT) impact. To this end, we evaluate the capabilities of Multi-Channel MitM and review every reported attack in Encryption the state of the art. We examine practical issues that hamper the total adoption of protection mechanisms, i.e., Security security patches and Protected Management Frames (PMF), and review available defense mechanisms in FragAttacks confronting the Multi-Channel MitM enabled attacks in the IoT context. Finally, we highlight the potential research problems and identify future research lines in this field.
A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning
Venue: arXiv cs.CV 2022
Year: 2022

Automatic localization and segmentation of organs-at-risk (OAR) in CT are essential pre-processing steps in medical image analysis tasks, such as radiation therapy planning. For instance, the segmentation of OAR surrounding tumors enables the maximization of radiation to the tumor area without compromising the healthy tissues. However, the current medical workflow requires manual delineation of OAR, which is prone to errors and is annotator-dependent. In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation rather than novel localization or segmentation architectures. To the best of our knowledge, our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging. In the first step, a 3D multi-variate regression network predicts organs' centroids and bounding boxes. Secondly, 3D organ-specific segmentation networks are leveraged to generate a multi-organ segmentation map. Our method achieved an overall Dice score of $0.9260\pm 0.18 \%$ on the VISCERAL dataset containing CT scans with varying fields of view and multiple organs.
Variational Autoencoders Without the Variation
Venue: arXiv cs.LG 2022
Year: 2022

Variational autoencdoers (VAE) are a popular approach to generative modelling. However, exploiting the capabilities of VAEs in practice can be difficult. Recent work on regularised and entropic autoencoders have begun to explore the potential, for generative modelling, of removing the variational approach and returning to the classic deterministic autoencoder (DAE) with additional novel regularisation methods. In this paper we empirically explore the capability of DAEs for image generation without additional novel methods and the effect of the implicit regularisation and smoothness of large networks. We find that DAEs can be used successfully for image generation without additional loss terms, and that many of the useful properties of VAEs can arise implicitly from sufficiently large convolutional encoders and decoders when trained on CIFAR-10 and CelebA.
A hardware-software co-design approach to minimize the use of memory resources in multi-core neuromorphic processors
Venue: arXiv cs.NE 2022
Year: 2022

Both in electronics and biology, physical implementations of neural networks have severe energy and memory constraints. We propose a hardware-software co-design approach for minimizing the use of memory resources in multi-core neuromorphic processors, by taking inspiration from biological neural networks. We use this approach to design new routing schemes optimized for small-world networks and to provide guidelines for designing novel application-specific multi-core neuromorphic chips. Starting from the hierarchical routing scheme proposed, we present a hardware-aware placement algorithm that optimizes the allocation of resources for arbitrary network models. We validate the algorithm with a canonical small-world network and present preliminary results for other networks derived from it.
Hierarchical Reinforcement Learning with AI Planning Models
Venue: arXiv cs.AI 2022
Year: 2022

Two common approaches to sequential decision-making are AI planning (AIP) and reinforcement learning (RL). Each has strengths and weaknesses. AIP is interpretable, easy to integrate with symbolic knowledge, and often efficient, but requires an up-front logical domain specification and is sensitive to noise; RL only requires specification of rewards and is robust to noise but is sample inefficient and not easily supplied with external knowledge. We propose an integrative approach that combines high-level planning with RL, retaining interpretability, transfer, and efficiency, while allowing for robust learning of the lower-level plan actions. Our approach defines options in hierarchical reinforcement learning (HRL) from AIP operators by establishing a correspondence between the state transition model of AI planning problem and the abstract state transition system of a Markov Decision Process (MDP). Options are learned by adding intrinsic rewards to encourage consistency between the MDP and AIP transition models. We demonstrate the benefit of our integrated approach by comparing the performance of RL and HRL algorithms in both MiniGrid and N-rooms environments, showing the advantage of our method over the existing ones.
Maximum Flow and Minimum-Cost Flow in Almost-Linear Time
Venue: arXiv cs.DS 2022
Year: 2022

We give an algorithm that computes exact maximum flows and minimum-cost flows on directed graphs with $m$ edges and polynomially bounded integral demands, costs, and capacities in $m^{1+o(1)}$ time. Our algorithm builds the flow through a sequence of $m^{1+o(1)}$ approximate undirected minimum-ratio cycles, each of which is computed and processed in amortized $m^{o(1)}$ time using a new dynamic graph data structure. Our framework extends to algorithms running in $m^{1+o(1)}$ time for computing flows that minimize general edge-separable convex functions to high accuracy. This gives almost-linear time algorithms for several problems including entropy-regularized optimal transport, matrix scaling, $p$-norm flows, and $p$-norm isotonic regression on arbitrary directed acyclic graphs.
Tricks and Plugins to GBM on Images and Sequences
Venue: arXiv cs.LG 2022
Year: 2022

Convolutional neural networks (CNNs) and transformers, which are composed of multiple processing layers and blocks to learn the representations of data with multiple abstract levels, are the most successful machine learning models in recent years. However, millions of parameters and many blocks make them difficult to be trained, and sometimes several days or weeks are required to find an ideal architecture or tune the parameters. Within this paper, we propose a new algorithm for boosting Deep Convolutional Neural Networks (BoostCNN) to combine the merits of dynamic feature selection and BoostCNN, and another new family of algorithms combining boosting and transformers. To learn these new models, we introduce subgrid selection and importance sampling strategies and propose a set of algorithms to incorporate boosting weights into a deep learning architecture based on a least squares objective function. These algorithms not only reduce the required manual effort for finding an appropriate network architecture but also result in superior performance and lower running time. Experiments show that the proposed methods outperform benchmarks on several fine-grained classification tasks.
Low-Cost On-device Partial Domain Adaptation (LoCO-PDA): Enabling efficient CNN retraining on edge devices
Venue: arXiv cs.CV 2022
Year: 2022

With the increased deployment of Convolutional Neural Networks (CNNs) on edge devices, the uncertainty of the observed data distribution upon deployment has led researchers to to utilise large and extensive datasets such as ILSVRC'12 to train CNNs. Consequently, it is likely that the observed data distribution upon deployment is a subset of the training data distribution. In such cases, not adapting a network to the observed data distribution can cause performance degradation due to negative transfer and alleviating this is the focus of Partial Domain Adaptation (PDA). Current works targeting PDA do not focus on performing the domain adaptation on an edge device, adapting to a changing target distribution or reducing the cost of deploying the adapted network. This work proposes a novel PDA methodology that targets all of these directions and opens avenues for on-device PDA. LoCO-PDA adapts a deployed network to the observed data distribution by enabling it to be retrained on an edge device. Across subsets of the ILSVRC12 dataset, LoCO-PDA improves classification accuracy by 3.04pp on average while achieving up to 15.1x reduction in retraining memory consumption and 2.07x improvement in inference latency on the NVIDIA Jetson TX2. The work is open-sourced at \emph{link removed for anonymity}.
TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors
Venue: arXiv cs.RO 2022
Year: 2022

Inspired by the human ability to perform complex manipulation in the complete absence of vision (like retrieving an object from a pocket), the robotic manipulation field is motivated to develop new methods for tactile-based object interaction. However, tactile sensing presents the challenge of being an active sensing modality: a touch sensor provides sparse, local data, and must be used in conjunction with effective exploration strategies in order to collect information. In this work, we focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM (TActile exploration aNd DEcision Making), an architecture to learn efficient exploration strategies in conjunction with decision making. Our approach is based on separate but co-trained modules for exploration and discrimination. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on binary contact signals alone. TANDEM achieves higher accuracy with fewer actions than alternative methods and is also shown to be more robust to sensor noise.
NESTANets: Stable, accurate and efficient neural networks for analysis-sparse inverse problems
Venue: arXiv cs.LG 2022
Year: 2022

Solving inverse problems is a fundamental component of science, engineering and mathematics. With the advent of deep learning, deep neural networks have significant potential to outperform existing state-of-the-art, model-based methods for solving inverse problems. However, it is known that current data-driven approaches face several key issues, notably hallucinations, instabilities and unpredictable generalization, with potential impact in critical tasks such as medical imaging. This raises the key question of whether or not one can construct deep neural networks for inverse problems with explicit stability and accuracy guarantees. In this work, we present a novel construction of accurate, stable and efficient neural networks for inverse problems with general analysis-sparse models, termed NESTANets. To construct the network, we first unroll NESTA, an accelerated first-order method for convex optimization. The slow convergence of this method leads to deep networks with low efficiency. Therefore, to obtain shallow, and consequently more efficient, networks we combine NESTA with a novel restart scheme. We then use compressed sensing techniques to demonstrate accuracy and stability. We showcase this approach in the case of Fourier imaging, and verify its stability and performance via a series of numerical experiments. The key impact of this work is demonstrating the construction of efficient neural networks based on unrolling with guaranteed stability and accuracy.