OpenRaiser/Intern-Atlas huggingface dataset:
AACP: Model Compression by Accurate and Automatic Channel Pruning
Venue: arXiv cs.CV 2021
Year: 2021

Channel pruning is formulated as a neural architecture search (NAS) problem recently. However, existing NAS-based methods are challenged by huge computational cost and inflexibility of applications. How to deal with multiple sparsity constraints simultaneously and speed up NAS-based channel pruning are still open challenges. In this paper, we propose a novel Accurate and Automatic Channel Pruning (AACP) method to address these problems. Firstly, AACP represents the structure of a model as a structure vector and introduces a pruning step vector to control the compressing granularity of each layer. Secondly, AACP utilizes Pruned Structure Accuracy Estimator (PSAE) to speed up the performance estimation process. Thirdly, AACP proposes Improved Differential Evolution (IDE) algorithm to search the optimal structure vector effectively. Because of IDE, AACP can deal with FLOPs constraint and model size constraint simultaneously and efficiently. Our method can be easily applied to various tasks and achieve state of the art performance. On CIFAR10, our method reduces $65\%$ FLOPs of ResNet110 with an improvement of $0.26\%$ top-1 accuracy. On ImageNet, we reduce $42\%$ FLOPs of ResNet50 with a small loss of $0.18\%$ top-1 accuracy and reduce $30\%$ FLOPs of MobileNetV2 with a small loss of $0.7\%$ top-1 accuracy. The source code will be released after publication.
Toward Blockchain-Enabled Supply Chain Anti-Counterfeiting and Traceability
Venue: arXiv cs.CY 2021
Year: 2021

Innovative solutions addressing product anti-counterfeiting and record provenance have been deployed across today's internationally spanning supply chain networks. These product anti-counterfeiting solutions are developed and implemented with centralized system architecture relying on centralized authorities or any form of intermediaries. Vulnerabilities of centralized product anti-counterfeiting solutions could possibly lead to system failure or susceptibility of malicious modifications performed on product records or various potential attacks to the system components by dishonest participant nodes traversing along the supply chain. Blockchain technology has progressed from merely with a use case of immutable ledger for cryptocurrency transactions to a programmable interactive environment of developing decentralized and reliable applications addressing different use cases globally. In this research, so as to facilitate trustworthy data provenance retrieval, verification and management, as well as strengthening capability of product anti-counterfeiting, key areas of decentralization and feasible mechanisms of developing decentralized and distributed product anti-counterfeiting and traceability ecosystems utilizing blockchain technology, are identified via a series of security and threat analyses performed mainly against NFC-Enabled Anti-Counterfeiting System (NAS) which is one of the solutions currently implemented in the industry with centralized architecture. A set of fundamental system requirements are set out for developing a blockchain-enabled autonomous and decentralized solution for supply chain anti-counterfeiting and traceability, as a secure and immutable scientific data provenance tracking and management platform in which provenance records, providing compelling properties on data integrity of luxurious goods, are recorded and verified automatically, for supply chain industry.
Improving Accountability in Recommender Systems Research Through Reproducibility
Venue: arXiv cs.IR 2021
Year: 2021

Reproducibility is a key requirement for scientific progress. It allows the reproduction of the works of others, and, as a consequence, to fully trust the reported claims and results. In this work, we argue that, by facilitating reproducibility of recommender systems experimentation, we indirectly address the issues of accountability and transparency in recommender systems research from the perspectives of practitioners, designers, and engineers aiming to assess the capabilities of published research works. These issues have become increasingly prevalent in recent literature. Reasons for this include societal movements around intelligent systems and artificial intelligence striving towards fair and objective use of human behavioral data (as in Machine Learning, Information Retrieval, or Human-Computer Interaction). Society has grown to expect explanations and transparency standards regarding the underlying algorithms making automated decisions for and around us. This work surveys existing definitions of these concepts, and proposes a coherent terminology for recommender systems research, with the goal to connect reproducibility to accountability. We achieve this by introducing several guidelines and steps that lead to reproducible and, hence, accountable experimental workflows and research. We additionally analyze several instantiations of recommender system implementations available in the literature, and discuss the extent to which they fit in the introduced framework. With this work, we aim to shed light on this important problem, and facilitate progress in the field by increasing the accountability of research.
A Machine Learning Approach to Optimal Inverse Discrete Cosine Transform (IDCT) Design
Venue: arXiv cs.MM 2021
Year: 2021

The design of the optimal inverse discrete cosine transform (IDCT) to compensate the quantization error is proposed for effective lossy image compression in this work. The forward and inverse DCTs are designed in pair in current image/video coding standards without taking the quantization effect into account. Yet, the distribution of quantized DCT coefficients deviate from that of original DCT coefficients. This is particularly obvious when the quality factor of JPEG compressed images is small. To address this problem, we first use a set of training images to learn the compound effect of forward DCT, quantization and dequantization in cascade. Then, a new IDCT kernel is learned to reverse the effect of such a pipeline. Experiments are conducted to demonstrate that the advantage of the new method, which has a gain of 0.11-0.30dB over the standard JPEG over a wide range of quality factors.
Commutative Monads for Probabilistic Programming Languages
Venue: arXiv cs.LO 2021
Year: 2021

A long-standing open problem in the semantics of programming languages supporting probabilistic choice is to find a commutative monad for probability on the category DCPO. In this paper we present three such monads and a general construction for finding even more. We show how to use these monads to provide a sound and adequate denotational semantics for the Probabilistic FixPoint Calculus (PFPC) -- a call-by-value simply-typed lambda calculus with mixed-variance recursive types, term recursion and probabilistic choice. We also show that in the special case where we consider continuous dcpo's, then all three monads coincide with the valuations monad of Jones and we fully characterise the induced Eilenberg-Moore categories by showing that they are all isomorphic to the category of continuous Kegelspitzen of Keimel and Plotkin.
Co-Seg: An Image Segmentation Framework Against Label Corruption
Venue: arXiv cs.LG 2021
Year: 2021

Supervised deep learning performance is heavily tied to the availability of high-quality labels for training. Neural networks can gradually overfit corrupted labels if directly trained on noisy datasets, leading to severe performance degradation at test time. In this paper, we propose a novel deep learning framework, namely Co-Seg, to collaboratively train segmentation networks on datasets which include low-quality noisy labels. Our approach first trains two networks simultaneously to sift through all samples and obtain a subset with reliable labels. Then, an efficient yet easily-implemented label correction strategy is applied to enrich the reliable subset. Finally, using the updated dataset, we retrain the segmentation network to finalize its parameters. Experiments in two noisy labels scenarios demonstrate that our proposed model can achieve results comparable to those obtained from supervised learning trained on the noise-free labels. In addition, our framework can be easily implemented in any segmentation algorithm to increase its robustness to noisy labels.
Generative and Discriminative Deep Belief Network Classifiers: Comparisons Under an Approximate Computing Framework
Venue: arXiv cs.LG 2021
Year: 2021

The use of Deep Learning hardware algorithms for embedded applications is characterized by challenges such as constraints on device power consumption, availability of labeled data, and limited internet bandwidth for frequent training on cloud servers. To enable low power implementations, we consider efficient bitwidth reduction and pruning for the class of Deep Learning algorithms known as Discriminative Deep Belief Networks (DDBNs) for embedded-device classification tasks. We train DDBNs with both generative and discriminative objectives under an approximate computing framework and analyze their power-at-performance for supervised and semi-supervised applications. We also investigate the out-of-distribution performance of DDBNs when the inference data has the same class structure yet is statistically different from the training data owing to dynamic real-time operating environments. Based on our analysis, we provide novel insights and recommendations for choice of training objectives, bitwidth values, and accuracy sensitivity with respect to the amount of labeled data for implementing DDBN inference with minimum power consumption on embedded hardware platforms subject to accuracy tolerances.
GraphEBM: Molecular Graph Generation with Energy-Based Models
Venue: arXiv cs.LG 2021
Year: 2021

We note that most existing approaches for molecular graph generation fail to guarantee the intrinsic property of permutation invariance, resulting in unexpected bias in generative models. In this work, we propose GraphEBM to generate molecular graphs using energy-based models. In particular, we parameterize the energy function in a permutation invariant manner, thus making GraphEBM permutation invariant. We apply Langevin dynamics to train the energy function by approximately maximizing likelihood and generate samples with low energies. Furthermore, to generate molecules with a desirable property, we propose a simple yet effective strategy, which pushes down energies with flexible degrees according to the properties of corresponding molecules. Finally, we explore the use of GraphEBM for generating molecules with multiple objectives in a compositional manner. Comprehensive experimental results on random, goal-directed, and compositional generation tasks demonstrate the effectiveness of our proposed method.
Boosting the Predictive Accurary of Singer Identification Using Discrete Wavelet Transform For Feature Extraction
Venue: arXiv cs.SD 2021
Year: 2021

Facing the diversity and growth of the musical field nowadays, the search for precise songs becomes more and more complex. The identity of the singer facilitates this search. In this project, we focus on the problem of identifying the singer by using different methods for feature extraction. Particularly, we introduce the Discrete Wavelet Transform (DWT) for this purpose. To the best of our knowledge, DWT has never been used this way before in the context of singer identification. This process consists of three crucial parts. First, the vocal signal is separated from the background music by using the Robust Principal Component Analysis (RPCA). Second, features from the obtained vocal signal are extracted. Here, the goal is to study the performance of the Discrete Wavelet Transform (DWT) in comparison to the Mel Frequency Cepstral Coefficient (MFCC) which is the most used technique in audio signals. Finally, we proceed with the identification of the singer where two methods have experimented: the Support Vector Machine (SVM), and the Gaussian Mixture Model (GMM). We conclude that, for a dataset of 4 singers and 200 songs, the best identification system consists of the DWT (db4) feature extraction introduced in this work combined with a linear support vector machine for identification resulting in a mean accuracy of 83.96%.
Quantifying the Global Support Network for Non-State Armed Groups (NAGs)
Venue: arXiv cs.SI 2021
Year: 2021

Human history has been shaped by armed conflicts. Rather than large-scale interstate wars, low-intensity attacks have been more prevalent in the post-World War era. These attacks are often carried out by non-state armed groups (NAGs), which are supported by host states (HSs). We analyze the global bipartite network of NAG-HS support and its evolution over the period of 1945-2010. We find striking parallels to ecological networks such as mutualistic and parasitic forms of support, and a nested and modular network architecture. The nestedness emerges from preferential behaviors: highly connected players are more likely to both gain and lose connections. Long-persisting major modules are identified, reflecting both regional and trans-regional interests, which show significant turnover in their membership, contrary to the transitory ones. Revealing this architecture further enables the identification of actor's roles and provide insights for effective intervention strategies.
Interpretable Reinforcement Learning Inspired by Piaget's Theory of Cognitive Development
Venue: arXiv cs.AI 2021
Year: 2021

Endeavors for designing robots with human-level cognitive abilities have led to different categories of learning machines. According to Skinner's theory, reinforcement learning (RL) plays a key role in human intuition and cognition. Majority of the state-of-the-art methods including deep RL algorithms are strongly influenced by the connectionist viewpoint. Such algorithms can significantly benefit from theories of mind and learning in other disciplines. This paper entertains the idea that theories such as language of thought hypothesis (LOTH), script theory, and Piaget's cognitive development theory provide complementary approaches, which will enrich the RL field. Following this line of thinking, a general computational building block is proposed for Piaget's schema theory that supports the notions of productivity, systematicity, and inferential coherence as described by Fodor in contrast with the connectionism theory. Abstraction in the proposed method is completely upon the system itself and is not externally constrained by any predefined architecture. The whole process matches the Neisser's perceptual cycle model. Performed experiments on three typical control problems followed by behavioral analysis confirm the interpretability of the proposed method and its competitiveness compared to the state-of-the-art algorithms. Hence, the proposed framework can be viewed as a step towards achieving human-like cognition in artificial intelligent systems.
Stochastic Geometry Analysis of Spatial-Temporal Performance in Wireless Networks: A Tutorial
Venue: arXiv cs.NI 2021
Year: 2021

The performance of wireless networks is fundamentally limited by the aggregate interference, which depends on the spatial distributions of the interferers, channel conditions, and user traffic patterns (or queueing dynamics). These factors usually exhibit spatial and temporal correlations and thus make the performance of large-scale networks environment-dependent (i.e., dependent on network topology, locations of the blockages, etc.). The correlation can be exploited in protocol designs (e.g., spectrum-, load-, location-, energy-aware resource allocations) to provide efficient wireless services. For this, accurate system-level performance characterization and evaluation with spatio-temporal correlation are required. In this context, stochastic geometry models and random graph techniques have been used to develop analytical frameworks to capture the spatio-temporal interference correlation in large-scale wireless networks. The objective of this article is to provide a tutorial on the stochastic geometry analysis of large-scale wireless networks that captures the spatio-temporal interference correlation (and hence the signal-to-interference ratio (SIR) correlation). We first discuss the importance of spatio-temporal performance analysis, different parameters affecting the spatio-temporal correlation in the SIR, and the different performance metrics for spatio-temporal analysis. Then we describe the methodologies to characterize spatio-temporal SIR correlations for different network configurations (independent, attractive, repulsive configurations), shadowing scenarios, user locations, queueing behavior, relaying, retransmission, and mobility. We conclude by outlining future research directions in the context of spatio-temporal analysis of emerging wireless communications scenarios.
Spatiotemporal Ground Reaction Force Analysis using Convolutional Neural Networks to Analyze Parkinsonian Gait
Venue: arXiv cs.LG 2021
Year: 2021

Parkinson's disease (PD) is a non-curable disease that commonly found among elders that greatly reduce their quality of life. PD primarily affects the gait pattern and slowly changes the walking gait from the normality to disability. The early diagnosing of PD is important for treatments and gait pattern analysis is used as a technique to diagnose PD. The present paper has identified the raw spatiotemporal ground reaction force (GRF) as a key parameter to identify the changes in human gait patterns associated with PD. The changes in GRF are identified using a convolutional neural network through pre-processing, conversion, recognition, and performance evaluation. The proposed algorithm is capable of identifying the severity of the PD and distinguishing the parkinsonian gait from the healthy gait. The technique has shown a 97% of accuracy in automatic decision-making process.
Control and Navigation Framework for a Hybrid Steel Bridge Inspection Robot
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous navigation of steel bridge inspection robots is essential for proper maintenance. The majority of existing robotic solutions for steel bridge inspection requires human intervention to assist in the control and navigation. In this paper, a control and navigation framework has been proposed for the steel bridge inspection robot developed by the Advanced Robotics and Automation (ARA)to facilitate autonomous real-time navigation and minimize human intervention. The ARA robot is designed to work in two modes: mobile and inch-worm. The robot uses mobile mode when moving on a plane surface and inch-worm mode when jumping from one surface to the other. To allow the ARA robot to switch between mobile and inch-worm modes, a switching controller is developed with 3D point cloud data based. The surface detection algorithm is proposed to allow the robot to check the availability of steel surfaces (plane, area, and height) to determine the transformation from mobile mode to inch-worm one. To have the robot safely navigate and visit all steel members of the bridge, four algorithms are developed to process the data from a depth camera, segment it into clusters, estimate the boundaries, construct a graph representing the structure, generate the shortest inspection path with any starting and ending points, and determine available robot configuration for path planning. Experiments on steel bridge structures setup highlight the effective performance of the algorithms, and the potential to apply to the ARA robot to run on real bridge structures.
Over-the-Air Computation with Spatial-and-Temporal Correlated Signals
Venue: arXiv cs.IT 2021
Year: 2021

Over-the-air computation (AirComp) leveraging the superposition property of wireless multiple-access channel (MAC), is a promising technique for effective data collection and computation of large-scale wireless sensor measurements in Internet of Things applications. Most existing work on AirComp only considered computation of spatial-and-temporal independent sensor signals, though in practice different sensor measurement signals are usually correlated. In this letter, we propose an AirComp system with spatial-and-temporal correlated sensor signals, and formulate the optimal AirComp policy design problem for achieving the minimum computation mean-squared error (MSE). We develop the optimal AirComp policy with the minimum computation MSE in each time step by utilizing the current and the previously received signals. We also propose and optimize a low-complexity AirComp policy in closed form with the performance approaching to the optimal policy.
On conditional Sibson's $α$-Mutual Information
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we analyse how to define a conditional version of Sibson's $α$-Mutual Information. Several such definitions can be advanced and they all lead to different information measures with different (but similar) operational meanings. We will analyse in detail one such definition, compute a closed-form expression for it and endorse it with an operational meaning while also considering some applications. The alternative definitions will also be mentioned and compared.
Benchmarking of Deep Learning Irradiance Forecasting Models from Sky Images -- an in-depth Analysis
Venue: arXiv cs.CV 2021
Year: 2021

A number of industrial applications, such as smart grids, power plant operation, hybrid system management or energy trading, could benefit from improved short-term solar forecasting, addressing the intermittent energy production from solar panels. However, current approaches to modelling the cloud cover dynamics from sky images still lack precision regarding the spatial configuration of clouds, their temporal dynamics and physical interactions with solar radiation. Benefiting from a growing number of large datasets, data driven methods are being developed to address these limitations with promising results. In this study, we compare four commonly used Deep Learning architectures trained to forecast solar irradiance from sequences of hemispherical sky images and exogenous variables. To assess the relative performance of each model, we used the Forecast Skill metric based on the smart persistence model, as well as ramp and time distortion metrics. The results show that encoding spatiotemporal aspects of the sequence of sky images greatly improved the predictions with 10 min ahead Forecast Skill reaching 20.4% on the test year. However, based on the experimental data, we conclude that, with a common setup, Deep Learning models tend to behave just as a 'very smart persistence model', temporally aligned with the persistence model while mitigating its most penalising errors. Thus, despite being captured by the sky cameras, models often miss fundamental events causing large irradiance changes such as clouds obscuring the sun. We hope that our work will contribute to a shift of this approach to irradiance forecasting, from reactive to anticipatory.
Side-Channel Trojan Insertion -- a Practical Foundry-Side Attack via ECO
Venue: arXiv cs.CR 2021
Year: 2021

Design companies often outsource their integrated circuit (IC) fabrication to third parties where ICs are susceptible to malicious acts such as the insertion of a side-channel hardware trojan horse (SCT). In this paper, we present a framework for designing and inserting an SCT based on an engineering change order (ECO) flow, which makes it the first to disclose how effortlessly a trojan can be inserted into an IC. The trojan is designed with the goal of leaking multiple bits per power signature reading. Our findings and results show that a rogue element within a foundry has, today, all means necessary for performing a foundry-side attack via ECO.