Reviewerly/exHarmony huggingface dataset:
Decentralized Zeroth-Order Constrained Stochastic Optimization Algorithms: Frank–Wolfe and Variants With Applications to Black-Box Adversarial Attacks

Zeroth-order optimization algorithms are an attractive alternative for stochastic optimization problems, when gradient computations are expensive or when closed-form loss functions are not available. Recently, there has been a surge of activity in utilizing zeroth-order optimization algorithms in myriads of applications including black-box adversarial attacks on machine learning frameworks, reinforcement learning, and simulation-based optimization, to name a few. In addition to utilizing the simplicity of a typical zeroth-order optimization scheme, distributed implementations of zeroth-order schemes so as to exploit data parallelizability are getting significant attention recently. This article presents an overview of recent work in the area of distributed zeroth-order optimization, focusing on constrained optimization settings and algorithms built around the Frank-Wolfe framework. In particular, we review different types of architectures, from master-worker-based decentralized to fully distributed, and describe appropriate zeroth-order projection-free schemes for solving constrained stochastic optimization problems catered to these architectures. We discuss performance issues including convergence rates and dimension dependence. In addition, we also focus on more refined extensions such as by employing variance reduction and describe and quantify convergence rates for a variance-reduced decentralized zeroth-order optimization method inspired by martingale difference sequences. We discuss limitations of zeroth-order optimization frameworks in terms of dimension dependence. Finally, we illustrate the use of distributed zeroth-order algorithms in the context of adversarial attacks on deep learning models.
Joint semantic segmentation of road objects and lanes using Convolutional Neural Networks

This paper presents a multi-task instance segmentation neural network able to provide both road lane and road participants detection. The multi-task approach, ERFNet-based, allows feature sharing and reduces the computational requirements of the overall detection architecture, allowing real time performance even in configurations with limited hardware. The proposed method includes an ad-hoc training procedure and automatic dataset creation mechanism that is also introduced in this paper. The proposed solution has been tested and validated through a newly generated public dataset derived from the BDD100K of 19K images, and in real scenarios. The results obtained prove the viability of the work for road application and its real time performance.
A Throughput-Optimized Channel-Oriented Processing Element Array for Convolutional Neural Networks

Over the past decade, significant developments have taken place in the field of deep learning. State-of-the-art convolutional neural networks (CNNs), a branch of deep learning, have been increasingly applied in various fields such as image classification, speech recognition, and natural language processing. Due to the high computational complexity of CNNs, lots of works have proposed their CNN accelerators to address this issue. Besides, a processing element (PE) array has been recently further focused and discussed since it is responsible for the entire computations as the core of CNN accelerators. Therefore, the specialized design of a PE array becomes one of the main researches on CNN accelerators for energy efficiency and high throughput. In this brief, a throughput-optimized PE array for CNNs based on the channel-oriented data pattern is proposed. The proposed PE array features fully PE interconnection which achieves scalability. Besides, any sized convolution can be processed in the PE array while maximizing the utilization of PEs by exploiting the channel-oriented data pattern. Compared to previous works, this brief achieves 1.22× and 1.25× improvement in the throughput density on AlexNet and VGG-16 respectively.
A novel 3D medical image super-resolution method based on densely connected network

High-quality and high-resolution medical images can help doctors make more accurate diagnoses, but the resolution of medical images is often limited by a variety of factors such as device, operation and compression rate. To deal with this issue, in this paper, we propose a novel densely connected network for super-resolution reconstruction of 3D medical images. In order to obtain multiscale information, we first adopt 3D dilated convolution with different dilation rates to extract shallow features. To better handle these hierarchical features, we combine local residual learning with densely connected layers, which apply 3D asymmetric convolution to improve performance without increasing inference time. Meanwhile, an improved attention module, which considers both channel-wise and spatial information, is applied to enhance attention of the channels and regions with more high-frequency details. Finally, a feature fusion module which contains three parallel dilated convolution is applied to fuse hierarchical features. Compared with the state-of-the-art methods, such as SRCNN, FSRCNN, SRResnet, DCSRN, ReCNN and DCED, our experimental results show that the proposed method has better performance in both objective metrics and visual effect.
Human-Like Constrained-Mating to Make Genetic Algorithm More Explorative

A genetic algorithm (GA) is widely used to solve many optimization problems. It does not promise accurate results but provides an acceptable ones in various practical applications. Sometimes, it is trapped at a premature convergence or a local optimum for a complex problem. Hence, a Human-Like Constrained-Mating Genetic Algorithm (HLCMGA) is proposed in this paper to tackle such a problem. HLCMGA can be simply described as a crossover with human-like constrained mating to improve exploration ability. Computer simulation on ten benchmark multi-modal functions shows that it performs better than the simple GA (SGA). Compared to a state-of-the-art Rao algorithm on five benchmark functions, it reaches the same performances on the four functions and just loses on one function. The simulation also informs that it has a higher exploration ability to converge at the global optimum on various complex search spaces.
Semi-discrete Matrix Factorization

Discrete matrix factorization (DMF) has been a promising solution to improve the inferring efficiency of matrix factorization (MF) against the rapidly growing numbers of users and items. However, DMF suffers from a serious encoding loss due to its oversimplified modeling on the original data geometry. In this article, we propose a semi-discrete matrix factorization (SDMF) model to combine the predicting efficacy of MF with the inferring efficiency of DMF. It first learns real-valued latent features by MF, and then, taking them as group-wise and point-wise smoothness, learns binary codes in the DMF framework, for preserving the geometrical structures collectively hidden in users and items, as well as aligning binary codes originated from Hamming space with their real-valued counterparts learned from vector space. Particularly, we devise a computationally efficient optimization algorithm to estimate model parameters. Extensive evaluations on three real-world datasets clearly demonstrate the superiority of our SDMF model over state-of-the-art hash-based recommendation methods.
An Efficient Blockchain-Based Bidirectional Friends Matching Scheme in Social Networks

In social networks, the personal attributes or hobbies of the users are exposed to the server to establish the relationships. Service providers may store these information for commercial purpose or statistical analysis. Furthermore, the server may expose to external attacks, which may disclose users' privacy information. In this paper, we present a hierarchical blockchain-based attribute matching scheme, which realizes privacy-preserving attribute matching under multiple semi-trusted servers. The scheme employs CP-ABE and bloom filter to satisfy the requirements of the users to make friend discovery, and reduces the computation cost of users by outsourcing decryption of CP-ABE. Besides, the hierarchical blockchain only implements the consensus and storage of matching results on the blockchain, while the complex calculations and a large amount of data storage are off-chain, which reduces the consumption of the blockchain and improves the operation efficiency. Finally, we prove the scheme can resist single point failure, collusion attack, internal attack and external attack, the experimental results demonstrate the proposed scheme is feasibility and efficiency.
PLEDGE: A Proof-of-Honesty based Consensus Protocol for Blockchain-based IoT Systems

Exhibition of malicious behavior during blockchain consensus, threats against reputation systems, and high TX latency are significant issues for blockchain-based IoT systems. Hence, to mitigate such challenges we propose "Pledge", a unique Proof-of-Honesty based consensus protocol. Initial experimentation shows that Pledge is economical with low computations and communications complexity and low latency in transaction confirmation.
Automatic localization and segmentation of focal cortical dysplasia in FLAIR‐negative patients using a convolutional neural network

Abstract Purpose Focal cortical dysplasia (FCD) is a common cause of epilepsy; the only treatment is surgery. Therefore, detecting FCD using noninvasive imaging technology can help doctors determine whether surgical intervention is required. Since FCD lesions are small and not obvious, diagnosing FCD through visual evaluations of magnetic resonance imaging (MRI) scans is difficult. The purpose of this study is to detect and segment histologically confirmed FCD lesions in images of normal fluid‐attenuated inversion recovery (FLAIR)‐negative lesions using convolutional neural network (CNN) technology. Methods The technique involves training a six‐layer CNN named Net‐Pos, which consists of two convolutional layers (CLs); two pooling layers (PLs); and two fully connected (FC) layers, including 60 943 learning parameters. We employed activation maximization (AM) to optimize a series of pattern image blocks (PIBs) that were most similar to a lesion image block by using the trained Net‐Pos. We developed an AM and convolutional localization (AMCL) algorithm that employs the mean PIBs combined with convolution to locate and segment FCD lesions in FLAIR‐negative patients. Five evaluation indices, namely, recall, specificity, accuracy, precision, and the Dice coefficient, were applied to evaluate the localization and segmentation performance of the algorithm. Results The PIBs most similar to an FCD lesion image block were identified by the trained Net‐Pos as image blocks with brighter central areas and darker surrounding image blocks. The technique was evaluated using 18 FLAIR‐negative lesion images from 12 patients. The subject‐wise recall of the AMCL algorithm was 83.33% (15/18). The Dice coefficient for the segmentation performance was 52.68. Conclusion We developed a novel algorithm referred to as the AMCL algorithm with mean PIBs to effectively and automatically detect and segment FLAIR‐negative FCD lesions. This work is the first study to apply a CNN‐based model to detect and segment FCD lesions in images of FLAIR‐negative lesions.
Fourier interferometry of multi-layer sample using wavelength tuning and partially negative window

When measuring the thickness of a transparent plate using wavelength tuning-based phase modulation fringe analysis, the nonlinearity of the phase modulation can cause both a nonuniform phase error and a bias phase error. In this study, a method of eliminating the bias phase error using the characteristic polynomial theory was developed and a flexible 4 N − 1 algorithm, consisting of a partially negative window and a discrete Fourier transform term, was constructed. The behaviors of the 4 N − 1 algorithm were visualized in the frequency domain. The error compensation ability was estimated by comparing it to conventional algorithms. The thickness and surface profile of a lithium niobate wafer were simultaneously measured using a Fizeau interferometer and the proposed 4 N − 1 algorithm. The experimental results showed that the 4 N − 1 algorithm performs better than other algorithms in terms of the repeatability error and suppression of the bias phase error.
Improving cloud data security through hybrid verification technique based on biometrics and encryption system

Cloud computing is a unique network or environment where access, maintenance and process can be done from any part of the world. It is a customized internet-based computer server. It is a current trend of modern technology. For the massive computational power, it is the best option for storing data. There is no uncertainty that Cloud Data Server offers quick and solid types of assistance to its customers. When data is storing in the cloud storage the most important thing comes is the security of data. So, in recent years, cloud security is so much important issue because of the increasing of data. For the security of cloud computing, in this research, a hybrid verification technique is made which is based on biometric and encryption systems. In order to achieve strong and secure technique, this work use fingerprint as biometric technique and advanced encryption standard as a trustworthy encryption system. The primary goal of this paper is to avert information access from cloud information stockpiling focuses by unapproved clients. This new data security system can provide efficient authentication of cloud computing.
DAF: Dependency-Aware FaaSifier for Node.js Monolithic Applications

Porting an existing monolith to a serverless platform ("FaaSification") can be challenging because monoliths commonly have many interdependencies and are highly cohesive. This article introduces a Dependency-Aware FaaSifier (DAF), which allows developers to annotate a method that should be faasified.
AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks

ShuffleNet is a state-of-the-art light weight convolutional neural network architecture. Its basic operations include group, channel-wise convolution and channel shuffling. However, channel shuffling is manually designed on empirical grounds. Mathematically, shuffling is a multiplication by a permutation matrix. In this paper, we propose to automate channel shuffling by learning permutation matrices in network training. We introduce an exact Lipschitz continuous non-convex penalty so that it can be incorporated in the stochastic gradient descent to approximate permutation at high precision. Exact permutations are obtained by simple rounding at the end of training and are used in inference. The resulting network, referred to as AutoShuffleNet, achieved improved classification accuracies on data from CIFAR-10, CIFAR-100 and ImageNet while preserving the inference costs of ShuffleNet. In addition, we found experimentally that the standard convex relaxation of permutation matrices into stochastic matrices leads to poor performance. We prove theoretically the exactness (error bounds) in recovering permutation matrices when our penalty function is zero (very small). We present examples of permutation optimization through graph matching and two-layer neural network models where the loss functions are calculated in closed analytical form. In the examples, convex relaxation failed to capture permutations whereas our penalty succeeded.
Parallel DNN Inference Framework Leveraging a Compact RISC-V ISA-based Multi-core System

RISC-V is an open-source instruction set and now has been examined as a universal standard to unify the heterogeneous platforms. However, current research focuses primarily on the design and fabrication of general-purpose processors based on RISC-V, despite the fact that in the era of IoT (Internet of Things), the fusion of heterogeneous platforms should also take application-specific processors into account. Accordingly, this paper proposes a collaborative RISC-V multi-core system for Deep Neural Network (DNN) accelerators. To the best of our knowledge, this is the first time that a multi-core scheduling architecture for DNN acceleration is formulated and RISC-V is explored as the ISA of a multi-core system to bridge the gap between the memory and the DNN Processor in order to increase the entire system throughput. The experiment realizes a four-stage design of the RISC-V core, and further reveals that a multi-core design along with an appropriate scheduling algorithm can efficiently decrease the runtime and elevate the throughput. Moreover, the experiment also provides us with a constructive suggestion regarding the ideal proportion of the cores to Process Engines (PE), which provides us with significant assistance in building highly efficient AI System-on-Chips (SoCs) in resource-aware situations.
Introduction to Computer Vision and Real Time Deep Learning-based Object Detection

Computer vision (CV) is a field of artificial intelligence that trains computers to interpret and understand the visual world for a variety of exciting downstream tasks such as self-driving cars, checkout-less shopping, smart cities, cancer detection, and more. The field of CV has been revolutionized by deep learning over the last decade. This tutorial looks under the hood of modern day CV systems, and builds out some of these tech pipelines in a Jupyter Notebook using Python, OpenCV, Keras and Tensorflow. While the primary focus is on digital images from cameras and videos, this tutorial will also introduce 3D point clouds, and classification and segmentation algorithms for processing them.
Comprehensive Criteria-Based Generalized Steganalysis Feature Selection Method

Redundant steganalysis feature components in high-dimensional steganalysis feature of images increase the spatio-temporal complexity of steganalysis and even reduce the detection accuracy of the stego images.In order to reduce the image steganalysis feature dimension, improve the detection accuracy of the stego images and achieve fast feature selection, this paper proposes a general method for image steganalysis feature selection.Firstly, a feature metric algorithm based on the difference function is given, and this algorithm measures the difference of the steganalysis feature components between the cover image class and the stego image class, which provides the basis for selecting the steganalysis feature components contributing greatly to detect the stego images.Secondly, the Pearson correlation coefficient is improved and used to measure the correlation between the steganalysis feature components and the image classification results to provide the basis for removing the redundant steganalysis feature components.And then, by setting thresholds, the steganalysis feature components with a large difference function value are selected and with a small Pearson correlation coefficient are deleted.Finally, the steganalysis feature components retained are trained and detected as the final steganalysis feature.A series of experimental results indicate that this method can reduce the feature dimension effectively and the spatio-temporal complexity of steganalysis, while maintaining or even improving the detection accuracy of the stego images.Compared to existing steganalysis feature selection methods such as Fisher-GFR and Improved-Fisher, this method has a higher detection accuracy of the stego images after simplification.
Generic Outlier Detection in Multi-Armed Bandit

In this paper, we study the problem of outlier arm detection in multi-armed bandit settings, which finds plenty of applications in many high-impact domains such as finance, healthcare, and online advertising. For this problem, a learner aims to identify the arms whose expected rewards deviate significantly from most of the other arms. Different from existing work, we target the generic outlier arms or outlier arm groups whose expected rewards can be larger, smaller, or even in between those of normal arms. To this end, we start by providing a comprehensive definition of such generic outlier arms and outlier arm groups. Then we propose a novel pulling algorithm named GOLD to identify such generic outlier arms. It builds a real-time neighborhood graph based on upper confidence bounds and catches the behavior pattern of outliers from normal arms. We also analyze its performance from various aspects. In the experiments conducted on both synthetic and real-world data sets, the proposed algorithm achieves 98% accuracy while saving 83% exploration cost on average compared with state-of-the-art techniques.
A direct approach for function approximation on data defined manifolds

In much of the literature on function approximation by deep networks, the function is assumed to be defined on some known domain, such as a cube or a sphere. In practice, the data might not be dense on these domains, and therefore, the approximation theory results are observed to be too conservative. In manifold learning, one assumes instead that the data is sampled from an unknown manifold; i.e., the manifold is defined by the data itself. Function approximation on this unknown manifold is then a two stage procedure: first, one approximates the Laplace–Beltrami operator (and its eigen-decomposition) on this manifold using a graph Laplacian, and next, approximates the target function using the eigen-functions. Alternatively, one estimates first some atlas on the manifold and then uses local approximation techniques based on the local coordinate charts. In this paper, we propose a more direct approach to function approximation on unknown, data defined manifolds without computing the eigen-decomposition of some operator or an atlas for the manifold, and without any kind of training in the classical sense. Our constructions are universal; i.e., do not require the knowledge of any prior on the target function other than continuity on the manifold. We estimate the degree of approximation. For smooth functions, the estimates do not suffer from the so-called saturation phenomenon. We demonstrate via a property called good propagation of errors how the results can be lifted for function approximation using deep networks where each channel evaluates a Gaussian network on a possibly unknown manifold.
Hybrid sparsity learning for image restoration: An iterative and trainable approach

State-of-the-art approaches toward image restoration can be classified into model-based and learning-based. The former class strives to exploit internal prior knowledge about the unknown target images by constructing advanced mathematical models; while the latter class leverages external image prior from a training dataset through powerful neural networks. It is natural to explore their middle ground and pursue a principled approach to learning the hybrid image prior in order to combine the strengths from both worlds. In this paper, we present a systematic approach to achieving this goal called Structured Analysis Sparsity Learning (SASL). Inspired by the strategy of iterative regularization, we propose to learn a hybrid sparse prior from both a collection of reference images (external prior) and the given degraded image (internal prior). Unlike previous hybrid approaches that simply take the average of restored images by different priors, we advocate our approach of combining complementary structured sparse priors in an iterative and trainable manner. By incorporating the knowledge from both domains (internal vs. external), we demonstrate that our iterative and trainable image restoration with a hybrid prior can boost the performance of common tasks including denoising and deblurring. Experimental results show that the proposed SASL image restoration techniques perform comparably with and often better than current state-of-the-art techniques.