Reviewerly/exHarmony huggingface dataset:
On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics

Stochastic gradient Langevin dynamics (SGLD) is a fundamental algorithm in stochastic optimization. Recent work by Zhang et al. [2017] presents an analysis for the hitting time of SGLD for the first and second order stationary points. The proof in Zhang et al. [2017] is a two-stage procedure through bounding the Cheeger's constant, which is rather complicated and leads to loose bounds. In this paper, using intuitions from stochastic differential equations, we provide a direct analysis for the hitting times of SGLD to the first and second order stationary points. Our analysis is straightforward. It only relies on basic linear algebra and probability theory tools. Our direct analysis also leads to tighter bounds comparing to Zhang et al. [2017] and shows the explicit dependence of the hitting time on different factors, including dimensionality, smoothness, noise strength, and step size effects. Under suitable conditions, we show that the hitting time of SGLD to first-order stationary points can be dimension-independent. Moreover, we apply our analysis to study several important online estimation problems in machine learning, including linear regression, matrix factorization, and online PCA.
Group visualization of class-discriminative features

Research explaining the behavior of convolutional neural networks (CNNs) has gained a lot of attention over the past few years. Although many visualization methods have been proposed to explain network predictions, most fail to provide clear correlations between the target output and the features extracted by convolutional layers. In this work, we define a concept, i.e., class-discriminative feature groups, to specify features that are extracted by groups of convolutional kernels correlated with a particular image class. We propose a detection method to detect class-discriminative feature groups and a visualization method to highlight image regions correlated with particular output and to interpret class-discriminative feature groups intuitively. The experiments showed that the proposed method can disentangle features based on image classes and shed light on what feature groups are extracted from which regions of the image. We also applied this method to visualize “lost” features in adversarial samples and features in an image containing a non-class object to demonstrate its ability to debug why the network failed or succeeded.
The First Vietnamese FOSD-Tacotron-2-based Text-to-Speech Model Dataset

Recent trends in voicebot application development have enabled utilization of both speech-to-text and text-to-speech (TTS) generation techniques. In order to generate a voice response to a given speech, one needs to use a TTS engine. The recently developed TTS engines are shifting towards end-to-end approaches utilizing models such as Tacotron, Tacotron-2, WaveNet, and WaveGlow. The reason is that it enables a TTS service provider to focus on developing training and validating datasets comprising of labelled texts and recorded speeches instead of designing an entirely new model that outperforms the others which is time-consuming and costly. In this context, this work introduces the first Vietnamese FPT Open Speech Data (FOSD)-Tacotron-2-based TTS model dataset. This dataset comprises of a configuration file in *.json format; training and validating text input files (in *.csv format); a 225,000-step checkpoint of the trained model; and several sample generated audios. The published dataset is extremely worth for serving as a model for benchmarking with other newly developed TTS models / engines. In addition, it opens an entirely new TTS research optimization problem to be addressed: How to effectively generate speech from text given: a black box TTS (trained) model and its training and validation input texts.
A novel noise-tolerant Zhang neural network for time-varying Lyapunov equation

Abstract The Zhang neural network (ZNN) has become a benchmark solver for various time-varying problems solving. In this paper, leveraging a novel design formula, a noise-tolerant continuous-time ZNN (NTCTZNN) model is deliberately developed and analyzed for a time-varying Lyapunov equation, which inherits the exponential convergence rate of the classical CTZNN in a noiseless environment. Theoretical results show that for a time-varying Lyapunov equation with constant noise or time-varying linear noise, the proposed NTCTZNN model is convergent, no matter how large the noise is. For a time-varying Lyapunov equation with quadratic noise, the proposed NTCTZNN model converges to a constant which is reciprocal to the design parameter. These results indicate that the proposed NTCTZNN model has a stronger anti-noise capability than the traditional CTZNN. Beyond that, for potential digital hardware realization, the discrete-time version of the NTCTZNN model (NTDTZNN) is proposed on the basis of the Euler forward difference. Lastly, the efficacy and accuracy of the proposed NTCTZNN and NTDTZNN models are illustrated by some numerical examples.
Attentive Pooling with Learnable Norms for Text Representation

Pooling is an important technique for learning text representations in many neural NLP models. In conventional pooling methods such as average, max and attentive pooling, text representations are weighted summations of the L1 or L∞ norm of input features. However, their pooling norms are always fixed and may not be optimal for learning accurate text representations in different tasks. In addition, in many popular pooling methods such as max and attentive pooling some features may be over-emphasized, while other useful ones are not fully exploited. In this paper, we propose an Attentive Pooling with Learnable Norms (APLN) approach for text representation. Different from existing pooling methods that use a fixed pooling norm, we propose to learn the norm in an end-to-end manner to automatically find the optimal ones for text representation in different tasks. In addition, we propose two methods to ensure the numerical stability of the model training. The first one is scale limiting, which re-scales the input to ensure non-negativity and alleviate the risk of exponential explosion. The second one is re-formulation, which decomposes the exponent operation to avoid computing the real-valued powers of the input and further accelerate the pooling operation. Experimental results on four benchmark datasets show that our approach can effectively improve the performance of attentive pooling.
Born-again Tree Ensembles

The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.
Elimination of All Bad Local Minima in Deep Learning

In this paper, we theoretically prove that adding one special neuron per output unit eliminates all suboptimal local minima of any deep neural network, for multi-class classification, binary classification, and regression with an arbitrary loss function, under practical assumptions. At every local minimum of any deep neural network with these added neurons, the set of parameters of the original neural network (without added neurons) is guaranteed to be a global minimum of the original neural network. The effects of the added neurons are proven to automatically vanish at every local minimum. Moreover, we provide a novel theoretical characterization of a failure mode of eliminating suboptimal local minima via an additional theorem and several examples. This paper also introduces a novel proof technique based on the perturbable gradient basis (PGB) necessary condition of local minima, which provides new insight into the elimination of local minima and is applicable to analyze various models and transformations of objective functions beyond the elimination of local minima.
From pedestrian to group retrieval via siamese network and correlation

In many public security applications such as anomaly detection, it is important to re-identify a group of pedestrians by other surveillance cameras, which ascribes to the group retrieval problem. Most previous studies focus on single-person re-identification (re-id) and ignore the correlations among group members, and they lack a large and comprehensive group retrieval benchmark to associate these two tasks. To address this issue, this paper focuses on solving the group retrieval problem and uses it to improve re-id. First, the paper build a comprehensive benchmark for both group retrieval and the group-aided re-id task by proposing a novel pedestrian group retrieval dataset named “SYSU-Group” and a corresponding group-associated re-id dataset named “Group-reID”, which introduces realistic challenges such as variations of pose, viewpoint, illumination, and intra-group layout. The paper then proposes the Siamese Verification-Identification-based Group Retrieval (SVIGR) method, which combines verification and identification modules in a Siamese network to extract robust person features and follows the principle of minimum distance matching to realize group retrieval. Finally, a group-guided re-id method named group retrieval correlation (GRC) is proposed to improve re-id with additional group information. Experimental results on three various group retrieval benchmarks demonstrate the superiority and effectiveness of our method.
An Explainable Machine Learning Approach for Definition Extraction

With the outburst of available data, Definition Extraction has emerged as an important technique as it is a precursor to many other tasks like ontology generation, glossary creation, and question answering. Definition Extraction is most commonly treated as a binary classification problem of definitional and non-definitional sentences. Traditional techniques for definition extraction involve rule-based approaches, which did not yield good results because of the overwhelming complexity of natural language. Incorporating linguistic information via syntactic dependencies turned out to be useful in identifying sentences containing a definition. In this paper, we explore the performance of Transformer based architectures, like Bidirectional Encoder Representations from Transformers (BERT), which produce state-of-the-art results on many Natural Language Processing (NLP) tasks. Experiments on an annotated dataset of definitional sentences prove that BERT obtains results comparable to the state-of-the-art benchmark. In further experiments, we look under the hood of BERT, trying to figure out the reason for its success. Analyzing the outputs of the attention heads reveals that BERT captures not only syntactic dependencies but many other relevant dependencies within the words of the sentence, which proves beneficial in Definition Extraction.
Multi-scale and multi-branch feature representation for person re-identification

Multi-scale feature fusion has been proven effective in substantial person re-identification (ReID) works. However, the existing multi-scale feature fusion is based on features of different semantic levels. We propose a novel multi-scale and multi-branch feature representation for person ReID, namely Ms-Mb. It merges the features of the same semantic level and integrates attention modules to learn robust and representative feature representations. Through the heterogeneous losses supervision, the final feature representation of the image is more discriminative for person ReID. Sufficient ablation study has proven that the multi-scale feature fusion, the attention module and heterogeneous losses training strategy contribute to the performance boost of Ms-Mb. We have conducted experiments on four mainstream benchmarks including Market1501, DukeMTMC-reID, CUHK03 and MSMT17. Extensive experimental results show that our approach Ms-Mb achieves state-of-the-art performances on Market1501 (Rank-1 = 95.8%, mAP = 89.9%), DukeMTMC-reID (Rank-1 = 90.8%, mAP = 82.2%) and MSMT17 (Rank-1 = 81.9%, mAP = 59.3%) without using additional external data or re-ranking. Our approach yields competitive results compared to the state-of-the-art method on CUHK03 (Rank-1 = 75.4%, mAP = 72.9%) and surpasses the other methods by a large margin.
Virtual Reality for Immersive Human Machine Teaming with Vehicles

We present developments in constructing a 3D environment and integrating a virtual reality headset in our Project Aquaticus platform. We designed Project Aquaticus to examine the interactions between human-robot teammate trust, cognitive load, and perceived robot intelligence levels while they compete in games of capture the flag on the water. Further, this platform will allows us to study human learning of tactical judgment under a variety of robot capabilities. To enable human-machine teaming (HMT), we created a testbed where humans operate motorized kayaks while the robots are autonomous catamaran-style surface vehicles. MOOS-IvP provides autonomy for the robots. After receiving an order from a human, the autonomous teammates can perform tasks conducive to capturing the flag, such as defending or attacking a flag. In the Project Aquaticus simulation, the humans control their virtual vehicle with a joystick and communicate with their robots via radio. Our current simulation is not engaging or realistic for participants because it presents a top-down, omniscient view of the field. This fully observable representation of the world is well suited for managing operations from the shore and teaching new players game mechanics and strategies; however, it does not accurately reflect the limited and almost chaotic view of the world a participant experiences while in their motorized kayak on the water. We present creating a 3D visualization through Unity that users experience through a virtual reality headset. Such a system allows us to perform experiments without the need for a significant investment in on-water experiment resources while also permitting us to gather data year-round through the cold winter months.
Internet of Things device authentication via electromagnetic fingerprints

Abstract We quickly approach a future where Internet of Things (IoT) devices are the norm. In this scenario, humans are surrounded by a multitude of heterogeneous devices that assist them in almost every aspect of their daily routines. The realization of this future demands strong authentication guarantees to ensure that these devices are not abused and that their users are not endangered. However, providing authentication for these systems is challenging due to the high heterogeneity of IoT applications. In this paper, we first review several IoT application scenarios and promising authentication methods for each. We identify th e key characteristics of each IoT application scenario, present the strengths and weaknesses of prominent authentication methods from the literature, and review which authentication methods have been proposed in the literature for each application. Then, we present a novel authentication method for IoT based on electromagnetic noise. The key advantage of electromagnetic noise is that any electronic device intrinsically generates electromagnetic noise during normal operation. We extract features from these electromagnetic emanations and use machine learning algorithms to identify devices based on these features. Our method achieves 77% accuracy when identifying devices among a set of seven devices.
The Quantum Cost Optimized Design of 2:4 Decoder Using the New Reversible Logic Block

Aim and Objective: This paper presents the quantum cost, garbage output, constant input and number of reversible gate optimized 2:4 decoder using 4×4 new reversible logic gate which is named as reversible decoder block or RD block. Method: The proposed block is implemented with a quantum circuit and quantum cost of the proposed RD block is 8. The proposed decoder can be designed using only one new proposed block. Results and Conclusion: The quantum cost, garbage output, constant input and gate number of the proposed 2:4 decoder is 9, 0, 2 and 1 which is better w.r.t previously reported work. The improvement % of quantum cost, garbage output, constant input and number of gates are 12.5 – 77.148 %, 100 %, 33.33 – 75 % and 0 – 85.71%.
Simple Baseline for Vehicle Pose Estimation: Experimental Validation

Significant progress on human and vehicle pose estimation has been achieved in recent years. The performance of these methods has evolved from poor to remarkable in just a couple of years. This improvement has been obtained from increasingly complex architectures. In this paper, we explore the applicability of simple baseline methods by adding a few deconvolutional layers on a backbone network to estimate heat maps that correspond to the vehicle keypoints. This approach has been proven to be very effective for human pose estimation. The results are analyzed on the PASCAL3D+ dataset, achieving state-of-the-art results. In addition, a set of experiments has been conducted to study current shortcomings in vehicle keypoints labelling, which adversely affect performance. A new strategy for defining vehicle keypoints is presented and validated with our customized dataset with extended keypoints.
Locally Private Hypothesis Selection

We initiate the study of hypothesis selection under local differential privacy.\n Given samples from an unknown probability distribution $p$ and a set of $k$ probability distributions $\mathcal{Q}$, we aim to output, under the constraints of $\varepsilon$-differential privacy, a distribution from $\mathcal{Q}$ whose total variation distance to $p$ is comparable to the best such distribution.\n This is a generalization of the classic problem of $k$-wise simple hypothesis testing, which corresponds to when $p \in \mathcal{Q}$, and we wish to identify $p$.\n Absent privacy constraints, this problem requires $O(\log k)$ samples from $p$, and it was recently shown that the same complexity is achievable under (central) differential privacy.\n However, the naive approach to this problem under local differential privacy would require $\tilde O(k^2)$ samples.\n\n We first show that the constraint of local differential privacy incurs an exponential increase in cost: any algorithm for this problem requires at least $\Omega(k)$ samples.\n Second, for the special case of $k$-wise simple hypothesis testing, we provide a non-interactive algorithm which nearly matches this bound, requiring $\tilde O(k)$ samples.\n Finally, we provide sequentially interactive algorithms for the general case, requiring $\tilde O(k)$ samples and only $O(\log \log k)$ rounds of interactivity.\n Our algorithms are achieved through a reduction to maximum selection with adversarial comparators, a problem of independent interest for which we initiate study in the parallel setting.\n For this problem, we provide a family of algorithms for each number of allowed rounds of interaction $t$, as well as lower bounds showing that they are near-optimal for every $t$.\n Notably, our algorithms result in exponential improvements on the round complexity of previous methods.
Comparative Performance Analysis of Neural Network Base Training Algorithm and Neuro-Fuzzy System with SOM for the Purpose of Prediction of the Features of Superconductors

In this paper, the Neural Network based training algorithm has been discussed briefly. Correlation of coefficient (Training, Validation, Testing) using the Levenberg-Marquardt algorithm for superconductivity has been observed graphically. The same operation has been performed applying Bayesian Regularization algorithm and Scaled Conjugate Gradient algorithm. Mean Square Error and Regression has been calculated according to training, validation and testing using Bayesian Regularization, Scaled Conjugate Gradient and Levenberg-Marquardt algorithm for superconductor dataset. The target variable is the critical temperature of the superconductor. The regression value of Scaled Conjugate Gradient, Bayesian Regularization, Levenberg-Marquardt algorithm for superconductor dataset is 0.809214,0,0.854644, respectively which concludes that the Levenberg-Marquardt algorithm provides comparatively larger regression (R) value among them in validation state. Error histogram with 20 bins has been explained visually with simulation Bayesian Regularization, Levenberg-Marquardt, and Scaled Conjugate Gradient algorithm. Neuro-Fuzzy system structure and Self-Organizing Maps (SOM) has also been implemented in this paper which provides the supremacy of the proposed work. The main benefit of SOM is that it is a useful multivariate visualization technique that permits the multidimensional data to be exposed as a 2-dimensional map.
Software Defect Prediction Using Software Metrics with Naïve Bayes and Rule Mining Association Methods

Software defect prediction (SDP) can help testers decide allocation of resources rationally to find defects effectively, so as to improve software quality. Naive Bayes (NB) is one of the most used classification algorithms because of the simplicity of the algorithm and easy to implement. The purpose of this study is to add the process of selecting features using ARM in the software prediction process using the NB method in the hope that it can improve the performance of the method using software metrics. Software metrics have an association with one another in completing software, so this cannot be ignored. Results of the empirical evaluation of scenario 1 (one) showed an increase with the values of parameter precision, recall, f-measure and accuracy of 0.101, 0.190, 0.154 and 0.180, and scenario 2 (second) also increased by 0.106, 0.182, 0.159 and 0.163, also as in scenario 3 (third) the proposed method shows good performance compared to using SVM, NN and DTREE with an average performance of 0.960 while the others are 0.855, 0.859 and 0.861. From the empirical results of the three scenarios made, the proposed performance method is better than the other methods.
A Summary of the Development of Cyber Security Threat Intelligence Sharing

In recent years, the sharing of cybersecurity threat intelligence (hereinafter referred to as threat intelligence) has received increasing attention from national network security management organizations and network security enterprises. Academia and industry have conducted research on threat intelligence analysis and sharing. This paper first introduces the value and significance of threat intelligence. Then it introduces the commonly used threat intelligence analysis model. Then it organizes and classifies the threat intelligence sharing norms and threat intelligence vendors. Then it starts from the main problems faced by threat intelligence sharing. A solution to build regional network security capabilities is presented; finally, the future research direction of threat intelligence sharing is explored.