Reviewerly/exHarmony huggingface dataset:
Performance Evaluation of Training Algorithms in Backpropagation Neural Network Approach to Blast-Induced Ground Vibration Prediction

Abstract&#x0D; Backpropagation Neural Network (BPNN) is an artificial intelligence technique that has seen several applications in many fields of science and engineering. It is well-known that, the critical task in developing an effective and accurate BPNN model depends on an appropriate training algorithm, transfer function, number of hidden layers and number of hidden neurons. Despite the numerous contributing factors for the development of a BPNN model, training algorithm is key in achieving optimum BPNN model performance. This study is focused on evaluating and comparing the performance of 13 training algorithms in BPNN for the prediction of blast-induced ground vibration. The training algorithms considered include: Levenberg-Marquardt, Bayesian Regularisation, Broyden–Fletcher–Goldfarb–Shanno (BFGS) Quasi-Newton, Resilient Backpropagation, Scaled Conjugate Gradient, Conjugate Gradient with Powell/Beale Restarts, Fletcher-Powell Conjugate Gradient, Polak-Ribiére Conjugate Gradient, One Step Secant, Gradient Descent with Adaptive Learning Rate, Gradient Descent with Momentum, Gradient Descent, and Gradient Descent with Momentum and Adaptive Learning Rate. Using ranking values for the performance indicators of Mean Squared Error (MSE), correlation coefficient (R), number of training epoch (iteration) and the duration for convergence, the performance of the various training algorithms used to build the BPNN models were evaluated. The obtained overall ranking results showed that the BFGS Quasi-Newton algorithm outperformed the other training algorithms even though the Levenberg Marquardt algorithm was found to have the best computational speed and utilised the smallest number of epochs.&#x0D; &#x0D; Keywords: Artificial Intelligence, Blast-induced Ground Vibration, Backpropagation Training Algorithms
Sharper Bounds for Uniformly Stable Algorithms

Deriving generalization bounds for stable algorithms is a classical question in learning theory taking its roots in the early works of Vapnik and Chervonenkis (1974) and Rogers and Wagner (1978). In a series of recent breakthrough papers by Feldman and Vondrak (2018, 2019), it was shown that the best\nknown high probability upper bounds for uniformly stable learning algorithms due to Bousquet and Elisseef (2002) are sub-optimal in some natural regimes. To do so, they proved two generalization bounds that significantly outperform the simple generalization bound of Bousquet and Elisseef (2002). Feldman and Vondrak also asked if it is possible to provide sharper bounds and prove corresponding high probability lower bounds. This paper is devoted to these questions: firstly,\ninspired by the original arguments of Feldman and Vondrak, we provide a short proof of the moment bound that implies the generalization bound stronger than both recent results Feldman and Vondrak (2018, 2019). Secondly, we prove general lower bounds, showing that our moment bound is sharp (up to a logarithmic factor) unless some additional properties of the corresponding random variables are used. Our main probabilistic result is a general concentration inequality for weakly\ncorrelated random variables, which may be of independent interest.
Sparse graph based self-supervised hashing for scalable image retrieval

In recent years, learning-based image hashing techniques have elicited wide interest among researchers because they can be applied in high-dimensional data such as videos and images. Supervised hashing techniques can achieve a satisfactory retrieval performance, but are excessively reliant on label information, resulting in the appearance of unsupervised hashing methods. However, existing unsupervised hashing methods often fail to obtain effective similarity information from the training set; this causes a large degradation in their retrieval performance. Some pseudo label-based methods partially alleviate this problem, but are sensitive to the pre-defined number of categories. Therefore, in this paper, we mainly discuss a solution to the above-mentioned problems regarding image hashing. We propose a sparse graph based self-supervised hashing method in which a sparse graph is constructed, which not only circumvents the requirement of a predefined number of categories as compared with pseudo label-based methods, but also significantly reduces the memory demand compared with a dense graph-based method. In addition, we also exploit a self-supervised reconstruction constraint to further preserve the semantic information. These items are combined in a linear manner and optimized using an iterative strategy. Four representative datasets, including two single-label and two multi-label datasets, are employed to evaluate our method. The results, based on multiple metrics, show that our method can outperform other state-of-the-art methods.
Enhancing the Role of Large-Scale Recommendation Systems in the IoT Context

The Internet of Things (IoT) connects heterogeneous physical devices with the ability to collect data using sensors and actuators. These data can infer useful information for decision-makers in many applications systems monitoring, healthcare, transportations, data storage, smart homes, and many others. In the Era of the Internet of Things (IoT), recommender systems can support scenarios such as recommending apps, IoT workflows, services, sensor equipment, hotels, and drugs to users and customers. Current state-of-art recommendation systems, including collaborative filtering methods, suffer from scalability and sparsity problems. This article proposes a clustering-based recommendation system that adopts the vector space model from information retrieval to obtain highly accurate recommendations. The proposed algorithm uses four well-known clustering techniques as k-means (KM), fuzzy c-mean (FCM), Single-Linkage (SLINK), and Self-Organizing-Maps (SOM). Various experiments and benchmarking on seven IoT rating datasets from different fields are conducted to assess the performance of the proposed recommender system. The experimental results using both error and prediction metrics indicate that the proposed algorithm outperforms the traditional collaborative filtering approach. Besides, adopting the self-organizing strategy obtains recommendations of significant accuracy as compared to the partitional learning approaches.
Dynamic network embedding via incremental skip-gram with negative sampling

Network representation learning, as an approach to learn low dimensional representations of vertices, has attracted considerable research attention recently. It has been proven extremely useful in many machine learning tasks over large graph. Most existing methods focus on learning the structural representations of vertices in a static network, but cannot guarantee an accurate and efficient embedding in a dynamic network scenario. The fundamental problem of continuously capturing the dynamic properties in an efficient way for a dynamic network remains unsolved. To address this issue, we present an efficient incremental skip-gram algorithm with negative sampling for dynamic network embedding, and provide a set of theoretical analyses to characterize the performance guarantee. Specifically, we first partition a dynamic network into the updated, including addition/deletion of links and vertices, and the retained networks over time. Then we factorize the objective function of network embedding into the added, vanished and retained parts of the network. Next we provide a new stochastic gradient-based method, guided by the partitions of the network, to update the nodes and the parameter vectors. The proposed algorithm is proven to yield an objective function value with a bounded difference to that of the original objective function. The first order moment of the objective difference converges in order of $$\mathbb{O}(\frac{1}{n^{2}})$$ , and the second order moment of the objective difference can be stabilized in order of $$\mathbb{O}(1)$$ . Experimental results show that our proposal can significantly reduce the training time while preserving the comparable performance. We also demonstrate the correctness of the theoretical analysis and the practical usefulness of the dynamic network embedding. We perform extensive experiments on multiple real-world large network datasets over multi-label classification and link prediction tasks to evaluate the effectiveness and efficiency of the proposed framework, and up to 22 times speedup has been achieved.
Federated Simulation for Medical Imaging

Labelling data is expensive and time consuming especially for domains such as medical imaging that contain volumetric imaging data and require expert knowledge. Exploiting a larger pool of labeled data available across multiple centers, such as in federated learning, has also seen limited success since current deep learning approaches do not generalize well to images acquired with scanners from different manufacturers. We aim to address these problems in a common, learning-based image simulation framework which we refer to as Federated Simulation. We introduce a physics-driven generative approach that consists of two learnable neural modules: 1) a module that synthesizes 3D cardiac shapes along with their materials, and 2) a CT simulator that renders these into realistic 3D CT Volumes, with annotations. Since the model of geometry and material is disentangled from the imaging sensor, it can effectively be trained across multiple medical centers. We show that our data synthesis framework improves the downstream segmentation performance on several datasets.
Shopper intent prediction from clickstream e-commerce data with minimal browsing information

Abstract We address the problem of user intent prediction from clickstream data of an e-commerce website via two conceptually different approaches: a hand-crafted feature-based classification and a deep learning-based classification. In both approaches, we deliberately coarse-grain a new clickstream proprietary dataset to produce symbolic trajectories with minimal information. Then, we tackle the problem of trajectory classification of arbitrary length and ultimately, early prediction of limited-length trajectories, both for balanced and unbalanced datasets. Our analysis shows that k -gram statistics with visibility graph motifs produce fast and accurate classifications, highlighting that purchase prediction is reliable even for extremely short observation windows. In the deep learning case, we benchmarked previous state-of-the-art (SOTA) models on the new dataset, and improved classification accuracy over SOTA performances with our proposed LSTM architecture. We conclude with an in-depth error analysis and a careful evaluation of the pros and cons of the two approaches when applied to realistic industry use cases.
Distilling Dense Representations for Ranking using Tightly-Coupled Teachers

We present an approach to ranking with dense representations that applies knowledge distillation to improve the recently proposed late-interaction ColBERT model. Specifically, we distill the knowledge from ColBERT's expressive MaxSim operator for computing relevance scores into a simple dot product, thus enabling single-step ANN search. Our key insight is that during distillation, tight coupling between the teacher model and the student model enables more flexible distillation strategies and yields better learned representations. We empirically show that our approach improves query latency and greatly reduces the onerous storage requirements of ColBERT, while only making modest sacrifices in terms of effectiveness. By combining our dense representations with sparse representations derived from document expansion, we are able to approach the effectiveness of a standard cross-encoder reranker using BERT that is orders of magnitude slower.
Genuine Einstein-Podolsky-Rosen steering of three-qubit states by multiple sequential observers

We investigate the possibility of multiple uses of a single copy of a three-qubit state for detecting genuine tripartite Einstein-Podolsky-Rosen (EPR) steering. A pure three-qubit state of either the Greenberger-Horne-Zeilinger (GHZ) type or $W$ type is shared between two fixed observers in two wings and a sequence of multiple observers in the third wing who perform unsharp or nonprojective measurements. The measurement settings of each of the multiple observers in the third wing are independent and uncorrelated with the measurement settings and outcomes of the previous observers. In such setup, we investigate all possible types of ($2\ensuremath{\rightarrow}1$) and ($1\ensuremath{\rightarrow}2$) genuine tripartite EPR steering. For each case, we obtain an upper limit on the number of observers on the third wing who can demonstrate genuine EPR steering through the quantum violation of an appropriate tripartite steering inequality. We show that the GHZ state allows for a higher number of observers compared to the $W$ state. Additionally, ($1\ensuremath{\rightarrow}2$) genuine steering is possible for a larger range of sharpness parameters compared to ($2\ensuremath{\rightarrow}1$) genuine steering cases.
Meta-Learning for Short Utterance Speaker Recognition with Imbalance Length Pairs

In practical settings, a speaker recognition system needs to identify a speaker given a short utterance, while the enrollment utterance may be relatively long. However, existing speaker recognition models perform poorly with such short utterances. To solve this problem, we introduce a meta-learning framework for imbalance length pairs. Specifically, we use a Prototypical Networks and train it with a support set of long utterances and a query set of short utterances of varying lengths. Further, since optimizing only for the classes in the given episode may be insufficient for learning discriminative embeddings for unseen classes, we additionally enforce the model to classify both the support and the query set against the entire set of classes in the training set. By combining these two learning schemes, our model outperforms existing state-of-the-art speaker verification models learned with a standard supervised learning framework on short utterance (1-2 seconds) on the VoxCeleb datasets. We also validate our proposed model for unseen speaker identification, on which it also achieves significant performance gains over the existing approaches. The codes are available at this https URL.
Understanding the Talent Management Intricacies of Remote Cybersecurity Teams in Covid-19 Induced Telework Organizational Ecosystems

Abstract The unanticipated disruption triggered by Coronavirus, also known as COVID-19, has accelerated the shift to virtual workplace ecosystems for employees in the government, business, and the military. The accelerated move to whole organization telework has also created new complex challenges around managing cybersecurity risks. Complex business and governmental organizational ecosystems have several significant and critical job tasks around cybersecurity. These roles have the involved responsibility of functioning as effective teams to handle incident responses, intrusion evaluations, crisis management, crisis communications, forensic data examinations, malware analyzations, firewall assessments, penetration testing, disaster recovery, emergency response planning, risk management, human factor analyzation, training assessment, and active network monitoring. This qualitative research study explores the nuances of employee engagement, organizational development, and the management of virtual and remote cybersecurity teams in ways that enhance complex business and organizational ecosystems in the world of professional practice.
On the near-optimality of one-shot classical communication over quantum channels

We study the problem of transmission of classical messages through a quantum channel in several network scenarios in the one-shot setting. We consider both the entanglement assisted and unassisted cases for the point to point quantum channel, the quantum multiple-access channel, the quantum channel with a state, and the quantum broadcast channel. We show that it is possible to near-optimally characterize the amount of communication that can be transmitted in these scenarios, using the position-based decoding strategy introduced in a prior study (A. Anshu, R. Jain, and N. Warsi, https://ieee.org/document/8399830). In the process, we provide a short and elementary proof of the converse for entanglement-assisted quantum channel coding in terms of the quantum hypothesis testing divergence [obtained earlier in the work of W. Matthews and S. Wehner, IEEE Trans. Inf. Theory 60, 7317–7329 (2014)]. Our proof has the additional utility that it naturally extends to various network scenarios mentioned above. Furthermore, none of our achievability results require a simultaneous decoding strategy, existence of which is an important open question in quantum Shannon theory.
META: Metadata-Empowered Weak Supervision for Text Classification

Recent advances in weakly supervised learning enable training high-quality text classifiers by only providing a few user-provided seed words.Existing methods mainly use text data alone to generate pseudo-labels despite the fact that metadata information (e.g., author and timestamp) is widely available across various domains.Strong label indicators exist in the metadata and it has been long overlooked mainly due to the following challenges: (1) metadata is multi-typed, requiring systematic modeling of different types and their combinations, (2) metadata is noisy, some metadata entities (e.g., authors, venues) are more compelling label indicators than others.In this paper, we propose a novel framework, META, which goes beyond the existing paradigm and leverages metadata as an additional source of weak supervision.Specifically, we organize the text data and metadata together into a text-rich network and adopt network motifs to capture appropriate combinations of metadata.Based on seed words, we rank and filter motif instances to distill highly label-indicative ones as "seed motifs", which provide additional weak supervision.Following a bootstrapping manner, we train the classifier and expand the seed words and seed motifs iteratively.Extensive experiments and case studies on real-world datasets demonstrate superior performance and significant advantages of leveraging metadata as weak supervision.
Kibble-Zurek scaling in quantum speed limits for shortcuts to adiabaticity

Geometric quantum speed limits quantify the tradeoff between the rate at which quantum states can change and the resources that are expended during the evolution. Counterdiabatic driving is a unique tool from shortcuts to adiabaticity to speed up quantum dynamics while completely suppressing nonequilibrium excitations. We show that the quantum speed limit for counterdiabatically driven systems undergoing quantum phase transitions fully encodes the Kibble-Zurek mechanism by correctly predicting the transition from adiabatic to impulse regimes. Our findings are demonstrated for three scenarios, namely the transverse field Ising model, the Landau-Zener model, and the Lipkin-Meshkov-Glick model.Received 15 June 2020Accepted 30 June 2020DOI:https://doi.org/10.1103/PhysRevResearch.2.032020Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasNonequilibrium statistical mechanicsQuantum controlQuantum phase transitionsQuantum quenchCondensed Matter, Materials & Applied PhysicsGeneral PhysicsStatistical PhysicsQuantum Information
Self-explaining AI as an Alternative to Interpretable AI

The ability to explain decisions made by AI systems is highly sought after, especially in domains where human lives are at stake such as medicine or autonomous vehicles. While it is often possible to approximate the input-output relations of deep neural networks with a few human-understandable rules, the discovery of the double descent phenomena suggests that such approximations do not accurately capture the mechanism by which deep neural networks work. Double descent indicates that deep neural networks typically operate by smoothly interpolating between data points rather than by extracting a few high level rules. As a result, neural networks trained on complex real world data are inherently hard to interpret and prone to failure if asked to extrapolate. To show how we might be able to trust AI despite these problems we introduce the concept of self-explaining AI. Self-explaining AIs are capable of providing a human-understandable explanation of each decision along with confidence levels for both the decision and explanation. Some difficulties with this approach along with possible solutions are sketched. Finally, we argue it is important that deep learning based systems include a "warning light" based on techniques from applicability domain analysis to warn the user if a model is asked to extrapolate outside its training distribution.
BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation

Incorporating prior knowledge like lexical constraints into the model's output to generate meaningful and coherent sentences has many applications in dialogue system, machine translation, image captioning, etc. However, existing RNN-based models incrementally generate sentences from left to right via beam search, which makes it difficult to directly introduce lexical constraints into the generated sentences. In this paper, we propose a new algorithmic framework, dubbed BFGAN, to address this challenge. Specifically, we employ a backward generator and a forward generator to generate lexically constrained sentences together, and use a discriminator to guide the joint training of two generators by assigning them reward signals. Due to the difficulty of BFGAN training, we propose several training techniques to make the training process more stable and efficient. Our extensive experiments on two large-scale datasets with human evaluation demonstrate that BFGAN has significant improvements over previous methods.
Tutorial: Applying Machine Learning in Behavioral Research

Machine-learning algorithms hold promise for revolutionizing how educators and clinicians make decisions. However, researchers in behavior analysis have been slow to adopt this methodology to further develop their understanding of human behavior and improve the application of the science to problems of applied significance. One potential explanation for the scarcity of research is that machine learning is not typically taught as part of training programs in behavior analysis. This tutorial aims to address this barrier by promoting increased research using machine learning in behavior analysis. We present how to apply the random forest, support vector machine, stochastic gradient descent, and k-nearest neighbors algorithms on a small dataset to better identify parents of children with autism who would benefit from a behavior analytic interactive web training. These step-by-step applications should allow researchers to implement machine-learning algorithms with novel research questions and datasets.