Reviewerly/exHarmony huggingface dataset:
A Study of Generalization of Stochastic Mirror Descent Algorithms on Overparameterized Nonlinear Models

We study the convergence, the implicit regularization and the generalization of stochastic mirror descent (SMD) algorithms in overparameterized nonlinear models, where the number of model parameters exceeds the number of training data points. Due to overpa-rameterization, the training loss has infinitely many global minima where they define a manifold of interpolating solutions. To have an understanding of the generalization performance of SMD algorithms, it is important to characterize which global minima the SMD algorithms converge to. In this work, we first theoretically show that in the overparameterized nonlinear setting, if the initialization is close enough to the manifold of global minima, which is usually the case in the high overparameterization setting, using sufficiently small step size, SMD converges to a global minimum. We further prove that this global minimum is approximately the closest one to the initialization in Bregman divergence, demonstrating the approximate implicit regularization of SMD. We then empirically confirm that these theoretical results are observed in practice. Finally, we provide an extensive study of the generalization of SMD algorithms. In our experiments, we show that on the CIFAR-10 dataset, SMD with ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">10</sub> norm potential (as a surrogate for ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> ) consistently general-izes better than SGD (corresponding to an ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm potential), which in turn consistently outperforms SMD with ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm potential.
Answering Event-Related Questions over Long-Term News Article Archives

Long-term news article archives are valuable resources about our past, allowing people to know detailed information of events that occurred at specific time points. To make better use of such heritage collections, this work considers the task of large scale question answering on long-term news article archives. Questions on such archives are often event-related. In addition, they usually exhibit strong temporal aspects and can be roughly categorized into two types: (1) ones containing explicit temporal expressions, and (2) ones only implicitly associated with particular time periods. We focus on the latter type as such questions are more difficult to be answered, and we propose a retriever-reader model with an additional module for reranking articles by exploiting temporal information from different angles. Experimental results on carefully constructed test set show that our model outperforms the existing question answering systems, thanks to an additional module that finds more relevant documents.
Improving Deep Learning Classification of JPEG2000 Images Over Bandlimited Networks

JPEG2000 (j2k) is a highly popular format for image and video compression. It plays a major role in the rapidly growing applications of cloud based image classification. Considering limited network bandwidth, we propose an end-to-end deep learning framework to achieve faster and more accurate classification by directly training a deep CNN image classifier using the CDF 9/7 Discrete Wavelet Transformed (DWT) coefficients from j2k-compressed images without image reconstruction. We demonstrate additional computation savings by utilizing shallower CNN to achieve classification of good accuracy. Furthermore, we present DWT-centric augmentation transformations to achieve more accurate classification without added cost. Achieving faster and more accurate classification for j2k encoded images, the proposed solution is well suited for joint compression and cloud-based image and video classification over limited channel bandwidth.
Using Randomness to Improve Robustness of Tree-Based Models Against Evasion Attacks

Machine learning models have been widely used in security applications. However, it is well-known that adversaries can adapt their attacks to evade detection. There has been some work on making machine learning models more robust to such attacks. However, one simple but promising approach called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">randomization</i> is under-explored. In addition, most existing works focus on models with differentiable error functions while tree-based models do not have such error functions but are quite popular because they are easy to interpret. This paper proposes a novel randomization-based approach to improve robustness of tree-based models against evasion attacks. The proposed approach incorporates randomization into both model training time and model application time (meaning when the model is used to detect attacks). We also apply this approach to random forest, an existing ML method which already has incorporated randomness at training time but still often fails to generate robust models. We proposed a novel weighted-random-forest method to generate more robust models and a clustering method to add randomness at model application time. We also proposed a theoretical framework to provide a lower bound for adversaries’ effort. Experiments on intrusion detection and spam filtering data show that our approach further improves robustness of random-forest method.
Autonomic Feature Selection using Computational Intelligence

This paper introduces an autonomic method to optimize Feature Selection (FS) in autonomic systems while also presenting a taxonomy of FS techniques. Feature selection is a dimension reduction technique that has been proven to lead to improved performance by avoiding overfitting and to address complexity, thus providing faster and cost-effective algorithms. To be successful, the current FS methods are heavily reliant on two key elements: (1) a well defined and static learning objective, and (2) a relevant dataset. Current FS approaches involve mostly a manual process and do not take into account the constant change in the state of the underlying system. However, the method to automate the FS process presented in this paper, the Autonomic Feature Selection (AFS), uses representation and transfer learning concepts borrowed from the deep learning area. Representations of various types of monitoring data from a system are learned, and the resulting knowledge is transferred and re-used. The AFS is targeted to self-aware software systems, where the learning objective and (or) the availability of the relevant data are subject to change. The results suggest that although building and transferring learned representations is computationally expensive, its benefits outweigh the cost, by offering the potential to build flexible and re-usable knowledge structures.
CAT: Corner Aided Tracking With Deep Regression Network

Single object tracking in visual media is an important yet challenging task. Various challenges, especially target scale variation, shape deformation and occlusion, can have large effects on the performances of trackers. Current deep regression based trackers only pay close attention to regression on the center key point of the tracking target, meanwhile employ the image pyramid based multi-scale testing method to deal with scale estimation. Such procedure can not properly handle the three challenges. We address these challenges in a principled way by the aid of auxiliary regressions on the four bounding box corners of the tracking target. In this work, we propose the novel Corner Aided Tracker with deep regression network, abbreviated as CAT. Different from RPN-based trackers, in CAT, four corners along with the center key point of the bounding box for tracking target are simultaneously obtained by five corresponding response maps. Furthermore, to robustly and accurately generate tight bounding boxes for the tracking target and collect reliable samples for online training of the network, we propose an adaptive key point selection method to select the subset of reliable key points and drop the unreliable ones, based on the qualities of their corresponding response maps as well as the constraints from shape, scale and location. We demonstrate that the regressed corners can help naturally locate the tracking target with tight bounding boxes. The challenges of scale variation, shape deformation and occlusion can be handled explicitly. The commonly used time-consuming image pyramid based multi-scale testing method can also be discarded. Extensive experiments on OTB2013, OTB2015, UAV123, LaSOT, VOT2016 and VOT2018 datasets are conducted to report new state-of-the-art performances and demonstrate the effectiveness of CAT.
An Optimized Approach for Breast Cancer Classification for Histopathological Images Based on Hybrid Feature Set

Breast cancer is considered as one of the most perilous sickness among females worldwide and the ratio of new cases is increasing yearly. Many researchers have proposed efficient algorithms to diagnose breast cancer at early stages, which have increased the efficiency and performance by utilizing the learned features of gold standard histopathological images.Most of these systems have either used traditional handcrafted or deep features, which had a lot of noise and redundancy, and ultimately decrease the performance of the system.A hybrid approach is proposed by fusing and optimizing the properties of handcrafted and deep features to classify the breast cancer images. HOG and LBP features are serially fused with pre-trained models VGG19 and InceptionV3. PCR and ICR are used to evaluate the classification performance of the proposed method.The method concentrates on histopathological images to classify the breast cancer. The performance is compared with the state-of-the-art techniques, where an overall patient-level accuracy of 97.2% and image-level accuracy of 96.7% is recorded.The proposed hybrid method achieves the best performance as compared to previous methods and it can be used for the intelligent healthcare systems and early breast cancer detection.
Uncovering the cyber security challenges in healthcare

The now infamous WannaCry hack of May 2017 not only sent shockwaves through the healthcare industry in the UK, but also caused cyber security concerns to understandably balloon across all sectors. It made people sit up and question the extent to which large, household name organisations like the National Health Service were acting to protect the critical data held about individuals under their care.
Special Aspects of Matrix Operation Implementations for Low-Precision Neural Network Model on the Elbrus Platform

This paper investigates the possibility of effective implementation of calculations in lowprecision neural network models on the Elbrus platform with the VLIW architecture.Such models are widely used in practice to increase the computational efficiency of recognition and well suit computers with the x86 and ARM architectures.In this paper, we consider an 8-bit neural network model, in which matrix multiplication is the most resource-intensive part of the implementation.This paper presents an effective implementation of matrix multiplication that takes into account the features of the Elbrus architecture: the presence of several computational channels with various arithmetic and logic devices, an array prefetch buffer, and its own SIMD extension.We carry out theoretical and experimental comparisons of the computational efficiency of low-precision and classical neural network models, which show that Elbrus processors have much more capabilities for performing fast floating point calculations and require the development of new approaches to increase the computational efficiency of neural network models.
Planning Under Uncertainty Applications in Power Plants Using Factored Markov Decision Processes

Due to its ability to deal with non-determinism and partial observability, represent goals as an immediate reward function and find optimal solutions, planning under uncertainty using factored Markov Decision Processes (FMDPs) has increased its importance and usage in power plants and power systems. In this paper, three different applications using this approach are described: (i) optimal dam management in hydroelectric power plants, (ii) inspection and surveillance in electric substations, and (iii) optimization of steam generation in a combined cycle power plant. For each case, the technique has demonstrated to find optimal action policies in uncertain settings, present good response and compilation times, deal with stochastic variables and be a good alternative to traditional control systems. The main contributions of this work are as follows, a methodology to approximate a decision model using machine learning techniques, and examples of how to specify and solve problems in the electric power domain in terms of a FMDP.
Deep Learning-Based Hypothesis Generation Model and Its Application on Virtual Chinese Calligraphy-Writing Robot

In recent years, a tremendous amount of effort has been devoted to modeling the cognition of human brain, particularly hypothesis generation process.Most research of the hypothesis generation model is probability-based.However, computation of human brains is still neuron-based instead of calculating the probability.As an attempt to solve this problem in this paper, we propose a novel neuron-based hypothesis generation model, called hypothesis generation net, to model human cognition, including how to make decisions and how to do actions.Basically, the proposed hypothesis generation model consists of two parts, i.e., a hypothesis model and an evaluation model.When these two models interact, the system is able to generate hypotheses to solve complex tasks based on historical experiences.To validate the feasibility of the proposed hypothesis generation model, we show a virtual robot with its cognition system can learn how to write Chinese calligraphy in a simulation environment, where an image-to-action translation via a cognitive framework is proposed to learn the pattern of Chinese characters.Based on the proposed deep thinking and learning mechanism, the virtual robot is able to write Chinese calligraphy well, which is a difficult task requiring extremely complicated motions, through thinking and practicing according to a human writing sample.
Single-view Object Shape Reconstruction Using Deep Shape Prior and Silhouette.

3D shape reconstruction from a single image is a highly ill-posed problem. Modern deep learning based systems try to solve this problem by learning an end-to-end mapping from image to shape via a deep network. In this paper, we aim to solve this problem via an online optimization framework inspired by traditional methods. Our framework employs a deep autoencoder to learn a set of latent codes of 3D object shapes, which are fitted by a probabilistic shape prior using Gaussian Mixture Model (GMM). At inference, the shape and pose are jointly optimized guided by both image cues and deep shape prior without relying on an initialization from any trained deep nets. Surprisingly, our method achieves comparable performance to state-of-the-art methods even without training an end-to-end network, which shows a promising step in this direction.
Design and implementation of deep learning strategy based smart signature verification System

The signature verification is broadly used for personal identification. The person is identified automatically using signature verification method to avoid forgery persons. The signature verification is classified into the static method and the dynamic method. The static verification method is based on stored images and the dynamic verification method is based on dynamic features of the signature. The integer wavelet transformation method is used to identify the breath and height ratio of the signature features. In addition to that spurious noise also removed before extracting the signature feature. And the signature is isolated from the background of the images. The extracted feature is analyzed using integer wavelet transformation and a neural network is selected to decide according to that original and forgery signature. As compared with the conventional system the proposed found to be about 20% error ratio. The database SVC2004 is selected to verify the signature.
Survey on Blockchain Consensus Strategies

Blockchain is a distributed data store that offers a trusted infrastructure for mutually distrusting parties to network and transact, eliminating the need of any central authority or third party intermediary. The trust element in blockchain is due to the tamper proof record keeping in blockchain through efficient hashed block-linking mechanism and the consensus protocol among the network peers to create any new record in the distributed ledger. The consensus mechanism in distributed data store ensures that every participant node in the network accepts on a common agreement over a transaction or state transition in a system by gathering confirmation from all or at-least majority of the members. Usually some form of voting or lottery mechanism is employed for the leader election process to implement consensus among the participants in a business network. Consensus mechanism is also responsible for the smooth functioning of the distributed network in the event of some arbitrary faults or node failures or even any malicious attacks in the network. Generally a consensus mechanism implementation is a tailored solution based on application or use case. It ensures the correctness of data in the trusted distributed ledger through the peer agreement process. In this work, we explore the evolution of state of the art consensus protocols for various blockchain implementations, their application, advantages and drawbacks as well.
Role of Artificial Intelligence (AI) in Understanding the Behavior Pattern: A Study on E-Commerce

In the good old days when traditional buying medium was the only option available to the consumers, there was hardly any role of intermediaries involved in the buying and selling processes that could influence the consumers. There has been a significant change in the present day scenario. Along with the enormous advancement of technology, in the era of Industrial Revolution 4.0, consumers’ buying decisions and behavioural patterns are effectively influenced by various other means. For instance before buying anything these days, people would visit the product websites, compare the products, engage in chat forums to have a detailed analysis of product specifications, have expert opinions on the same while some might even attend related webinars. Precisely, for any retailer, the situation has become altogether much more complex with the cumulative research done by the buyers beforehand, hence establishing the fact that only with the human intelligence and the salesperson involved, retailers would not be able to get their deals cracked and succeed in the highly competitive environment. Customer satisfaction is one of the pillars of strength for any successful retailer, hence the infiltration of AI it becomes easier in understanding and predicting consumer behaviour patterns and trends in E-commerce as with the use on Analytics only, it failed to demonstrate the behavioural patterns precisely due to the humungous quantity and variance of data paving the path for AI to enter the scenario as coupling with Analytics it can unleash its values to simplify. Purpose of the Research is to study the role of AI in understanding the behaviour pattern of E-commerce Customers. To understand how Artificial Intelligence can add value to increase the sales of an E-commerce Company—Creating more personalized marketing across multiple devices is by using artificial intelligence to eliminate redundancies and automate processes and enable trade in conversations.
Cross-Graph Convolution Learning for Large-Scale Text-Picture Shopping Guide in E-Commerce Search

In this work, a new e-commerce search service named text-picture shopping guide (TPSG) is investigated and deployed to one of the most popular shopping platforms called Taobao. Different from traditional services that only contain text options, the TPSG provides pairs of text terms and user-friendly pictures for shopping guide, named text-picture options (TPOs). Instead of manually labeling pictures, we aim to automatically recommend personalized pictures in TPOs. To this end, we build a large-scale graph model on a great amount of data about users, pictures, and terms. Accordingly, a cross-graph convolution learning (CGCL) method is proposed to facilitate the accurate and efficient inference on the constructed graph. To separate the cue of personalized preferences of users to commodities, we factorize the entire mixture-relation graph involving attributes/relations of users and commodities into the user graph, the commodity graph, and the cross user-commodity graph which just characterizes the preferences. Further, we introduce powerful graph convolution to learn more effective representation of these graphs. To reduce the computation burden, specifically, we generalize graph convolution and propose a tensor graph convolution method to learn representation on cross graphs. We conduct extensive offline and online experiments on the large-scale datasets. The results show that the proposed CGCL is very effective and the TPOs recommendation method outperforms manual/advanced selection methods.
Discriminating Cognitive Disequilibrium and Flow in Problem Solving: A Semi-Supervised Approach Using Involuntary Dynamic Behavioral Signals

Problem solving is one of the most important 21st century skills. However, effectively coaching young students in problem solving is challenging because teachers must continuously monitor their cognitive and affective states, and make real-time pedagogical interventions to maximize their learning outcomes. It is an even more challenging task in social environments with limited human coaching resources. To lessen the cognitive load on a teacher and enable affect-sensitive intelligent tutoring, many researchers have investigated automated cognitive and affective detection methods. However, most of the studies use culturally-sensitive indices of affect that are prone to social editing such as facial expressions, and only few studies have explored involuntary dynamic behavioral signals such as gross body movements. In addition, most current methods rely on expensive labelled data from trained annotators for supervised learning. In this paper, we explore a semi-supervised learning framework that can learn low-dimensional representations of involuntary dynamic behavioral signals (mainly gross-body movements) from a modest number of short time series segments. Experiments on a real-world dataset reveal a significant advantage of these representations in discriminating cognitive disequilibrium and flow, as compared to traditional complexity measures from dynamical systems literature, and demonstrate their potential in transferring learned models to previously unseen subjects.