Reviewerly/exHarmony huggingface dataset:
Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate

Stochastic Gradient Descent (SGD) is a central tool in machine learning. We prove that SGD converges to zero loss, even with a fixed (non-vanishing) learning rate --- in the special case of homogeneous linear classifiers with smooth monotone loss functions, optimized on linearly separable data. Previous works assumed either a vanishing learning rate, iterate averaging, or loss assumptions that do not hold for monotone loss functions used for classification, such as the logistic loss. We prove our result on a fixed dataset, both for sampling with or without replacement. Furthermore, for logistic loss (and similar exponentially-tailed losses), we prove that with SGD the weight vector converges in direction to the $L_2$ max margin vector as $O(1/\log(t))$ for almost all separable datasets, and the loss converges as $O(1/t)$ --- similarly to gradient descent. Lastly, we examine the case of a fixed learning rate proportional to the minibatch size. We prove that in this case, the asymptotic convergence rate of SGD (with replacement) does not depend on the minibatch size in terms of epochs, if the support vectors span the data. These results may suggest an explanation to similar behaviors observed in deep networks, when trained with SGD.
Preface to the Focus Section on Machine Learning in Seismology

Research Article| February 13, 2019 Preface to the Focus Section on Machine Learning in Seismology Karianne J. Bergen; Karianne J. Bergen aDepartment of Earth and Planetary Sciences, Harvard University, Cambridge, Massachusetts 02138 U.S.A., karianne_bergen@fas.harvard.edu Search for other works by this author on: GSW Google Scholar Ting Chen; Ting Chen bLos Alamos National Laboratory, Los Alamos, New Mexico 87545 U.S.A. Search for other works by this author on: GSW Google Scholar Zefeng Li Zefeng Li cSeismological Laboratory, California Institute of Technology, Pasadena, California 91125 U.S.A. Search for other works by this author on: GSW Google Scholar Author and Article Information Karianne J. Bergen aDepartment of Earth and Planetary Sciences, Harvard University, Cambridge, Massachusetts 02138 U.S.A., karianne_bergen@fas.harvard.edu Ting Chen bLos Alamos National Laboratory, Los Alamos, New Mexico 87545 U.S.A. Zefeng Li cSeismological Laboratory, California Institute of Technology, Pasadena, California 91125 U.S.A. Publisher: Seismological Society of America First Online: 13 Feb 2019 Online Issn: 1938-2057 Print Issn: 0895-0695 © Seismological Society of America Seismological Research Letters (2019) 90 (2A): 477–480. https://doi.org/10.1785/0220190018 Article history First Online: 13 Feb 2019 Cite View This Citation Add to Citation Manager Share Icon Share Facebook Twitter LinkedIn MailTo Tools Icon Tools Get Permissions Search Site Citation Karianne J. Bergen, Ting Chen, Zefeng Li; Preface to the Focus Section on Machine Learning in Seismology. Seismological Research Letters 2019;; 90 (2A): 477–480. doi: https://doi.org/10.1785/0220190018 Download citation file: Ris (Zotero) Refmanager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentBy SocietySeismological Research Letters Search Advanced Search Machine learning (ML) is a collection of algorithms and statistical models that enable computers to extract relevant patterns and information from large data sets. Unlike physical modeling approaches, in which scientists develop theories based on physical laws to compare with real‐world observations, ML approaches learn directly from data without explicitly reasoning about the underlying physical mechanisms. ML algorithms are often categorized into supervised and unsupervised learning (see fig. 2 in Kong et al., 2018). Supervised learning algorithms build a model from existing labeled input data with the goal of predicting the labels of new data. A key challenge... You do not have access to this content, please speak to your institutional administrator if you feel you should have access.
Sender Gender Influences Emoji Interpretation in Text Messages

With the rise in social media use, emojis have become a popular addition to text-based commu-nication. The sudden increase in the number and variety of emojis used raises questions about how individuals interpret messages containing emojis. To explore perceptions of emoji usage, we conducted a 2 (Sender Gender: Female or Male) x 2 (Emoji Type: Affectionate or Friendly) be-tween-groups experiment to examine the appropriateness and likability of each of four hypo-thetical text messages sent to a woman from either a male or female coworker. In general, we predicted that text messages containing affectionate emojis (i.e., kissing-face and heart emoji) would be perceived as more appropriate and likable when they came from female than from male senders, whereas messages containing less overtly affectionate (but still friendly) emojis (i.e., smiling-face emoji) would be considered equally appropriate and likable whether it came from female or male senders. As predicted, the results confirmed that texts with affectionate emojis were judged as more appropriate and likable when they came from women than from men. How-ever, texts with less affectionate but friendly emojis were judged as equally appropriate–but more likable–when they came from men than when they came from women. Taken together, our results indicate that gender and emoji choice influence perceptions, and therefore people should consider how emoji choice could impact the reception of their message.
A mathematical model of production facilities location

A mathematical model of the industrial enterprises location in the region is formalized and studied in this paper. It is necessary to locate objects polluting the environment in such way as to maximize the total income of the players. The total income is calculated as income from the activities of enterprises, minus the funds that are spent on the recover damage to the environment. This problem is formalized as a non-cooperative game with n players that exploit common sources – natural objects.
Crow Search Algorithm Based on Neighborhood Search of Non-Inferior Solution Set

Characterized by less parameter settings, easy implementation, and strong optimization capacity, the crow search algorithm has been successfully applied to solve the optimization problem. As the basic crow search algorithm is a new kind of swarm intelligent algorithm only based on the crow's memory foraging mode, it also contains defects like slow search speed and low optimization precision in later iterations, which are especially obvious for the optimization of high-dimensional functions. In order to overcome these shortcomings, a new crow search algorithm based on neighborhood search of non-inferior solution set (NICSA) is proposed. The proposed algorithm makes the crow individual choose the memory search mode or neighborhood search mode automatically in the course of evolution by the determination factor of non-inferior solution. With this strategy, the local exploitation and the global exploration of the algorithm became more balanced. In the neighborhood search, the selectivity factor is used to guide non-inferior solutions to adaptively execute neighborhood search of Levy flight or Gaussian flight, to enhance the neighborhood searchability of the algorithm and improve the optimization precision. The result of simulation experiments with 23 benchmark test functions verifies that the proposed algorithm has good optimization effect in the aspects of search veracity, convergence rate and robustness.
Efficient Softmax Hardware Architecture for Deep Neural Networks

Deep neural network (DNN) has become a pivotal machine learning and object recognition technology in the big data era. The softmax layer is one of the key component layers for completing multi-classification tasks. However, the softmax layer contains complex exponential and division operations, resulting in low accuracy and long critical paths in hardware accelerator design. In order to solve the above issues, we present a softmax hardware architecture with proper accuracy, good trade-off and strong expansibility. We summarize the classification rules of neural network and balance the calculation accuracy between resource consumption. On this basis, we proposed an exponential calculation unit based on the group lookup table, and improve a natural logarithmic calculation unit based on the Maclaurin series and the data preprocessing scheme matching them. The experimental results show that the softmax hardware architecture proposed in this paper can achieve the calculation accuracy of 3 decimal fraction and the classification accuracy of $99.01%$. Theoretically, it can accomplish the classification task of infinite categories.
Impact of Different Sensory Stimuli on Presence in Credible Virtual Environments

Multiple factors can affect presence in virtual environments, such as the number of human senses engaged in a given experience or the extent to which the virtual experience is credible. The purpose of the present work is to study how the inclusion of credible multisensory stimuli affects the sense of presence, namely, through the use of wind, passive haptics, vibration, and scent. Our sample consisted of 37 participants (27 men and 10 women) whose ages ranged from 17 to 44 years old and were mostly students. The participants were divided randomly into 3 groups: Control Scenario (visual and auditory - N = 12), Passive Haptic Scenario (visual, auditory, and passive haptic - N = 13) and Multisensory Scenario (visual, auditory, wind, passive haptic, vibration, and scent - N = 12). The results indicated a significant increase in the involvement subscale when all multisensory stimuli were delivered. We found a trend where the use of passive haptics by itself has a positive impact on presence, which should be the subject of further work.
FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

While dynamic scene reconstruction has made revolutionary progress from the earliest setup using a mass of static cameras in studio environment to the latest egocentric or hand-held moving camera based schemes, it is still restricted by the recording volume, user comfortability, human labor and expertise. In this paper, a novel solution is proposed through a real-time and robust dynamic fusion scheme using a single flying depth camera, denoted as FlyFusion. By proposing a novel topology compactness strategy for effectively regularizing the complex topology changes, and the Geometry And Motion Energy (GAME) metric for guiding the viewpoint optimization in the volumetric space, FlyFusion succeeds to enable intelligent viewpoint selection based on the immediate dynamic reconstruction result. The merit of FlyFusion lies in its concurrent robustness, efficiency, and adaptation in producing fused and denoised 3D geometry and motions of a moving target interacting with different non-rigid objects in a large space.
Activity Recognition for IoT Devices Using Fuzzy Spatio-Temporal Features as Environmental Sensor Fusion

The IoT describes a development field where new approaches and trends are in constant change. In this scenario, new devices and sensors are offering higher precision in everyday life in an increasingly less invasive way. In this work, we propose the use of spatial-temporal features by means of fuzzy logic as a general descriptor for heterogeneous sensors. This fuzzy sensor representation is highly efficient and enables devices with low computing power to develop learning and evaluation tasks in activity recognition using light and efficient classifiers. To show the methodology’s potential in real applications, we deploy an intelligent environment where new UWB location devices, inertial objects, wearable devices, and binary sensors are connected with each other and describe daily human activities. We then apply the proposed fuzzy logic-based methodology to obtain spatial-temporal features to fuse the data from the heterogeneous sensor devices. A case study developed in the UJAmISmart Lab of the University of Jaen (Jaen, Spain) shows the encouraging performance of the methodology when recognizing the activity of an inhabitant using efficient classifiers.
Bridging Machine Learning and Logical Reasoning by Abductive Learning

Perception and reasoning are two representative abilities of intelligence that are integrated seamlessly during human problem-solving processes. In the area of artificial intelligence (AI), the two abilities are usually realised by machine learning and logic programming, respectively. However, the two categories of techniques were developed separately throughout most of the history of AI. In this paper, we present the abductive learning targeted at unifying the two AI paradigms in a mutually beneficial way, where the machine learning model learns to perceive primitive logic facts from data, while logical reasoning can exploit symbolic domain knowledge and correct the wrongly perceived facts for improving the machine learning models. Furthermore, we propose a novel approach to optimise the machine learning model and the logical reasoning model jointly. We demonstrate that by using abductive learning, machines can learn to recognise numbers and resolve unknown mathematical operations simultaneously from images of simple hand-written equations. Moreover, the learned models can be generalised to longer equations and adapted to different tasks, which is beyond the capability of state-of-the-art deep learning models.
Kernel Spectral Embedding Transfer Ensemble for Heterogeneous Defect Prediction

Cross-project defect prediction (CPDP) refers to predicting defects in the target project lacking of defect data by using prediction models trained on the historical defect data of other projects (i.e., source data). However, CPDP requires the source and target projects have common metric set (CPDP-CM). Recently, heterogeneous defect prediction (HDP) has drawn the increasing attention, which predicts defects across projects having heterogeneous metric sets. However, building high-performance HDP methods remains a challenge owing to several serious challenges including class imbalance problem, nonlinear, and the distribution differences between source and target datasets. In this paper, we propose a novel kernel spectral embedding transfer ensemble (KSETE) approach for HDP. KSETE first addresses the class-imbalance problem of the source data and then tries to find the latent common feature space for the source and target datasets by combining kernel spectral embedding, transfer learning, and ensemble learning. Experiments are performed on 22 public projects in both HDP and CPDP-CM scenarios in terms of multiple well-known performance measures such as, AUC, G-Measure, and MCC. The experimental results show that (1) KSETE improves the performance over previous HDP methods by at least 22.7, 138.9, and 494.4 percent in terms of AUC, G-Measure, and MCC, respectively. (2) KSETE improves the performance over previous CPDP-CM methods by at least 4.5, 30.2, and 17.9 percent in AUC, G-Measure, and MCC, respectively. It can be concluded that the proposed KSETE is very effective in both the HDP scenario and the CPDP-CM scenario.
Consortium Blockchain-Based Microgrid Market Transaction Research

The microgrid trading market can effectively solve the problem of in-situ consumption of distributed energy and reduce the impact of distributed generation (DG) on the grid. However, the traditional microgrid trading model has some shortcomings, such as high operation cost and poor security. Therefore, in this paper, a microgrid market trading model was developed using consortium blockchain technology and Nash game equilibrium theory. Firstly, blockchain container is used to authenticate the users who want to participate in the transaction. Then, the pricing system collects and integrates transaction requests submitted by users, then formulates transaction pricing strategy of microgrid using Nash equilibrium theory. Finally, the price, trading volume, and user information are submitted to the blockchain container for transaction matching to achieve the transaction. After the transaction is completed, its related information is recorded in the hyperledger and the dispatch system is called. The scene simulation was implemented on Fabric 1.1 platform and the results analyzed. Results show that the trading model proposed in this paper greatly reduces the cost of electricity purchase and improves the benefits of electricity sales. Besides, the model is far more capable of handling transactions than the models based on Bitcoin and Ethereum.
Which Category Is Better: Benchmarking Relational and Graph Database Management Systems

Abstract Over decades, relational database management systems (RDBMSs) have been the first choice to manage data. Recently, due to the variety properties of big data, graph database management systems (GDBMSs) have emerged as an important complement to RDBMSs. As pointed out in the existing literature, both RDBMSs and GDBMSs are capable of managing graph data and relational data; however, the boundaries of them still remain unclear. For this reason, in this paper, we first extend a unified benchmark for RDBMSs and GDBMSs over the same datasets using the same query workload under the same metrics. We then conduct extensive experiments to evaluate them and make the following findings: (1) RDBMSs outperform GDMBSs by a substantial margin under the workloads which mainly consist of group by, sort, and aggregation operations, and their combinations; (2) GDMBSs show their superiority under the workloads that mainly consist of multi-table join, pattern match, path identification, and their combinations.
Polynomial-Based Google Map Graphical Password System against Shoulder-Surfing Attacks in Cloud Environment

Text password systems are commonly used for identity authentication to access different kinds of data resources or services in cloud environment. However, in the text password systems, the main issue is that it is very hard for users to remember long random alphanumeric strings due to the long-term memory limitation of the human brain. To address this issue, graphical passwords are accordingly proposed based on the fact that humans have better memory for images than alphanumeric strings. Recently, a Google map graphical password (GMGP) system is proposed, in which a specific location of Google Map is preset as a password for authentication. Unfortunately, the use of graphical passwords increases the risk of exposing passwords under shoulder-surfing attacks. A snooper can easily look over someone’s shoulder to get the information of a location on map than a text password from a distance, and thus the shoulder-surfing attacks are more serious for graphical passwords than for text passwords. To overcome this issue, we design a polynomial-based Google map graphical password (P-GMGP) system. The proposed P-GMGP system can not only resist the shoulder-surfing attacks effectively, but also need much fewer challenge-response rounds than the GMGP system for authentication. Moreover, the P-GMGP system is extended to allow a user to be authenticated in cloud environment effectively and efficiently.
Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers

Strong theoretical guarantees of robustness can be given for ensembles of classifiers generated by input randomization. Specifically, an $\ell_2$ bounded adversary cannot alter the ensemble prediction generated by an additive isotropic Gaussian noise, where the radius for the adversary depends on both the variance of the distribution as well as the ensemble margin at the point of interest. We build on and considerably expand this work across broad classes of distributions. In particular, we offer adversarial robustness guarantees and associated algorithms for the discrete case where the adversary is $\ell_0$ bounded. Moreover, we exemplify how the guarantees can be tightened with specific assumptions about the function class of the classifier such as a decision tree. We empirically illustrate these results with and without functional restrictions across image and molecule datasets.
Remove Some Noise: On Pre-processing of Side-channel Measurements with Autoencoders

In the profiled side-channel analysis, deep learning-based techniques proved to be very successful even when attacking targets protected with countermeasures. Still, there is no guarantee that deep learning attacks will always succeed. Various countermeasures make attacks significantly more complex, and such countermeasures can be further combined to make the attacks even more challenging. An intuitive solution to improve the performance of attacks would be to reduce the effect of countermeasures.This paper investigates whether we can consider certain types of hiding countermeasures as noise and then use a deep learning technique called the denoising autoencoder to remove that noise. We conduct a detailed analysis of six different types of noise and countermeasures separately or combined and show that denoising autoencoder improves the attack performance significantly.
Building an Apparatus: Refractive, Reflective, and Diffractive Readings of Trace Data

We propose a set of methodological principles and strategies for the use of trace data, i.e., data capturing performances carried out on or via information systems, often at a fine level of detail. Trace data comes with a number of methodological and theoretical challenges associated with the inseparable nature of the social and material. Drawing on Haraway and Barad’s distinctions among refraction, reflection, and diffraction, we compare three approaches to trace data analysis. We argue that a diffractive methodology allows us to explore how trace data are not given but created through the construction of a research apparatus to study trace data. By focusing on the diffractive ways in which traces ripple through an apparatus, it is possible to explore some of the taken-for-granted, invisible dynamics of sociomateriality. Equally important, this approach allows us to describe what distinctions emerge and when, within entwined phenomena in the research process. Empirically, we illustrate the guiding methodological principles and strategies by analyzing trace data from Gravity Spy, a crowdsourced citizen science project on Zooniverse.org. We conclude by suggesting that a diffractive methodology helps us draw together quantitative and qualitative research practices in new and productive ways that allow us to study and design for the entwined and dynamic sociomaterial practices found in contemporary organizations.
Robust forgery detection for compressed images using CNN supervision

Images available on online sharing platforms have a high probability of being modified, with additional global transformations such as compression, resizing or filtering covering the possible alteration. Such manipulations impose many constraints on forgery detection algorithms. This article presents a framework improving robustness for image forgery detection. The most important step of our framework is to take into account the image quality corresponding to the chosen application. Therefore, we relied on a camera identification model based on convolutional neural networks. Lossy compression such as JPEG being considered as the most common type of intentional or inadvertent concealment of image forgery, that leads us to experiment our proposal on this manipulation. Thus, our trained CNN is fed with a mixture of different qualities of compressed and uncompressed images. Experimental results showed the importance of this step to improve the effectiveness of our approach against recent literature approaches. To better interpret our trained CNN, we proposed an in-depth supervision by first a visualization of the layer and an experimental analysis of the influence of the learned features. This analysis led us to a more robust and accurate framework. Finally, we applied this improved system on an image forgery detection application and showed some promising results.