Reviewerly/exHarmony huggingface dataset:
Multi-level region-based Convolutional Neural Network for image emotion classification

Analyzing emotional information of visual content has attracted growing attention for the tendency of internet users to share their feelings via images and videos online. In this paper, we investigate the problem of affective image analysis, which is very challenging due to its complexity and subjectivity. Previous research reveals that image emotion is related to low-level to high-level visual features from both global and local view, while most of the current approaches only focus on improving emotion recognition performance based on single-level visual features from a global view. Aiming to utilize different levels of visual features from both global and local view, we propose a multi-level region-based Convolutional Neural Network (CNN) framework to discover the sentimental response of local regions. We first employ Feature Pyramid Network (FPN) to extract multi-level deep representations. Then, an emotional region proposal method is used to generate proper local regions and remove excessive non-emotional regions for image emotion classification. Finally, to deal with the subjectivity in emotional labels, we propose a multi-task loss function to take the probabilities of images belonging to different emotion classes into consideration. Extensive experiments show that our method outperforms the state-of-the-art approaches on various commonly used benchmark datasets.
A Novel Bat Algorithm with Multiple Strategies Coupling for Numerical Optimization

A bat algorithm (BA) is a heuristic algorithm that operates by imitating the echolocation behavior of bats to perform global optimization. The BA is widely used in various optimization problems because of its excellent performance. In the bat algorithm, the global search capability is determined by the parameter loudness and frequency. However, experiments show that each operator in the algorithm can only improve the performance of the algorithm at a certain time. In this paper, a novel bat algorithm with multiple strategies coupling (mixBA) is proposed to solve this problem. To prove the effectiveness of the algorithm, we compared it with CEC2013 benchmarks test suits. Furthermore, the Wilcoxon and Friedman tests were conducted to distinguish the differences between it and other algorithms. The results prove that the proposed algorithm is significantly superior to others on the majority of benchmark functions.
MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation

We provide a benchmark for digital Media Forensics Challenge (MFC) evaluations. Our comprehensive data comprises over 176,000 high provenance (HP) images and 11,000 HP videos; more than 100,000 manipulated images and 4,000 manipulated videos; 35 million internet images and 300,000 video clips. We have designed and generated a series of development, evaluation, and challenge datasets, and used them to assess the progress and thoroughly analyze the performance of diverse systems on a variety of media forensics tasks in the past two years. In this paper, we first introduce the objectives, challenges, and approaches to building media forensics evaluation datasets. We then discuss our approaches to forensic dataset collection, annotation, and manipulation, and present the design and infrastructure to effectively and efficiently build the evaluation datasets to support various evaluation tasks. Given a specified query, we build an infrastructure that selects the customized evaluation subsets for the targeted analysis report. Finally, we demonstrate the evaluation results in the past evaluations.
Is All Formulaic Language Created Equal? Unpacking the Processing Advantage for Different Types of Formulaic Sequences

Research into recurrent, highly conventionalized “formulaic” sequences has shown a processing advantage compared to “novel” (non-formulaic) language. Studies of individual types of formulaic sequence often acknowledge the contribution of specific factors, but little work exists to compare the processing of different types of phrases with fundamentally different properties. We use eye-tracking to compare the processing of three types of formulaic phrases—idioms, binomials, and collocations—and consider whether overall frequency can explain the advantage for all three, relative to control phrases. Results show an advantage, as evidenced through shorter reading times, for all three types. While overall phrase frequency contributes much of the processing advantage, different types of phrase do show additional effects according to the specific properties that are relevant to each type: frequency, familiarity, and decomposability for idioms; predictability and semantic association for binomials; and mutual information for collocations. We discuss how the results contribute to our understanding of the representation and processing of multiword lexical units more broadly.
Real time violence detection framework for football stadium comprising of big data analysis and deep learning through bidirectional LSTM

Football is the most popular sport in the world with four billion fans all over the world. Reportedly, the violence incidence rates are high during or after the matches. The violent or destructive behavior carried out by a person or player, who watches or plays the game in the stadium is known as football hooliganism. To prevent or control the violence, a real time violence detection system is exclusively needed to monitor the behavior of the crowd and players to take necessary action before the violence is about to happen. Even it is necessary for the system to find whether the attack is non-intentional or intentional in the game. In this paper, a real time violence detection system is proposed which processes the huge input streaming data and recognize the violence with human intelligence simulation. The input to the system is the enormous amount of real time video streams from different sources which is processed in Spark framework. In the Spark framework, the frames are separated and the features of individual frames are extracted by using HOG (Histogram of Oriented Gradients) function. Then the frames are labeled based on features as violence model, human part model and negative model, which are used to train the Bidirectional Long Short-Term Memory (BDLSTM) network for recognition of violence scenes. The bidirectional LSTM can access the information both in forward and reverse direction. Thus the output is generated in context to both past and future information. The network is trained with the violent interaction dataset (VID), containing 2314 videos with 1077 fight ones and 1237 no-fight ones. Moreover to make the model robust to violence detection, we have created a dataset with 410 video clips having non-violence scenes and 409 video clips having violence scenes, acquired from the football stadium. The performance of this model is validated and it proves the sturdiness of the system with an accuracy of 94.5 percentage in recognizing the violent action.
Computing Resource Trading for Edge-Cloud-Assisted Internet of Things

Optimal computing resource allocation for edge-cloud-assisted Internet of things (IoT) in blockchain network is attracting increasing attention. Auction is a classical algorithm which guarantees that the computing resources are allocated to the buyers of the computing resource. However, the traditional auction algorithm only guarantees the revenue gains for the sellers of the computing resource. How to guarantee the seller and the buyer of the computing resource when both are willing to trade and moreover, bid truthfully, is still an open problem in computing resource trading for edge-cloud-assisted IoT. In this paper, we introduce a broker with sparse information to manage and adjust the trading market. We then propose an iterative double-sided auction scheme for computing resource trading, where the broker solves an allocation problem to determine how much computing resource is traded and designs a specific price rule to induce the buyers and sellers of the computing resource to submit bids in a truthful way. Thus, hidden information can be extracted gradually to obtain optimal computing resource allocation and trading prices. Hence, the proposed algorithm can achieve the maximum social welfare meanwhile protecting the privacies of the buyers and the sellers. Our theoretical analysis and simulations demonstrate that the proposed algorithm is efficient, i.e., it achieves the maximum social welfare. In addition, the proposed algorithm can provide effective trading strategies for the buyers and sellers of the computing resource, leading to the proposed algorithm satisfying incentive compatibility, individual rationality, and budget balance.
Bayesian Networks for Risk Prediction Using Real-World Data: A Tool for Precision Medicine

The fields of medicine and public health are undergoing a data revolution. An increasing availability of data has brought about a growing interest in machine-learning algorithms. Our objective is to present the reader with an introduction to a knowledge representation and machine-learning tool for risk estimation in medical science known as Bayesian networks (BNs).In this article we review how BNs are compact and intuitive graphical representations of joint probability distributions (JPDs) that can be used to conduct causal reasoning and risk estimation analysis and offer several advantages over regression-based methods. We discuss how BNs represent a different approach to risk estimation in that they are graphical representations of JPDs that take the form of a network representing model random variables and the influences between them, respectively.We explore some of the challenges associated with traditional risk prediction methods and then describe BNs, their construction, application, and advantages in risk prediction based on examples in cancer and heart disease.Risk modeling with BNs has advantages over regression-based approaches, and in this article we focus on three that are relevant to health outcomes research: (1) the generation of network structures in which relationships between variables can be easily communicated; (2) their ability to apply Bayes's theorem to conduct individual-level risk estimation; and (3) their easy transformation into decision models.Bayesian networks represent a powerful and flexible tool for the analysis of health economics and outcomes research data in the era of precision medicine.
Integrating Neural Networks with a Quantum Simulator for State Reconstruction

We demonstrate quantum many-body state reconstruction from experimental data generated by a programmable quantum simulator, by means of a neural network model incorporating known experimental errors. Specifically, we extract restricted Boltzmann machine (RBM) wavefunctions from data produced by a Rydberg quantum simulator with eight and nine atoms in a single measurement basis, and apply a novel regularization technique to mitigate the effects of measurement errors in the training data. Reconstructions of modest complexity are able to capture one- and two-body observables not accessible to experimentalists, as well as more sophisticated observables such as the R\'enyi mutual information. Our results open the door to integration of machine learning architectures with intermediate-scale quantum hardware.
Privacy-preserving cloud computing on sensitive data: A survey of methods, products and challenges

The increasing volume of personal and sensitive data being harvested by data controllers makes it increasingly necessary to use the cloud not just to store the data, but also to process them on cloud premises. However, security concerns on frequent data breaches, together with recently upgraded legal data protection requirements (like the European Union’s General Data Protection Regulation), advise against outsourcing unprotected sensitive data to public clouds. To tackle this issue, this survey covers technologies that allow privacy-aware outsourcing of storage and processing of sensitive data to public clouds. Specifically and as a novelty, we review masking methods for outsourced data based on data splitting and anonymization, in addition to cryptographic methods covered in other surveys. We then compare these methods in terms of operations supported on the masked outsourced data, overhead, accuracy preservation, and impact on data management. Furthermore, we list several research projects and available products that have materialized some of the surveyed solutions. Finally, we identify outstanding research challenges.
Divide and Conquer the Embedding Space for Metric Learning

Learning the embedding space, where semantically similar objects are located close together and dissimilar objects far apart, is a cornerstone of many computer vision applications. Existing approaches usually learn a single metric in the embedding space for all available data points, which may have a very complex non-uniform distribution with different notions of similarity between objects, e.g. appearance, shape, color or semantic meaning. Approaches for learning a single distance metric often struggle to encode all different types of relationships and do not generalize well. In this work, we propose a novel easy-to-implement divide and conquer approach for deep metric learning, which significantly improves the state-of-the-art performance of metric learning. Our approach utilizes the embedding space more efficiently by jointly splitting the embedding space and data into K smaller sub-problems. It divides both, the data and the embedding space into K subsets and learns K separate distance metrics in the non-overlapping subspaces of the embedding space, defined by groups of neurons in the embedding layer of the neural network. The proposed approach increases the convergence speed and improves generalization since the complexity of each sub-problem is reduced compared to the original one. We show that our approach outperforms the state-of-the-art by a large margin in retrieval, clustering and re-identification tasks on CUB200-2011, CARS196, Stanford Online Products, In-shop Clothes and PKU VehicleID datasets. Source code: https://bit.ly/dcesml.
Recurrent Neural Networks With Intra-Frame Iterations for Video Deblurring

Recurrent neural networks (RNNs) are widely used for sequential data processing. Recent state-of-the-art video deblurring methods bank on convolutional recurrent neural network architectures to exploit the temporal relationship between neighboring frames. In this work, we aim to improve the accuracy of recurrent models by adapting the hidden states transferred from past frames to the frame being processed so that the relations between video frames could be better used. We iteratively update the hidden state via re-using RNN cell parameters before predicting an output deblurred frame. Since we use existing parameters to update the hidden state, our method improves accuracy without additional modules. As the architecture remains the same regardless of iteration number, fewer iteration models can be considered as a partial computational path of the models with more iterations. To take advantage of this property, we employ a stochastic method to optimize our iterative models better. At training time, we randomly choose the iteration number on the fly and apply a regularization loss that favors less computation unless there are considerable reconstruction gains. We show that our method exhibits state-of-the-art video deblurring performance while operating in real-time speed.
Auditing Data Provenance in Text-Generation Models

To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new model auditing technique that helps users check if their data was used to train a machine learning model. We focus on auditing deep-learning models that generate natural-language text, including word prediction and dialog generation. These models are at the core of popular online services and are often trained on personal data such as users' messages, searches, chats, and comments. We design and evaluate a black-box auditing method that can detect, with very few queries to a model, if a particular user's texts were used to train it (among thousands of other users). We empirically show that our method can successfully audit well-generalized models that are not overfitted to the training data. We also analyze how text-generation models memorize word sequences and explain why this memorization makes them amenable to auditing.
Towards Efficient and Privacy-Preserving Federated Deep Learning

Deep learning has been applied in many areas, such as computer vision, natural language processing and emotion analysis. Differing from the traditional deep learning that collects users' data centrally, federated deep learning requires participants to train the networks on private datasets and share the training results, and hence has more gratifying efficiency and stronger security. However, it still presents some privacy issues since adversaries can deduce users' privacy from local outputs, such as gradients. While the problem of private federated deep learning has been an active research issue, the latest research findings are still inadequate in terms of security, accuracy and efficiency. In this paper, we propose an efficient and privacy-preserving federated deep learning protocol based on stochastic gradient descent method by integrating the additively homomorphic encryption with differential privacy. Specifically, users add noises to each local gradients before encrypting them to obtain the optical performance and security. Moreover, our scheme is secure to honest-but-curious server setting even if the cloud server colludes with multiple users. Besides, our scheme supports federated learning for large-scale users scenarios and extensive experiments demonstrate our scheme has high efficiency and high accuracy compared with non-private model.
Recommendations on designing practical interval type-2 fuzzy systems

Interval type-2 (IT2) fuzzy systems have become increasingly popular in the last 20 years. They have demonstrated superior performance in many applications. However, the operation of an IT2 fuzzy system is more complex than that of its type-1 counterpart. There are many questions to be answered in designing an IT2 fuzzy system: Should singleton or non-singleton fuzzifier be used? How many membership functions (MFs) should be used for each input? Should Gaussian or piecewise linear MFs be used? Should Mamdani or Takagi–Sugeno–Kang (TSK) inference be used? Should minimum or product t-norm be used? Should type-reduction be used or not? How to optimize the IT2 fuzzy system? These questions may look overwhelming and confusing to IT2 beginners. In this paper we recommend some representative starting choices for an IT2 fuzzy system design, which hopefully will make IT2 fuzzy systems more accessible to IT2 fuzzy system designers.
RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators From RL Policies

This letter addresses two challenges facing samplingbased kinodynamic motion planning: a way to identify good candidate states for local transitions and the subsequent computationally intractable steering between these candidate states. Through the combination of sampling-based planning, a Rapidly Exploring Randomized Tree (RRT) and an efficient kinodynamic motion planner through machine learning, we propose an efficient solution to long-range planning for kinodynamic motion planning. First, we use deep reinforcement learning to learn an obstacle-avoiding policy that maps a robot's sensor observations to actions, which is used as a local planner during planning and as a controller during execution. Second, we train a reachability estimator in a supervised manner, which predicts the RL policy's time to reach a state in the presence of obstacles. Lastly, we introduce RL-RRT that uses the RL policy as a local planner, and the reachability estimator as the distance function to bias tree-growth towards promising regions. We evaluate our method on three kinodynamic systems, including physical robot experiments. Results across all three robots tested indicate that RL-RRT outperforms state of the art kinodynamic planners in efficiency, and also provides a shorter path finish time than a steering function free method. The learned local planner policy and accompanying reachability estimator demonstrate transferability to the previously unseen experimental environments, making RL-RRT fast because the expensive computations are replaced with simple neural network inference.
Neural RGB®D Sensing: Depth and Uncertainty From a Video Camera

Depth sensing is crucial for 3D reconstruction and scene understanding. Active depth sensors provide dense metric measurements, but often suffer from limitations such as restricted operating ranges, low spatial resolution, sensor interference, and high power consumption. In this paper, we propose a deep learning (DL) method to estimate per-pixel depth and its uncertainty continuously from a monocular video stream, with the goal of effectively turning an RGB camera into an RGB-D camera. Unlike prior DL-based methods, we estimate a depth probability distribution for each pixel rather than a single depth value, leading to an estimate of a 3D depth probability volume for each input frame. These depth probability volumes are accumulated over time under a Bayesian filtering framework as more incoming frames are processed sequentially, which effectively reduces depth uncertainty and improves accuracy, robustness, and temporal stability. Compared to prior work, the proposed approach achieves more accurate and stable results, and generalizes better to new datasets. Experimental results also show the output of our approach can be directly fed into classical RGB-D based 3D scanning methods for 3D scene reconstruction.
Fuzzy AHP based prioritization and taxonomy of software process improvement success factors in global software development

The phenomena of global software development (GSD) face many challenges that are specifically associated with software process improvement (SPI). The main objective of this work is to develop the prioritization-based taxonomy of the SPI success factors using the fuzzy AHP approach. Total of twenty-one factors were extracted from the available literature that were further evaluated by conducting questionnaire survey with the SPI experts. In the second stage, multi-criterion decision making (MCDM) fuzzy AHP tool was used to prioritize and develop the taxonomy of the identified factors and their categories. The implications of fuzzy AHP approach are novel in this research area as it has been successfully used previously in different other domains e.g. electrical and electronics, supplier selection, agile software development and personnel selection. The contribution of this study is not only limited to the development of the factor’s taxonomy, but also their proper prioritization by introducing the novel fuzzy AHP approach in the research field of GSD and SPI, which assist to remove the vagueness and uncertainty in the opinion of the process improvement experts.