Reviewerly/exHarmony huggingface dataset:
A gray-box modeling methodology for runtime prediction of Apache Spark jobs

Abstract Apache Spark jobs are often characterized by processing huge data sets and, therefore, require runtimes in the range of minutes to hours. Thus, being able to predict the runtime of such jobs would be useful not only to know when the job will finish, but also for scheduling purposes, to estimate monetary costs for cloud deployment, or to determine an appropriate cluster configuration, such as the number of nodes. However, predicting Spark job runtimes is much more challenging than for standard database queries: cluster configuration and parameters have a significant performance impact and jobs usually contain a lot of user-defined code making it difficult to estimate cardinalities and execution costs. In this paper, we present a gray-box modeling methodology for runtime prediction of Apache Spark jobs. Our approach comprises two steps: first, a white-box model for predicting the cardinalities of the input RDDs of each operator is built based on prior knowledge about the behavior and application parameters such as applied filters data, number of iterations, etc. In the second step, a black-box model for each task constructed by monitoring runtime metrics while varying allocated resources and input RDD cardinalities is used. We further show how to use this gray-box approach not only for predicting the runtime of a given job, but also as part of a decision model for reusing intermediate cached results of Spark jobs. Our methodology is validated with experimental evaluation showing a highly accurate prediction of the actual job runtime and a performance improvement if intermediate results can be reused.
Credit Card Fraud Detection using Data Pre-processing on Imbalanced Data - both Oversampling and Undersampling

Online or Offline purchasing using credit or debit card has been popularized rapidly for its ease and flexibility of use. With the growing rate of online purchasing, fraudulent transactions have also increased with an alarming rate. Researchers have investigated the reason and remedies of these kind of acts. However, credit card fraud detection can be categorized into two sections in application fraud and behavioral fraud aspects. While an application is being compromised by the cyber-crimes, then, the user behavior analytic system has come to the limelight in recent days. In this research, a fraud detection system has been built, which can distinguish three types of user from pre-processed transaction logs. To prepare a balanced dataset, undersampling of majority class and synthetic minority over-sampling technique on minority class applied on feature vectors. K-nearest neighbor is used to distinguish the legal, fraudulent and enigmatic users. This system outperforms current state-of-art techniques to get behavioral analytics.
Supervised Sentiment Analysis of Twitter Handle of President Trump with Data Visualization Technique

The approval rating of the President of the United States (POTUS) can be used to gauge public support for the current administration. Public support, in turn, is highly dynamic and erratic, and can be influenced by current events, including political/economic policy announcements, ongoing scandals, treatment in the media, and general propaganda. The current POTUS, Donald Trump, is unique because, for the first time, the office of POTUS has direct access to social media platforms that offer a direct avenue of communication with the general population. Can the social media presence of POTUS influence the public approval of the administration? In this paper, we analyze the relationship between tweets generated by POTUS and his approval rating using sentiment-analytics and data visualization tools. The twitter feed of POTUS is mined, cleaned, and given a quantitative measure based on word content, called the “sentiment score”. By comparing tweets before the election, between election and inauguration, and after the inauguration, we find that the sentiment score for Mr. Trump's feed has increased on average with time by a factor of 60%. Using cross-correlation analysis, we find a preliminary causative relationship between POTUS twitter activity and approval rating. The findings provide a new perspective on the forces that influences public opinion, and the general strategy can be used for other analyses in the future.
Segregating Hazardous Waste Using Deep Neural Networks in Real-Time Video

Sustaining a society requires reusing, reducing, and recycling waste. Waste disposal has always been a problem in developing countries because of inadequate infrastructure. By utilizing artificial intelligence to detect hazardous waste, more individuals will be protected from the negative effects of it. To help mitigate this problem, we experimented with Keras, to create a convolutional neural network, and OpenCV, to create real-time videos, that identifies hazardous waste from other recyclable materials. Through the use of machine learning, our model is able to categorize different recyclable materials with about 90% accuracy. Objects within the video receive a prediction for 3 classifications which includes batteries, syringes, and nonhazardous waste. Then, the category with the highest category is what the network will classify it as. In conclusion, the model is able to identify hazardous objects and recyclable items within a pile of trash to help protect all individuals.
Testing the reliability of inter-rater reliability

Analyses of learning often rely on coded data. One important aspect of coding is establishing reliability. Previous research has shown that the common approach for establishing coding reliability is seriously flawed in that it produces unacceptably high Type I error rates. This paper focuses on testing whether or not these error rates correspond to specific reliability metrics or a larger methodological problem. Our results show that the method for establishing reliability is not metric specific, and we suggest the adoption of new practices to control Type I error rates associated with establishing coding reliability.
2D Geometric Moment Invariants from the Point of View of the Classical Invariant Theory

The aim of this paper is to clear up the question of the connection between the geometric moment invariants and the invariant theory, considering a problem of describing the 2D geometric moment invariants as a problem of the classical invariant theory. We give a precise statement of the problem of computation of the 2D geometric invariant moments, introducing the notions of the algebras of simultaneous 2D geometric moment invariants, and prove that they are isomorphic to the algebras of joint $$\hbox {SO}(2)$$ -invariants of several binary forms. Also, to simplify the calculating of the invariants, we proceed from an action of Lie group $$\hbox {SO}(2)$$ to an action of its Lie algebra $${{\mathfrak {so}}}_2$$ . Though the 2D geometric moments are not as effective as the orthogonal ones are, the author hopes that the results will be useful to the researchers in the fields of image analysis and pattern recognition.
Privacy-Preserving Deep Neural Networks Using Pixel-Based Image Encryption Without Common Security Keys

We present a novel privacy-preserving scheme for deep neural networks (DNNs) that enables us not to only apply images without visual information to DNNs but to also consider the use of independent encryption keys, for both training and testing images for the first time. In this paper, a novel pixel-based image encryption method, which considers maintaining the properties of original images, is first proposed for privacy-preserving DNNs. For training, a DNN model is trained with images encrypted by using the proposed method under the use of independent keys. For testing, the model enables us to applied both encrypted images and plain images for image classification. Therefore, there is no need to manage the keys. In an experiment, the proposed method is applied to a well-known network, deep residual networks, for image classification. The experimental results demonstrate that the proposed method with independent encryption keys has robustness against ciphertext-only attack (COA) and can provide almost the same classification performance as that of using plain images. Moreover, the results confirm that the proposed scheme is able to classify plain images as well as encrypted images.
Cell image analysis for malaria detection using deep convolutional network

Malaria is a protozoan disease that is affecting the 200 million lives of the people around the world and around 4 lakhs death per year due to this which raises our concern and we have tried to target the most affected part in the world i.e. Africa. In the paper approach is to maximize the recent d evelopments in the area of malaria detection using cell images using Convolutional Neural Network (CNN). We have tried to automate the processes which are indulged in the detection of malaria. The method with no pre-processing and no high ended GPU dependency produces an accuracy of 97% proving it to be an efficient as well as low cost detection algorithm. The given implementation can easily detect malaria even from blurred images with no initial pre-processing needed. The algorithm is further compared with standard classification algorithms and stands out be highly efficient in terms of precision, recall, F1 score and computation time.
Evolutionary game for task mapping in resource constrained heterogeneous environments

Power-aware computing is becoming popular using heterogeneous ecosystem. For recent execution units, the heterogeneity is exhibited via the hybrid cores, as well as via virtual and physical asymmetric cores. The inherent performance disparity between different types of execution units at their different clock frequencies offers a great resources scheduling challenge. Multiple metrics (such as throughput, latency, energy cost) are used to decide whether the scheduling is an optimal solution or not. However, in heterogeneous ecosystem, tasks distribution aiming at optimizing costs is not trivial. During the task mapping, one of the primary challenges is to dynamically identify and map the inherent advantages/features of the heterogeneous or hybrid architectures for each individual task. In this work we deal with the task mapping problem using a multi-objective formulation based on evolutionary game theory to optimize a suitable payoff function. This payoff accounts for the power, workload imbalance, task resource affinity and data offloading costs (from host to accelerator). Here, we report that in a very restrictive resource usage scenario (supporting both over and under subscription), the proposed formulation based on Evolutionary Games on Network equation (EGN) can outperform the traditional resource allocation heuristics (such as best-fit, first-fit). Using an extensive set of simulations, we show that our proposed model can outperform first-fit algorithm from more than 5% up to 34.6% and best-fit algorithm from 4% up to 35.7%.
Locality preserving projection based on Euler representation

Locality preserving projection (LPP) is a widely used linear dimensionality reduction method, which preserves the locality structure of the original data. Motivated by the fact that kernel technique can capture nonlinear similarity of features and help to improve separability between nearby data points, this paper proposes locality preserving projection model based on Euler representation (named as ELPP). This model first projects the data into a complex space with Euler representation, then learns the dimensionality reduction projection with preserving locality structure in this complex space. We also extend ELPP to F-ELPP by replacing the squared F-norm with F-norm, which will weaken the exaggerated errors and be more robustness to outliers. The optimization algorithms of the two models are given, and the convergence of F-ELPP is proved. A large number of experiments on several public databases have demonstrated that the two proposed models have good robustness and feature extraction ability.
A Multi Sensor Real-time Tracking with LiDAR and Camera

Self driving cars are equipped with various driverassistive technologies (ADAS) like Forward Collision Warning system (FCW), Adaptive Cruise Control and Collision Mitigation by Breaking (CMbB) to ensure safety.Tracking plays an important role in ADAS systems for understanding dynamic environment.This paper proposes 3D multi-target tracking method by following a lean way of implementation using object detection with aim of real time.Object Tracking is an integral part of environment sensing, which enables the vehicle to estimate the surrounding object's trajectories to accomplish motion planning.The advancement in the object detection methodologies benefits greatly when following the tracking by detection approach.The proposed method implemented 2D tracking on camera data and 3D tracking on LiDAR point cloud data.The estimated state from each sensors are fused together to come with a more optimal state of objects present in the surrounding.The multi object tracking performance has evaluated on publicly available KITTI dataset.
On the Robustness to Adversarial Examples of Neural ODE Image Classifiers

The vulnerability of deep neural networks to adversarial attacks currently represents one of the most challenging open problems in the deep learning field. The NeurIPS 2018 work that obtained the best paper award proposed a new paradigm for defining deep neural networks with continuous internal activations. In this kind of networks, dubbed Neural ODE Networks, a continuous hidden state can be defined via parametric ordinary differential equations, and its dynamics can be adjusted to build representations for a given task, such as image classification. In this paper, we analyze the robustness of image classifiers implemented as ODE Nets to adversarial attacks and compare it to standard deep models. We show that Neural ODE are natively more robust to adversarial attacks with respect to state-of-the-art residual networks, and some of their intrinsic properties, such as adaptive computation cost, open new directions to further increase the robustness of deep-learned models. Moreover, thanks to the continuity of the hidden state, we are able to follow the perturbation injected by manipulated inputs and pinpoint the part of the internal dynamics that is most responsible for the misclassification.
Emerging Issues and Applications of Type-2 Fuzzy Sets and Systems

Due to its complex and unstable nature, stock market prediction is one of the most challenging problems. Lots of research has tried to explore the possibility of predicting stock trends using numerous machine learning methods. However, little effort has been made to predict the future trend online. In this paper, we propose an incremental type-2 fuzzy rule-base classifier capable of considering a high amount of uncertainty in stock data in addition to self-adaptation in an online manner. In this approach, interval type-2 fuzzy sets are built by combining type-1 sets that are directly extracted from the data. The proposed classifier can dynamically modify its parameters in order to react to the alteration of concepts without re-training the model. The efficiency of the proposed method is tested on some market indexes such as SP500, TAIEX, etc. and compared to various incremental methods. Moreover, to show the effectiveness of the proposed technique even in offline mode, we have compared it against some outstanding offline approaches, including deep neural networks. The obtained results demonstrate that our proposed method performs well under different settings.
Monte Carlo tree search with optimal computing budget allocation

We analyze a tree search problem with an underlying Markov decision process, in which the goal is to identify the best action at the root that achieves the highest cumulative reward. We present a new tree policy that optimally allocates a limited computing budget to maximize a lower bound on the probability of correctly selecting the best action at each node. Compared to the widely used Upper Confidence Bound (UCB) type of tree policies, the new tree policy presents a more balanced approach to manage the exploration and exploitation trade-off when the sampling budget is limited. Furthermore, UCB assumes that the support of reward distribution is known, whereas our algorithm relaxes this assumption, and can be applied to game trees with mild modifications. A numerical experiment is conducted to demonstrate the efficiency of our algorithm in selecting the best action at the root.
Research on real-time analysis technology of urban land use based on support vector machine

One of the main problems that traditional support vector machine (SVM) has to solve is how to dynamically determine the kernel parameters and penalty parameters of the kernel function in time, along with the increasing amount of data and the changing data structure and characteristics. A new method is proposed for dynamic acquisition of SVM parameters by fruit fly optimization algorithm (FOA) based on the analysis of the classification and aggregation of land use data in urban industry. FOA-SVM aims at the relationship between feature words in the classification process and the core words of different activity semantics in context. In an incomplete date set of initial feature words, FOA-SVM can extract new feature words from the semantic association of feature words to improve the feature word date set. The dynamic parameters of SVM can be obtained through continuous training with FOA, and the accuracy of classification can be improved. The experimental results showed that FOA-SVM can process multi-feature synchronous classification according to different activity semantics and efficiently control the operation of the whole classification process, so as to obtain higher classification accuracy and stronger robustness in multi-source web date categorization. The efficiency of land use real-time analysis is improved.
Game Theoretic Approach for Real-Time Task Scheduling in Cloud Computing Environment

Cloud computing is one of the emerging technology in the field of distributed computing and is designed as per the requirement and demand of the user. It uses the virtualization technique to create multiple virtual machines that are the basis of computation in the cloud. One of the major issues in cloud computing is to efficiently schedule the tasks and completing their execution before the deadline to maximize the utilization of the processor, maximize the throughput and reduce the waiting time of the task. In this paper, first, we propose a system model and proposed a game-theoretic framework for scheduling realtime tasks in the cloud computing environment to reduce the total completion time and total waiting time. In our game model task act as a player, the virtual machine act as a strategy and the payoff of the player is represented by completion time and waiting time. We have performed our experiments using the non-cooperative and cooperative game model. Our experimental results show that the total execution time and total waiting time are less in the cooperative game model than a non-cooperative game model.
Remote consultation based on mixed reality technology

Telemedicine includes remote teleradiology, remote ultrasound diagnostics, telesurgery, telemedicine consultation, and other forms. Telemedicine consultation is the most used form of telemedicine. However, the traditional telemedicine consultation is limited by a lack of communication and presentation methods, and its wide application is greatly limited. Mixed reality technology cuts through the boundaries between virtual reality and actual reality, bringing a new method of remote consultation.
On Malicious Agents in Non-Bayesian Social Learning with Uncertain Models

Social Learning is the process of cooperatively aggregating information between agents in order to collectively estimate or learn an unknown value. Most all research in social learning assume that the likelihoods of the private observations given the possible hypotheses are known with absolute certainty. However, these likelihoods must be machine learned before the social learning process. Recent work has extended social learning for uncertain likelihoods. Such likelihoods are only known within Dirichlet distributions due to limited training samples available to learn them. This paper investigates the effects of malicious agents when both the good and malicious agents are uncertain about their likelihoods. Such malicious agents are trying to drive the consensus to accept an incorrect hypothesis and reject the correct hypothesis. This paper also presents and evaluates a method to identify and remediate against the effects of the malicious agents.
A novel quantum image parallel searching algorithm

In this paper, a novel quantum image searching algorithm is proposed. The newly proposed searching algorithm makes full use of the features of the quantum image model OQIM to compare and search the images. Since the quantum image model OQIM encodes the image with the sorting orders, the real coordinates and colors can be respectively employed for comparison and searching, i.e., two freedoms for comparison and searching, as will be more flexible and better suited for the more complicated cases such as luminance variation and orthogonal rotation occurred in the images to be searched during the comparison and searching. Compared with the traditional image searching methods in the classical computers, the proposed method can complete the searching task via only one single quantum operation, i.e., the proposed searching algorithm is free of the size of the image and the number of the images to be searched. Compared with the existing quantum image searching methods, the proposed searching algorithm’s advantages are twofold: one is that the searching algorithm is luminance-invariable, and the other one is that the searching algorithm is also free of some geometric transform such as orthogonal rotation and mirror transform and so on. Experiments and theoretical analysis show that the proposed quantum image searching algorithm model has the more powerful searching superiority than the traditional image searching methods in the classical computers and the existing quantum image searching methods.
A Comparison of The Fuzzy Time Series Methods of Chen, Cheng and Markov Chain in Predicting Rainfall in Medan

Abstract Medan has a high rainfall variability. The amount of rainfall affects the welfare of life such as in the fields of health, economy, agriculture, industry, transportation, tourism and so on. To find out changes in rainfall patterns, a prediction of rainfall levels is designed to see and analyze the rainfall patterns that will form in the future. Forecasting is the art and science of predicting future events by taking historical data and projecting it into the future by using some form of mathematical model. One of the methods used to predict an event is the fuzzy time series method. Fuzzy time series is a concept that can be used to predict problems where historical data is formed in linguistic values. While the latest data as a result are in the form of real numbers. The purpose of this research is to implement the fuzzy time series method to predict rainfall in Medan by comparing several developments of the fuzzy time series method, namely Fuzzy Time Series Chen, Markov Chain and Cheng. In determining the interval in the Fuzzy time series Avergae Based rules are used to get the best results. In this study the result is MAPE value of each method. Chen’s method give MAPE=8.002%, Markov chain’s method give MAPE=30.12% and cheng’s method give MAPE=34.5 %. So the best method for forecasting rainfall is Chen Method.