Reviewerly/exHarmony huggingface dataset:
Job scheduling for large-scale machine learning clusters

With the rapid proliferation of Machine Learning (ML) and Deep learning (DL) applications running on modern platforms, it is crucial to satisfy application performance requirements such as meeting deadline and ensuring accuracy. To this end, researchers have proposed several job schedulers for ML clusters. However, none of the previously proposed schedulers consider ML model parallelism, though it has been proposed as an approach to increase the efficiency of running large-scale ML and DL jobs. Thus, in this paper, we propose an ML job Feature based job Scheduling system (MLFS) for ML clusters running both data parallelism and model parallelism ML jobs. MLFS first uses a heuristic scheduling method that considers an ML job's spatial and temporal features to determine task priority for job queue ordering in order to improve job completion time (JCT) and accuracy performance. It uses the data from the heuristic scheduling method for training a deep reinforcement learning (RL) model. After the RL model is well trained, it then switches to the RL method to automatically make decisions on job scheduling. Furthermore, MLFS has a system load control method that selects tasks from overloaded servers to move to underloaded servers based on task priority, and also intelligently removes the tasks that generate little or no improvement on the desired accuracy performance when the system is overloaded to improve JCT and accuracy by job deadline. Real experiments and large-scale simulation based on real trace show that MLFS reduces JCT by up to 53% and makespan by up to 52%, and improves accuracy by up to 64% when compared with existing ML job schedulers. We also open sourced our code.
Hyperparameter-Free Out-of-Distribution Detection Using Cosine Similarity

The ability to detect out-of-distribution (OOD) samples is vital to secure the reliability of deep neural networks in real-world applications. Considering the nature of OOD samples, detection methods should not have hyperparameters that need to be tuned depending on incoming OOD samples. However, most recently proposed methods do not meet this requirement, leading to a compromised performance in real-world applications. In this paper, we propose a simple and computationally efficient, hyperparameter-free method that uses cosine similarity. Although recent studies show its effectiveness for metric learning, it remains uncertain if cosine similarity works well also for OOD detection. There are several differences in the design of output layers from the metric learning methods; they are essential to achieve the best performance. We show through experiments that our method outperforms the existing methods on the evaluation test recently proposed by Shafaei et al. which takes the above issue of hyperparameter dependency into account; it achieves at least comparable performance to the state-of-the-art on the conventional test, where other methods but ours are allowed to use explicit OOD samples for determining hyperparameters. Lastly, we provide a brief discussion of why cosine similarity works so well, referring to an explanation by Hsu et al.
Applicability of MMRR load balancing algorithm in cloud computing

One of cloud computing’s fundamental problems is the balancing of loads, which is essential for evenly distributing the workload across all nodes. This study proposes a new load balancing algorithm, which combines maximum-minimum and round-robin (MMRR) algorithm so that tasks with long execution time are allocated using maximum-minimum and tasks with lowest execution task will be assigned using round-robin. Cloud analyst tool was used to introduce the new load balancing techniques, and a comparative analysis with the existing algorithm was conducted to optimize cloud services to clients. The study findings indicate that ’MMRR has brought significant changes to cloud services. MMRR performed better from the algorithms tested based on the whole response time and cost-effectiveness (89%). The study suggested that MMRR should be implemented for enhancing user satisfaction in the cloud service.
AutoGraph: Automated Graph Neural Network

Graphs play an important role in many applications. Recently, Graph Neural Networks (GNNs) have achieved promising results in graph analysis tasks. Some state-of-the-art GNN models have been proposed, e.g., Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), etc. Despite these successes, most of the GNNs only have shallow structure. This causes the low expressive power of the GNNs. To fully utilize the power of the deep neural network, some deep GNNs have been proposed recently. However, the design of deep GNNs requires significant architecture engineering. In this work, we propose a method to automate the deep GNNs design. In our proposed method, we add a new type of skip connection to the GNNs search space to encourage feature reuse and alleviate the vanishing gradient problem. We also allow our evolutionary algorithm to increase the layers of GNNs during the evolution to generate deeper networks. We evaluate our method in the graph node classification task. The experiments show that the GNNs generated by our method can obtain state-of-the-art results in Cora, Citeseer, Pubmed and PPI datasets.
Hybrid machine learning approach for popularity prediction of newly released contents of online video streaming services

In the industry of video content providers such as VOD and IPTV, predicting the popularity of video contents in advance is critical, not only for marketing but also for network usage. By successfully predicting user preferences, contents can be optimally deployed among servers which ultimately leads to network cost reduction. Many previous studies have predicted view-counts for this purpose. However, they normally make predictions based on historical view-count data from users, given the assumption that contents are already published to users. This can be a downside for newly released contents, which inherently does not have historical data. To address the problem, this research proposes a hybrid machine learning approach for the popularity prediction of unpublished video contents. In this paper, we propose a framework which effectively predicts the popularity of video contents, via a combination of various methods. First, we divide the entire dataset into two types, according to the characteristics of the contents. Next, the popularity prediction is performed by either using XGBoost or neural net with category embedding, which helps resolving the sparsity of categorical variables and requiring the system to learn efficiently for the specified deep neural net model. In addition, we use the FTRL model to alleviate the volatility of view-counts. Experiments are carried out with a dataset from one of the top streaming service companies, and results display overall better performance compared to various standalone methods.
Learning Predictive Models from Observation and Interaction

Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes. However, learning a model that captures the dynamics of complex skills represents a major challenge: if the agent needs a good model to perform these skills, it might never be able to collect the experience on its own that is required to learn these delicate and complex behaviors. Instead, we can imagine augmenting the training set with observational data of other agents, such as humans. Such data is likely more plentiful, but cannot always be combined with data from the original agent. For example, videos of humans might show a robot how to use a tool, but (i) are not annotated with suitable robot actions, and (ii) contain a systematic distributional shift due to the embodiment differences between humans and robots. We address the first challenge by formulating the corresponding graphical model and treating the action as an observed variable for the interaction data and an unobserved variable for the observation data, and the second challenge by using a domain-dependent prior. In addition to interaction data, our method is able to leverage videos of passive observations in a driving dataset and a dataset of robotic manipulation videos to improve video prediction performance. In a real-world tabletop robotic manipulation setting, our method is able to significantly improve control performance by learning a model from both robot data and observations of humans.
Artificial intelligence and multi agent based distributed ledger system for better privacy and security of electronic healthcare records

Application of Artificial Intelligence (AI) and the use of agent-based systems in the healthcare system have attracted various researchers to improve the efficiency and utility in the Electronic Health Records (EHR). Nowadays, one of the most important and creative developments is the integration of AI and Blockchain that is, Distributed Ledger Technology (DLT) to enable better and decentralized governance. Privacy and security is a critical piece in EHR implementation and/or adoption. Health records are updated every time a patient visits a doctor as they contain important information about the health and wellbeing of the patient and describes the history of care received during the past and to date. Therefore, such records are critical to research, hospitals, emergency rooms, healthcare laboratories, and even health insurance providers.In this article, a platform employing the AI and the use of multi-agent based systems along with the DLT technology for privacy preservation is proposed. The emphasis of security and privacy is highlighted during the process of collecting, managing and distributing EHR data.This article aims to ensure privacy, integrity and security metrics of the electronic health records are met when such copies are not only immutable but also distributed. The findings of this work will help guide the development of further techniques using the combination of AI and multi-agent based systems backed by DLT technology for secure and effective handling EHR data. This proposed architecture uses various AI-based intelligent based agents and blockchain for providing privacy and security in EHR. Future enhancement in this work can be the addition of the biometric based systems for improved security.
Unsupervised domain adaptation for person re-identification with iterative soft clustering

In this work, we propose to address the unsupervised domain adaptive (UDA) person re-id problem in which the model learns from an unlabeled target domain using a fully annotated source domain. Current approaches mainly address domain shift problem or the inter/intra-domain variation of the two domains. However, they have neglected to integrate the-easy-to-learn label distribution of the target domain into the model to improve its performance. Moreover, the automatic label assignment for the unlabeled target data currently used in UDA methods does not reflect the underlying data. To address these issues, we introduce a technique that enforces three properties: (1) target instance invariance that considers the target data and uses a key–value memory to guess the label distribution that is later used as the supervision signal. (2) a camera invariance, formed by unlabeled target images, and their camera-style transferred. Here, a new loss function is proposed to control overconfident predictions on the styled images. Lastly, (3) a hierarchical clustering-based optimization technique that exploits the similarities between the target images to constrain the supervision information of the first property. Here, we randomly allocate each target image to a separate cluster, then gradually incorporate similarity within each identity as we group similar images into clusters and use the cluster-IDs as the new target labels. We iteratively refine the guessed label distribution of the target domain by making predictions on the unlabeled target domain and then train the network with these new samples. Extensive experimental results on the concurrent use of these three properties demonstrate that the proposed model can achieve the state-of-the-art on unsupervised domain adaptive person re-id. Our work is important for knowledge discovery and knowledge transfer.
Pipeline FPGA-Based Implementations of ANNs for the Prediction of up to 600-Steps-Ahead of Chaotic Time Series

Chaotic time series prediction can be performed by applying different architectures of artificial neural networks (ANNs) that can be implemented on field-programmable gate arrays (FPGAs). However, the main challenges are the reduction of hardware resources to develop faster ANNs and the prediction capabilities for large horizons. In this manner, the contribution is devoted to introduce pipeline architectures in which some registers are placed between combinational blocks to divide the logic into shorter stages that can run with a faster clock. The cases of study are the multilayer perceptron (MLP), nonlinear autoregressive with exogenous input (NARX), and echo state network (ESN). In addition, another contribution is devoted to introduce the application of the decimation technique to extend the prediction horizon of the ANNs from 12 to 600-steps-ahead. The prediction capabilities of the MLP, NARX and ESN are compared by using eight chaotic time series with different maximum Lyapunov exponents. The pipeline FPGA-based implementations show that the ESN with a reservoir of at least 30 neurons guarantees a large prediction horizon of 600-steps-ahead. Another important advantage of the ESN is that its FPGA-based implementation can be performed by reusing one neuron, thus requiring the lowest quantity of hardware resources.
PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click

Existing interactive object segmentation methods mainly take spatial interactions such as bounding boxes or clicks as input. However, these interactions do not contain information about explicit attributes of the target-of-interest and thus cannot quickly specify what the selected object exactly is, especially when there are diverse scales of candidate objects or the target-of-interest contains multiple objects. Therefore, excessive user interactions are often required to reach desirable results. On the other hand, in existing approaches attribute information of objects is often not well utilized in interactive segmentation. We propose to employ phrase expressions as another interaction input to infer the attributes of target object. In this way, we can 1) leverage spatial clicks to locate the target object and 2) utilize semantic phrases to qualify the attributes of the target object. Specifically, the phrase expressions focus on "what" the target object is and the spatial clicks are in charge of "where" the target object is, which together help to accurately segment the target-of-interest with smaller number of interactions. Moreover, the proposed approach is flexible in terms of interaction modes and can efficiently handle complex scenarios by leveraging the strengths of each type of input. Our multi-modal phrase+click approach achieves new state-of-the-art performance on interactive segmentation. To the best of our knowledge, this is the first work to leverage both clicks and phrases for interactive segmentation.
An OWASP Top Ten Driven Survey on Web Application Protection Methods

Web applications (WAs) are constantly evolving and deployed at broad scale. However, they are exposed to a variety of attacks. The biggest challenge facing organizations is how to develop a WA that fulfills their requirements with respect to sensitive data exchange, E-commerce, and secure workflows. This paper identifies the most critical web vulnerabilities according to OWASP Top Ten, their corresponding attacks, and their countermeasures. The application of these countermeasures will guarantee the protection of the WAs against the most severe attacks and prevent several unknown exploits.
Cryogenic CMOS Circuits and Systems: Challenges and Opportunities in Designing the Electronic Interface for Quantum Processors

Quantum computing could potentially offer faster solutions for some of today's classically intractable problems using quantum processors as computational support for quantum algorithms [1]. Quantum processors, in the most frequent embodiment, comprise an array of quantum bits (qubits), the fundamental computational unit of a quantum computer. Unlike conventional bits, qubits can take a coherent state ranging from |0> to |1> on a continuous sphere, known as the Bloch sphere (Figure 1). When the state of the qubit, represented by a vector on the Bloch sphere, is on the equator of such a sphere, qubits are said to be in maximum superposition. Entanglement is the second important quantum mechanical property of qubit states, where knowing the state of one qubit implies knowing the state of the other one as well.
String Diagram Rewrite Theory I: Rewriting with Frobenius Structure

String diagrams are a powerful and intuitive graphical syntax, originating in theoretical physics and later formalised in the context of symmetric monoidal categories. In recent years, they have found application in the modelling of various computational structures, in fields as diverse as Computer Science, Physics, Control Theory, Linguistics, and Biology. In several of these proposals, transformations of systems are modelled as rewrite rules of diagrams. These developments require a mathematical foundation for string diagram rewriting: whereas rewrite theory for terms is well-understood, the two-dimensional nature of string diagrams poses quite a few additional challenges. This work systematises and expands a series of recent conference papers, laying down such a foundation. As a first step, we focus on the case of rewrite systems for string diagrammatic theories that feature a Frobenius algebra. This common structure provides a more permissive notion of composition than the usual one available in monoidal categories, and has found many applications in areas such as concurrency, quantum theory, and electrical circuits. Notably, this structure provides an exact correspondence between the syntactic notion of string diagrams modulo Frobenius structure and the combinatorial structure of hypergraphs. Our work introduces a combinatorial interpretation of string diagram rewriting modulo Frobenius structures in terms of double-pushout hypergraph rewriting. We prove this interpretation to be sound and complete and we also show that the approach can be generalised to rewriting modulo multiple Frobenius structures. As a proof of concept, we show how to derive from these results a termination strategy for Interacting Bialgebras, an important rewrite theory in the study of quantum circuits and signal flow graphs.
Comparison of Naive Bayes Algorithm and Support Vector Machine using PSO Feature Selection for Sentiment Analysis on E-Wallet Review

Abstract Cashless payment habits have been widely applied to the transportation system, restaurants and shops in the mall area. So, it is normal if the growth of mobile payment services is currently very rapid. The ease of doing transactions and promotional offers in the form of points and cashback in digital wallet applications (e-wallets) is very beneficial for its users. One of the most popular e-wallets is OVO. With so many reviews about OVO customer opinions on social media, there has also been a lot of public opinion. These opinions can produce negative or positive statements. Sentiment analysis is the mining of opinions or text to classify opinions or user reviews, of a brand reviews, product reviews, or service reviews into the category of positive or negative opinion. The methods used in this research are Naive Bayes and SVM. Both of these algorithms are the best algorithms widely used in text classification research. However, both of these algorithms have weaknesses in several parameters. So, in this study Feature Selection is used to improve its performance. The evaluation was carried out using 10-fold cross validation. Measurement accuracy is measured by confusion matrix and ROC curves. This study uses 500 positive reviews and 500 negative reviews as data training. The results of this study indicate that the use of PSO-based Naive Bayes algorithm produces an accuracy value of 93.10 percent with an AUC value of 0.750. While the results of research from the PSO-based SVM algorithm are 91.30 percent with an AUC value of 0.970. Based on these results the accuracy value generated by the Naive Bayes algorithm is classified as Fair Classification and SVM is classified as Excellent Classification. The AUC value generated by the Naive Bayes algorithm is also smaller than SVM. Therefore, in this study found that SVM is the best algorithm in classifying text.
Stain Style Transfer of Histopathology Images via Structure-Preserved Generative Learning

Computational histopathology image diagnosis becomes increasingly popular and important. While pathologists do not struggle with color variations in slides, computational solutions usually suffer from this critical issue. In this regard, this study proposes two stain style transfer models, SSIM-GAN and DSCSI-GAN, based on the generative adversarial networks. By cooperating structural preservation metrics and feedback of an auxiliary diagnosis net in learning, medical-relevant information presented by image texture, structure, and chroma-contrast features is preserved in color-normalized images. Particularly, the smart treat of chromatic image content in our DSCSI-GAN model helps to achieve noticeable normalization improvement in image regions where stains mix due to histological substances co-localization. Extensive experimentation on public histopathology image sets indicates that our methods outperform prior arts in terms of generating more stain-consistent images, better preserving histological information in images, and obtaining significantly higher learning efficiency. Our python implementation is published on https://github.com/hanwen0529/DSCSI-GAN.
Beyond Fixed Grid: Learning Geometric Image Representation with a Deformable Grid

In modern computer vision, images are typically represented as a fixed uniform grid with some stride and processed via a deep convolutional neural network. We argue that deforming the grid to better align with the high-frequency image content is a more effective strategy. We introduce Deformable Grid (DefGrid), a learnable neural network module that predicts location offsets of vertices of a 2-dimensional triangular grid, such that the edges of the deformed grid align with image boundaries. We showcase our DefGrid in a variety of use cases, i.e., by inserting it as a module at various levels of processing. We utilize DefGrid as an end-to-end learnable geometric downsampling layer that replaces standard pooling methods for reducing feature resolution when feeding images into a deep CNN. We show significantly improved results at the same grid resolution compared to using CNNs on uniform grids for the task of semantic segmentation. We also utilize DefGrid at the output layers for the task of object mask annotation, and show that reasoning about object boundaries on our predicted polygonal grid leads to more accurate results over existing pixel-wise and curve-based approaches. We finally showcase DefGrid as a standalone module for unsupervised image partitioning, showing superior performance over existing approaches. Project website: http://www.cs.toronto.edu/ jungao/def-grid .