Reviewerly/exHarmony huggingface dataset:
Virtual Machine Placement Strategy Using Clustering and Genetic Algorithm for increasing cloud performance and power saving

Cloud computing is one of the most critical technologies of the century. However, it faces many challenges in both the adoption and operation phases. In the bigger picture, resolutions leading to increasing performance and decreasing costs are highly valued. Cloud computing has two types of expenses, Initial costs and routine expenditures such as human resources, maintenance fees, and electricity bills which form a significant part of the total costs of cloud infrastructure. Improving efficiency using optimal resource usage and decreasing energy waste would be possible through correct resource allocation. Intelligent algorithms can achieve maximum efficiency by placing virtual machines in the minimum number of physical servers and using the resources of these servers to the best. The main goal of this research is to reduce electricity consumption and the cooling required for under-utilized servers while preventing network congestions. The proposed algorithm of this research uses a multi-objective genetic algorithm and clusters the virtual machines to reduce the genetic algorithm execution time. The evaluation of the proposed algorithm implementation indicates that in this method convergence time is faster than the algorithms that lack clustering. As the secondary objective, the proposed algorithm distributes network traffic between physical servers to reduce network bottlenecks.
A drug recommender system Based on Collaborative Filtering for Covid-19 patients

The epidemic caused by a new mutation of the coronavirus family called Covid-19 has created a global crisis involving all the world's countries. This disease has become a severe danger to everyone due to its unknown nature, high spread, and inability to detect the infected. In this regard, one of the important issues facing patients with Covid-19 is the prescription of Drugs according to the severity of the disease and considering the records of underlying diseases in people. In recent years, recommender systems have been developed significantly along with the advancement in information technology and artificial intelligence, which is one of its applications in various fields of medical sciences. Among them, we can refer to recommending systems for the prevention, control, and treatment of diseases. In this research, using the collaborative filtering approach as one of the types of recommender systems as well as the K-means clustering algorithm, a Drug recommendation system for patients with Covid-19 in the treatment stage of the disease is presented. The results of this research show that this recommender system has an acceptable performance based on the evaluation criteria of precision, recall, and F1-score compared to the opinions of experts in this field.
Using Structured Pruning to Find Winning Lottery Tickets

In recent years, deep neural networks have successfully solved artificial intelligence problems. However, large models need more memory and computational resources. Recent research has proved that the deep models are over-parameterized and have redundancy in their parameterization. The lottery ticket hypothesis paper by Frankle and Carbin offers that based on pruning, we can achieve subnetworks with initializations that are capable of training from scratch. Still, they have used unstructured pruning, and the resulting architectures are sparse that need special hardware/software for compression and speedup. On the other hand, structured pruning methods in the convolutional neural networks (CNNs) preserved the structure of the convolution layers. Therefore, we do not need special hardware/software (HW/SW) libraries. In this work, we examined the lottery ticket hypothesis with structured pruning techniques and used these methods with different architectures.
Real-time Steal Recognition on CCTV-Based Videos for Embedded Systems

Action Recognition is a computer vision task in which a given video has to be classified. As far as videos should be processed, this task is computationally more expensive than the other common tasks of computer vision such as classification and object detection. There will be many issues to address when this should be implemented, such as how to handle the computational costs of this task while working in a real-time manner, especially when it is being conducted on embedded devices as well. This paper explores surveillance as one of the situations in which action recognition becomes so critical. A CNN and RNN-based solution have been introduced. Then some experiments have been conducted in order to determine the best architecture choice for each of the CNN and RNN parts. As a result, the final can be used on embedded devices real time maintaining high accuracies.
Predicting Users' Demographic Features Based on Searched Queries and Installed Apps and Games

Employing various strategies to catch interest in using online app stores has become a common trend in recent decades. One of the dominant factors in determining the success of online businesses in this area is whether they have information about users' demographic features such as gender or age. In this study, we try to detect these features based on the lists of installed applications, installed games, and searched queries collected from an Iranian mobile application store. For this goal, we use a wide range of machine learning techniques to identify which model has the highest performance in these classification and regression tasks. Our findings show that we can detect genders with a balanced accuracy of 0.76. We also achieve 10.02 as the RMSE for age predictions and the ROC AUC of 0.81 in determining users' age groups.
OutCLIP, A New Multi-Outfit CLIP Based Triplet Network

Choosing a proper outfit is one of the problems we deal with every day. Today, people tend to use online websites for shopping, and the COVID-19 situation forced this condition more than before. In this research, we proposed a new architecture for multi-fashion item retrieval from a website database. We deployed a CLIP transformer model instead of convolutional neural networks in a triplet network. We also added a long short-term memory network (LSTM) to automatically extract and code the image features to generate descriptive text for each input image. Our OutCLIP model succeeded in doing its task with 83% precision and 85% recall accuracy in multi-item retrieval. This model can be trained and used in fashion retrieval problems and improve the former proposed models. Considering the descriptive text and the image together gives the model a better understanding of the concept and improves its generalization.
A Noval Reduced Particle Swarm Optimization With Improved Learning Strategy and Crossover Operator

In terms of balancing the exploration and exploitation capabilities of the PSO method in order to increase its resilience, this work provides a unique particle swarm optimization with enhanced learning techniques and a crossover operator (LSCPSO). Each particle is updated depending on the simplified equations in the first stage. The proposed LSCPSO method then employs a self-learning technique in which each particle (personal best) learns from k better particles in the current population. Then, a crossover step is introduced to the algorithm in the subsequent stage. After taking the k global best (gbest particle), the crossover is performed. This method strengthens the LSCPSO algorithm's capacity for social learning and global exploration. In subsequent trials, the performance of the LSCPSO algorithm is compared to that of five sample PSO variations. The benchmark function test results show that the proposed ILSPSO algorithm has much better overall performance than the other PSO variations that were looked at.
A Data-Centric Approach for Improving Adversarial Training Through the Lens of Out-of-Distribution Detection

Current machine learning models achieve super-human performance in many real-world applications. Still, they are susceptible against imperceptible adversarial perturbations. The most effective solution for this problem is adversarial training that trains the model with adversarially perturbed samples instead of original ones. Various methods have been developed over recent years to improve adversarial training such as data augmentation or modifying training attacks. In this work, we examine the same problem from a new data-centric perspective. For this purpose, we first demonstrate that the existing model-based methods can be equivalent to applying smaller perturbation or optimization weights to the hard training examples. By using this finding, we propose detecting and removing these hard samples directly from the training procedure rather than applying complicated algorithms to mitigate their effects. For detection, we use maximum softmax probability as an effective method in out-of-distribution detection since we can consider the hard samples as the out-of-distribution samples for the whole data distribution. Our results on SVHN and CIFAR-10 datasets show the effectiveness of this method in improving the adversarial training without adding too much computational cost.
Multi-Objective Optimization for Neural Network Structure

This study presents a new algorithm for training flexible perceptron multilayer neural networks. This algorithm is based on the multi-objective evolutionary optimization and tries to find the smallest optimal structure simultaneously by reducing the network error. In this method, a compatibility is established between the mean squared error and the vector length of the parameters of the activation functions by using flexible neurons which cause a greater degree of freedom leading to a faster convergence of the neural network. Then, the network structure decreases and the problem of overfitting and local minimum is prevented based on the values of these parameters and the use of the integration method of neurons. Moreover, it increases the power of the generalizability of the neural network. This method was used for classification problems, and the results were compared with AMGA, BCPA, LASSO, and Early Stopping methods. Based on the results, the algorithm proposed in this study usually works better compared to the similar algorithms. In addition, the proposed algorithm is a systematic method for finding the optimal neural network structure.
Driving the Technology Value Stream by Analyzing App Reviews

An emerging feature of mobile application software is the need to quickly produce new versions to solve problems that emerged in previous versions. This helps adapt to changing user needs and preferences. In a continuous software development process, the user reviews collected by the apps themselves can play a crucial role to detect which components need to be reworked. This paper proposes a novel framework that enables software companies to drive their technology value stream based on the feedback (or reviews) provided by the end-users of an application. The proposed end-to-end framework exploits different Natural Language Processing (NLP) tasks to best understand the needs and goals of the end users. We also provide a thorough and in-depth analysis of the framework, the performance of each of the modules, and the overall contribution in driving the technology value stream. An analysis of reviews with sixteen popular Android Play Store applications from various genres over a long period of time provides encouraging evidence of the effectiveness of the proposed approach.
Real-Time Machine Learning for Multi-User Massive MIMO: Symbol Detection Using Multi-Mode StructNet

In this paper, we develop a learning-based symbol detection algorithm for massive MIMO-OFDM systems. To exploit the structure information inherited in the received signals from massive antenna array, multi-mode reservoir computing is adopted as the building block to facilitate over-the-air training in time domain. In addition, alternating recursive least square optimization method, and decision feedback mechanism are utilized in our algorithm to achieve the real-time learning capability. That is, the neural network is trained purely online with its weights updated on an OFDM symbol basis to promptly and adaptively track the dynamic environment. Furthermore, an online learning-based module is devised to compensate the nonlinear distortion caused by RF circuit components. On top of that, a learning-efficient classifier named StructNet is introduced in frequency domain to further improve the symbol detection performance by utilizing the QAM constellation structural pattern. Evaluation results demonstrate that our algorithm achieves substantial gain over traditional model-based approach and state-of-the-art learning-based techniques under dynamic channel environment and RF circuit nonlinear distortion. Moreover, empirical result reveals our NN model is robust to training label error, which benefits the decision feedback mechanism.
The Opportunities and Limitations of Extended Page Table Switching for Fine-Grained Isolation

Extended Page Table switching with VMFUNC is a hardware isolation mechanism available in Intel CPUs. VMFUNC is attractive for low overhead and the possibility to isolate privileged kernel code. However, many careful design decisions are needed to ensure the security of the isolation boundary.
A General Contextualized Rewriting Framework for Text Summarization

The rewriting method for text summarization combines the advantage of extractive and abstractive approaches, improving the conciseness and readability of extractive summaries. Exiting rewriting systems take extractive sentences as the only input and rewrite each sentence independently, which may lose critical background knowledge and break cross-sentence coherence of the summary. To this end, we propose contextualized rewriting to consume the entire document and maintain the summary coherence, representing extractive sentences as a part of the document encoding and introducing group-tags to align the extractive sentences to the summary. We further propose a general framework for rewriting with an external extractor and a joint internal extractor, representing sentence selection as a special token prediction. We demonstrate the framework's effectiveness by implementing three rewriter instances on various pre-trained models. Experiments show that contextualized rewriting significantly outperforms previous non-contextualized rewriting, achieving strong improvements on ROUGE scores upon multiple extractors. Empirical results further suggest that joint modeling of sentence selection and rewriting can largely enhance performance.
BD-SAS: Enabling Dynamic Spectrum Sharing in Low-Trust Environment

The spectrum access system (SAS) designated by the FCC follows a centralized server-client model where a spectrum user registers with one SAS service provider for spectrum allocation and other spectrum management functions. This model, however, is vulnerable to adversarial influence on individual SAS servers, causing concerns over system reliability and trustworthiness, especially when the ecosystem embraces a growing base of SAS service providers and heterogeneous user requirements. In this paper, we propose a blockchain-based decentralized SAS architecture called BD-SAS to provide SAS services securely and efficiently, without assuming trust in individual SAS servers. BD-SAS is backward compatible with the existing SAS infrastructure and supports the automatic execution of key spectrum management functions. A global blockchain network (G-Chain) is used for spectrum regulatory tasks while localized blockchain networks (L-Chains) are instantiated in individual spectrum zones for automating spectrum access assignment and other spectrum management activities. To further achieve security against an adaptive adversary, BD-SAS integrates a SAS server reshuffle scheme to resist adaptive corruptions on individual SAS servers. We implemented a BD-SAS prototype with practical blockchain platforms. Evaluation results demonstrate the feasibility and responsiveness of our system, wherein a spectrum access assignment can be finalized at the level of seconds.
Using a Graph Transformer Network to Predict 3D Coordinates of Proteins via Geometric Algebra Modelling

The state of the art in protein structure prediction (PSP) is currently achieved by complex deep learning pipelines that require several input features. In this paper, we demonstrate the relevance of Geometric Algebra (GA) for modelling protein features in PSP. We do so by proposing a novel GA metric based on the relative orientations of amino acid residues. We then employ this metric as an additional input feature to a Graph Transformer (GT) to aid the prediction of the 3D coordinates of a protein. Adding this GA-based orientational information improves the accuracy of the predicted coordinates even after few learning iterations and on a small dataset.
Split-Knit Convolution: Enabling Dense Evaluation of Transpose and Dilated Convolutions on GPUs

Transpose convolutions occur in several image-based neural network applications, especially those involving segmentation or image generation. Unlike regular (forward) convolutions, they result in data access and computation patterns that are less regular, and generally have poorer performance when implemented in software. We present split-knit convolution (SKConv) – a technique to replace transpose convolutions with multiple regular convolutions followed by interleaving. We show how existing software frameworks for GPU implementation of deep neural networks can be adapted to realize this computation, and compare against the standard techniques used by such frameworks.
High-Performance Truss Analytics in Arkouda

In graph analytics, a truss is a cohesive subgraph based on the number of triangles supporting each edge. It is widely used for community detection applications such as social networks and security analysis, and the performance of truss analytics highly depends on its triangle counting method. This paper proposes a novel triangle counting kernel named Minimum Search (MS). Minimum Search can select two smaller adjacency lists out of three and uses fine-grained parallelism to improve the performance of triangle counting. Then, two basic algorithms, MS-based triangle counting, and MS-based support updating are developed. Based on the novel triangle counting kernel and the two basic algorithms above, three fundamental parallel truss analytics algorithms are designed and implemented to enable different kinds of graph truss analysis. These truss algorithms include an optimized K-Truss algorithm, a Max-Truss algorithm, and a Truss Decomposition algorithm. Moreover, all proposed algorithms have been implemented in the parallel language Chapel and integrated into an open-source framework, Arkouda. Through Arkouda, data scientists can efficiently con-duct graph analysis through an easy-to-use Python interface and handle large-scale graph data in powerful back-end computing resources. Experimental results show that the proposed methods can significantly improve the performance of truss analysis on real-world graphs compared with the existing and widely adopted list intersection-based method. The implemented code is publicly available from GitHub (https://github.com/Bears-R-Us/arkouda-njit).
Building a Performance Model for Deep Learning Recommendation Model Training on GPUs

We devise a performance model for GPU training of Deep Learning Recommendation Models (DLRM), whose GPU utilization is low compared to other well-optimized CV and NLP models. We show that both the device active time (the sum of kernel runtimes) but also the device idle time are important components of the overall device time. We therefore tackle them separately by (1) flexibly adopting heuristic-based and ML-based kernel performance models for operators that dominate the device active time, and (2) categorizing operator overheads into five types to determine quantitatively their contribution to the device active time. Combining these two parts, we propose a critical-path-based algorithm to predict the per-batch training time of DLRM by traversing its execution graph. We achieve less than 10% geometric mean average error (GMAE) in all kernel performance modeling, and 4.61% and 7.96% geomean errors for GPU active time and overall E2E per-batch training time prediction with overheads from individual workloads, respectively. A slight increase of 2.19% incurred in E2E prediction error with shared overheads across workloads suggests the feasibility of using shared overheads in large-scale prediction. We show that our general performance model not only achieves low prediction error on DLRM, which has highly customized configurations and is dominated by multiple factors but also yields comparable accuracy on other compute-bound ML models targeted by most previous methods. Using this performance model and graph-level data and task dependency analysis, we show our system can provide more general model-system co-design than previous methods.
AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

Deep Learning (DL) has become a prominent machine learning technique due to the availability of efficient computational resources in the form of Graphics Processing Units (GPUs), large-scale datasets and a variety of models. The newer generation of GPUs are being designed with special emphasis on optimizing performance for DL applications. Also, the availability of easy-to-use DL frameworks—like PyTorch and TensorFlow— has enhanced productivity of domain experts to work on their custom DL applications from diverse domains. However, existing Deep Neural Network (DNN) training approaches may not fully utilize the newly emerging powerful GPUs like the NVIDIA A100—this is the primary issue that we address in this paper. Our motivating analyses show that the GPU utilization on NVIDIA A100 can be as low as 43% using traditional DNN training approaches for small-to-medium DL models and input data size. This paper proposes AccDP—a data-parallel distributed DNN training approach—to accelerate GPU-based DL applications. AccDP exploits the Message Passing Interface (MPI) communication library coupled with the NVIDIA’s Multi-Process Service (MPS) to increase the amount of work assigned to parallel GPUs resulting in higher utilization of compute resources. We evaluate our proposed design on different small-to-medium DL models and input sizes on the state-of-the-art HPC clusters. By injecting more parallelism into DNN training using our approach, the evaluation shows up to 58% improvement in training performance on a single GPU and up to 62% on 16 GPUs compared to regular DNN training. Furthermore, we conduct an in-depth characterization to determine the impact of several DNN training factors and best practices—including the batch size and the number of data loading workers— to optimally utilize GPU devices. To the best of our knowledge, this is the first work that explores the use of MPS and MPI to maximize the utilization of GPUs in distributed DNN training.