Reviewerly/exHarmony huggingface dataset:
PredictDDL: Reusable Workload Performance Prediction for Distributed Deep Learning

Accurately predicting the training time of deep learning (DL) workloads is critical for optimizing the utilization of data centers and allocating the required cluster resources for completing critical model training tasks before a deadline. The state-of-the-art prediction models, e.g., Ernest and Cherrypick, treat DL workloads as black boxes, and require running the given DL job on a fraction of the dataset. Moreover, they require retraining their prediction models every time a change occurs in the given DL workload. This significantly limits the reusability of prediction models across DL workloads with different deep neural network (DNN) architectures. In this paper, we address this challenge and propose a novel approach where the prediction model is trained only once for a particular dataset type, e.g., ImageNet, thus completely avoiding tedious and costly retraining tasks for predicting the training time of new DL workloads. Our proposed approach, called PredictDDL, provides an end-to-end system for predicting the training time of DL models in distributed settings. PredictDDL leverages Graph HyperNetworks, a class of neural networks that takes computational graphs as input and produces vector representations of their DNNs. PredictDDL is the first prediction system that eliminates the need of retraining a performance prediction model for each new DL workload and maximizes the reuse of the prediction model by requiring running a DL workload only once for training the prediction model. Our extensive evaluation using representative workloads shows that PredictDDL achieves up to 9.8× lower average prediction error and 10.3× lower inference time compared to the state-of-the-art system, i.e., Ernest, on multiple DNN architectures.
Rethinking Virtual Machines Live Migration for Memory Disaggregation

Resource underutilization has troubled data centers for several decades. Memory disaggregation provides an efficient way to improve memory utilization while leaving a missing puzzle piece on CPU underutilization. Live migration is an essential method for CPU resource reallocation. However, the state-of-the-art Virtual Machines (VM) live migration suffers from significant resource consumption. We discover the substantial potential for optimizing live migration in disaggregated memory systems. We propose Anemoi, a source management system incorporating VM live migration into memory disaggregation to fill in the missing piece. Disaggregated memory enables the read-only replica to be accessible from destination nodes, eliminating considerable network transmission for memory pages and saving migration time. Regarding the potential overwhelming memory consumption of duplicate read-only replicas, we design a dedicated compression algorithm. The evaluation shows that Anemoi reduces the network bandwidth use and the migration time by 69% and 83%, respectively, compared to VM live migration. The compression can achieve a space-saving of 83.6%.
Gland Segmentation in H&amp;E Histopathological Images using U-Net with Attention Module

Gland segmentation refers to the process of identifying and delineating glandular regions within histopathology images. However, gland segmentation in histopathology images is a challenging task due to several factors such as shape variations and overlapping structures. Despite numerous research efforts dedicated to addressing this challenge, it remains an unresolved problem. To address this issue, we introduce a deep gland segmentation method by designing an enhanced U-Net variant combined with an Attention module. By incorporating the Attention module into the U-Net architecture, the model becomes more adept at capturing contextual information and fine details within the histopathology images. To verify the effectiveness of the introduced method, we compare it with well-known semantic segmentation architectures on the GLAS and GRAG datasets. According to the results, our approach demonstrates a high success rate in gland segmentation, effectively competing with existing state-of-the-art methods.
Enhanced DFT-Based Chaotic Image Block Cipher with Several Modes of Operation

Chaotic image encryption has been investigated extensively in the literature in different domains, including the frequency domain using Discrete Fourier Transform (DFT). Despite DFT-based chaotic image cryptosystems exhibiting good confusion properties, they suffer from poor diffusion characteristics and non-uniform histograms. Therefore, they are vulnerable to many attacks, such as differential attacks, and statistical cryptanalytic attacks. This paper presents an enhanced block cipher image cryptosystem with several modes of operation based on the concept of bit reversal using the 2D chaotic Baker map and DFT. The proposed enhancement develops an improved image encryption scheme that involves properly diffusion-confusion enhanced operations within modified versions of the known block cipher modes of operation, namely Cipher Block Chaining Mode (CBC), Cipher Feedback Mode (CFB), and Output Feedback Mode (OFB). These enhanced operations are based on a 2D chaotic baker map in the Fourier domain. The proposed enhancement is validated using a MATLAB simulation. Results show that the proposed work manages to overcome the abovementioned problems outstandingly and outperforms the existing results in the literature in many encryption quality metrics.
DoW-KV: A DPU-offloaded and Write-optimized Key-Value Store on Disaggregated Persistent Memory

Disaggregated Persistent Memory (DPM) is a promising technology offering elasticity, high resource utilization, persistent data storage, and lower power consumption. While building KV stores on the DPM benefits from these merits, achieving efficient writes also faces two primary challenges: 1) limited scalability caused by the underused PM bandwidth, and 2) limited CPU resources the persistent memory server (PMS) can provide. Integrating the SmartNIC such as the Data Processing Unit (DPU) into the DPM gives developers the chance to optimize writing to KV stores by utilizing both the memory and processor of DPU. However, simple offloading cannot make full use of the DPU’s potential capacity. To address these challenges, we propose DoW-KV, a persistent hash KV store on DPM. DoW-KV employs a two-tier hash index consisting of a DPU cache table in DPU memory and multiple PM persistent tables on the PM. It relocates small random writes to the DPU memory and consolidates them to the PM at a coarse granularity. Furthermore, DoW-KV uses DPU-offloaded step merge and a coroutine-based asynchronous processing framework to efficiently manage the PM persistent tables. DoW-KV also introduces a client-mixed read strategy to boost key searching on the two-tier hash index. Experimental results show that DoW-KV outperforms the state-of-the-art DINOMO by 2.1× and 1.3× in the Put and Get operations, respectively.
Accelerating Distributed ML Training via Selective Synchronization

In distributed training, deep neural networks (DNNs) are launched over multiple workers concurrently and aggregate their local updates on each step in bulk-synchronous parallel (BSP) training.However, BSP does not linearly scale-out due to high communication cost of aggregation.To mitigate this overhead, alternatives like Federated Averaging (FedAvg) and Stale-Synchronous Parallel (SSP) either reduce synchronization frequency or eliminate it altogether, usually at the cost of lower final accuracy.In this paper, we present SelSync, a practical, low-overhead method for DNN training that dynamically chooses to incur or avoid communication at each step either by calling the aggregation op or applying local updates based on their significance.We propose various optimizations as part of SelSync to improve convergence in the context of semi-synchronous training.Our system converges to the same or better accuracy than BSP while reducing training time by up to 14×.
CCDMOT: An Optimized Multi-Object Tracking Method for Unmanned Vehicles Pedestrian Tracking

Multi-object tracking (MOT) is pivotal for under-standing environments in which unmanned vehicles function. The Joint Detection and Embedding (JDE) paradigm, merging target motion and appearance for data association, stands as a cornerstone in MOT. Despite its recognition, conventional methods exhibit limitations, particularly when faced with intricate traffic settings, diverse pedestrian appearances, and recurrent occlusions. Addressing these challenges, this paper introduces CCDMOT, an optimized MOT algorithm. Inspired by FairMOT, CCDMOT incorporates a Criss-Cross Attention Block to refine feature extraction and boost the relationship between varied dimensional features. Moreover, our innovative Dynamic Re-ID Embedding network augments the Re-ID tasks, amplifying stability in congested settings and mitigating target losses. Comparative assessments on the MOT17 validation set indicate a significant 1.5 MOTA improvement over FairMOT, with ID switches reducing from 404 to 340, emphasizing CCDMOT's enhanced stability. Additionally, CCDMOT also achieved satisfactory results on the MOT challenge. Finally, qualitative evaluation on the autonomous driving dataset KITTI confirmed that the framework proposed in this study can effectively be applied to pedestrian tracking in unmanned vehicles, demonstrating the practical significance and applicability of our work.
Exploration of Deep Methods for Cross-Domain Aerial Image Matching

Image matching is an attractive area for researchers. This field has been propelled recently due to the advancement of imaging devices with multi-spectral capabilities, compute power, and the evolution of the deep learning era. Cross-domain or cross-platform matching remains challenging due to the multiple type modalities associated and the need for relevant datasets. Learning or training in one domain (e.g., spectral band) and matching across domains are the real challenges for the imaging community today. The wide availability of Google Earth imagery, the development of cross-domain matching algorithms, and high-speed devices encourage addressing this issue. This paper explored deep learning methods for a cross-domain image-matching perspective. We have exploited latent and feature space, particularly for standard deep methods with several preprocessing steps, optimizer settings, and post-processing steps. We evaluated performance over standard cross-platform dataset using metrics available in the literature.
Robust Real-Time Curb Detection for Autonomous Sanitation Vehicles

Curb detection is a key enabling functionality for the precise curb-following sanitation control of autonomous sanitation vehicles. The robust and efficient curb detection in complex environments is still a challenging issue. In this paper, we propose a novel semantic segmentation-based framework for curb detection using monocular bird's eye-view images. We employ a lightweight segmentation network based on HRNet to extract the drivable area. A zero-shot post-processing approach is proposed to extract a candidate point set from the segmented image for robust curve fitting. In addition, we propose a modified RANSAC fitting approach that accounts for outlier points to achieve dynamic order curve fitting and curb representation. Experimental results in complex sanitation scenarios demonstrate the efficiency, accuracy, and robustness of the proposed approach.
ExplSched: Maximizing Deep Learning Cluster Efficiency for Exploratory Jobs

Resource management for Deep Learning (DL) clusters is essential for system efficiency and model training quality. Existing schedulers provided by DL frameworks are mostly adaptations from traditional HPC clusters and usually work on jobs’ makespan, assuming that DL training jobs finish completely. Unfortunately, it is reported that a fair amount of training jobs are exploratory jobs and often finish unsuccessfully (over 30%) in production clusters. This is due to the distinct characteristic of Deep Neural Network (DNN) training that it is an exploratory process of frequent user interventions, such as adjusting model structures, tuning hyperparameters, and exploring feature validity. Existing DL cluster schedulers using offline algorithms are not suitable for exploratory jobs when unexpected early terminations can cause noticeable resource waste. Moreover, DL training jobs are iterative and usually yield diminishing returns as they progress. Equally allocating resource among training iterations is not efficient, especially when dealing with exploratory jobs where it can worsen the degradation of system efficiency. The fundamental goal of a DL training job is to gain model quality improvement, usually indicated by the loss reduction (job profit) of a DNN model. This paper introduces a novel scheduling problem for exploratory jobs that seeks to maximize the overall training profit of a DL cluster. We propose ExplSched, an online scheduling solution based on the primal-dual framework, resulting in a competitive ratio of 2α that belongs to O(ln n). It uses a resource price function that emphasizes the importance of job profit to resource consumption ratio to make quick resource allocation decisions. Experimental results show that ExplSched achieved an average system utility improvement of 87.28% compared with other related work.
Feature Mutual Reconstruction for Semi-Supervised Few-Shot Remote Sensing Image Scene Classification

Few-shot remote sensing image scene classification (RSISC) faces the challenge of too few labeled samples. In recent years, semi-supervised few-shot learning (SSFSL) have expanded the labeling information by assigning pseudo-labels to unlabelled samples. However, obtaining high-confidence pseudo-labels with limited labeling samples also remains challenging. For this problem, this work proposes a feature mutual reconstruction method for semi-supervised few-shot RSISC. Specifically, the proposed feature mutual reconstruction module uses the similarity between unlabeled and labeled data to reconstruct labeled data, and the reconstructed labeled data classify the unlabeled data by joint representation error to obtain high-confidence pseudo-labels. In addition, the proposed prototype contrast correction module adjusts the spatial position of the prototypes by constructing a contrast learning model through pseudo-labels. Experimental results on three public datasets demonstrate that our method can effectively distinguish unlabelled data and achieve excellent classification results.
A Data Augmentation Method Based on Saliency Measures and Feature Transformation for Few-Shot Learning

Few-shot learning is devoted to making a deep neural network recognize unseen categories with few labels. Because the number of labeled samples is limited, the actual data distribution is difficult to be reflected. As a result, the model has the problems of overfitting and poor generalization performance. To solve the issues above, a data augmentation method based on saliency measures and feature transformation is proposed for few-shot learning. In our method, the salient region measure module is utilized to select discriminative information and reduce redundant information, which makes contributions to improving the quality of generated pseudo data. Further, in order to make the augmented data help perfect the data distribution, a contextual feature transformation module is proposed to augment data in feature space. The results obtained by experimenting on two datasets indicate that the proposed data augmentation method performs better than other comparison methods in few-shot learning.
Communication-Avoiding Recursive Aggregation

Recursive aggregation has been of considerable interest due to its unifying a wide range of deductive-analytic workloads, including social-media mining and graph analytics. For example, Single-Source Shortest Paths (SSSP), Connected Components (CC), and PageRank may all be expressed via recursive aggregates. Implementing recursive aggregation has posed a serious algorithmic challenge, with state-of-the-art work identifying sufficient conditions (e.g., pre-mappability) under which implementations may push aggregation within recursion, avoiding the serious materialization overhead inherent to traditional reachability-based methods (e.g., Datalog).State-of-the-art implementations of engines supporting recursive aggregates focus on large unified machines, due to the challenges posed by mixing semi-naïve evaluation with distribution. In this work, we present an approach to implementing recursive aggregates on high-performance clusters which avoids the communication overhead inhibiting current-generation distributed systems to scale recursive aggregates to extremely high process counts. Our approach leverages the observation that aggregators form functional dependencies, allowing us to implement recursive aggregates via a high-parallel local aggregation to ensure maximal throughput. Additionally, we present a dynamic join planning mechanism, which customizes join order per-iteration based on dynamic relation sizes. We implemented our approach in PARALAGG, a library which allows the declarative implementation of queries which utilize recursive aggregates and executes them using our MPI-based runtime. We evaluate PARALAGG on a large unified node and leadership-class supercomputers, demonstrating scalability up to 16,384 processes.
Prophet: Fine-grained Load Balancing for Parallel Training of Large-scale MoE Models

Mixture of Expert (MoE) has received increasing attention for scaling DNN models to extra-large size with negligible increases in computation. The MoE model has achieved the highest accuracy in several domains. However, a significant load imbalance occurs in the device during the training of a MoE model, resulting in significantly reduced throughput. Previous works on load balancing either harm model convergence or suffer from high execution overhead. To address these issues, we present Prophet: a fine-grained load balancing method for parallel training of large-scale MoE models, which consists of a planner and a scheduler. Prophet planner first employs a fine-grained resource allocation method to determine the possible scenarios for the expert placement in a fine-grained manner, and then efficiently searches for a well-balanced expert placement to balance the load without introducing additional overhead. Prophet scheduler exploits the locality of the token distribution to schedule the resource allocation operations using a layer-wise fine-grained schedule strategy to hide their overhead. We conduct extensive experiments in four clusters and five representative models. The results indicate that Prophet gains up to 2.3x speedup compared to the state-of-the-art MoE frameworks including Deepspeed-MoE and FasterMoE. Additionally, Prophet achieves a load balancing enhancement of up to 12.06x when compared to FasterMoE.
Optimizing Federated Learning Approach: Literature Survey and Open Points

Federated Learning (FL) is a decentralized machine learning strategy that ensures training data stays on personal devices while also facilitating collaborative machine learning of complicated models among dispersed devices. FL is used to circumvent Mobile Edge Computing’s constraints (MEC). In FL, mobile devices utilize their local data to train an ML model that is controlled by the Federated Learning server. The model changes, i.e. the model’s weights, are subsequently sent to the FL server for collection. This stage is performed several times until the desired precision is obtained. Federated Learning has several issues, including communication costs, resource allocation, privacy, and security. The Federated Learning method is outlined in this work, and existing solutions to overcome its constraints are categorized and reviewed. Furthermore, there are several research directions and unanswered questions that must be addressed while considering a better method to Maximize Federated Learning Models in The Mobile Edge Computing Environment.
Requirements Correlation Identification Method for Launch Vehicle Based on Word Mover's Distance

In the forward design process of launch vehicle based on requirement engineering, the identification of requirements correlation directly affects the design quality and efficiency. Aiming at the problems of the traditional identification strategies' lacking efficiency and accuracy, a requirements correlation identification method using Word Mover's Distance is proposed. The hierarchical requirements model of the rocket system is constructed. In addition, Word Mover's Distance is adaptively improved to calculate the semantic distance between the requirement items, which accelerates the requirements correlation identification process. The accuracy and reliability of the proposed method is verified by the experiment. The research lays a foundation for improving the intelligent level of requirement modeling in the launch vehicle field.
A Multi-Dimensional Dynamic Evaluation Method for the Intelligence of Unmanned Aerial Vehicle Swarm

With the emergence of unmanned system and swarm intelligence technology, how to describe and evaluate the intelligence of unmanned aerial vehicle swarm (UAVS) has become a challenging topic. An intelligence description framework is established involving the mission execution complexity and mission performance indexes. Multiple dimensions index system is built to express the internal intelligence behavior of the UAVS. To dynamically evaluate the intelligence capability of UAVS from multi-dimension, a dynamic indexes framework based on data yard and combined weighting method is proposed. Intelligence performance under multiple dynamic execution periods and complexities are synthesized to evaluate the intelligence capability. Aiming at discussing UAVS intelligence across missions, the evaluated intelligence capability is transformed to intelligence level by the designed intelligence classification framework and the standard swarm intelligence. An intelligence evaluation instance is provided to verify the effectiveness of the proposed method. The intelligence evaluation framework and method can fully consider the dynamic performance and mission execution complexities of the swarm system, and evaluate the intelligent capabilities of UAVS from multiple dimensions.
Hate speech detection by classic machine learning

It is becoming increasingly important for society to identify hate speech on social media. Differentiating hate speech from other instances involving offensive language is a significant difficulty for automatic hate speech tracking on social media. To distinguish between these categories, we train various classical machine learning models such as logistic regression, decision trees, random forest, naive Bayes, k-nearest neighbors, and support vector machines (SVM) - support vector classifier (SVC) on a dataset divided into three groups: those containing hate speech, those containing only offensive language, and those containing neither. From our practical trials, we found that the Logistic Regression algorithm and the SVM-SVC algorithm perform well in detecting hate speech and offensive language.