Reviewerly/exHarmony huggingface dataset:
An Improved cGAN with Self-Supervised Guidance Encoder for Generation of High-Resolution Facial Expression Images

The recent spread of smartphones and social networking services has increased the means of seeing images of human faces. Particularly, in the field of facial images, the facial expression conversion has already been realized using deep learning-based approaches. However, in the conventional deep learning-based methods, only low-resolution images can be generated due to limited computational resources. As a result, the generated images are blurry or aliasing. To solve low-resolution problem, we proposed a serial method to enhance the resolution of the generated facial images by combining a super-resolution network following the generative model in our previous work. In this paper, we propose a new model with two encoders that integrates a self-supervised guidance encoder into cGAN to further improve the accuracy of the generated results. We used structural similarity and peak signal-to-noise ratio as evaluation metrics and were able to improve image quality compared to our previous baseline model.
XSSMitigate: Deep Packet Inspection based XSS Attack Quarantine in Software Defined Networks

Web applications are used by millions everyday for day-to-day activities. These applications deal with personal and financial data and security of these applications becomes important. Cross-Site Scripting (XSS) is a type of attack which target web applications. These attacks can escalate user privileges and can provide access to unauthorized data compromising users' privacy. In this paper, we present a technique to detect and mitigate the XSS attacks in Software Defined Networks (SDN). Our method has two phases as detection and mitigation. For detecting the attack, we use deep packet inspection on network packets collected at a switch placed in front of web server. Once the attack is detected we block the source of attack by installing appropriate rules at the switch through controller. We experiment within a lab setup and also with real world traces to validate the attack detection. Through simulations we show the feasibility of blocking the attack source for quarantine.
Comparison of Modern Compression Standards on Medical Images for Telehealth Applications

Telehealth applications, such as remote diagnosis and examination, have become more and more popular nowadays. However, the generated large number of medical images and their impractical digital size when it comes to telehealth have brought pressure on communication infrastructures. More specifically, the image processing time has increased dramatically due to the size of the digitized images, while requiring substantial transmission bandwidth and storage space. For these reasons, medical image compression has become a hot research topic. While image compression techniques have evolved over several generations, it remains an open question how these standards will perform when it comes to sensitive oversized medical image content. In this paper, the emerging compression standard High Efficiency Video Coding (HEVC) and the next generation standard Versatile Video Coding (VVC) are evaluated on two different medical image datasets, covering the entire range of quality levels. The results of this study show that the VVC standard overperforms HEVC on raw medical image compression both visually and quantitatively, with the performance gap narrowing when dealing with pre-compressed medical images.
Addressing Straggler Problem Through Dynamic Partial All-Reduce for Distributed Deep Learning in Heterogeneous GPU Clusters

Distributed deep learning is an inevitable choice in learning large-scale deep learning models today. Beyond a certain level, training deep learning models can take days or months, which can lead to catastrophic consequences in applications that require rapid trend reflection or decision-making. Distributed deep learning is largely divided into synchronous and asynchronous methods according to the synchronization method at the time of parameter update. The former updates the parameter with the average value of gradient calculated by all workers, and there is a problem that the processing speed is matched to the slowest worker. The latter is faster because it updates parameters without waiting for the slowest worker, but can converge more slowly to the optimal state due to the stale gradient problem. In this paper, we propose Dynamic Partial All-Reduce, a distributed learning algorithm that uses a synchronous method but dynamically manages whether workers participate in global synchronization to autonomously control the effects of the straggler problem. In this algorithm, if a slow worker is detected, the influence of straggler is limited by excluding the worker from global communication and allowing the remaining workers to update the parameters. Then, when the slow worker recovers the normal speed, it returns to the synchronization group again. In this way, the decision is made by comparing what causes the greater loss in speed and convergence between the loss of computational power due to the exclusion of one GPU from the learning process, the omission of as much learning data as distributed to the worker, and the slowdown due to straggler. We implemented this algorithm based on PyTorch and Horovod, and all experiments were conducted on Tencent Cloud.
Knowledge Transferring in Deep Learning of Wearable Dynamics

Wearable dynamics learning is attracting intensive interests now in the era of smart health. One challenge is how the learning model can be effective trained with limited amounts of data. Targeting this, we in this study propose a deep transfer learning framework, to leverage the knowledge obtained from non-target users and boost the performance on the target-user. More specifically, we have designed and pre-trained a convolutional neural network on the non-target database, and then fine-tuned the model on a small portion of the target database. The framework has been evaluated on a wearable biomechanical learning application for physical activity detection. Compared with direct target-data-based learning, the proposed deep transfer learning approach great boosts the detection accuracy. This study will advance the wearable dynamics learning applications through deep knowledge transferring.
Improving Memory Utilization by Sharing DNN Models for Serverless Inference

Serving deep neural network (DNN) models requires a lot of memory and has a relatively long response time. Researchers nowadays try to integrate DNN inference services with the serverless architecture, which workload is suited for the serverless, because of its short-lived and burst characteristics. In addition, the cases of deploying DNN services to edge clouds are also increasing, to reduce the response time. However, serving DNN models in a serverless manner has a problem of excessive memory usage deriving from the data duplication, which is a more serious problem on strongly resource-constrained edge cloud. To address this problem, we designed ShmFaas on the open-source serverless platform running on the Kubernetes with minimal code changes. First, we implemented the serverless system with lightweight memory isolation by sharing DNN models in-memory, in order to avoid the model duplication problem. Also, we designed an LRU-based model eviction algorithm for efficient memory usage on the edge cloud. As a result, in our experiment, the system's memory usage is reduced by more than 29.4% compared to the existing system, and it shows that the overhead due to the proposed system is negligible enough to be used for real-world workloads.
Exploration of Machine Learning Attacks in Automotive Systems Using Physical and Mixed Reality Platforms

Adversarial attacks on Deep Neural Networks represent a critical challenge in the adoption of DNNs in critical applications. However, - and in spite of its great need, - there is significant mystery surrounding attacks on DNNs. One reason for this is the lack of a platform that enables users to get a hands-on, intuitive understanding of the attacks. In this paper, we address this problem by designing an extensible, configurable exploration platform for studying various attacks on DNNs. Our platform specifically focuses on DNNs deployed in Computer Vision modules of automotive systems. Using the platform, the user can perform various adversarial machine learning attacks, such as evasion attacks and image-perturbation attacks, and comprehend their adversarial effects on autonomous vehicles. The platform can be used to plug and play with various neural network models developed for Traffic Sign Recognition systems in autonomous vehicles. The infrastructure includes both physical and mixed-reality variants, and we demonstrate the usage of the platform on two traffic sign recognition models with different adversarial attacks.
Crowd Multi Prediction: Single Network for Crowd Counting, Localization and Anomaly Detection

In this study, we propose a neural network to solve crowd counting, localization and abnormal event detection problems together. Our proposed model combines P2P-Net with a novel crowd anomaly detection module. The final network has a single backbone and multiple head structure. Synthetic datasets are used for training and evaluation. Results show that our model gets high accuracy in crowd anomaly detection task.
Design and Implementation of Image-Sensing Based Learning Assistant System for Self-Practice Dynamic Yoga on Embedded GPU Device

In this study, by using OpenPose technology and fuzzy rules, the developed system provides an image-sensing learning system by combining the back-end with front-end functions about self-practice Yoga sports at household. In the front-end side, users can select the Yoga exercises which they want on the web page at the first level, after selecting the exercise, the website page will go into the web page at the second level, users select the options that the system provides the start, skeleton, and arrows functions. In the back-end side, firstly the the frame for initialization, the key frames, and the frames for synchronization of instructor are selected based on Euclidean distances in pre-processing. Secondly, in system processing, the frames obtained in pre-processing are utilized to compute the difference of angles between the instructor and user, then taking the angle difference value as input into the fuzzy-rule based scoring system to get each score, Finally, the system synchronizes the instructor and user at the time domain. By NVIDIA Jetson Nano device, the experimental results show that user can interact with the system through the user interface (UI), and the back-end operations work effectively by the detection of dynamic Yoga poses.
Fast Hyperparameter Tuning for Ising Machines

In this paper, we propose a novel technique to accelerate Ising machines hyperparameter tuning. Firstly, we define Ising machine performance and explain the goal of hyperparameter tuning in regard to this performance definition. Secondly, we compare well-known hyperparameter tuning techniques, namely random sampling and Treestructured Parzen Estimator (TPE) on different combinatorial optimization problems. Thirdly, we propose a new convergence acceleration method for TPE which we call “FastConvergence”. It aims at limiting the number of required TPE trials to reach best performing hyperparameter values combination. We compare FastConvergence to previously mentioned well-known hyperparameter tuning techniques to show its effectiveness. For experiments, well-known Travel Salesman Problem (TSP) and Quadratic Assignment Problem (QAP) instances are used as input. The Ising machine used is Fujitsu's third generation Digital Annealer (DA). Results show, in most cases, FastConvergence can reach similar results to TPE alone within less than half the number of trials.
Invisible Encoded Backdoor attack on DNNs using Conditional GAN

Deep Learning (DL) models deliver superior performance and have achieved remarkable results for classification and vision tasks. However, recent research focuses on exploring these Deep Neural Networks (DNNs) weaknesses as these can be vulnerable due to transfer learning and outsourced training data. This paper investigates the feasibility of generating a stealthy invisible backdoor attack during the training phase of deep learning models. For developing the poison dataset, an interpolation technique is used to corrupt the sub-feature space of the conditional generative adversarial network. Then, the generated poison dataset is mixed with the clean dataset to corrupt the training images dataset. The experiment results show that by injecting a 3% poison dataset combined with the clean dataset, the DL models can effectively fool with a high degree of model accuracy.
Adversarial attacks and robust defenses in deep learning

Deep learning models have shown exceptional performance in many applications, including computer vision, natural language processing, and speech processing. However, if no defense strategy is considered, deep learning models are vulnerable to adversarial attacks. In this chapter, we will first describe various typical adversarial attacks. Then we will describe different adversarial defense methods for image classification and object detection tasks.
Quaternion Extreme Learning Machine Based on Real Augmented Representation

Widely linear modeling is an important quaternion signal processing technique for capturing the complete second-order statistics of quaternion signals. However, the algorithms based on widely linear modeling are computationally expensive due to the augmented variables and statistics. In this letter, a fast estimation technique based on a real augmented representation of widely linear modeling is proposed in the context of quaternion extreme learning machine with augmented hidden layer (QELMAH), resulting in a reduction of almost 93% of multiplications and 75% of additions. An equivalence between the proposed algorithm and the original QELMAH is theoretically established by proving that the trained networks with the proposed algorithm and the original one mathematically implement identical mapping. Such a technique is also applicable to the widely linear quaternion recursive least squares algorithm. The theoretical analysis and the effectiveness of the proposed algorithms are validated by simulations on two benchmark problems.
Conservation Laws and Quantum Error Correction: Toward a Generalized Matching Decoder

Decoding algorithms are essential to fault-tolerant quantum-computing architectures.In this perspective we explore decoding algorithms for the surface code; a prototypical quantum low-density parity-check code that underlies many of the leading efforts to demonstrate scalable quantum computing.Central to our discussion is the minimum-weight perfect-matching decoder.The decoder works by exploiting underlying structure that arises due to materialised symmetries among surface-code stabilizer elements.By concentrating on these symmetries, we begin to address the question of how a minimum-weight perfect-matching decoder might be generalised for other families of codes.We approach this question first by investigating examples of matching decoders for other codes.These include decoding algorithms that have been specialised to correct for noise models that demonstrate a particular structure or bias with respect to certain codes.In addition to this, we propose a systematic way of constructing a minimum-weight perfect-matching decoder for codes with certain characteristic properties.The properties we make use of are common among topological codes.We discuss the broader applicability of the proposal, and we suggest some questions we can address that may show us how to design a generalised matching decoder for arbitrary stabilizer codes.
A Privacy Protection Scheme for Bidding Users of Peer-to-Peer Electricity Call Auction Trading in Microgrids

Renewable energy generation is greatly affected by the weather, and electricity is not easily stored in large quantities. So, the call auction trading based on the rule of the maximum trading volume is of great significance to promote the electricity consumption of microgrids. Therefore, in microgrid electricity call auction trading, aiming at the problems, such as bidding messages may leak users’ privacy, they may also be eavesdropped or tampered with in the bidding process, and the balance between privacy protection and supervision cannot be achieved, a privacy protection scheme for bidding users of peer-to-peer electricity call auction trading in microgrids is proposed. It consists of a market layer and a blockchain layer. Based on the improved certificateless ring signcryption and timed-release encryption algorithms, the market layer is used to hide the user's real identity while encrypting the bidding message and track illegal users. The blockchain layer is used for trading settlement and trading record storage to ensure data storage security. The security is proved theoretically. Performance analysis and simulation results indicate that computational and communication costs are low. Especially the verification cost is low and increases slowly with the increase in the number of ring members.
A Highly Interpretable Framework for Generic Low-Cost UAV Attack Detection

The increasing prevalence of cyber-attacks on unmanned aerial vehicles (UAVs) has led to research on effective detection methods. However, current approaches often lack transferability and interoperability, which limits their effectiveness. This study proposes a CNN-BiLSTM-Attention (CBA) model for efficient attack detection using real-time UAV sensor data. Additionally, the SHapley Additive exPlanations (SHAP) method is used to improve the interpretability of the model. The proposed approach is tested on real attack scenarios, including denial-of-service (DoS) attacks and global positioning system (GPS) spoofing attacks, and demonstrates both effectiveness and interpretability.
Time-Aware Dynamic Graph Embedding for Asynchronous Structural Evolution

Dynamic graphs refer to graphs whose structure dynamically changes over time.Despite the benefits of learning vertex representations (i.e., embeddings) for dynamic graphs, existing works merely view a dynamic graph as a sequence of changes within the vertex connections, neglecting the crucial asynchronous nature of such dynamics where the evolution of each local structure starts at different times and lasts for various durations.To maintain asynchronous structural evolutions within the graph, we innovatively formulate dynamic graphs as temporal edge sequences associated with joining time of vertices (ToV) and timespan of edges (ToE).Then, a time-aware Transformer is proposed to embed vertices' dynamic connections and ToEs into the learned vertex representations.Meanwhile, we treat each edge sequence as a whole and embed its ToV of the first vertex to further encode the time-sensitive information.Extensive evaluations on several datasets show that our approach outperforms the state-of-the-art in a wide range of graph mining tasks.At the same time, it is very efficient and scalable for embedding large-scale dynamic graphs.
API comparison knowledge extraction via prompt-tuned language model

Application Programming Interfaces (APIs) are frequent in software engineering domain texts, such as API references and Stack Overflow. These APIs and the comparison knowledge between them are not only important for solving programming issues (e.g., question answering), but they are also organized into structured knowledge to support many software engineering tasks (e.g., API misuse detection). As a result, extracting API comparison knowledge (API entities and semantic relations) from texts is essential. Existing rule-based and sequence labeling-based approaches must manually enumerate all linguistic patterns or label a large amount of data. Therefore, they involve a significant labor overhead and are exacerbated by morphological and common-word ambiguity. In contrast to matching or labeling API entities and relations, we formulates heterogeneous API extraction and API relation extraction tasks as a sequence-to-sequence generation task. It proposes APICKnow, an API entity-relation joint extraction model based on the large language model. To improve our model’s performance and quick learning ability, we adopt the prompt learning method to stimulate APICKnow to recognize API entities and relations. We systematically evaluate APICKnow on a set of sentences from Stack Overflow. The experimental results show that APICKnow can outperform the state-of-the-art baselines, and APICKnow has a quick learning ability and strong generalization ability.
Study on Volleyball-Movement Pose Recognition Based on Joint Point Sequence

With the high-speed operation of society and the increasing development of modern science, people's quality of life continues to improve. Contemporary people are increasingly concerned about their quality of life, pay attention to body management, and strengthen physical exercise. Volleyball is a sport that is loved by many people. Studying volleyball postures and recognizing and detecting them can provide theoretical guidance and suggestions for people. Besides, when it is applied to competitions, it can also help the judges to make fair and reasonable decisions. At present, pose recognition in ball sports is challenging in action complexity and research data. Meanwhile, the research also has an important application value. Therefore, this article studies human volleyball pose recognition by combining the analysis and summary of the existing human pose recognition studies based on joint point sequences and long short-term memory (LSTM). This article proposes a data preprocessing method based on the angle and relative distance feature enhancement and a ball-motion pose recognition model based on LSTM-Attention. The experimental results show that the data preprocessing method proposed here can further improve the accuracy of gesture recognition. For example, the joint point coordinate information of the coordinate system transformation significantly improves the recognition accuracy of the five ball-motion poses by at least 0.01. In addition, it is concluded that the LSTM-attention recognition model is not only scientific in structure design but also has considerable competitiveness in gesture recognition performance.
Decentralized Federated Learning: A Defense Against Gradient Inversion Attack

Federated learning (FL) is a machine learning technique that enables data to be stored and calculated on geographically distributed local clients. In centralized FL, there is an orchestrating system server responsible for aggregating local client parameters. Such a design is vulnerable to gradient inversion attacks where a malicious central server can restore the client's data through the model gradients. This paper proposes a Decentralized Federated Learning (DFL) method to mitigate the gradient inversion attack. We design a federated learning framework in a decentralized structure, where only peer-to-peer communication is adopted to transfer model parameters for aggregating and updating local models. Extensive experiments and detailed case studies are conducted on a real dataset, through which we demonstrate that the proposed DFL mechanism has excellent performance and is resistant to gradient inversion attack.