Reviewerly/exHarmony huggingface dataset:
Research Paper Classification and Recommendation System based-on Fine-Tuning BERT

In this paper, we compare the performance of two popular NLP models, pre-train fine-tuned BERT and BiLSTM with combined CNN, in terms of the classification and recommendation tasks of research papers. We conduct the performance evaluation of these two models with research journal benchmark dataset. Performance results show that the pre-train fine-tuned BERT model is superior to CNN-BiLSTM combined model in terms of classification performance.
Improving the Reusability of Pre-trained Language Models in Real-world Applications

The reusability of state-of-the-art Pre-trained Language Models (PLMs) is often limited by their generalization problem, where their performance drastically decreases when evaluated on examples that differ from the training dataset, known as Out-of-Distribution (OOD)/unseen examples. This limitation arises from PLMs’ reliance on spurious correlations, which work well for frequent example types but not for general examples. To address this issue, we propose a training approach called Mask-tuning, which integrates Masked Language Modeling (MLM) training objectives into the fine-tuning process to enhance PLMs’ generalization. Comprehensive experiments demonstrate that Mask-tuning surpasses current state-of-the-art techniques and enhances PLMs’ generalization on OOD datasets while improving their performance on in-distribution datasets. The findings suggest that Mask-tuning improves the reusability of PLMs on unseen data, making them more practical and effective for real-world applications.
Ensemble Learning for UAV Detection: Developing a Multi-Class Multimodal Dataset

Unmanned aerial vehicles (UAVs) are a growing threat to public safety if used maliciously. In this study, we present our multimodal data set containing image, audio, and radio frequency (RF) data, which can serve as a valuable resource for researchers and developers in the field of UAV detection. We present a multiclass multimodal ensemble approach to address the need to improve UAV identification and detection. Our approach is novel as we integrated multiple deep-learning classifiers into a single ensemble classifier. We evaluate the performance of our proposed solution with a hard-voting model and a soft-voted model to evaluate the effectiveness of the proposed solution. Overall, our ensemble approach performed better than the single-modality classifier and when combined, could mitigate the low accuracy of the RF (CNN) accuracy score of 67%. This study has shown how effective ensemble approaches can be used to mitigate limitations when predicting UAV based on multimodal signatures.
Speech-to-speech Low-resource Translation

Speech-to-speech translation (S2ST), particularly in the context of low-resource languages, plays a vital role in facilitating global communication. However, comprehensive research in this emerging field is lacking, especially concerning translation without the use of text. The objective of this study is to bridge the gap by conducting a systematic review of existing literature on S2ST for low-resource languages. We discovered 455 articles by searching the Scopus, IEEE Xplore, and ACM Digital Library databases, focusing on identifying research trends. The results highlight significant topics covered in the literature, marking a transition from traditional neural network methodologies to advanced transformer-based models. Our findings provide a robust overview of the S2ST landscape, identifying challenges and potential solutions for future research, particularly regarding the application of this technology in low-resource settings. The research contribution of this study is the insights gleaned will benefit academics and professionals seeking a comprehensive understanding of S2ST for low-resource languages.
Meme it Up: Patterns of Emoji Usage on Twitter

As emojis have grown in their popularity in social media over the last decades, they not only enrich messaging with emotional connotations but offer a convenient system for studying the effects of memes on ideas’ survival. In this work, we treat emojis like standardized memes to test the impact of their usage on different facets of success within social media. Specifically, we extracted random individual tweets from Twitter to construct a list of emojis used within each tweet. With this dataset, we aimed to address three distinct questions: (1) whether there are specific patterns of emoji usage that increase tweet popularity; (2) whether emojis usage on tweeter can be a good predictor of the stock market trading volume; and (3) whether there is a specific subset of emojis associated with low-quality tweets (e.g., spam). We found no evidence of the positive effects of emoji usage on tweet popularity. However, there was a reason to claim that negative emojis may trigger an intensive response from the audience. For some companies, we were able to accurately predict stock patterns based on emoji usage. Finally, there clearly was a specific subset of emojis used in low-quality tweets. This work may serve as a starting point for a deep investigation of the emoji-meme system, as this topic seems to be relatively fresh in the literature.
Enhancing Model Explainability in Financial Trading Using Training Aid Samples: A CNN-Based Candlestick Pattern Recognition Approach

Candlestick pattern recognition is a widely adopted technique in financial trading, leveraging visual patterns to analyze price movements. Deep Convolutional Neural Networks (CNNs) have exhibited remarkable accuracy in this domain. However, the increasing demand for transparency and explainability in CNN-based models raises concerns regarding their applicability in trading decision-making. This paper addresses these concerns by presenting a framework that enhances the explainability of CNN-based candlestick pattern recognition models. Our approach introduces an innovative data augmentation method to generate training aid samples, facilitating the model’s learning process within human domains. By incorporating this framework, traders gain valuable insights into the decision-making process, comprehending the rationale behind the model’s predictions. Our proposed approach exposes the inherent “black box” nature of CNN-based models, improving their interpretability and empowering traders to make well-informed decisions based on transparent and understandable insights. This advancement holds significant potential for enhancing decision-making processes in financial trading and fostering trust among traders.
Old Dog, New Trick: Reservoir Computing Advances Machine Learning for Climate Modeling

Abstract Physics‐informed machine learning (ML) applied to geophysical simulation is developing explosively. Recently, graph neural net and vision transformer architectures have shown 1–7 days global weather forecast skill superior to any conventional model with integration times over 1,000 times faster, but longer simulations rapidly degrade. ML that achieves high skill in both weather and climate applications is a tougher goal. This Commentary was inspired by Arcomano et al. (2023, https://doi.org/10.1029/2022GL102649 ), who show impressive progress toward that goal using hybrid ML, combining reservoir computing (RC) to a coarse‐grid climate model and coupling to a separate data‐driven RC model that interactively predicts sea‐surface temperature. This opens new horizons; where will the next ML breakthrough come from, and is conventional climate modeling about to be disrupted?
Federated Learning in Computer Vision

Federated Learning (FL) has recently emerged as a novel machine learning paradigm allowing to preserve privacy and to account for the distributed nature of the learning process in many real-world settings. Computer vision tasks deal with huge datasets often with critical privacy issues, therefore many federated learning approaches have been presented to exploit its distributed and privacy-preserving nature. Firstly, this paper introduces the different FL settings used in computer vision and the main challenges that need to be tackled. Then, it provides a comprehensive overview of the different strategies used for FL in vision applications and presents several different approaches for image classification, object detection, semantic segmentation and for focused settings in face recognition and medical imaging. For the various approaches the considered FL setting, the employed data and methodologies and the achieved results are thoroughly discussed.
Optimizing Reinforcement Learning Control Model in Furuta Pendulum and Transferring it to Real-World

Reinforcement learning does not require explicit robot modeling as it learns on its own based on data, but it is temporal and spatial constraints when transfer it to real-world environments. In this research, we train a balancing Furuta pendulum problem, which is difficult to modeling, in a virtual environment (Unity) and transfer to real-world. The balancing Furuta pendulum problem is maintaining balance of the pendulum’s end effector at a vertical position. We resolved the temporal and spatial by performing reinforcement learning in a virtual environment. Furthermore, we design a novel reward function that enabled faster and more stable problem-solving compared to existing two reward functions. We validate each reward function by applying it to soft actor critic (SAC) and proximal policy optimization (PPO). The experimental result shows that cosine reward function is trained faster and more stable. Finally, SAC algorithm model using a cosine reward function in the virtual environment is an optimized controller. Additionally, we evaluated the robustness of this model by transferring it to the real environment.
Fast and Scalable Multicore YOLOv3-Tiny Accelerator Using Input Stationary Systolic Architecture

This article proposes a scalable accelerator for deep learning (DL) implementation on edge computing, which is often limited by power, storage, and computation speed. The accelerator is based on systolic array cores with 126 processing elements (PEs) and optimized for YOLOv3-Tiny with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$448\times448$ </tex-math></inline-formula> input images. Two multicast (MC) network architectures, feature map multicasting and weight multicasting, are introduced to control data stream distribution within the multicores. Results show that the proposed weight multicast (W-MC) systems outperformed the feature map multicast (FMAP-MC) systems in multicore scenarios, with up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.23\times $ </tex-math></inline-formula> frame rates per second (FPS). The 4-core W-MC system achieved the best efficiency with an overall frame rate of 13.73 FPS/W and an overall throughput of 35.83 GOPS/W. The 8-core W-MC system delivered the best performance, with a frame rate of 38.50 FPS after normalization to the standard YOLOv3-Tiny network. The proposed accelerator offers better computational efficiency and greater accelerator utilization in real-world inference scenarios, compared to previous state-of-the-art works.
A Controllable Pipeline Framework of Block Ciphers on GPU for Streaming Data

With the application and development of real-time multimedia information technology such as online meetings and short videos, the demand for secure transmission of streaming media data based on high throughput and low latency has increased significantly. Since servers must encrypt large amounts of data in real-time without compromising their service capabilities, Graphic Processing Units (GPUs) are considered the best candidates for cryptographic accelerators to handle large amounts of data in this situation. For the characteristics of streaming data, this paper proposes a GPU-based controllable pipelined encryption framework, which guarantees security during streaming data transmission while satisfying the requirements of high throughput and low latency. Firstly, in order to make the CPU in the traditional heterogeneous system understand the intermediate state of the GPU’s operation in real-time, this paper adopts the cyclic checkpoint mechanism to realize the interrupt control; secondly, real-time signal feedback between the host and the device each other during the execution of the kernel function is realized by real-time monitoring and other modules; through the above ways, the CPU-GPU pipeline control framework is realized. Finally, based on AES and SM4 encryption algorithms, this paper designs and realizes the pipeline encryption and decryption process of streaming data based on CUDA in Python environment, and verifies the group automation pipeline framework based on interrupt monitoring and controlling through experiments; and under the condition of using 5 threads grouping, the data throughput rate reaches 1.01Gbps, which has a high throughput rate.
Digital Twin Enabled Asynchronous SplitFed Learning in E-Healthcare Systems

The advancement of Industrial Internet of Things (IIoT) technology has resulted in the fourth industrial revolution, or Industry 4.0, enabling industries to enhance productivity. However, despite the benefits, there remain significant challenges, such as resource heterogeneity, communication efficiency, and data privacy, that limit the applications of IIoT in privacy-sensitive domains like healthcare. In order to protect data privacy, Federated Learning (FL) has been suggested as a solution, involving the sharing of model parameters rather than data itself. Current FL applications, however, still struggle with cost efficiency, especially when IIoT devices with heterogenous resources are involved. To address this, this paper proposes Digital Twin (DT) enabled Asynchronous SplitFed Learning (DT-ASFL) for classification tasks in the e-healthcare system over mobile networks. We first develop SplitFed Learning to introduce communication efficiency in the e-healthcare system, sending only extracted features during the learning process instead of the entire learning model. This enables resource-constrained devices to participate in the learning process by allowing the participants to train a partial learning model. DT is then employed to provide real-time statuses of IIoT devices deployed in the system, enabling asynchronous model updates in SplitFed Learning. The experimental results demonstrate the efficacy of DT-ASFL compared to the existing methods.
Improved Box Embeddings for Fine-Grained Entity Typing

Different from traditional vector-based fine-grained entity typing methods, the box-based method is more effective in capturing the complex relationships between entity mentions and entity types. The box-based fine-grained entity typing method projects entity types and entity mentions into high-dimensional box space, where entity types and entity mentions are embedded as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> -dimensional hyperrectangles. However, the impacts of entity types are not considered during classification in high-dimensional box space, and the model cannot be optimized precisely when two boxes are completely separated or overlapped in high-dimensional box space. Based on the above shortcomings, an <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> mproved <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</b> ox <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> mbeddings (IBE) method for fine-grained entity typing is proposed in this work. The IBE not only introduces the impacts of entity types during classification in high-dimensional box space, but also proposes a distance based module to optimize the model precisely when two boxes are completely separated or overlapped in high-dimensional box space. Experimental results on four fine-grained entity typing datasets verify the effectiveness of the proposed IBE, demonstrating that IBE is a state-of-the-art method for fine-grained entity typing.
NBSync: Parallelism of Local Computing and Global Synchronization for Fast Distributed Machine Learning in WANs

Recently, due to privacy concerns, distributed machine learning in Wide-Area Networks (DML-WANs) attracts increasing attention and has been widely deployed to promote the widespread application of intelligence services that rely on geographically distributed data. DML-WANs is essentially performing collaboratively federated learning over a combination of servers at both edge and cloud on a large spatial scale. However, efficient model training is challenging for DML-WANs because it is blocked by the high overhead of model parameter synchronization between computing servers over WANs. The reason is that there has a sequential dependency between local model computing and global model synchronization of traditional DML-WANs training methods intrinsically producing a sequential blockage between them, e.g., FedAvg. When the computing heterogeneity and the low WAN bandwidth coexist, a long block of global model synchronization prolongs the training time and leads to low utilization of local computing. Despite many efforts on alleviating synchronization overhead with novel communication technologies and synchronization methods, they still use traditional training patterns with sequential dependency and thereby have very limited improvements, such as FedAsync and ESync. In this article, we propose NBSync, a novel training algorithm for DML-WANs, which greatly speeds up the model training by the parallelism of local computing and global synchronization. NBSync employs a well-designed pipelining scheme, which can properly relax the sequential dependency of local computing and global synchronization and process them in parallel so as to overlap their operating overhead in the time dimension. NBSync also realizes flexible, differentiated and dynamical local computing for workers to maximize the overlap ratio in dynamically heterogeneous training environments. Convergence analysis shows that the convergence rate of NBSync training process is asymptotically equal to that of SSGD, and NBSync has a better convergence efficiency. We implemented the prototype of NBSync based on a popular parameter server system, i.e., MXNET's PS-LITE library, and evaluate its performance on a DML-WANs testbed. Experimental results show that NBSync speeds up training about 1.43×–2.79× than state-of-the-art distributed training algorithms (DTAs) in DML-WANs scenarios where computing heterogeneity and low WAN bandwidth coexist.
An Improved Data Privacy Diagnostic Framework for Multiple Machinery Components Data Based on Swarm Learning Algorithm

The continuous operation of the equipment degrades the performance of the critical parts, which can cause the equipment to fail and stop at a particular and unexpected moment. Timely diagnosis of the equipment is vital for condition monitoring and maintenance. Due to the small amount of data and data collection limitation, it is difficult to train an efficient diagnosis model for real-time tracking within only one piece of equipment data. This study proposes an improved data privacy diagnostic framework for multiple same type of machinery components, solving the insufficient data, data protection, and multiple machines’ fault information exchange. First, the swarm learning framework integrates various data sources to enrich the data contained within a solo diagnosis network. Second, the different training nodes utilize various local diagnosis models to improve data protection efficiency further and realize the interaction of data information. Third, we developed three different local diagnosis models which can mutually exchange the partially faulty information with each other to make up for the inefficiency of model diagnosis caused by single insufficient data. The experimental demonstration is conducted on the bearing fault datasets, proving that the proposed method can be more flexible and reliable in more industrial scenarios.
Artificial Bee Colony Based on Adaptive Search Strategies and Elite Selection Mechanism

In the field of optimization algorithms, artificial bee colony algorithm (ABC) shows strong search ability on many optimization problems. However, ABC still has a few shortcomings. It exhibits weak exploitation and slow convergence. In the late search stage, the original probability selection for onlooker bees may not work. Due to the above deficiencies, a modified ABC using adaptive search strategies and elite selection mechanism (namely ASESABC) is presented. Firstly, a strategy pool is created using three different search strategies. A tolerance-based strategy selection method is used to select a sound search strategy at each iteration. Then, to choose better solutions for further search, an elite selection means is utilized in the stage of onlooker bees. To examine the capability of ASESABC, 22 classical benchmark functions are tested. Results show ASESABC surpasses five other ABCs according to the quality of solutions.
MM-VTON: A Multi-stage Virtual Try-on Method Using Multiple Image Features

Virtual try-on allows users to see how they look without actually trying the clothes on during their purchase. This technology has numerous applications in the display of clothing effects and is especially useful during the pandemic, because it enables remote try-on without physical contact. The major limitations of current virtual try-on methods, however, lie in the difficulty of addressing clothing deformation, edge synthesis, etc. In this study, we present a new three-stage virtual try-on method to reduce the reliance on clothing regions in human images. To achieve this, we design a new semantic prediction module to fully remove clothing-related information from human images. Additionally, we introduce a new try-on module to fuse the extracted features using an adversarial loss, resulting in significant improvements on the try-on image quality. Experimental results have demonstrated the effectiveness of our method, which achieves competitive results in comparison to state-of-the-art methods.
Dynamic Multi-objective Prediction Strategy for Transfer Learning Based on Imbalanced Data Classification

Prediction strategies based on transfer learning have been proved to be effective in solving Dynamic Multi-Objective Optimization Problems (DMOPs). However, slow running speed impedes the development of this method. To address this issue, this paper proposes a transfer learning method based on imbalanced data classification and combines it with a decomposition-based multi-objective optimization algorithm, referred to as ICTr-MOEA/D. This method combines the prediction strategies based on transfer learning with knee points to save computational resources. In order to prevent the prediction accuracy from being affected by the insufficient of knee points, ELM classifier selects more high-quality points from a large number of random solutions. Moreover, SMOTE resolves the class imbalance problem during the ELM training process. The simulation results demonstrate that ICTr-DMOEA shows good competitiveness.