Reviewerly/exHarmony huggingface dataset:
The incremental online k-means clustering algorithm and its application to color quantization

Color quantization is a common image processing operation with various applications in computer graphics, image processing, and computer vision. Color quantization is essentially a large-scale combinatorial optimization problem. Many clustering algorithms, both of hierarchical and partitional types, have been applied to this problem since the 1980s. In general, hierarchical color quantization algorithms are faster, whereas partitional ones produce better results provided that they are initialized properly. In this paper, we propose a novel partitional color quantization algorithm based on a binary splitting formulation of MacQueen’s online k-means algorithm. Unlike MacQueen’s original algorithm, the proposed algorithm is both deterministic and free of initialization. Experiments on a diverse set of public test images demonstrate that the proposed algorithm is significantly faster than two popular batch k-means algorithms while yielding nearly identical results. In other words, unlike previously proposed k-means variants, our algorithm addresses both the initialization and acceleration issues of k-means without sacrificing the simplicity of the algorithm. The presented algorithm may be of independent interest as a general-purpose clustering algorithm.
What do they capture?

Recently, many pre-trained language models for source code have been proposed to model the context of code and serve as a basis for downstream code intelligence tasks such as code completion, code search, and code summarization. These models leverage masked pre-training and Transformer and have achieved promising results. However, currently there is still little progress regarding interpretability of existing pre-trained code models. It is not clear why these models work and what feature correlations they can capture. In this paper, we conduct a thorough structural analysis aiming to provide an interpretation of pre-trained language models for source code (e.g., CodeBERT, and GraphCodeBERT) from three distinctive perspectives: (1) attention analysis, (2) probing on the word embedding, and (3) syntax tree induction. Through comprehensive analysis, this paper reveals several insightful findings that may inspire future studies: (1) Attention aligns strongly with the syntax structure of code. (2) Pre-training language models of code can preserve the syntax structure of code in the intermediate representations of each Transformer layer. (3) The pre-trained models of code have the ability of inducing syntax trees of code. Theses findings suggest that it may be helpful to incorporate the syntax structure of code into the process of pre-training for better code representations.
A Deep Learning-Based Cryptocurrency Price Prediction Model That Uses On-Chain Data

Cryptocurrency has recently attracted substantial interest from investors due to its underlying philosophy of decentralization and transparency.Considering cryptocurrency's volatility and unique characteristics, accurate price prediction is essential for developing successful investment strategies.To this end, the authors of this work propose a novel framework that predicts the price of Bitcoin (BTC), a dominant cryptocurrency.For stable prediction performance in unseen price range, the change point detection technique is employed.In particular, it is used to segment time-series data so that normalization can be separately conducted based on segmentation.In addition, on-chain data, the unique records listed on the blockchain that are inherent in cryptocurrencies, are collected and utilized as input variables to predict prices.Furthermore, this work proposes self-attention-based multiple long short-term memory (SAM-LSTM), which consists of multiple LSTM modules for on-chain variable groups and the attention mechanism, for the prediction model.Experiments with real-world BTC price data and various method setups have proven the proposed framework's effectiveness in BTC price prediction.The results are promising, with the highest MAE, RMSE, MSE, and MAPE values of 0.3462, 0.5035, 0.2536, and 1.3251, respectively.
Weakly Alignment-Free RGBT Salient Object Detection With Deep Correlation Network

RGBT Salient Object Detection (SOD) focuses on common salient regions of a pair of visible and thermal infrared images. Existing methods perform on the well-aligned RGBT image pairs, but the captured image pairs are always unaligned and aligning them requires much labor cost. To handle this problem, we propose a novel deep correlation network (DCNet), which explores the correlations across RGB and thermal modalities, for weakly alignment-free RGBT SOD. In particular, DCNet includes a modality alignment module based on the spatial affine transformation, the feature-wise affine transformation and the dynamic convolution to model the strong correlation of two modalities. Moreover, we propose a novel bi-directional decoder model, which combines the coarse-to-fine and fine-to-coarse processes for better feature enhancement. In particular, we design a modality correlation ConvLSTM by adding the first two components of modality alignment module and a global context reinforcement module into ConvLSTM, which is used to decode hierarchical features in both top-down and button-up manners. Extensive experiments on three public benchmark datasets show the remarkable performance of our method against state-of-the-art methods.
Multiple Events Detection Using Context-Intelligence Features

Event detection systems are mainly used to observe and monitor human behavior via red green blue (RGB) images and videos. Event detection using RGB images is one of the challenging tasks of the current era. Human detection, position and orientation of human body parts in RGB images is a critical phase for numerous systems models. In this research article, the detection of human body parts by extracting context-aware energy features for event recognition is described. For this, silhouette extraction, estimation of human body parts, and context-aware features are extracted. To optimize the context-intelligence vector, we applied an artificial intelligence-based self-organized map (SOM) while a genetic algorithm (GA) is applied for multiple event detection. The experimental results on challenging RGB images and video-based datasets were promising. Three datasets were used. Event recognition and body parts detection accuracy rates for the University of central Florida’s (UCF) dataset were 88.88% and 86.75% respectively. 90.0% and 87.37% for event recognition and body parts detection were achieved over the University of Texas (UT) dataset. 87.89% and 85.87% for event recognition and body parts detection were achieved for the sports videos in the wild (SVW) dataset. The proposed system performs better than other current state-of-the-art approaches in terms of body parts and event detection and recognition outcomes.
Gaussian Focal Loss: Learning Distribution Polarized Angle Prediction for Rotated Object Detection in Aerial Images

With the increasing availability of aerial data, object detection in aerial images has aroused more and more attention in remote sensing community. The difficulty lies in accurately predicting the angular information for each target when using the oriented bounding boxes to represent the arbitrary oriented objects, as the periodicity of the angle could cause inconsistency between target angle values. To resolve the problem, recent works propose to perform angular prediction from a regression problem to a classification task with circular smooth label. However, we find that current loss functions applying to binary soft labels need to approximate the soft label values at each position. When summed over all the negative angle categories, these relatively insignificant loss values can overwhelm the target angle category, thus preventing the network from predicting precise angle information. In this paper, we propose a novel loss function that acts as a more effective alternative to the classification-based rotated detectors. By constructing the classification loss with adaptive Gaussian attenuation on the negative locations, our training objective can not only avoid discontinuous angle boundaries but also enable the network to obtain more accurate angle predictions with higher response at peaks. Moreover, an aspect ratio-aware factor was proposed based on our loss function to enhance the robustness of the model for determining the orientation for square-like objects. Extensive experiments on aerial image datasets DOTA, HRSC2016, and UCAS-AOD demonstrated the effectiveness and superior performances of our approaches.
Mean-Field Learning for Edge Computing in Mobile Blockchain Networks

Blockchain has been leveraged to secure transactions for the m-commerce. However, the intensive computation in the mining process restricts the participation of mobile devices. Currently, some studies have deployed edge computing services to support the mining process, where edge servers managed by one Service Provider (SP) are considered. This paper investigates a more practical scenario with multiple SPs, where servers managed by different SPs have distinct capacities and prices, making miners’ offloading decisions rather complicated. To tackle the above challenges, we consider task offloading, block propagation and miner mobility comprehensively to maximize utilities of miners. Specifically, we first formulate a Markov game, and then design a learning-based offloading algorithm for off-chain computation, where a novel learning model is constructed by integrating Deep Reinforcement Learning (DRL) and Mean Field Theory (MFT) to guarantee a Nash equilibrium. Different from existing studies, each miner merely needs to respond to the average effect from others in our system, insteading of knowing policies of others. Finally, both theoritical and performance results show that our designed algorithm has superiority on average miner utilities and algorithm convergence time compared with other representative algorithms.
High-Precision Bootstrapping for Approximate Homomorphic Encryption by Error Variance Minimization

The Cheon-Kim-Kim-Song (CKKS) scheme (Asiacrypt’17) is one of the most promising homomorphic encryption (HE) schemes as it enables privacy-preserving computing over real (or complex) numbers. It is known that bootstrapping is the most challenging part of the CKKS scheme. Further, homomorphic evaluation of modular reduction is the core of the CKKS bootstrapping. As modular reduction is not represented by the addition and multiplication of complex numbers, approximate polynomials for modular reduction should be used. The best-known techniques (Eurocrypt’21) use a polynomial approximation for trigonometric functions and their composition. However, all the previous methods are based on an indirect approximation, and thus it requires lots of multiplicative depth to achieve high accuracy. This paper proposes a direct polynomial approximation of modular reduction for CKKS bootstrapping, which is optimal in error variance and depth. Further, we propose an efficient algorithm, namely the lazy baby-step giant-step (BSGS) algorithm, to homomorphically evaluate the approximate polynomial, utilizing the lazy relinearization/rescaling technique. The lazy-BSGS reduces the computational complexity by half compared to the ordinary BSGS algorithm. The performance improvement for the CKKS scheme by the proposed algorithm is verified by implementation using HE libraries. The implementation results show that the proposed method has a multiplicative depth of 10 for modular reduction to achieve the state-of-the-art accuracy, while the previous methods have depths of 11 to 12. Moreover, we achieve higher accuracy within a small multiplicative depth, for example, 93-bit within multiplicative depth 11.
Impact of Block Data Components on the Performance of Blockchain-Based VANET Implemented on Hyperledger Fabric

Blockchain, a vital technology in today's era, changed the way we share, manage and exchange our data in a centralized way to decentralized architecture.With the increasing demand for Blockchain, various platforms are available to implement public, private, consortium, or permissioned, permissionless Blockchain.Hyperledger, an open-source, permissioned, distributed ledger-based Blockchain, was hosted by Linux.This paper explores Hyperledger Fabric Private Blockchain Network (HFPBN).The architecture of HFPBN with its components and transaction flow is explored in detail.The Blockchain in HFPBN comprises multiple blocks that are linked to each other.The block elements are discussed in detail with their type and size, and after that, the total size of the block depending upon various parameters is calculated.Further, one application of Blockchain, i.e., Vehicular Ad-hoc Networks (VANETs), is discussed in this paper as a case study.The VANET application is being implemented on the Hyperledger Fabric platform.Formulas showing the dependency of various parameters like endorsement policy, number of transactions, and number of reads and writes on block size are derived and shown in their relationship through the graph for the VANET system.The impact of block size on various performance parameters like throughput, latency, memory, and CPU utilization for the VANET system is then analyzed using Hyperledger Caliper.An optimal required value of throughput and latency is achieved for Blockchainbased VANET.Also, the Hyperledger Fabric platform seems suitable for many applications as it creates separate Blockchain for different applications.
LVE-S2D: Low-Light Video Enhancement From Static to Dynamic

Recently, deep-learning-based low-light video enhancement methods have drawn wide attention and achieved remarkable performance. However, limited by the difficulty in collecting dynamic low-light and well-lighted video pairs in real scenes, how to construct video sequences for supervised learning and design a low-light enhancement network for real dynamic video remains a challenge. In this paper, we propose a simple yet effective low-light video enhancement method (LVE-S2D), which generates dynamic video training pairs from static videos, and enhances the low-light video by mining dynamic temporal information. To obtain low-light and well-lighted video pairs, a sliding window-based dynamic video generation mechanism is designed to produce pseudo videos with rich dynamic temporal information. Then, a siamese dynamic low-light video enhancement network is presented, which effectively utilizes temporal correlation between adjacent frames to enhance the video frames. Extensive experimental results demonstrate that the proposed method not only achieves superior performance on static low-light videos, but also outperforms the state-of-the-art methods on real dynamic low-light videos.
AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation

Recently, medical report generation, which aims to automatically generate a long and coherent descriptive paragraph of a given medical image, has received growing research interests. Different from the general image captioning tasks, medical report generation is more challenging for data-driven neural models. This is mainly due to 1) the serious data bias: the normal visual regions dominate the dataset over the abnormal visual regions, and 2) the very long sequence. To alleviate above two problems, we propose an AlignTransformer framework, which includes the Align Hierarchical Attention (AHA) and the Multi-Grained Transformer (MGT) modules: 1) AHA module first predicts the disease tags from the input image and then learns the multi-grained visual features by hierarchically aligning the visual regions and disease tags. The acquired disease-grounded visual features can better represent the abnormal regions of the input image, which could alleviate data bias problem; 2) MGT module effectively uses the multi-grained features and Transformer framework to generate the long medical report. The experiments on the public IU-Xray and MIMIC-CXR datasets show that the AlignTransformer can achieve results competitive with state-of-the-art methods on the two datasets. Moreover, the human evaluation conducted by professional radiologists further proves the effectiveness of our approach.
Anchor Retouching via Model Interaction for Robust Object Detection in Aerial Images

Object detection has made tremendous strides in computer vision. Small object detection with appearance degradation is a prominent challenge, especially for aerial observations. To collect sufficient positive/negative samples for heuristic training, most object detectors preset region anchors in order to calculate Intersection-over-Union (IoU) against the ground-truthed data. In this case, small objects are frequently abandoned or mislabeled. In this paper, we present an effective Dynamic Enhancement Anchor (DEA) network to construct a novel training sample generator. Different from the other state-of-the-art techniques, the proposed network leverages a sample discriminator to realize interactive sample screening between an anchor-based unit and an anchor-free unit to generate eligible samples. Besides, multi-task joint training with a conservative anchor-based inference scheme enhances the performance of the proposed model while reducing computational complexity. The proposed scheme supports both oriented and horizontal object detection tasks. Extensive experiments on two challenging aerial benchmarks (i.e., DOTA and HRSC2016) indicate that our method achieves state-of-the-art performance in accuracy with moderate inference speed and computational overhead for training. On DOTA, our DEA-Net which integrated with the baseline of RoI-Transformer surpasses the advanced method by 0.40% mean-Average-Precision (mAP) for oriented object detection with a weaker backbone network (ResNet-101 vs ResNet-152) and 3.08% mean-Average-Precision (mAP) for horizontal object detection with the same backbone. Besides, our DEA-Net which integrated with the baseline of ReDet achieves the state-of-the-art performance by 80.37%. On HRSC2016, it surpasses the previous best model by 1.1% using only 3 horizontal anchors.
Privacy-Preserving Deep Learning With Homomorphic Encryption: An Introduction

Privacy-preserving deep learning with homomorphic encryption (HE) is a novel and promising research area aimed at designing deep learning solutions that operate while guaranteeing the privacy of user data.Designing privacypreserving deep learning solutions requires one to completely rethink and redesign deep learning models and algorithms to match the severe technological and algorithmic constraints of HE.This paper provides an introduction to this complex research area as well as a methodology for designing privacy-preserving convolutional neural networks (CNNs).This methodology was applied to the design of a privacy-preserving version of the well-known LeNet-1 CNN, which was successfully operated on two benchmark datasets for image classification.Furthermore, this paper details and comments on the research challenges and software resources available for privacy-preserving deep learning with HE.
Consortium Blockchain-Based Spectrum Trading for Network Slicing in 5G RAN: A Multi-Agent Deep Reinforcement Learning Approach

Network slicing (NS) is envisioned as an emerging paradigm for accommodating different virtual networks on a common physical infrastructure. Considering the integration of blockchain and NS, a secure decentralized spectrum trading platform can be established for autonomous radio access network (RAN) slicing. Moreover, the realization of proper incentive mechanisms for fair spectrum trading is crucial for effective RAN slicing. This paper proposes a novel hierarchical framework for blockchain-empowered spectrum trading for NS in RAN. Specifically, we deploy a consortium blockchain platform for spectrum trading among spectrum providers and buyers for slice creation, and autonomous slice adjustment. For slice creation, the spectrum providers are infrastructure providers (InPs) and buyers are mobile virtual network operators (MVNOs). Then, underloaded MVNOs with extra spectrum to spare, trade with overloaded MVNOs, for slice spectrum adjustment. For proper incentive maximization, we propose a three-stage Stackelberg game framework among InPs, seller MVNOs, and buyer MVNOs, for joint optimal pricing and demand prediction strategies. Then, a multi-agent deep reinforcement learning (MADRL) method is designed to achieve a Stackelberg equilibrium (SE). Security assessment and extensive simulation results confirm the security and efficacy of our proposed method in terms of players’ utility maximization and fairness, compared with other baselines.
Unified End-to-End YOLOv5-HR-TCM Framework for Automatic 2D/3D Human Pose Estimation for Real-Time Applications

Three-dimensional human pose estimation is widely applied in sports, robotics, and healthcare. In the past five years, the number of CNN-based studies for 3D human pose estimation has been numerous and has yielded impressive results. However, studies often focus only on improving the accuracy of the estimation results. In this paper, we propose a fast, unified end-to-end model for estimating 3D human pose, called YOLOv5-HR-TCM (YOLOv5-HRet-Temporal Convolution Model). Our proposed model is based on the 2D to 3D lifting approach for 3D human pose estimation while taking care of each step in the estimation process, such as person detection, 2D human pose estimation, and 3D human pose estimation. The proposed model is a combination of best practices at each stage. Our proposed model is evaluated on the Human 3.6M dataset and compared with other methods at each step. The method achieves high accuracy, not sacrificing processing speed. The estimated time of the whole process is 3.146 FPS on a low-end computer. In particular, we propose a sports scoring application based on the deviation angle between the estimated 3D human posture and the standard (reference) origin. The average deviation angle evaluated on the Human 3.6M dataset (Protocol #1-Pro #1) is 8.2 degrees.
Machine translation and foreign language education

Online machine translation tools have great potential to transform foreign language education. This essay will synthesize systematic research on the role of machine translation conducted in the field of educational linguistics. After describing approaches developed that promote the integration of machine translation into language learning environments, the essay will briefly outline lingering concerns associated with the integration of MT tools into educational settings. We will propose future R&D priorities that can generate products based on existing technologies that have the potential to support language learners more optimally compared to existing machine translation tools. We conclude that an acknowledgment of the difficulties of MT tools to handle socio-culturally complex source text would pave the way for the development of MT-based pedagogical tools.