Reviewerly/exHarmony huggingface dataset:
Towards adaptive quality-aware Complex Event Processing in the Internet of Things

This paper investigates how to complement Complex Event Processing (CEP) with dynamic quality monitoring mechanisms and support the dynamic integration of suitable sensory data sources. In the proposed approach, queries to detect complex events are annotated with consumer-definable quality policies that are evaluated and used to autonomously assign (or even configure) suitable data sources of the sensing infrastructure. We present and study different forms of expressing quality policies and explore how they affect the process of quality monitoring including different modes of assessing and applying quality-related adaptations. A performance study in an IoT scenario shows that the proposed mechanisms in supporting quality policy monitoring and adaptively selecting suitable data sources succeed in enhancing the acquired quality of results while fulfilling consumers' quality requirements. We show that the quality-based selection of sensor sources also extends the network's lifetime by optimizing the data sources' energy consumption.
Multi-timescale History Modeling for Temporal Knowledge Graph Completion

Temporal knowledge graph (TKG) has received great attention in recent years. However, the TKG is not always complete due to the missing of important facts, which has seriously hindered its wide application. Inferring missing facts in TKG is a critical and challenging task due to its highly dynamic nature. Most of the existing methods mainly focus on modeling the structural features and temporal dependencies of TKG to solve the temporal knowledge graph completion problem (TKGC). However, those methods only operate at a single timescale without considering the latent time variability of TKG and thus limit the performance of TKGC solutions. Therefore, we propose a novel method named MtGCN (Multi-timescale history modeling framework based on Graph Convolutional Networks) for completing TKG by self-adaptively modeling the multi-timescale history of the incomplete TKG. Firstly, MtGCN uses a structural encoder with a graph convolutional network to mine the latent semantic information and structural features of the TKG. Secondly, MtGCN uses GRU-based temporal encoder to learn the historical information at various timescales of the TKG. Finally, it generates effective entity and relation representations to infer the missing facts for the originally incomplete TKG. By conducting comprehensive experiments on 5 public datasets, the experimental results show that our proposed method MtGCN significantly outperforms the baselines by achieving the highest MRR and HITS@1,3,10.
Design of Facial Expression Recognition Algorithm Based on CNN Model

This paper proposes an improved CNN model with the main process of face image preprocessing, face image feature extraction, input test sample training, acquisition of test sample features, face image feature classification, restoration of face expression images, and recognition results. In the convolution activation function, the cumulative sum operation is performed on three aspects of facial expression image pixels, the number of convolution operations, and the number of pooling operations, and a highly integrated facial expression image feature extraction is realized. It is merged with the pooling operation. In the classification loss function, two processes of convolution and pooling operations are considered, and the weight and bias updates in the convolution, pooling, and classification loss functions are included. Relevant experiments and experimental statistics show that the classification accuracy of the facial image expression feature extraction algorithm proposed in this paper is about 87%, and the recognition accuracy of the improved CNN facial expression recognition model proposed in this paper is about 88%.
Privacy Preserved Federated Learning for Skin Cancer Diagnosis

In recent years, the use of artificial intelligence to assist the medical industry in making more accurate diagnoses has become popular. To achieve a more accurate diagnosis, more data is often needed to constantly improve the model. However, when data needs to be shared by patients from different hospitals, patients may be reluctant to share their privacy with unknown institutions that they do not trust. This situation creates a Data Silo problem that hinders progress in AI diagnostics. Therefore, in this paper, we propose to use the Federated Learning framework to train models with private patient data on local clients and aggregate models on a central server, allowing participants to jointly train models without sharing data and technically breaking down Data Silos and enabling AI collaboration. Experiments demonstrate that compared to traditional CNN training, the CNN model base on the Federated Learning framework sacrifices 8% of accuracy to accomplish the above requirements. Due to the inherent nature of the distributed system, the training time and node communication time is also favorable.
Back-projection Residual Low-light Image Enhancement Network with Color Correction Matrix

Pictures taken in low light conditions suffer from low brightness, feature loss, and color distortion. To address the problem, a Back-projection Residual Network with Color Correction Matrix (BRCCM-Net) for Low-Light Image Enhancement is designed through the following two stages. First, the BRCCM-Net extracts shallow features and inputs into numerous Residual Lightning Back Projection (RLBP) to adjust brightness and learn the residuals of normal light estimations together with the Feature Recalibration module (FRM) to capture and fuse more important feature information, which helps to recover the dark areas of low-light images. Second, a color correction matrix (CCM) model was designed to calibrate luminance and color. The experimental results on the LOL dataset show that the designed network has greater subjective visual effects such as brightness improvement, less noise, color distortion, and sharper and more natural image details. Comparatively, the results for objective indicators (PSNR, SSIM, and MSE) are also superior to those of other existing techniques.
Yoga action recognition based on STF-ResNet

Aiming at the disadvantages of traditional two-stream convolutional networks which are difficult to learn spatial-temporal correlation information and low recognition accuracy, a yoga action recognition algorithm based on Spatial-Temporal Fusion Residual Network (STF-ResNet) was proposed. The spatial-temporal features are complemented by mixing spatial and temporal stream features with residuals, and the information loss of high-level features is compensated by low-level features. The convolutional block attention module (CBAM) is added before mixing to filter the yoga action features again in both channel and spatial dimensions. A convolutional block attention module is added before the fusion to filter the yoga action features again from both channel and spatial dimensions. Experimentally validated on a custom Yoga dataset, the algorithm improves the accuracy of yoga recognition with an average accuracy of 98.6%, a 6.3% improvement compared to traditional two-stream convolutional networks.
Less Is More: Similarity Models for Content-Based Video Retrieval

The concept of object-to-object similarity plays a crucial role in interactive content-based video retrieval tools. Similarity (or distance) models are core components of several retrieval concepts, e.g. Query by Example or relevance feedback. In these scenarios, the common approach is to apply some feature extractor that transforms the object to a vector of features, i.e., positions it into an induced latent space. The similarity is then based on some distance metric in this space. Historically, feature extractors were mostly based on some color histograms or hand-crafted descriptors such as SIFT, but nowadays state-of-the-art tools mostly rely on some deep learning (DL) approaches. However, so far there were no systematic study of how suitable are individual feature extractors in the video retrieval domain. Or, in other words, to what extent are human-perceived and model-based similarities concordant. To fill this gap, we conducted a user study with over 4000 similarity judgements comparing over 20 variants of feature extractors. Results corroborate the dominance of deep learning approaches, but surprisingly favor smaller and simpler DL models instead of larger ones.
Visual–Textual Alignment for Generalizable Person Reidentification in Internet of Things

Person reidentification (re-id) has gained increased attention for its important application in surveillance. Yet one of the major obstacles for the re-id approaches deploying in practical Internet of Things systems is their weak generalization ability. In spite that the current methods have a high performance under supervised setting, their discrimination ability in unseen domains meets decline. Due to the immutability of attributes among different domains, we attempt to exploit the alignment between the pedestrians’ attributes and visual features to enhance our model’s generalization ability. Furthermore, for the existing methods cannot fully extract the attribute information, we formulate a more effective NLP-based method for attribute feature extraction. Thus, the generated features are termed as textual features and our proposed method are called visual–textual alignment (VTA). As for alignment, two strategies are adopted: 1) metric learning-based alignment and 2) adversarial learning-based alignment. The former is designed to adjust the metric relationship of different persons in feature space. And the latter is aimed to guide our model’s domain-invariant feature learning. The experimental results demonstrate the effectiveness and superiority of our proposed method compared to the state-of-the-art methods.
Energy Transfer Contrast Network for Unsupervised Domain Adaption

The main goal of unsupervised domain adaptation is to improve the classification performance on unlabeled data in target domains. Many methods try to reduce the domain gap by treating multiple domains as one to enhance the generalization of a model. However, aligning domains as a whole does not account for instance-level alignment, which might lead to sub-optimal results. Currently, many researchers utilize meta-learning and instance segmentation approaches to tackle this problem. But it can only obtain a further optimized the domain-invariant feature learned by the model, rather than achieve instance-level alignment. In this paper, we interpret unsupervised domain adaptation from a new perspective, which exploits the energy difference between the source and target domains to reduce the performance drops caused by the domain gap. At the same time, we improve and exploit the contrastive learning loss, which can push the target domain away from the decision boundary. The experimental results on different benchmarks against a range of the state-of-the-art approaches justify the performance and the effectiveness of the proposed method.
Serpens: A High Performance FaaS Platform for Network Functions

More and more enterprises deploy applications on Function-as-a-Service (FaaS) platforms to improve resource efficiency and save monetary costs. Network Functions (NFs) suffer from staggered peaks of traffic patterns and could benefit from fine-grained resource multiplexing in FaaS platform. However, naively exploring existing FaaS platforms to support NFs can introduce significant performance overheads in three aspects, including slow instance startup, remote state access for NFs, and costly packet delivery between NFs. To address these problems, we propose <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> , a high performance FaaS platform for NFs. First, <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> proposes a reusable NF runtime design to slash instance startup overhead. Second, <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> designs a novel state management mechanism to support local state access. Third, <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> introduces an advanced service chaining approach to avoid extra packet delivery. Besides, <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> designs an NF scaling mechanism to minimize performance fluctuation. We have implemented a prototype of <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> and conducted comprehensive experiments. Compared with the NFs and Service Function Chains (SFCs) that run on existing FaaS platforms, <inline-formula><tex-math notation="LaTeX">${\sf Serpens}$</tex-math></inline-formula> can improve the throughput by more than 10× and reduce the latency by more than 90%.
Artificial neural network encoding of molecular wavefunctions for quantum computing

Molecular wavefunction encoded with artificial neural networks (BM2, BM3, and RBM) is prepared by the quantum algorithm.
3D-Online Generalized Sensed Shape Expansion: A Probabilistically Complete Motion Planner in Obstacle-Cluttered Unknown Environments

We present an online motion planning algorithm (3D-OGSSE) for generating smooth, collision-free trajectories over multiple planning iterations for a 3-D agent operating in an unknown, obstacle-cluttered, 3-D environment. In each planning iteration, 3D-OGSSE constructs an obstacle-free region termed “generalized sensed shape” based on the locally-sensed environment information and the notion of generalized shape. A collision-free path is computed by sampling points in the generalized sensed shape and is used to generate a smooth, time-parametrized trajectory by minimizing snap. The generated trajectory at every planning iteration is constrained to lie within generalized sensed shape, which ensures the agent maneuvers in locally obstacle-free space. As the agent reaches the boundary of the generalized sensed shape in a planning iteration, a re-plan is triggered by a receding horizon planning mechanism that also enables the initialization of the next planning iteration. We also present a theoretical guarantee for probabilistic completeness of the developed algorithm over the entire environment and for completely collision-free trajectory generation. We evaluate the proposed method in simulation on complex 3-D environments with varied obstacle-densities. Further, we also evaluate it in scenarios with sensor noise and constraints on the on-board sensor's field-of-view (FOV). We observe that each planning iteration computation takes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim 14$</tex-math></inline-formula> milliseconds on a single thread of an Intel Core i5-8500 3.0 GHz CPU, which is significantly faster than several existing algorithms. In addition, we also observe 3D-OGSSE to be less conservative in complex scenarios such as narrow passages.
QP-LDP for better global model performance in federated learning

With the deployment of local differential privacy (LDP), federated learning (FL) has gained stronger privacy-preserving capability against inference-type attacks. However, existing LDP methods reduce global model performance. In this paper, we propose a QP-LDP algorithm for FL to obtain a better-performed global model without losing privacy guarantees defined by the original LDP. Different from previous LDP methods for FL, QP-LDP improves the global model performance by precisely disturbing the non-common components of quantized local contributions. In addition, QP-LDP comprehensively protects two types of local contributions. Through security analysis, QP-LDP provides the probability indistinguishability of clients' private local contributions at a component-level. More importantly, ingenious experiments show that with the deployment of QP-LDP, the global model outperforms that in the original LDP-based FL in terms of prediction accuracy and convergence rate.
DPFed: Toward Fair Personalized Federated Learning with Fast Convergence

Instead of training a single global model to fit the needs of all clients, personalized federated learning aims to train multiple client-specific models to better account for data disparities across participating clients. However, existing solutions suffer from serious unfairness among clients in terms of model accuracy and slow convergence under non-lID data. In this paper, we propose a novel personalized federated learning framework, called D PFed, which employs deep reinforcement learning (D RL) to identify relationship between clients and enable closer collaboration among similar clients. By exploiting such relationships, DPFed can personalize model aggregation for each client and achieve fast convergence. Moreover, by regularizing the reward function of DRL, we can reduce the variance of model accuracy across clients and achieve a higher level of fairness. Finally, we conduct extensive experiments to evaluate the effectiveness of our proposed framework under a variety of datasets and degrees of non-lID data distribution. The results demonstrate that DPFed outperforms other alternatives in terms of convergence speed, model accuracy, and fairness.
A Multi-Stream Fusion Network for Image Splicing Localization

In this paper, we address the problem of image splicing localization with a multi-stream network architecture that processes the raw RGB image in parallel with other handcrafted forensic signals. Unlike previous methods that either use only the RGB images or stack several signals in a channel-wise manner, we propose an encoder-decoder architecture that consists of multiple encoder streams. Each stream is fed with either the tampered image or handcrafted signals and processes them separately to capture relevant information from each one independently. Finally, the extracted features from the multiple streams are fused in the bottleneck of the architecture and propagated to the decoder network that generates the output localization map. We experiment with two handcrafted algorithms, i.e., DCT and Splicebuster. Our proposed approach is benchmarked on three public forensics datasets, demonstrating competitive performance against several competing methods and achieving state-of-the-art results, e.g., 0.898 AUC on CASIA.
MetaSpeech: Speech Effects Switch Along with Environment for Metaverse

Metaverse expands the physical world to a new dimension, and the physical environment and Metaverse environment can be directly connected and entered. Voice is an indispensable communication medium in the real world and Metaverse. Fusion of the voice with environment effects is important for user immersion in Metaverse. In this paper, we proposed using the voice conversion based method for the conversion of target environment effect speech. The proposed method was named MetaSpeech, which introduces an environment effect module containing an effect extractor to extract the environment information and an effect encoder to encode the environment effect condition, in which gradient reversal layer was used for adversarial training to keep the speech content and speaker information while disentangling the environmental effects. From the experiment results on the public dataset of LJSpeech with four environment effects, the proposed model could complete the specific environment effect conversion and outperforms the baseline methods from the voice conversion task.
Next Point-of-Interest Recommendation for Cold-start Users with Spatial-temporal Meta-Learning

Next point-of-interest (POI) recommendation is to recommend the next POI that the users may want to visit according to their check-in trajectories. When there are few check-ins in the trajectories, it will cause the cold-start problem that traditional next POI recommendation methods can't capture users' preferences accurately and no longer has the ability to forecast the next POI. Based on the few-shot meta-learning method of MAML, this paper proposes a new method named STMeta to recommend the next POI for cold-start users. Following few-shot learning, the main idea of the proposed method is to train a model based on the long trajectories of other users with the aim to learn some transferrable and generalized knowledge which can be reused under the cold-start environment. For this purpose, it constructs a lot of short trajectories with only two POI checkins next to each other from the long trajectories to simulate the cold-start environment, and uses them to train a MLP with the ability of measuring the spatial-temporal transition information between two POI check-ins. This information indicates the probability that the users visit the next POI from current POI. And by computing such transition information between users' current POI and all candidate POIs, the most nearest POI can be recommended to users. Based on the public datasets of Gowalla and Brightkite, our experiments have shown that the proposed method achieves better performance in recommending next POIs for cold-start users when compared with related methods.
GIS Switch State Detection Based on Coupling Current and Heterogeneous Information Fusion

Because the disconnector in GIS equipment is sealed in the metal shell, its exact position cannot be directly seen by naked eyes. After running in for a long time or severe external influence, the opening and closing position of GIS equipment knife switch may change, and the equipment and power grid will be seriously damaged when the knife switch is not in place. Therefore, this paper quickly and accurately judges the opening and closing status of GIS equipment knife switch by collecting and fusing the grounding current of GIS equipment knife switch, lightning arrester leakage current and shell vibration. First of all, this paper adopts the fast research and judgment method of GIS equipment opening and closing states based on coupling capacitance current, and uses particle swarm optimization to find the super parameters of the subsequent classification CNN network to speed up the network research and judgment. In addition, it further accurately studies and judges the network through the time-frequency memory recurrent neural network based on IMF energy entropy.
In-Air Handwritten Chinese Text Recognition with Attention Convolutional Recurrent Network

In-air handwriting is a new and more humanized way of human-computer interaction, which has a broad application prospect. One of the existing online handwritten Chinese text recognition model is to convert the trajectory data into image-like representation and use two-dimensional convolutional neural network (2DCNN) for feature extraction, and the another one directly process trajectory sequence with Long Short-Term Memory (LSTM). However, when using 2DCNN, many information will be lost in the process of conversion into images. When using LSTM, LSTM network is easy to cause gradient problem. So we propose an attention convolutional recurrent network (ACRN) for in-air handwritten Chinese text, which introduces one-dimensional convolutional neural network (1DCNN) containing dilation convolution for feature extraction of trajectory data directly. After that, the ACRN uses LSTM combined with multihead attention mechanism to focus on some key words in handwritten Chinese text, mines multi-level dependencies and outputs to softmax for classification. Finally the ACRN uses the Connectionist Temporal Classification (CTC) objective function without input-output alignment to decode the coding results. We conduct experiments on the CASIA-OLHWDB2.0-2.2 dataset and in-air handwritten Chinese text dataset IAHCT-UCAS2018. Experimental results demonstrate that compared with previous methods, our method obtains a more compact model with a higher recognition accuracy.