Reviewerly/exHarmony huggingface dataset:
PPBR-FL: A Privacy-Preserving and Byzantine-Robust Federated Learning System

As a distributed machine learning framework, federated learning enables a multitude of participants to train a joint model privately by keeping training data locally. The federated learning is emerging as a promising alternative to solve data privacy protection, but it has been proven that federated learning is vulnerable to attacks from malicious clients, especially Byzantine attackers sending poisoned parameters during the training phase. To address this problem, several aggregation approaches against Byzantine failures have been proposed. However, these existing aggregation methods are only of limited utility in the setting of privacy protection. This paper proposes a privacy-preserving and Byzantine-robust federated learning framework (PPBR-FL) which achieves the objective to satisfy privacy and robustness simultaneously. We use the local differential privacy mechanism to realize privacy protection for the clients and propose a Byzantine-robust aggregation rule named as TPM (Trimmed Padding Mean) to realize robustness. Extensive experiments on two benchmark datasets demonstrate that the TPM outperforms the classical aggregation approaches in terms of robustness and privacy. When less than half of the workers are Byzantine attackers, the final global model not only achieves satisfactory performance against Byzantine attack, but also provide privacy protection on parameters to prevent privacy disclosure.
DBGARE: Across-Within Dual Bipartite Graph Attention for Enhancing Distantly Supervised Relation Extraction

Recently, a kind of method based on single bipartite graph combined with single attention mechanism was proposed to solve the challenges of noisy labels and long-tailed distributions in distantly supervised relation extraction, which achieved competitive results. However, we argue that single bipartite graph will lose interactive information within each bipartite graph partition and single attention will cause attention bias. In order to improve the robustness of this method for noisy labels and long-tailed distributions, we propose a novel framework DBGARE, whereby graphs across and within each bipartite graph partition are constructed to enrich unbalanced relations dependency, based on which, a novel technique dual attention mechanism is devised to avoid attention bias and promote information dissemination from data-riches to data-poors. Multiple experiments on various widely-used real-world datasets show the state-of-the-art effectiveness of our framework in both challenges above. Furthermore, almost previous benchmark datasets are in English, we provide CIPDS $$*$$ , a new Chinese dataset collected from Chinese Corpus in Chinese search engine, to expand distantly supervised relation extraction research field.
Improving the Generalization of Deep Learning Classification Models in Medical Imaging Using Transfer Learning and Generative Adversarial Networks

Data sets for medical images are generally imbalanced and limited in sample size because of high data collection costs, time-consuming annotations, and patient privacy concerns. The training of deep neural network classification models on these data sets to improve the generalization ability does not produce the desired results for classifying the medical condition accurately and often overfit the data on the majority of class samples. To address the issue, we propose a framework for improving the classification performance metrics of deep neural network classification models using transfer learning: pre-trained models, such as Xception, InceptionResNet, DenseNet along with the Generative Adversarial Network (GAN) - based data augmentation. Then, we trained the network by combining traditional data augmentation techniques, such as randomly flipping the image left to right and GAN-based data augmentation, and then fine-tuned the hyper-parameters of the transfer learning models, such as the learning rate, batch size, and the number of epochs. With these configurations, the Xception model outperformed all other pre-trained models achieving a test accuracy of 98.7%, the precision of 99%, recall of 99.3%, f1-score of 99.1%, receiver operating characteristic (ROC) - area under the curve (AUC) of 98.2%.
Implementation of Real-Time Human Tracking System Based on Deep Learning Using Kinect Camera

In this study, an effective real-time tracking target human method using Kinect camera is developed in different condition environments using deep learning. The proposed algorithm combines the YOLOv3 and the deep SORT method. To be more detailed, the YOLOv3 algorithm is adopted to detect the target human. To trace the identified human, the deep SORT is employed. The efficiency of the proposed system is tested with the complex environments with four cases: low light, normal light, outdoor, and rain environments. The experiment results indicate that the suggested solution performed effectively in different complex environments. Besides, the developed algorithm can detect the human target in the situation of partial occlusion and loss of the human target.
A Multi-level Attention-Based LSTM Network for Ultra-short-term Solar Power Forecast Using Meteorological Knowledge

Deep learning has been widely applied to modeling, prediction, control, and optimization in smart energy systems, especially in solar energy. Accurate forecasting of photovoltaic (PV) power is essential for power system dispatch. However, PV power output is greatly affected by weather conditions. Therefore integrating domain knowledge to improve power prediction accuracy has become a major challenge. To tackle this problem, this paper proposes an effective deep learning model for solar power forecasting based on temporal correlation and meteorological knowledge. The model adopts an encoder-decoder architecture with multi-level attention machines and long short-term memory units. The encoder is designed to dynamically extract the historical features of the in situ measurements of the PV plant, whereas the decoder is used to capture the temporal features of multi-source variables. The domain knowledge (e.g., clearness index, numerical weather prediction, and short-wave radiation) is integrated into the decoder for forecasting solar PV power. A case study is conducted using the dataset collected from real-world PV plants in different weather conditions. The experiment results demonstrate that the forecasting accuracy of the proposed model outperforms three baselines, and the prediction error is reduced by 9.5% on average compared to others.
Towards Better Personalization: A Meta-Learning Approach for Federated Recommender Systems

Recommender systems (RSs) seek to suggest items that may be of interest to users by primarily modeling their historical interactions. In today’s context, due to privacy and security constraints, it is becoming increasingly attractive to implement RSs based on federated learning frameworks, in which users’ private data need not be uploaded to a central server. However, two major limitations remain unaddressed by existing works about federated RSs. First, canonical federated learning only develops a common output for all users and does not adapt the model to each user. This is an important missing feature, especially given the heterogeneity of the distribution of historical interaction data across users. Second, many current advanced recommendation algorithms, such as self-attention-based methods, have not been applied to federated RSs due to mobile devices’ limited storage and computing power. To address these issues, we propose a personalized variant of the well-known Federated Averaging algorithm, where the Model-Agnostic Meta-Learning (MAML) framework is applied to provide a more personalized recommendation model for each user. Meanwhile, we investigate a sparse attention mechanism for sequential recommendation that significantly reduces the time complexity and memory usage when computing attention scores. Extensive experiments on six benchmark datasets reveal that our method consistently outperforms other federated RSs.
Vietnamese Spelling Error Detection and Correction Using BERT and N-gram Language Model

Vietnamese spelling error detection and correction is a crucial task in Natural language processing, it plays an important role in many different real-world applications. Although there is a lot of research on it, dealing with diverse types of errors in Vietnamese is still a challenge. In this paper, we propose a model to help detect and correct some specific Vietnamese spelling errors by combining a pre-trained neural network-based Vietnamese language model and N-gram language model. We also provide a clear definition of handleable error types, error generation rules in the training set and evaluate our proposed model on a Vietnamese benchmark dataset at the word level. The experimental results show that our model achieves higher than from 1% to 14% f1-score than other neural network-based pre-trained language models in detection and make comparisons with bi, tri and 4-g language models to choose the best model for correction.
Enhanced Side-Channel Analysis on ECDSA Employing Fixed-Base Comb Method

Table-based scalar multiplication provides practical security for ECDSA signature generation. However, a novel key recovery attack against this form of ECDSA signature generation that exploits the collisions between entries was recently proposed at CHES 2021. This attack is possible even if table entries are unknown, such as with random permutated entry ordering. In this paper, we enhance the efficiency of the key recovery attack against secure ECDSA signature generation based on fixed-base comb scalar multiplication. We significantly reduce the required number of traces by compressing collision information using the mathematical relationship between table entry collisions. We verify this is a practical threat by performing an experiment on fixed-base comb method with window width <inline-formula><tex-math notation="LaTeX">$w=4$</tex-math></inline-formula>. Using our method, up to 27 traces are needed, much fewer than 1,019 traces required in the CHES publication. We cluster real traces measured using 32-bit STM32F4 microcontroller. In the experiment, we provide a selection method of points of interest using variance traces and unsupervised clustering-based leakage detection. With the selection method, we succeed in clustering leakages into 16 classes with a 100&#x0025; success rate with 32-bit MCU. This represents the first experiment to cluster the more leakage classes with a 32-bit MCU than in literature.
Towards Explainable Reinforcement Learning Using Scoring Mechanism Augmented Agents

Deep reinforcement learning (DRL) is increasingly used in application areas such as medicine and finance. However, the direct mapping from state to action in DRL makes it challenging to explain why decisions are made. Existing algorithms for explaining DRL policy are posteriori, explaining to an agent after it has been trained. As a common limitation, these posteriori methods fail to improve training with the deduced knowledge. Face with that, an end-to-end trainable explanation method is proposed, in which an Adaptive Region Scoring Mechanism (ARS) is embedded into DRL system. The ARS explains the agent's action by evaluating the features of the input state that are most relevant action before DRL re-learn from task-related regions. The proposed method is validated on Atari games. Experiments demonstrate that agent using the explainable proposed mechanism outperforms the original models.
Mitigating Targeted Bit-Flip Attacks via Data Augmentation: An Empirical Study

As deep neural networks (DNNs) become more widely used in various safety-critical applications, protecting their security has been an urgent and important task. Recently, one critical security issue is proposed that DNN models are vulnerable to targeted bit-flip attacks. This kind of sophisticated attack tries to inject backdoors into models via flipping only a few bits of carefully chosen model parameters. In this paper, we propose a gradient obfuscation-based data augmentation method to mitigate these targeted bit-flip attacks as an empirical study. Particularly, we mitigate such targeted bit-flip attacks by preprocessing only input samples to break the link between the features carried by triggers of input samples with the modified model parameters. Moreover, our method can keep an acceptable accuracy on benign samples. We show that our method is effective against two targeted bit-flip attacks by experiments on two widely-used structures (ResNet-20 and VGG-16) with one famous dataset (CIFAR-10).
KEAN: Knowledge-Enhanced and Attention Network for News Recommendation

An important goal of online news system is to provide accurate personalized recommendation for users from mass news. One of the main problems in news recommendation is to obtain accurate news representation. Generally speaking, news is full of intellectual entities and common sense. However, existing news recommendation tends to ignore external knowledge and fail to fully detect potential knowledge links between news. In addition, news recommendation also faces the problem of user interest diversity. To solve the above problems, this paper proposes a knowledge enhancement and attention network based bnews recommendation model (KEAN) to enhance news title representation and user interest. The proposed model is mainly processed from two perspectives. First, for news titles, the entity embedding is enriched by aggregating information from its neighborhood in the knowledge graph, so as to obtain richer contextual embedding. Secondly, in order to meet the different interests in users, attention network is used to dynamically aggregate the information which is related to the current candidate news in user history. KEAN model can get rich feature representation by dealing with the correlation between entities and users’ interests. The experimental results show that the proposed model performs better than the other compared models with F1 and AUC being increased by 0.8%–1.2%, 0.6%–1.8% in Bing News and MIND dataset.
CKGAC: A Commonsense Knowledge Graph About Attributes of Concepts

This paper presents a method for building a large commonsense knowledge graph about attributes of concepts, called CKGAC. CKGAC contains triples that connect concepts (nouns) with attribute values (adjectives) via corresponding attributes like color, taste and age. We first manually construct a seed commonsense knowledge graph for CKGAC based on a limited set of Chinese adjectives. Then we add new triples to this seed commonsense knowledge graph by extracting < concept, adjective> pairs from unstructured Web documents. We further extend the commonsense knowledge graph using isa relations between concepts. Finally, we obtain CKGAC which consists of 4,490,768 high-quality commonsense triples covering 187 concept attributes. Experimental results show the effectiveness of our approach in terms of both precision and coverage through human evaluation. In order to prove the utility of CKGAC, we demonstrate an application of it to commonsense knowledge acquisition. CKGAC is available at https://zenodo.org/record/6084143 .
LAM: Lightweight Attention Module

The attention mechanisms have been widely used in existing methods due to their effectiveness. In the field of computer vision, these mechanisms can be grouped as 1) channel attention mechanisms, which highlight the important channels for images, and 2) spatial attention mechanisms, which focus on the location features for all channels of images. These two groups of mechanisms, which have various strategies for capturing features, actually play complementary roles in image classification. Existing lightweight models based on one group of attention mechanisms have fewer parameters than convolutional networks. However, few works consider their integration and maintain their merits for lightweight neural networks. In this paper, we propose a new Lightweight Attention Module (LAM) for lightweight convolutional neural networks to efficiently integrate these attention mechanisms. Specifically, we use element-wise addition and smaller convolutional kernels in the spatial module, avoiding the vanishing gradient problem. Besides, we replace the multi-layer perceptron (MLP) layer with squeeze-and-excitation layers in the channel module, alleviating the problem of channel dependencies. Finally, we adopt a parallel mechanism to coordinate these two attention modules with low computational complexity. Experimental results on benchmark datasets demonstrate the effectiveness of LAM in terms of image classification tasks, ablation study and robustness analysis.
A Benchmark Sentinel-1 SAR Dataset for Airport Detection

Airports are important targets in both military and civilian tasks. Synthetic aperture radar (SAR) based airport detection has received increasing attention in recent years. However, due to the high cost of the SAR imaging and annotation process, there is no publicly available SAR dataset for airport detection, which leads to the fact that deep learning methods have not been applied to airport detection and no unified evaluation benchmark for traditional methods. To provide a benchmark for airport detection studies in SAR images, a large SAR airport dataset (SAD) is presented in this paper. It contains 624 SAR images from Sentinel 1B, covering 104 instances of airports with different scales, orientations and shapes, which can realistically reflect the real world. Experiments with four deep learning-based methods and three traditional methods on this dataset demonstrate its effectiveness and challenge. It serves for the development of state-of-the-art airport detection algorithms or other related tasks. In addition, we found that airport runways in SAR images always have some parallel line segments. Inspired by this, the fusion of deep features and line segments is achieved by designing a line segment detector branch, which further improves the accuracy of airport detection.
How to Search and Contextualize Scenes Inside Videos for Enriched Watching Experience: Case Stories of the Second World War Veterans

This demo paper demonstrates the idea of publishing and watching videos on the Semantic Web. An in-use application, WarMemoirSampo, is presented that enables scene segments in videos to be searched by their semantic content. While watching a video, additional contextual information is provided dynamically. The system is based on a SPARQL endpoint whose knowledge graph has been extracted automatically from timestamped natural language descriptions of the video contents.
Topic and Reference Guided Keyphrase Generation from Social Media

Automatic keyphrase generation can help human efficiently understand or process critical information from massive social media posts. Seq2Seq-based generation models that can produce both present and absent keyphrases have achieved remarkable performance. However, existing models are limited by the sparseness of posts that are widely exhibited in social media language. Sparseness makes it difficult for models to obtain useful features and generate accurate keyphrases. To address this problem, we propose the Topic and Reference Guided Keyphrase Generation model(TRGKG) for social media posts, which enrich scarce posts features by corpus-level topics and post-level reference knowledge. The proposed model incorporates a contextual neural topic model to exploit topics and a heterogeneous graph to capture reference knowledge from retrieved related posts. To guide the decoding process, we introduce new topic-aware hierarchical attention and copy mechanism, which directly copies appropriate words from both the source post and its references. Experiments on two public datasets demonstrate that TRGKG achieves state-of-the-art performance.
MSSA-FL: High-Performance Multi-stage Semi-asynchronous Federated Learning with Non-IID Data

Federated Learning (FL) is an emerging distributed machine learning framework that allows edge devices to collaborative train a shared global model without transmitting their sensitive data to centralized servers. However, it is extremely challenging to apply FL in practical scenarios because the statistics of the data across edge devices are usually not independent and identically distributed (Non-IID), which will introduce the bias to global model. To solve the above data heterogeneity issue, we propose a novel Multi-Stage Semi-Asynchronous Federated Learning (MSSA-FL) framework. MSSA-FL benefits convergence accuracy through making the local model complete multi-stage training within the group guided by combination module. To improve the training efficiency of the framework, MSSA-FL adopts a semi-asynchronous update method. Meanwhile, proposed model assignment strategy and model aggregation method further boost the performance of MSSA-FL. Experiments on several public datasets show that MSSA-FL achieves higher accuracy and faster convergence than the comparison algorithms.
Towards UML-Style Visual Queries over Wikidata

We describe and demonstrate the options for visual UML-style presentation and visual-based creation of SPARQL queries over Wikidata, a central Linked Data resource employing a custom data classification encoding. We provide visual presentations of public Wikidata example queries, this way adding a visual dimension to their comprehensibility. The process of visual query creation within the tool is supported by auto-completion facilities that consider the context of the already created query part, where possible.
Option Discovery for Autonomous Generation of Symbolic Knowledge

In this work we present an empirical study where we demonstrate the possibility of developing an artificial agent that is capable to autonomously explore an experimental scenario. During the exploration, the agent is able to discover and learn interesting options allowing to interact with the environment without any assigned task, and then abstract and re-use the acquired knowledge to solve the assigned tasks. We test the system in the so-called Treasure Game domain described in the recent literature and we empirically demonstrate that the discovered options can be abstracted in an probabilistic symbolic planning model (using the PPDDL language), which allowed the agent to generate symbolic plans to achieve extrinsic goals.
Spotlight on Video Piracy Websites: Familial Analysis Based on Multidimensional Features

With the gradual increase in awareness of intellectual property protection in recent years, it has become imperative to strengthen the monitoring and regulation of digital piracy. The previous countermeasures suffer from low accuracy or passive data collection. Furthermore, the commonly adopted website clustering methods focus exclusively on a few attributes. The results obtained do not draw a comprehensive picture of the connections between websites within a family. In this paper, we aim to address the issue of digital piracy being challenging to identify, trace, monitor, and regulate in the current situation, utilizing video piracy websites targeting Chinese consumers as an example. The present architecture enables proactive discovery and detection of suspicious websites with a 96.2% accuracy, compensating for traditional digital piracy detection inadequacies. The proposed novel feature extraction method for clustering video piracy websites can synthesize multiple aspects in terms of layout, content, and infrastructure. The clustering results indicate that the websites belonging to the same family obtained by the proposed method show a more comprehensive similarity.