Reviewerly/exHarmony huggingface dataset:
Adversarial Attack Mitigation Strategy for Machine Learning-Based Network Attack Detection Model in Power System

The network attack detection model based on machine learning (ML) has received extensive attention and research in PMU measurement data protection of power systems. However, well-trained ML-based detection models are vulnerable to adversarial attacks. By adding meticulously designed perturbations to the original data, the attacker can significantly decrease the accuracy and reliability of the model, causing the control center to receive unreliable PMU measurement data. This paper takes the network attack detection model in the power system as a case study to analyze the vulnerability of the ML-based detection model under adversarial attacks. And then, a mitigation strategy for adversarial attacks based on causal theory is proposed, which can enhance the robustness of the detection model under different adversarial attack scenarios. Unlike adversarial training, this mitigation strategy does not require adversarial samples to train models, saving computing resources. Furthermore, the strategy only needs a small amount of detection model information and can be migrated to various models. Simulation experiments on the IEEE node systems verify the threat of adversarial attacks against different ML-based detection models and the effectiveness of the proposed mitigation strategy.
Domain-Aware Federated Social Bot Detection with Multi-Relational Graph Neural Networks

Social networks have been the widespread popular tools for communication and socialization, and it also been the ideal platform for bots to publish malicious information. Therefore, social bot detection is essential for the social network's security. Existing methods almost ignore the differences in bot behaviors in multiple domains. Thus, we first propose a DomainAware detection method with Multi-Relational Graph neural networks (DA-MRG) to improve detection performance. Specifically, DA-MRG constructs multi-relational graphs with users' features and relationships, obtains the user presentations with graph embedding and distinguishes bots from humans with domainaware classifiers. Meanwhile, considering the similarity between bot behaviors in different social networks, we believe that sharing data among them could boost detection performance. However, the data privacy of users needs to be strictly protected. To overcome the problem, we implement a study of federated learning framework for DA-MRG to achieve data sharing between different social networks and protect data privacy simultaneously. We conduct extensive experiments on TwiBot-20, and the results demonstrate that the proposed method can effectively achieve federated social bot detection.
Domain Adaptive Video Segmentation via Temporal Pseudo Supervision

Video semantic segmentation has achieved great progress under the supervision of large amounts of labelled training data. However, domain adaptive video segmentation, which can mitigate data labelling constraints by adapting from a labelled source domain toward an unlabelled target domain, is largely neglected. We design temporal pseudo supervision (TPS), a simple and effective method that explores the idea of consistency training for learning effective representations from unlabelled target videos. Unlike traditional consistency training that builds consistency in spatial space, we explore consistency training in spatiotemporal space by enforcing model consistency across augmented video frames which helps learn from more diverse target data. Specifically, we design cross-frame pseudo labelling to provide pseudo supervision from previous video frames while learning from the augmented current video frames. The cross-frame pseudo labelling encourages the network to produce high-certainty predictions, which facilitates consistency training with cross-frame augmentation effectively. Extensive experiments over multiple public datasets show that TPS is simpler to implement, much more stable to train, and achieves superior video segmentation accuracy as compared with the state-of-the-art. Code is available at https://github.com/xing0047/TPS .
Underwater Multitarget Tracking With Sonar Images Using Thresholded Sequential Monte Carlo Probability Hypothesis Density Algorithm

This letter presents a multi-target tracking algorithm—thresholded Sequential Monte Carlo probability hypothesis density (TH-SMC-PHD) algorithm. The TH-SMC-PHD aims to overcome the problem that underwater multi-target tracking is prone to missing tracking on sonar images, resulting in the breakage of trajectories. First, CA-CFAR and K-means are employed to detect potential underwater targets from sonar images, respectively. Then TH-SMC-PHD is applied to the detection results for tracking, using a continuously lost frame threshold to reduce the missing tracking rate and the Minimum-Sampling-Variance (MSV) resampling to improve tracking accuracy. An actual underwater multi-target tracking experiment using a forward-looking sonar was contracted in a water tank to evaluate the tracking performance. Compared with the other two PHD tracking algorithms, the results demonstrate that the proposed algorithm achieves high-precision, non-fracture, non-missing tracking of three targets. In addition, the TH-SMC-PHD is more stable and less affected by different detection algorithms, which has the potential for practical applications.
Privacy-Preserving Federated Learning based on Differential Privacy and Momentum Gradient Descent

To preserve participants' privacy, Federated Learning (FL) has been proposed to let participants collaboratively train a global model by sharing their training gradients instead of their raw data. However, several studies have shown that con-ventional FL is insufficient to protect privacy from adversaries, as even from gradients, useful information can still be recovered. To obtain stronger privacy protection, Differential Privacy (DP) has been proposed on the server's side and the clients' side. Although adding artificial noise to the raw data can enhance users' privacy, the accuracy performance of the FL is inevitably degraded. In addition, although the communication overhead caused by the FL is much smaller than that of centralized learning, it still becomes a bottleneck of the learning performance and utilization efficiency due to its frequent parameters exchange. To tackle these problems, we propose a new FL framework via applying DP both locally and centrally in order to strengthen the protection of par-ticipants' privacy. To improve the accuracy performance of the model, we also apply sparse gradients and Momentum Gradient Descent on the server's side and the clients' side. Moreover, using sparse gradients can reduce the total communication costs. We provide the experiments to evaluate our proposed framework and the results show that our framework not only outperforms other DP-based FL frameworks in terms of the model accuracy but also provides a more powerful privacy guarantee. Besides, our framework can save up to 90% of communication costs while achieving the best accuracy performance.
DORIAN in action

Existing automated machine learning solutions and intelligent discovery assistants are popular tools that facilitate the end-user with the design of data science (DS) pipelines. However, they yield limited applicability for a wide range of real-world use cases and application domains due to (a) the limited support of DS tasks; (b) a small, static set of available operators; and (c) restriction to evaluation processes with quantifiable loss functions. We demonstrate DORIAN, a human-in-the-loop approach for the assisted design of data science pipelines that supports a large and growing set of DS tasks, operators, and arbitrary user-defined evaluation processes. Based on the user query, i.e., a dataset and a DS task, DORIAN computes a ranked list of candidate pipelines that the end-user can choose from, alter, execute and evaluate. It stores executed pipelines in an experiment database and utilizes similarity-based search to identify relevant previously-run pipelines from the experiment database. DORIAN also takes user interaction into account to improve suggestions over time. We show how users can interact with DORIAN to create and compare DS pipelines on various real-world DS tasks without the need for writing any code.
Towards the Quantitative Verification of Deep Learning for Safe Perception

Deep learning (DL) is seen as an inevitable building block for perceiving the environment with sufficient detail and accuracy as required by automated driving functions. Despite this, its black-box nature and the therewith intertwined unpredictability still hinders its use in safety-critical systems. As such, this work addresses the problem of making this seemingly unpredictable nature measurable by providing a risk-based verification strategy, such as required by ISO 21448. In detail, a method is developed to break down acceptable risk into quantitative performance targets of individual DL-based components along the perception architecture. To verify these targets, the DL input space is split into areas according to the dimensions of a fine-grained operational design domain <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\mu \mathbf{ODD})$</tex> . As it is not feasible to reach full test coverage, the strategy suggests to distribute test efforts across these areas according to the associated risk. Moreover, the testing approach provides answers with respect to how much test coverage and confidence in the test result is required and how these figures relate to safety integrity levels (SILs).
Continual Semantic Segmentation via Structure Preserving and Projected Feature Alignment

Deep networks have been shown to suffer from catastrophic forgetting. In this work, we try to alleviate this phenomenon in the field of continual semantic segmentation (CSS). We observe that two main problems lie in existing arts. First, attention is only paid to designing constraints for encoder (i.e., the backbone of segmentation network) or output probabilities. But we find that forgetting also happens in the decoder head and harms the performance greatly. Second, old and new knowledge are entangled in intermediate features when learning new categories, making existing practices hard to balance between plasticity and stability. On these bases, we propose a framework driven by two novel constraints to address the aforementioned problems. First, a structure preserving loss is applied to the decoder's output to maintain the discriminative power of old classes from two different granularities in embedding space. Second, a feature projection module is adopted to disentangle the process of preserving old knowledge from learning new classes. Extensive evaluations on VOC2012 and ADE20K datasets show the effectiveness of our approach, which significantly outperforms existing state-of-the-art CSS methods.
HVC-Net: Unifying Homography, Visibility, and Confidence Learning for Planar Object Tracking

Robust and accurate planar tracking over a whole video sequence is vitally important for many vision applications. The key to planar object tracking is to find object correspondences, modeled by homography, between the reference image and the tracked image. Existing methods tend to obtain wrong correspondences with changing appearance variations, camera-object relative motions and occlusions. To alleviate this problem, we present a unified convolutional neural network (CNN) model that jointly considers homography, visibility, and confidence. First, we introduce correlation blocks that explicitly account for the local appearance changes and camera-object relative motions as the base of our model. Second, we jointly learn the homography and visibility that links camera-object relative motions with occlusions. Third, we propose a confidence module that actively monitors the estimation quality from the pixel correlation distributions obtained in correlation blocks. All these modules are plugged into a Lucas-Kanade (LK) tracking pipeline to obtain both accurate and robust planar object tracking. Our approach outperforms the state-of-the-art methods on public POT and TMT datasets. Its superior performance is also verified on a real-world application, synthesizing high-quality in-video advertisements.
Virtual Reality and Education – The Steam Panorama

Virtual Reality (VR) is increasingly being used as an education medium also enabled by advances in the consumer market, making the devices less expensive and the setup easier. The hardware is more accessible, and players can use Steam as the leading distribution platform for VR applications. Hence, more and more VR applications are being published, also for educational purposes. However, the VR educational games panorama is yet to be described due to the recent rise of this area, as well as users’ perceptions (both learners and educators) and issues specific to each learning domain. Towards this, we provide a first analysis of the Steam educational VR applications. We report an analysis of user reviews describing the positive and negative characteristics of VR experiences. Finally, we give an overview of popular educational VR experiences and what we can learn from reviewers about the design requirements for specific learning fields.
On the formation of electronic libraries reflecting the scientific heritage of Russia

The paper considers the problem of forming digital resources, which reflect the scientific heritage of Russia. One of the most developed resources in this area is the digital library “Scientific Heritage of Russia” (DL SHR), which development began in 2007 as a part of the special program of the Russian Academy of Sciences, headed by the Joint Supercomputer Centre of RAS (JSC RAS). The library is built on the principle of linked data and reflects information about scientists, their publications (including digitized full texts) and archival materials related to scientists and museum objects. The content of the library was formed by joint efforts of many scientific organizations, including academic libraries. DL SHR is publicly available on the Internet since 2010. By 2020, DL SHR new generation technology, its internal and external software shells, search algorithms were completely redesigned and put into operation. The article describes the current state of NNR DL, which includes two WEB-oriented blocks - administrative and user. The administrative block, available to authorized users, provides input, editing, quality control of the created resources; the user block, presented in the public domain, provides wide opportunities for searching for resources using Boolean logic; it provides obvious visualization and advanced navigation through related resources. A number of examples of work with DL SHR are given. Along with DL SHR, the article considers the electronic libraries “Scientific Heritage of the Urals” and “Scientific Heritage of the Far East”, presented, respectively, on the websites of the Central Scientific Libraries of the Ural Branch of the Russian Academy of Sciences and the Far Eastern Branch of the Russian Academy of Sciences. These libraries are compared with DL SHR, and the conclusion is made about the need to coordinate work on creating digital resources that reflect the scientific heritage of Russia and its regions.
Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning

Scene graph generation (SGG) is a fundamental task aimed at detecting visual relations between objects in an image. The prevailing SGG methods require all object classes to be given in the training set. Such a closed setting limits the practical application of SGG. In this paper, we introduce open-vocabulary scene graph generation, a novel, realistic and challenging setting, in which a model is trained on a set of base object classes but is required to infer relations for unseen target object classes. To this end, we propose a two-step method which firstly pre-trains on large amounts of coarse-grained region-caption data and then leverages two prompt-based techniques to finetune the pre-trained model without updating its parameters. Moreover, our method is able to support inference over completely unseen object classes, which existing methods are incapable of handling. On extensive experiments on three benchmark datasets, Visual Genome, GQA and Open-Image, our method significantly outperforms recent, strong SGG methods on the setting of Ov-SGG, as well as on the conventional closed SGG.
Learning to Anticipate Future with Dynamic Context Removal

Anticipating future events is an essential feature for in-telligent systems and embodied AI. However, compared to the traditional recognition task, the uncertainty of future and reasoning ability requirement make the anticipation task very challenging and far beyond solved. In this filed, previous methods usually care more about the model ar-chitecture design or but few attention has been put on how to train an anticipation model with a proper learning policy. To this end, in this work, we propose a novel training scheme called Dynamic Context Removal (DCR), which dynamically schedule the visibility of observed future in the learning procedure. It follows the human-like curriculum learning process, i.e., gradually removing the event context to increase the anticipation difficulty till satisfying the final anticipation target. Our learning scheme is plug-and-play and easy to integrate any reasoning model including transformer and LSTM, with advantages in both effectiveness and efficiency. In extensive experiments, the pro-posed method achieves state-of-the-art on four widely-used benchmarks. Our code and models are publicly released at https://github.com/AllenXuuuIDCR.
BTS: A Bi-lingual Benchmark for Text Segmentation in the Wild

As a prerequisite of many text-related tasks such as text erasing and text style transfer, text segmentation arouses more and more attention recently. Current researches mainly focus on only English characters and digits, while few work studies Chinese characters due to the lack of pub-lic large-scale and high-quality Chinese datasets, which limits the practical application scenarios of text segmentation. Different from English which has a limited alphabet of letters, Chinese has much more basic characters with com-plex structures, making the problem more difficult to deal with. To better analyze this problem, we propose the Bi-lingual Text Segmentation (BTS) dataset, a benchmark that covers various common Chinese scenes including 14,250 diverse and fine-annotated text images. BTS mainly focuses on Chinese characters, and also contains English words and digits. We also introduce Prior Guided Text Segmen-tation Network (PGTSNet), the first baseline to handle bi-lingual and complex-structured text segmentation. A plug-in text region highlighting module and a text perceptual dis-criminator are proposed in PGTSNet to supervise the model with text prior, and guide for more stable and finer text seg-mentation. A variation loss is also employed for suppressing background noise under complex scene. Extensive ex-periments are conducted not only to demonstrate the neces-sity and superiority of the proposed dataset BTS, but also to show the effectiveness of the proposed PGTSNet compared with a variety of state-of-the-art text segmentation methods.
Forward and Backward Inertial Anomaly Detector: A Novel Time Series Event Detection Method

Time series event detection is related to studying methods for detecting observations in a series with special meaning. These observations differ from the expected behavior of the data set. In data streaming scenarios, it is possible to observe an increase in the speed of data generation in time series. Therefore, adapting to time series changes becomes crucial. Thus, identifying events associated with these changes is essential for timely and correct decision-making. Although there are many methods to detect events, it is still possible to have difficulties detecting them correctly, particularly those associated with concept drift. In order to fill the gap in the literature, this work proposes a new method, named Forward and Backward Inertial Anomaly Detector (FBIAD), for detecting events in time series. It contributes by analyzing surrounding inertia around observations. FBIAD outperformed other methods both in accuracy and elapsed time.
Light-Weight EPINET Architecture for Fast Light Field Disparity Estimation

Recent deep learning-based light field disparity estimation algorithms require millions of parameters, which demand high computational cost and limit the model deployment. In this paper, an investigation is carried out to analyze the effect of depthwise separable convolution and ghost modules on state-of-the-art EPINET architecture for disparity estimation. Based on this investigation, four convolutional blocks are proposed to make the EPINET architecture a fast and light-weight network for disparity estimation. The experimental results exhibit that the proposed convolutional blocks have significantly reduced the computational cost of EPINET architecture by up to a factor of 3.89, while achieving comparable disparity maps on HCI Benchmark dataset.
Named Entity Recognition in Electronic Medical Records Based on Transfer Learning

Named entity recognition is the first step in clinical electronic medical record text mining, which is significant for clinical decision support and personalized medicine. However, the lack of annotated electronic medical record datasets limits the application of pre-trained language models and deep neural networks in this field. To alleviate the problem of data scarcity, we propose T-RoBERTa-BiLSTM-CRF, a transfer learning-based electronic medical record entity recognition model, which aggregates the characteristics of medical data from different sources and uses a small amount of electronic medical record data as target data for further training. Compared with existing models, our approach can model medical entities more effectively, and the extensive comparative experiments on the CCKS 2019 and DEMRC datasets show the effectiveness of our approach.
Smaug: A TEE-Assisted Secured SQLite for Embedded Systems

As one of the most popular relational databases for embedded devices, SQLite is lightweight to be embedded into applications without installing a specific database management system. However, simplicity and easy-to-use are double-edged swords; while bringing convenience, they also make data processing and storage risky. For example, an attacker can obtain data from a database file or memory and tamper with it once he has gained higher privileges, threatening the database's confidentiality and integrity. To address such security issues, based on a trusted execution environment (TEE) and a trusted platform module (TPM), we have proposed Smaug, a general secure scheme to ensure the confidentiality and integrity of SQLite and similar databases. With Smaug, all the critical data is stored in ciphertext, and data integrity protection is also provided. Besides, with TEE, all the sensitive operations are isolated from the untrusted environment, which can effectively resist attacks against memory. In addition, we use TPM to provide a solid root-of-trust (RoT) for the system. Finally, we have implemented a prototype system, and the performance evaluations have clarified the dominant factors that affect the system availability, providing a reference to the design and implementation of similar systems.