Reviewerly/exHarmony huggingface dataset:
Recognition of Oracle Bone Inscriptions by using Two Deep Learning Models

Oracle bone inscriptions (OBIs) contain some of the oldest characters in the world and were used in China about 3000 years ago. As an ancient form of literature, OBIs store a lot of information that can help us understand world history, character evaluations, and more. However, as OBIs were only discovered about 120 years ago, few studies have described them, and the aging process has made the inscriptions less legible. Hence, automatic character detection and recognition have become important issues. This paper aims to design an online OBI recognition system for helping cultural heritage preservation and organization. We have evaluated two deep learning models for OBI recognition and designed an application programming interface (API) that can be accessed online for the recognition. In the first stage, You Only Look Once (YOLO) is applied for detecting and recognizing OBIs. However, since not all of the OBIs can be detected correctly by YOLO in the first stage, we then utilize MobileNet to recognize the undetected OBIs by manually cropping the undetected OBIs in the image. MobileNet is selected for the second stage of recognition as our evaluation of ten state-of-the-art deep learning models showed that it is the best network for OBI recognition due to its superior performance in terms of accuracy, loss and time consumption. We have installed our system on a server as an API and opened it for OBI detection and recognition.
Weighted multi-view co-clustering (WMVCC) for sparse data

Multi-view clustering has gained importance in recent times due to the large-scale generation of data, often from multiple sources. Multi-view clustering refers to clustering a set of objects which are expressed by multiple set of features, known as views, such as movies being expressed by the list of actors or by a textual summary of its plot. Co-clustering, on the other hand, refers to the simultaneous grouping of data samples and features under the assumption that samples exhibit a pattern only under a subset of features. This paper combines multi-view clustering with co-clustering and proposes a new Weighted Multi-View Co-Clustering (WMVCC) algorithm. The motivation behind the approach is to use the diversity of features provided by multiple sources of information while exploiting the power of co-clustering. The proposed method expands the clustering objective function to a unified co-clustering objective function across all the multiple views. The algorithm follows the k-means strategy and iteratively optimizes the clustering by updating cluster labels, features, and view weights. A local search is also employed to optimize the clustering result using weighted multi-step paths in a graph. Experiments are conducted on several benchmark datasets. The results show that the proposed approach converges quickly, and the clustering performance significantly outperforms other recent and state-of-the-art algorithms on sparse datasets.
Model Inversion Attack: Analysis under Gray-box Scenario on Deep Learning based Face Recognition System

In a wide range of ML applications, the training data contains privacy-sensitive information that should be kept secure.Training the ML systems by privacy-sensitive data makes the ML model inherent to the data.As the structure of the model has been fine-tuned by training data, the model can be abused for accessing the data by the estimation in a reverse process called model inversion attack (MIA).Although, MIA has been applied to shallow neural network models of recognizers in literature and its threat in privacy violation has been approved, in the case of a deep learning (DL) model, its efficiency was under question.It was due to the complexity of a DL model structure, big number of DL model parameters, the huge size of training data, big number of registered users to a DL model and thereof big number of class labels.This research work first analyses the possibility of MIA on a deep learning model of a recognition system, namely a face recognizer.Second, despite the conventional MIA under the white box scenario of having partial access to the users' non-sensitive information in addition to the model structure, the MIA is implemented on a deep face recognition system by just having the model structure and parameters but not any user information.In this aspect, it is under a semi-white box scenario or in other words a gray-box scenario.The experimental results in targeting five registered users of a CNN-based face recognition system approve the possibility of regeneration of users' face images even for a deep model by MIA under a gray box scenario.Although, for some images the evaluation recognition score is low and the generated images are not easily recognizable, but for some other images the score is high and facial features of the targeted identities are observable.The objective and subjective evaluations demonstrate that privacy cyber-attack by MIA on a deep recognition system not only is feasible but also is a serious threat with increasing alert state in the future as there is considerable potential for integration more advanced ML techniques to MIA.
Random Differential Fault Attacks on the Lightweight Authenticated Encryption Stream Cipher Grain-128AEAD

Grain-128AEAD is a lightweight authenticated encryption stream cipher and one of the finalists in the National Institute of Standards and Technology (NIST) Lightweight Cryptography (LWC) project. This paper provides an independent third-party analysis of Grain-128AEAD against fault attacks. We investigate the application of three differential fault attack models on Grain-128AEAD. All these attacks can recover the initial state of Grain-128AEAD. First, we demonstrate an attack using a bit-flipping fault that requires access to 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">7.80</sup> faulty outputs to recover the initial state. Then, we demonstrate an attack with a more relaxed assumption of a random fault with a probabilistic approach. Our probabilistic random fault attack requires access to 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11.60</sup> faulty outputs and 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">10.45</sup> fault injections to recover the initial state with a success rate over 99%. Both of the above two attacks are based on precise control on the fault target. Finally, we apply a random fault attack with a deterministic approach (can conclusively determine the random fault value) and using different precision controls. For the precise control, we use existing approaches that have been applied to other ciphers, such as Tiaoxin-346. We also propose a technique for less stringent precision models, such as moderate control and no control, which are more practical than the precise control. Our result indicates that the deterministic random fault attack with a precise control requires an average of 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">7.64</sup> fault injections and a data complexity of 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">8.80</sup> . The deterministic random fault attack with moderate control requires a weak assumption on the fault injection and hence, is the best attack presented in this paper; and is expected to require about 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">9.39</sup> fault injections with a data complexity of about 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">12.98</sup> . All the attacks discussed in this paper are verified experimentally.
An Action Recognition Algorithm for Sprinters Using Machine Learning

The advancements in modern science and technology have greatly promoted the progress of sports science. Advanced technological methods have been widely used in sports training, which have not only improved the scientific level of training but also promoted the continuous growth of sports technology and competition results. With the development of sports science and the gradual deepening of sport practices, the use of scientific training methods and monitoring approaches has improved the effect of sports training and athletes’ performance. This paper takes sprint as the research problem and constructs the image of sprinter’s action recognition based on machine learning. In view of the shortcomings of traditional dual-stream convolutional neural network for processing long-term video information, the time-segmented dual-stream network, based on sparse sampling, is used to better express the characteristics of long-term motion. First, the continuous video frame data is divided into multiple segments, and a short sequence of data containing user actions is formed by randomly sampling each segment of the video frame sequence. Next, it is applied to the dual-stream network for feature extraction. The optical flow image extraction involved in the dual-stream network is implemented by the system using the Lucas–Kanade algorithm. The system in this paper has been tested in actual scenarios, and the results show that the system design meets the expected requirements of the sprinters.
Predicting Visual Search Task Success from Eye Gaze Data as a Basis for User-Adaptive Information Visualization Systems

Information visualizations are an efficient means to support the users in understanding large amounts of complex, interconnected data; user comprehension, however, depends on individual factors such as their cognitive abilities. The research literature provides evidence that user-adaptive information visualizations positively impact the users’ performance in visualization tasks. This study attempts to contribute toward the development of a computational model to predict the users’ success in visual search tasks from eye gaze data and thereby drive such user-adaptive systems. State-of-the-art deep learning models for time series classification have been trained on sequential eye gaze data obtained from 40 study participants’ interaction with a circular and an organizational graph. The results suggest that such models yield higher accuracy than a baseline classifier and previously used models for this purpose. In particular, a Multivariate Long Short Term Memory Fully Convolutional Network shows encouraging performance for its use in online user-adaptive systems. Given this finding, such a computational model can infer the users’ need for support during interaction with a graph and trigger appropriate interventions in user-adaptive information visualization systems. This facilitates the design of such systems since further interaction data like mouse clicks is not required.
Superconducting single-photon detector with a system efficiency of 93% operated in a 2.4 K space-application-compatible cryocooler

Abstract We report herein the development of a superconducting nanowire single-photon detector (SNSPD) that uses a hybrid cryocooler compatible with space applications. The hybrid cryocooler incorporates a two-stage high-frequency pulse-tube cryocooler and a Joule–Thomson loop, which produces a minimum temperature of 2.2 K. The SNSPD with the structure of sandwiched nanowires integrated in the cryocooler shows a maximum detection efficiency of 93% at 1550 nm, which makes it promising for deep-space communications and quantum information applications.
Geometric Calibration Updates to Landsat 7 ETM+ Instrument for Landsat Collection 2 Products

The Landsat 7 (L7) spacecraft and its instrument, the enhanced thematic mapper plus (ETM+), have been consistently characterized and calibrated since its launch in April of 1999. These performance metrics and calibration updates are determined through the U. S. Geological Survey (USGS) Landsat image assessment system (IAS), which has been performing this function since launch. Starting in 2016, the USGS adopted a tiered collection management structure for its Landsat data products that ensures a consistent method of processing for the Landsat archive within a given collection while allowing a set of calibration updates to be performed between any two given collections. The time frame between 2016 and the end of 2020 was part of the Landsat data Collection 1, in the middle of 2020 was the start of the Landsat Collection 2 data products. The start of a given collection initiates the reprocessing of the Landsat archive, which may involve one or more of a set of updated calibration parameters, improvements in the support data needed for product generation, and improved algorithms used in both the processing flow of products along with the characterization and calibration of the Landsat instruments and spacecraft. This paper discusses only the ETM+ geometric spacecraft and instrument calibration improvements for Collection 2. Three ETM+ calibration updates were made for the ETM+; updates to the thermal band odd-to-even detector alignment, sensor to attitude control system (ACS) alignment, and a cold-to-warm focal plane alignment adjustment. The sensor alignment updates impact only the accuracy of the systematic terrain products (L1GT), which are the products generated before applying any corrections based on the ground control used in registration. The band alignment changes impacted only bands 5, 6, and 7 within the focal plane. Other geometric calibration updates, such as scan mirror alignment, are done on a routine basis and are not part of the Collection 2 updates due to their more dynamic characteristics.
Infrared phase measuring deflectometry by using defocused binary fringe

Three-dimensional surface information acquisition of specular objects plays an important role in the fields of automobile industry, aerospace, cultural relic protection, intelligent robotics, equipment manufacturing, and so on. Most of the existing specular surface measurement methods are based on focused sinusoidal fringe patterns, so there are certain requirements for the range of the depth of field (DOF) of the camera on the focus position. However, for many specular surfaces with a large gradient, the tested objects may not always be in the DOF of the camera, so sinusoidal fringe patterns are defocused to be vulnerable to the noise. In this Letter, a new infrared phase measuring deflectometry (PMD) based on defocused binary fringe is proposed that combines a binary fringe defocusing technique and direct PMD. The measurement principle and the corresponding system calibration method are described. The feasibility and measurement accuracy of fringe defocus in specular measurement are studied in principle. The experimental results on several specular objects show that the proposed method can effectively measure specular surfaces out of the DOF of the camera.
Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction

Opinion target extraction and opinion term extraction are two fundamental tasks in Aspect Based Sentiment Analysis (ABSA).Many recent works on ABSA focus on Targetoriented Opinion Words (or Terms) Extraction (TOWE), which aims at extracting the corresponding opinion words for a given opinion target.TOWE can be further applied to Aspect-Opinion Pair Extraction (AOPE) which aims at extracting aspects (i.e., opinion targets) and opinion terms in pairs.In this paper, we propose Target-Specified sequence labeling with Multi-head Self-Attention (TSMSA) for TOWE, in which any pre-trained language model with multi-head self-attention can be integrated conveniently.As a case study, we also develop a Multi-Task structure named MT-TSMSA for AOPE by combining our TSMSA with an aspect and opinion term extraction module.Experimental results indicate that TSMSA outperforms the benchmark methods on TOWE significantly; meanwhile, the performance of MT-TSMSA is similar or even better than state-of-the-art AOPE baseline models.
Application of Deep-Learning Methods to Real Time Face Mask Detection

Due to the high rate of infection and the lack of a specific vaccine or medication for the new disease known as SARS-CoV2, the World Health Organization (WHO) has recommended the use of Personal Protective Equipment (PPE) as the main measure to avoid or reduce infections. One way to maximize compliance with this recommendation is through an automatic system that can recognize in real time whether a person is correctly using the corresponding PPE. This work presents the design, implementation and performance analysis of a system for recognizing the use of masks from image sequences, with the ability to operate in real time. Based on a generic object detection network, a training scheme is proposed for a detector of faces with masks and faces without masks, wherewith an average detection accuracy higher than 90% is obtained. This accuracy can be improved by using a network with a greater number of parameters, but with a longer computation time. The performance of the detector is validated with video sequences of people with and without facemasks, captured in different environments.
Scholar2vec: Vector Representation of Scholars for Lifetime Collaborator Prediction

While scientific collaboration is critical for a scholar, some collaborators can be more significant than others, e.g., lifetime collaborators. It has been shown that lifetime collaborators are more influential on a scholar’s academic performance. However, little research has been done on investigating predicting such special relationships in academic networks. To this end, we propose Scholar2vec, a novel neural network embedding for representing scholar profiles. First, our approach creates scholars’ research interest vector from textual information, such as demographics, research, and influence. After bridging research interests with a collaboration network, vector representations of scholars can be gained with graph learning. Meanwhile, since scholars are occupied with various attributes, we propose to incorporate four types of scholar attributes for learning scholar vectors. Finally, the early-stage similarity sequence based on Scholar2vec is used to predict lifetime collaborators with machine learning methods. Extensive experiments on two real-world datasets show that Scholar2vec outperforms state-of-the-art methods in lifetime collaborator prediction. Our work presents a new way to measure the similarity between two scholars by vector representation, which tackles the knowledge between network embedding and academic relationship mining.
DeriveNet for (Very) Low Resolution Image Classification

Images captured from a distance often result in (very) low resolution (VLR/LR) region of interest, requiring automated identification. VLR/LR images (or regions of interest) often contain less information content, rendering ineffective feature extraction and classification. To this effect, this research proposes a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DeriveNet</i> model for VLR/LR classification, which focuses on learning effective class boundaries by utilizing the class-specific domain knowledge. DeriveNet model is jointly trained via two losses: (i) proposed Derived-Margin softmax loss and (ii) the proposed Reconstruction-Center (ReCent) loss. The Derived-Margin softmax loss focuses on learning an effective VLR classifier while explicitly modeling the inter-class variations. The ReCent loss incorporates domain information by learning a HR reconstruction space for approximating the class variations for the VLR/LR samples. It is utilized to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">derive</i> inter-class margins for the Derived-Margin softmax loss. The DeriveNet model has been trained with a novel Multi-resolution Pyramid based data augmentation which enables the model to learn from varying resolutions during training. Experiments and analysis have been performed on multiple datasets for (i) VLR/LR face recognition, (ii) VLR digit classification, and (iii) VLR/LR face recognition from drone-shot videos. The DeriveNet model achieves state-of-the-art performance across different datasets, thus promoting its utility for several VLR/LR classification tasks.
Towards offensive language detection and reduction in four Software Engineering communities

Software Engineering (SE) communities such as Stack Overflow have become unwelcoming, particularly through members’ use of offensive language. Research has shown that offensive language drives users away from active engagement within these platforms. This work aims to explore this issue more broadly by investigating the nature of offensive language in comments posted by users in four prominent SE platforms – GitHub, Gitter, Slack and Stack Overflow (SO). It proposes an approach to detect and classify offensive language in SE communities by adopting natural language processing and deep learning techniques. Further, a Conflict Reduction System (CRS), which identifies offence and then suggests what changes could be made to minimize offence has been proposed. Beyond showing the prevalence of offensive language in over 1 million comments from four different communities which ranges from 0.07% to 0.43%, our results show promise in successful detection and classification of such language. The CRS system has the potential to drastically reduce manual moderation efforts to detect and reduce offence in SE communities.
UAG: Uncertainty-aware Attention Graph Neural Network for Defending Adversarial Attacks

With the increasing popularity of graph-based learning, graph neural networks (GNNs) emerge as the essential tool for gaining insights from graphs. However, unlike the conventional CNNs that have been extensively explored and exhaustively tested, people are still worrying about the GNNs' robustness under the critical settings, such as financial services. The main reason is that existing GNNs usually serve as a black-box in predicting and do not provide the uncertainty on the predictions. On the other side, the recent advancement of Bayesian deep learning on CNNs has demonstrated its success of quantifying and explaining such uncertainties to fortify CNN models. Motivated by these observations, we propose UAG, the first systematic solution to defend adversarial attacks on GNNs through identifying and exploiting hierarchical uncertainties in GNNs. UAG develops a Bayesian uncertainty technique to explicitly capture uncertainties in GNNs and further employs an uncertainty-aware attention technique to defend adversarial attacks on GNNs. Intensive experiments show that our proposed defense approach outperforms the state-of-the-art solutions by a significant margin.
Towards an Immersive Learning Knowledge Tree - a Conceptual Framework for Mapping Knowledge and Tools in the Field

The interdisciplinary field of immersive learning research is scattered.Combining efforts for better exploration of this field from the different disciplines requires researchers to communicate and coordinate effectively.We call upon the community of immersive learning researchers for planting the Knowledge Tree of Immersive Learning Research, a proposal for a systematization effort for this field, combining both scholarly and practical knowledge, cultivating a robust and ever-growing knowledge base and methodological toolbox for immersive learning.This endeavor aims at promoting evidence-informed practice and guiding future research in the field.This paper contributes with the rationale for three objectives: 1) Developing common scientific terminology amidst the community of researchers; 2) Cultivating a common understanding of methodology, and 3) Advancing common use of theoretical approaches, frameworks, and models.