Reviewerly/exHarmony huggingface dataset:
Visual Alignment Constraint for Continuous Sign Language Recognition

Vision-based Continuous Sign Language Recognition (CSLR) aims to recognize unsegmented signs from image streams. Overfitting is one of the most critical problems in CSLR training, and previous works show that the iterative training scheme can partially solve this problem while also costing more training time. In this study, we revisit the iterative training scheme in recent CSLR works and realize that sufficient training of the feature extractor is critical to solving the overfitting problem. Therefore, we propose a Visual Alignment Constraint (VAC) to enhance the feature extractor with alignment supervision. Specifically, the proposed VAC comprises two auxiliary losses: one focuses on visual features only, and the other enforces prediction alignment between the feature extractor and the alignment module. Moreover, we propose two metrics to reflect overfitting by measuring the prediction inconsistency between the feature extractor and the alignment module. Experimental results on two challenging CSLR datasets show that the proposed VAC makes CSLR networks end-to-end trainable and achieves competitive performance.
A comparative study on temporal phase unwrapping methods in high-speed fringe projection profilometry

Phase unwrapping, either spatial or temporal, plays an important role in fringe projection profilometry (FPP) to recover a continuous phase map. Compared with the spatial phase unwrapping (SPU), the temporal phase unwrapping (TPU), including multi/two-frequency, phase coding, and gray code methods, is more widely used due to its ability to measure discontinuous objects. However, multiple patterns are required in TPU, which limits the measurement speed and the application of dynamic object measurement. To overcome this problem, different techniques, such as binary defocusing projection, two-plus-one phase-shifting algorithm, geometry/continuity constrained phase unwrapping, and ternary/quaternary gray code phase unwrapping have been proposed and actively studied recently, which either enhance the projection speed or reduce the number of projected patterns. However, there are very few studies on how these techniques affect the accuracy of TPU and which TPU method is most accurate under the scenario of high-speed measurement. This paper compares the accuracy of the two-frequency (TF), phase coding (PC), and gray code (GC) methods with different situations, including the traditional 8-bit focused FPP (aFPP), the high-speed binary defocused FPP (bFPP), and the geometry/continuity constrained binary defocused FPP (cFPP). We classify the phase unwrapping errors caused by system noises into uniformly and non-uniformly distributed errors, and analyze their distributions and rates in different TPU methods and different FPP systems. By comparative simulations and experiments, we find that, for low-frequency phase unwrapping, all the three TPU methods have a good result, while for high-frequency phase unwrapping which is desired in high-quality measurement, GC in aFPP and TF in cFPP provide higher accuracy. Thus, for measurement where accuracy is more concerned than the speed GC in aFPP is preferred, for dynamic measurement where extreme high speed is required, TF in cFPP is suggested.
Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework

Semi-supervised learning on graphs is an important problem in the machine learning area. In recent years, state-of-the-art classification methods based on graph neural networks (GNNs) have shown their superiority over traditional ones such as label propagation. However, the sophisticated architectures of these neural models will lead to a complex prediction mechanism, which could not make full use of valuable prior knowledge lying in the data, e.g., structurally correlated nodes tend to have the same class. In this paper, we propose a framework based on knowledge distillation to address the above issues. Our framework extracts the knowledge of an arbitrary learned GNN model (teacher model), and injects it into a well-designed student model. The student model is built with two simple prediction mechanisms, i.e., label propagation and feature transformation, which naturally preserves structure-based and feature-based prior knowledge, respectively. In specific, we design the student model as a trainable combination of parameterized label propagation and feature transformation modules. As a result, the learned student can benefit from both prior knowledge and the knowledge in GNN teachers for more effective predictions. Moreover, the learned student model has a more interpretable prediction process than GNNs. We conduct experiments on five public benchmark datasets and employ seven GNN models including GCN, GAT, APPNP, SAGE, SGC, GCNII and GLP as the teacher models. Experimental results show that the learned student model can consistently outperform its corresponding teacher model by on average. Code and data are available at https://github.com/BUPT-GAMMA/CPF
Dual self-attention with co-attention networks for visual question answering

Visual Question Answering (VQA) as an important task in understanding vision and language has been proposed and aroused wide interests. In previous VQA methods, Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) are generally used to extract visual and textual features respectively, and then the correlation between these two features is explored to infer the answer. However, CNN mainly focuses on extracting local spatial information and RNN pays more attention on exploiting sequential architecture and long-range dependencies. It is difficult for them to integrate the local features with their global dependencies to learn more effective representations of the image and question. To address this problem, we propose a novel model, i.e., Dual Self-Attention with Co-Attention networks (DSACA), for VQA. It aims to model the internal dependencies of both the spatial and sequential structure respectively by using the newly proposed self-attention mechanism. Specifically, DSACA mainly contains three submodules. The visual self-attention module selectively aggregates the visual features at each region by a weighted sum of the features at all positions. The textual self-attention module automatically emphasizes the interdependent word features by integrating associated features among the sentence words. Besides, the visual-textual co-attention module explores the close correlation between visual and textual features learned from self-attention modules. The three modules are integrated into an end-to-end framework to infer the answer. Extensive experiments performed on three generally used VQA datasets confirm the favorable performance of DSACA compared with state-of-the-art methods.
Hybrid whale optimization algorithm with gathering strategies for high-dimensional problems

In order to solve the problems, such as insufficient search ability and low search efficiency, of Whale Optimization Algorithm (WOA) in solving high-dimensional problems, a novel Hybrid WOA with Gathering strategies (HWOAG) is proposed in this paper. Firstly, an individual-based updating way is used in HWOAG instead of the dimension-based updating one of WOA to reduce the computational complexity and to be more suitable for high-dimensional problems. Secondly, a random opposition learning strategy is embedded into the individual-based WOA to form an opposition learning WOA (OWOA), and Grey Wolf Optimizer (GWO) is integrated into OWOA to form an OWOA with GWO (OWOAG) so as to improve the global search ability of WOA. Finally, two standalone OWOAGs are formulated to balance exploration and exploitation better. The two OWOAGs adopt strategies such as switching parameter tuning, random differential disturbance and global-best spiral operator to get stronger search ability. A lot of experimental results on high-dimensional (i.e. 1000-, 2000-, 4000- and 8000- dimensional) benchmark functions and clustering datasets for Fuzzy C-Means (FCM) optimization show that HWOAG has stronger search ability and higher search efficiency than WOA and quite a few state-of-the-art algorithms and that all the strategies gathered to WOA are effective. The source codes of the proposed algorithm HWOAG are available at https://github.com/kangzhai/HWOAG.
Enhancing Arabic aspect-based sentiment analysis using deep learning models

Aspect-based sentiment analysis is a special type of sentiment analysis that aims to identify the discussed aspects and their sentiment polarities in a given review. In this paper, two deep learning models are proposed to address essential aspect-based sentiment analysis tasks: aspect-category identification and aspect-sentiment classification. For the first task, an identification model is proposed based on a convolutional neural network and stacked independent long-short term memory. For the second task, a classification model is proposed based on stacked bidirectional independent long-short term memory, a position-weighting mechanism, and multiple attention mechanism layers. The proposed models are evaluated using the Arabic SemEval-2016 dataset for the Hotels domain. Experimental results demonstrate that the proposed models outperform the baseline and other models, where the first model, C-IndyLSTM, achieves an F1 measure of 58.08%, and the second model, MBRA, achieves an accuracy measure of 87.31%.
Visual interpretability in 3D brain tumor segmentation network

Abstract Medical image segmentation is a complex yet one of the most essential tasks for diagnostic procedures such as brain tumor detection. Several 3D Convolutional Neural Network (CNN) architectures have achieved remarkable results in brain tumor segmentation. However, due to the black-box nature of CNNs, the integration of such models to make decisions about diagnosis and treatment is high-risk in the domain of healthcare. It is difficult to explain the rationale behind the model's predictions due to the lack of interpretability. Hence, the successful deployment of deep learning models in the medical domain requires accurate as well as transparent predictions. In this paper, we generate 3D visual explanations to analyze the 3D brain tumor segmentation model by extending a post-hoc interpretability technique. We explore the advantages of a gradient-free interpretability approach over gradient-based approaches. Moreover, we interpret the behavior of the segmentation model with respect to the input Magnetic Resonance Imaging (MRI) images and investigate the prediction strategy of the model. We also evaluate the interpretability methodology quantitatively for medical image segmentation tasks. To deduce that our visual explanations do not represent false information, we validate the extended methodology quantitatively. We learn that the information captured by the model is coherent with the domain knowledge of human experts, making it more trustworthy. We use the BraTS-2018 dataset to train the 3D brain tumor segmentation network and perform interpretability experiments to generate visual explanations.
Fake Profile Detection on Social Networking Websites: A Comprehensive Review

This article aims to summarize the recent advancement in the fake account detection methodology on social networking websites. Over the past decade, social networking websites have received huge attention from users all around the world. As a result, popular websites such as Facebook, Twitter, LinkedIn, Instagram, and others saw an unexpected rise in registered users. However, researchers claim that all registered accounts are not real; many of them are fake and created for specific purposes. The primary purpose of fake accounts is to spread spam content, rumor, and other unauthentic messages on the platform. Hence, it is needed to filter out the fake accounts, but it has many challenges. In the past few years, researchers applied many advanced technologies to identify fake accounts. In the survey presented in this article, we summarize the recent development of fake account detection technologies. We discuss the challenges and limitations of the existing models in brief. The survey may help future researchers to identify the gaps in the current literature and develop a generalized framework for fake profile detection on social networking websites.
Cyberbullying Detection on Social Networks Using Machine Learning Approaches

The use of social media has grown exponentially over time with the growth of the Internet and has become the most influential networking platform in the 21st century. However, the enhancement of social connectivity often creates negative impacts on society that contribute to a couple of bad phenomena such as online abuse, harassment cyberbullying, cybercrime and online trolling. Cyberbullying frequently leads to serious mental and physical distress, particularly for women and children, and even sometimes force them to attempt suicide. Online harassment attracts attention due to its strong negative social impact. Many incidents have recently occurred worldwide due to online harassment, such as sharing private chats, rumours, and sexual remarks. Therefore, the identification of bullying text or message on social media has gained a growing amount of attention among researchers. The purpose of this research is to design and develop an effective technique to detect online abusive and bullying messages by merging natural language processing and machine learning. Two distinct freatures, namely Bag-of Words (BoW) and term frequency-inverse text frequency (TFIDF), are used to analyse the accuracy level of four distinct machine learning algorithms.
COVID-19 contact tracing apps: UK public perceptions

In order to combat the COVID-19 pandemic, policymakers around the globe have increasingly invested in digital health technologies to support the 'test, track and trace' approach of containing the spread of the novel coronavirus. These technologies include mobile 'contact tracing' applications (apps), which can trace individuals likely to have come into contact with those who have reported symptoms or tested positive for the virus and request that they self-isolate. This paper takes a critical public health perspective that advocates for 'genuine participation' in public health interventions and emphasises the need to take citizen's knowledge into account during public health decision-making. In doing so, it presents and discusses the findings of a UK interview study that explored public views on the possibility of using a COVID-19 contact-tracing app public health intervention at the time the United Kingdom (UK) Government announced their decision to develop such a technology. Findings illustrated interviewees' range and degree of understandings, misconceptions, and concerns about the possibility of using an app. In particular, concerns about privacy and surveillance predominated. Interviewees associated these concerns much more broadly than health by identifying with pre-existent British national narratives associated with individual liberty and autonomy. In extending and contributing to ongoing sociological research with public health, we argue that understanding and responding to these matters is vital, and that our findings demonstrate the need for a forward-looking, anticipatory strategy for public engagement as part of the responsible innovation of the COVID-19 contact-tracing app in the UK.
Toward Blockchain for Edge-of-Things: A New Paradigm, Opportunities, and Future Directions

Blockchain is gaining momentum as a promising technology for many application domains, one of them being the Edge-of- Things (EoT) that is enabled by the integration of edge computing and the Internet-of-Things (IoT). Particularly, the amalgamation of blockchain and EoT leads to a new paradigm, called blockchain enabled EoT (BEoT) that is crucial for enabling future low-latency and high-security services and applications. This article envisions a novel BEoT architecture for supporting industrial applications under the management of blockchain at the network edge in a wide range of IoT use cases such as smart home, smart healthcare, smart grid, and smart transportation. The potentials of BEoT in providing security services are also explored, including access authentication, data privacy preservation, attack detection, and trust management. Finally, we point out some key research challenges and future directions in this emerging area.
Comparative Performance Assessments of Machine-Learning Methods for Artificial Seismic Sources Discrimination

Mankind is vulnerable to artificial seismic sources and accompanying explosions’ consequences. Recently, seismicity catalog contamination is among the main problems faced by seismologists. Since identifying artificial seismic sources is the first and always challenging stage, it is imperative to develop an automated control system that will discriminate tectonic from non-tectonic events. Detection and removal of the artificial seismic sources have become urgent. Early treatments and cleaning of contaminated seismicity catalogs are crucial to assist in accurate seismic hazard identification and enhance the planning of future urban developments. With the advancement of machine learning (ML) techniques, artificial seismic source detection accuracy has been improved. Today, there are different kinds of methods, ML techniques, and diverse processes like knowledge discovery are developed for discriminating artificial seismic sources and earthquakes. ML techniques offer various probabilistic and statistical methods that allow intelligent systems to learn from reoccurring experiences to detect and identify patterns from a dataset. This study aims to build an automated system that is able to detect the existence of artificial seismic sources in seismicity catalogs. More concretely, we classify seismic activity reports into two classes using classical and ensemble ML algorithms. Classical seismicity parameters or features are supplied to linear and nonlinear ML classifiers. The proposed scheme based on the four features (Latitude, Longitude, depth, and Magnitude) can enhance the performance. To assure the enhanced performance, we have examined the proposed scheme by both the accuracy of each model, ROC curves, Precision-Recall, and Calibration. The obtained results prove that the ensemble learning algorithms exhibit better results compared to other classical ML algorithms by having 98.14% testing accuracy.
A Two-Stage Underwater Enhancement Network Based on Structure Decomposition and Characteristics of Underwater Imaging

Due to the scattering and attenuation of light into the water, the underwater image usually appears with color distortion, blurred details, and low contrast. To address these problems, a novel two-stage underwater image convolutional neural network (CNN) based on structure decomposition (UWCNN-SD) for underwater image enhancement is proposed by considering the characteristics of underwater imaging. Specifically, the raw underwater image is decomposed into high-frequency and low-frequency based on theoretical analysis of the underwater imaging. Then, a two-stage underwater enhancement network including a preliminary enhancement network and a refinement network is proposed. In the first stage, the preliminary enhancement network, which contains the high-frequency and the low-frequency enhancement networks, is proposed. The high-frequency part is enhanced directly by a deep learning network, and the low-frequency enhancement network is based on the underwater imaging, which is integrated transmission map and background light into joint component map. In the second stage, the refinement network is designed to further optimize the color of the underwater image by considering complexity of underwater imaging. The experimental results of synthetic and real-world underwater images/videos demonstrate that the proposed UWCNN-SD method can perform color correction and enhancement on different types of underwater images. The ablation study verifies the effectiveness of each component, and application tests further illustrate that the proposed UWCNN-SD method can obtain underwater images with higher visual quality. The trained model is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wushengcong/UWCNN-SD</uri> .
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least approximately, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can recover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.
TapID: Rapid Touch Interaction in Virtual Reality using Wearable Sensing

Current Virtual Reality systems typically use cameras to capture user input from controllers or free-hand mid-air interaction. In this paper, we argue that this is a key impediment to productivity scenarios in VR, which require continued interaction over prolonged periods of time-a requirement that controller or free-hand input in mid-air does not satisfy. To address this challenge, we bring rapid touch interaction on surfaces to Virtual Reality-the input modality that users have grown used to on phones and tablets for continued use. We present TapID, a wrist-based inertial sensing system that complements headset-tracked hand poses to trigger input in VR. TapID embeds a pair of inertial sensors in a flexible strap, one at either side of the wrist; from the combination of registered signals, TapID reliably detects surface touch events and, more importantly, identifies the finger used for touch. We evaluated TapID in a series of user studies on event-detection accuracy (F1 = 0.997) and hand-agnostic finger-identification accuracy (within-user: F1 = 0.93; across users: F1 = 0.91 after 10 refinement taps and F1 = 0.87 without refinement) in a seated table scenario. We conclude with a series of applications that complement hand tracking with touch input and that are uniquely enabled by TapID, including UI control, rapid keyboard typing and piano playing, as well as surface gestures.