Reviewerly/exHarmony huggingface dataset:
Energy-constrained Self-training for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to transfer the knowledge on a labeled source domain distribution to perform well on an unlabeled target domain. Recently, the deep self-training involves an iterative process of predicting on the target domain and then taking the confident predictions as hard pseudo-labels for retraining. However, the pseudo-labels are usually unreliable, and easily leading to deviated solutions with propagated errors. In this paper, we resort to the energy-based model and constrain the training of the unlabeled target sample with the energy function minimization objective. It can be applied as a simple additional regularization. In this framework, it is possible to gain the benefits of the energy-based model, while retaining strong discriminative performance following a plug-and-play fashion. We deliver extensive experiments on the most popular and large scale UDA benchmarks of image classification as well as semantic segmentation to demonstrate its generality and effectiveness.
SqueezeNet and Fusion Network-Based Accurate Fast Fully Convolutional Network for Hand Detection and Gesture Recognition

Accurate fast hand detection and gesture recognition for hand understanding are still challenging tasks that are influenced by the diversity of hands and the complexity of the scene in color images.To address the above problem, we propose a novel SqueezeNet and fusion network-based fully convolutional network (SF-FCNet) to accurately and quickly perform hand detection and gesture recognition in color images.First, we introduce the first 17-layer structure in the lightweight SqueezeNet as the hand feature extraction network to accelerate the detection and recognition speed by greatly compressing the network parameters.Second, a precise hand prediction fusion network is designed by adding a residual structure to the deconvolutional network to integrate high-and low-level features of hands, and hand detection and gesture recognition are performed on a single convolutional layer at multiple scales to improve the precision and reduce the computational costs.The verification results on the Oxford hand dataset show that SF-FCNet can reach a precision of 84.1% and a speed of 32 FPS.The experimental results show that SF-FCNet can substantially enhance the precision and speed of hand detection and gesture recognition on three benchmark datasets and has a strong generalization ability on a homemade test set.
Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer

Recurrent Neural Network Transducer (RNN-T), like most end-to-end speech recognition model architectures, has an implicit neural network language model (NNLM) and cannot easily leverage unpaired text data during training. Previous work has proposed various fusion methods to incorporate external NNLMs into end-to-end ASR to address this weakness. In this paper, we propose extensions to these techniques that allow RNN-T to exploit external NNLMs during both training and inference time, resulting in 13-18% relative Word Error Rate improvement on Librispeech compared to strong baselines. Furthermore, our methods do not incur extra algorithmic latency and allow for flexible plug- and-play of different NNLMs without re-training. We also share in-depth analysis to better understand the benefits of the different NNLM fusion methods. Our work provides a reliable technique for leveraging unpaired text data to significantly improve RNN-T while keeping the system streamable, flexible, and lightweight.
Learning to Rank for Educational Search Engines

In this digital age, there is an abundance of online educational materials in public and proprietary platforms. To allow effective retrieval of educational resources, it is a necessity to build keyword-based search engines over these collections. In modern Web search engines, high-quality rankings are obtained by applying machine learning techniques, known as learning to rank (LTR). In this article, our focus is on constructing machine-learned ranking models to be employed in a search engine in the education domain. Our contributions are threefold. First, we identify and analyze a rich set of features (including click-based and domain-specific ones) to be employed in educational search. LTR models trained on these features outperform various baselines based on ad-hoc retrieval functions and two neural models. As our second contribution, we utilize domain knowledge to build query-dependent ranking models specialized for certain courses or education levels. Our experiments reveal that query-dependent models outperform both the general ranking model and other baselines. Finally, given well-known importance of user clicks in LTR, our third contribution is for handling singleton queries without any click information. To this end, we propose a new strategy to “propagate” click information from the other, similar, queries to the singleton queries. The proposed click propagation approach yields a better ranking performance than the general ranking model and another baseline from the literature. Overall, these findings reveal that both the general and query-dependent ranking models, trained using LTR approaches, yield high effectiveness in educational search, which may ultimately lead to a better learning experience.
Comparative Study of Machine Learning Algorithms for Fraud Detection in Blockchain

Fraudulent transactions have a huge impact on the economy and trust of a blockchain network. Consensus algorithms like proof of work or proof of stake can verify the validity of the transaction but not the nature of the users involved in the transactions or those who verify the transactions. This makes a blockchain network still vulnerable to fraudulent activities. One of the ways to eliminate fraud is by using machine learning techniques. Machine learning can be of supervised or unsupervised nature. In this paper, we use various supervised machine learning techniques to check for fraudulent and legitimate transactions. We also provide an extensive comparative study of various supervised machine learning techniques like decision trees, Naive Bayes, logistic regression, multilayer perceptron, and so on for the above task.
Streaming Multi-Speaker ASR with RNN-T

Recent research shows end-to-end ASR systems can recognize overlapped speech from multiple speakers. However, all published works have assumed no latency constraints during inference, which does not hold for most voice assistant interactions. This work focuses on multi-speaker speech recognition based on a recurrent neural network transducer (RNN-T) that has been shown to provide high recognition accuracy at a low latency online recognition regime. We investigate two approaches to multi-speaker model training of the RNN-T: deterministic output-target assignment and permutation invariant training. We show that guiding separation with speaker order labels in the former case enhances the high-level speaker tracking capability of RNN-T. Apart from that, with multi-style training on single- and multi-speaker utterances, the resulting models gain robustness against ambiguous numbers of speakers during inference. Our best model achieves a WER of 10.2% on simulated 2-speaker LibriSpeech data, which is competitive with the previously reported state-of-the-art non-streaming model (10.3%), while the proposed model could be directly applied for streaming applications.
Participatory Threat Modelling

We present “participatory threat modelling” as a feminist cybersecurity practice which allows technology research to centre traditionally marginalized and excluded experiences. We facilitated a series of community workshops in which we invited participants to define their own cybersecurity threats, implement changes to defend themselves, and reflect on the role cybersecurity plays in their lives. In doing so, we contest both hierarchical approaches to users in cybersecurity—which seek to ‘solve’ the problems of human behavior—and a tendency in HCI to equate action research with the development of novel technology solutions. Our findings draw highlight barriers to engaging with cybersecurity, the role of personal experiences (for instance of gender, race or sexuality) in shaping this engagement, and the benefits of communal approaches to cybersecurity.
Multi‐scale attention‐based convolutional neural network for classification of breast masses in mammograms

Purpose Breast cancer is the cancer with the highest incidence in women, and early detection can effectively improve the survival rate of patients. Mammography is an important method for physicians to screening breast cancer, but the diagnosis of mammograms by physicians depends largely on clinical practice experience. Studies have shown that using computer‐aided diagnosis techniques can help doctors diagnose breast cancer. Methods In this paper, the method of convolutional neural network is mainly used to classify benign and malignant breast masses in the mammograms. First, we use multi‐scale residual networks and densely connected networks as backbone networks to extract the features of global image patches and local image patches. Second, we use the attention module named convolutional block attention module (CBAM) to improve the two feature extraction networks to enhance the network's feature expression ability. Finally, we fuse the features of multi‐scale image patches to achieve the classification of benign and malignant breast masses. Results In the digital database for screening mammography (DDSM) database, the accuracy, sensitivity, AUC value and corresponding standard deviation of our method are 0.9626 ± 0.0110, 0.9719 ± 0.0126, and 0.9576 ± 0.0064, respectively. Compared with the commonly used ResNet (AUC = 0.8823 ± 0.0112) and DenseNet (AUC = 0.9141 ± 0.0085), the performance of our method has improved. In addition, we also used the INbreast database to train and validate the proposed method. The accuracy, sensitivity, AUC and corresponding standard deviations are 0.9554 ± 0.0296, 0.9605 ± 0.0228, and 0.9468 ± 0.0085, respectively. Conclusions Compared with the previous work, our proposed method uses multi‐scale image features, has better classification performance in breast mass patches classification tasks, and can effectively assist physicians in breast cancer diagnosis.
Multi-Attribute Regression Network for Face Reconstruction

In this paper, we propose a multi-attribute regression network (MARN) to investigate the problem of face reconstruction, especially in challenging cases when faces undergo large variations including severe poses, extreme expressions, and partial occlusions in unconstrained environments. The traditional 3DMM parametric regression method does not distinguish the learning of identity, expression, and attitude attributes, resulting in lacking geometric details in the reconstructed face. We propose to learn a face multi-attribute features during 3D face reconstruction from single 2D images. Our MARN enables the network to better extract the feature information of face identity, expression, and pose attributes. We introduce three loss functions to constrain the above three face attributes respectively. At the same time, we carefully design the geometric contour constraint loss function, using the constraints of sparse 2D face landmarks to improve the reconstructed geometric contour information. The experimental results show that our MARN has achieved significant improvements in 3D face reconstruction and face alignment on the AFLW2000-3D and AFLW datasets.
Tatt‐BiLSTM: Web service classification with topical attention‐based BiLSTM

Abstract With the rapid growth of the number of Web services on the Internet, how to classify Web services correctly and efficiently become particularly important in service management tasks, such as service discovery, service selection, service ranking, and service recommendation. Existing functionality‐based service classification techniques have some drawbacks: (1) the keyword order and context information are not considered; (2) the embedding features of keywords are taken as equal importance to learn the classification model; (3) the topic number is hard to determine manually. Due to these drawbacks, the accuracy of service classification needs to be improved further. At present, deep learning techniques show the strong power in modeling complex and nonlinear function relationship. Thus, to address the problems above, this paper exploits attention mechanism to combine the local implicit state vector of Bidirectional Long Short‐Term Memory Network (BiLSTM) and the global hierarchical Dirichlet process (HDP) topic vector, and proposes a Web service classification approach with topical attention‐based BiLSTM. Specifically, BiLSTM is used to automatically learn the keyword feature representations of Web services. Then, the topic vectors of Web service documents are obtained with HDP by offline training, and topic attention mechanism is adopted to strengthen the feature representation by discriminating the importance or weight of different keywords in Web service documents. Finally, the enhanced Web service feature representation is used as the input of a softmax neural network layer to perform the classification prediction for Web services. Extensive experiments are conducted to validate the effectiveness of the proposed approach.
IdBench: Evaluating Semantic Representations of Identifier Names in Source Code

Identifier names convey useful information about the intended semantics of code. Name-based program analyses use this information, e.g., to detect bugs, to predict types, and to improve the readability of code. At the core of name-based analyses are semantic representations of identifiers, e.g., in the form of learned embeddings. The high-level goal of such a representation is to encode whether two identifiers, e.g., len and size, are semantically similar. Unfortunately, it is currently unclear to what extent semantic representations match the semantic relatedness and similarity perceived by developers. This paper presents IdBench, the first benchmark for evaluating semantic representations against a ground truth created from thousands of ratings by 500 software developers. We use IdBench to study state-of-the-art embedding techniques proposed for natural language, an embedding technique specifically designed for source code, and lexical string distance functions. Our results show that the effectiveness of semantic representations varies significantly and that the best available embeddings successfully represent semantic relatedness. On the downside, no existing technique provides a satisfactory representation of semantic similarities, among other reasons because identifiers with opposing meanings are incorrectly considered to be similar, which may lead to fatal mistakes, e.g., in a refactoring tool. Studying the strengths and weaknesses of the different techniques shows that they complement each other. As a first step toward exploiting this complementarity, we present an ensemble model that combines existing techniques and that clearly outperforms the best available semantic representation.
Knowledge Distillation for Improved Accuracy in Spoken Question Answering

Spoken question answering (SQA) is a challenging task that requires the machine to fully understand the complex spoken documents. Automatic speech recognition (ASR) plays a significant role in the development of QA systems. However, the recent work shows that ASR systems generate highly noisy transcripts, which critically limit the capability of machine comprehension on the SQA task. To address the issue, we present a novel distillation framework. Specifically, we devise a training strategy to perform knowledge distillation (KD) from spoken documents and written counterparts. Our work aims at distilling rich knowledge from the language model to improve the performance of the student model by reducing the misalignment between automatic and manual transcripts. Experiments demonstrate that our approach outperforms several state-of-the-art language models on the Spoken-SQuAD dataset.
Comparative study for machine learning classifier recommendation to predict political affiliation based on online reviews

In the current era of social media, different platforms such as Twitter and Facebook have frequently been used by leaders and the followers of political parties to participate in political events, campaigns, and elections. The acquisition, analysis, and presentation of such content have received considerable attention from opinion-mining researchers. For this purpose, different supervised and unsupervised techniques have been used. However, they have produced less efficient results, which need to be improved by incorporating additional classifiers with the extended data sets. The authors investigate different supervised machine learning classifiers for classifying the political affiliations of users. For this purpose, a data set of political reviews is acquired from Twitter and annotated with different polarity classes. After pre-processing, different machine learning classifiers like K-nearest neighbor, naïve Bayes, support vector machine, extreme gradient boosting, and others, are applied. Experimental results illustrate that support vector machine and extreme gradient boosting have shown promising results for predicting political affiliations.
Adaptive strategy in differential evolution via explicit exploitation and exploration controls

Existing multi-strategy adaptive differential evolution (DE) commonly involves trials of multiple strategies and then rewards better-performing ones with more resources. However, the trials of an exploitative or explorative strategy may result in over-exploitation or over-exploration. To improve the performance, this paper proposes a new strategy adaptation method, named explicit adaptation scheme (Ea scheme), which separates multiple strategies and employs them on-demand. It is done by dividing the evolution process into several Selective-candidate with Similarity Selection (SCSS) generations and adaptive generations. In the SCSS generations, the exploitation and exploration needs are learnt by utilizing a balanced strategy. To meet these needs, in adaptive generations, two other strategies, exploitative or explorative is adaptively used. Experimental studies on benchmark functions demonstrate the effectiveness of Ea scheme when compared with its variants and other adaptation methods. Furthermore, performance comparisons with state-of-the-art evolutionary algorithms and swarm intelligence-based algorithms show that EaDE is very competitive.
NLP for Requirements Engineering: Tasks, Techniques, Tools, and Technologies

Requirements engineering (RE) is one of the most natural language-intensive fields within the software engineering area. Therefore, several works have been developed across the years to automate the analysis of natural language artifacts that are relevant for RE, including requirements documents, but also app reviews, privacy policies, and social media content related to software products. Furthermore, the recent diffusion of game-changing natural language processing (NLP) techniques and plat-forms has also boosted the interest of RE researchers. However, a reference framework to provide a holistic understanding of the field of NLP for RE is currently missing. Based on the results of a recent systematic mapping study, and stemming from a previous ICSE tutorial by one of the authors, this technical briefing gives an overview of NLP for RE tasks, available techniques, supporting tools and NLP technologies. It is oriented to both researchers and practitioners, and will gently guide the audience towards a clearer view of how NLP can empower RE, providing pointers to representative works and specialised tools.
Exploring latent states of problem‐solving competence using hidden Markov model on process data

Abstract The response process of problem‐solving items contains rich information about respondents' behaviours and cognitive process in the digital tasks, while the information extraction is a big challenge. The aim of the study is to use a data‐driven approach to explore the latent states and state transitions underlying problem‐solving process to reflect test‐takers' behavioural patterns, and to investigate how these states and state transitions could be associated with test‐takers' performance. We employed the Hidden Markov Modelling approach to identify test takers' hidden states during the problem‐solving process and compared the frequency of states and/or state transitions between different performance groups. We conducted comparable studies in two problem‐solving items with a focus on the US sample that was collected in PIAAC 2012, and examined the correlation between those frequencies from two items. Latent states and transitions between them underlying the problem‐solving process were identified and found significantly different by performance groups. The groups with correct responses in both items were found more engaged in tasks and more often to use efficient tools to solve problems, while the group with incorrect responses was found more likely to use shorter action sequences and exhibit hesitative behaviours. Consistent behavioural patterns were identified across items. This study demonstrates the value of data‐driven based HMM approach to better understand respondents' behavioural patterns and cognitive transmissions underneath the observable action sequences in complex problem‐solving tasks.
The Case for “Weird Social” in VR/XR

COVID underscores the potential of VR meeting tools to compensate for lack of embodied communication in applications like Zoom. But both research and commercial VR meeting environments typically seek to approximate physical meetings, instead of exploring new capacities of communication and coordination. We argue the most transformative features of VR (and XR more broadly) may look and feel very different from familiar social rituals of physical meetings. Embracing "weird" forms of sociality and embodiment, we incorporate inspiration from a range of sources including: (1) emerging rituals in commercial social VR, (2) existing research on social augmentation systems for meetings, (3) novel examples of embodied VR communication, and (4) a fictionalized vignette envisioning a future with aspects of "Weird Social XR" folded into everyday life. We call upon the research community to approach these speculative forms of alien sociality as opportunities to explore new kinds of social superpowers.
Intensity modulator for secure, stable, and high-performance decoy-state quantum key distribution

Abstract The decoy-state method substantially improves the performance of quantum key distribution (QKD) and perfectly solves crucial issues caused by multiphoton pulses. In recent years, the decoy-state method has occupied a key position in practicality, and almost all the QKD systems have employed the decoy-state method. However, the imperfections of traditional intensity modulators limit the performance of the decoy-state method and bring side channels. In this work, a special intensity modulator and its accompanying modulation method are designed and experimentally verified for the secure, stable, and high-performance decoy-state QKDs. The experimental result indicates that its stable and adjustable intensities, convenient two-level modulation, inherently high speed, and compact structure is extremely fit for future trends and will help the decoy-state method to be perfectly applied to QKD systems.