Reviewerly/exHarmony huggingface dataset:
End-to-End Mispronunciation Detection and Diagnosis From Raw Waveforms

Mispronunciation detection and diagnosis (MDD) is designed to identify pronunciation errors and provide instructive feedback to guide non-native language learners, which is a core component in computer-assisted pronunciation training (CAPT) systems. However, MDD often suffers from the data-sparsity problem due to that collecting non-native data and the associated annotations is time-consuming and labor-intensive. To address this issue, we explore a fully end-to-end (E2E) neural model for MDD, which processes learners&#x0027; speech directly based on raw waveforms. Compared to conventional hand-crafted acoustic features, raw waveforms retain more acoustic phenomena and potentially can help neural networks discover better and more customized representations. To this end, our MDD model adopts a co-called SincNet module to take input a raw waveform and covert it to a suitable vector representation sequence. SincNet employs the cardinal sine (sinc) function to implement learnable bandpass filters, drawing inspiration from the convolutional neural network (CNN). By comparison to CNN, SincNet has fewer parameters and is more amenable to human interpretation. Extensive experiments are conducted on the L2-ARCTIC dataset, which is a publicly-available non-native English speech corpus compiled for research on CAPT. We find that the sinc filters of SincNet can be adapted quickly for non-native language learners of different nationalities. Furthermore, our model can achieve comparable mispronunciation detection performance in relation to state-of-the-art E2E MDD models that take input the standard handcrafted acoustic features. Besides that, our model also provides considerable improvements on phone error rate (PER) and diagnosis accuracy.
DIY: Assessing the Correctness of Natural Language to SQL Systems

Designing natural language interfaces for querying databases remains an important goal pursued by researchers in natural language processing, databases, and HCI. These systems receive natural language as input, translate it into a formal database query, and execute the query to compute a result. Because the responses from these systems are not always correct, it is important to provide people with mechanisms to assess the correctness of the generated query and computed result. However, this assessment can be challenging for people who lack expertise in query languages. We present Debug-It-Yourself (DIY), an interactive technique that enables users to assess the responses from a state-of-the-art natural language to SQL (NL2SQL) system for correctness and, if possible, fix errors. DIY provides users with a sandbox where they can interact with (1) the mappings between the question and the generated query, (2) a small-but-relevant subset of the underlying database, and (3) a multi-modal explanation of the generated query. End-users can then employ a back-of-the-envelope calculation debugging strategy to evaluate the system's response. Through an exploratory study with 12 users, we investigate how DIY helps users assess the correctness of the system's answers and detect & fix errors. Our observations reveal the benefits of DIY while providing insights about end-user debugging strategies and underscore opportunities for further improving the user experience.
Sociodemographic determinants of the adoption of a contact tracing application during the COVID-19 epidemic in Delhi, India

The Aarogya Setu (ASA) is an open-source Indian smartphone application for contact tracing developed by the government of India. To determine the sociodemographic determinants of the use of the ASA among participants of two consecutive rounds of a state-wide seroepidemiological study. We analyzed secondary data of participants aged ≥16 years from the dataset of a repeated cross-sectional serosurvey conducted during August-September 2020 in Delhi, India. The participants reported on whether they had installed the ASA which was subsequently verified by assessing the app status on their mobile phones'. P < 0.05 was considered statistically significant. The Aarogya Setu application (ASA) was currently installed by 7437 (58.3%, 95% C.I. 57.4, 59.1, missing = 533) and 7,997 (55.6%, 95% C.I. 54.8, 56.4, missing = 566) participants during August and September, respectively. The mean (SD) age of the participants was 37.5 (14.6) years including 47.4% males and 52.6% females. The lack of a smartphone was reported by 17.3% of participants. On binary logistic regression analysis, the absence of ASA installation was independently associated with age≥40, female gender, lesser years of education, and lower per capita income. High acceptability of the Aarogya Setu contact tracing mobile phone application was observed in the residents of Delhi. Furthermore, the barriers in utilization of the associated technology were predominantly explained by the socioeconomic, educational and digital divide.
TPGN: A Time-Preference Gate Network for e-commerce purchase intention recognition

The studies on users’ purchase intentions based on e-commerce data are of great significance to marketers, buyers, and society. Current studies on users’ intentions with traditional machine learning methods usually focus on unique features and are time-consuming. Due to the characteristics of user behaviors and the importance of time sequence, deep learning methods are increasingly applied in relevant studies. In the study, in order to predict online user’s purchase intentions, based on Long-Short Term Memory (LSTM) model, we proposed Time-Preference Gate Network (TPGN). A pair of preference gates and a pair of time interval gates are added to the model. The preference gates are used to capture the users’ category preferences at different time and the time interval gates are used to capture the users’ long-term interest. In our model, through coupling the input gate with the forget gate, the parameters of the model are reduced and the performance is improved. In addition, the difference in user gender leads to the difference in the performance. Extended experiments with two real data sets confirmed that our model performed better than other baseline models.
Generalized <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e4213" altimg="si40.svg"><mml:mi>k</mml:mi></mml:math>-means in GLMs with applications to the outbreak of COVID-19 in the United States

Generalized k -means can be combined with any similarity or dissimilarity measure for clustering. Using the well known likelihood ratio or F -statistic as the dissimilarity measure, a generalized k -means method is proposed to group generalized linear models (GLMs) for exponential family distributions. Given the number of clusters k , the proposed method is established by the uniform most powerful unbiased (UMPU) test statistic for the comparison between GLMs. If k is unknown, then the proposed method can be combined with generalized liformation criterion (GIC) to automatically select the best k for clustering. Both AIC and BIC are investigated as special cases of GIC. Theoretical and simulation results show that the number of clusters can be correctly identified by BIC but not AIC. The proposed method is applied to the state-level daily COVID-19 data in the United States, and it identifies 6 clusters. A further study shows that the models between clusters are significantly different from each other, which confirms the result with 6 clusters.
Constructing classifiers for imbalanced data using diversity optimisation

Imbalanced data is challenging in classification. This paper proposes a new approach to address imbalanced data by adopting diversity optimisation to generate synthetic instances for over-sampling the minority class. Diversity optimisation assures that the generated instances are close to the minority group but not identical. It also ensures the optimal spread of the generated instances in the space. We develop two formulations named as Diversity-based Average Distance Over-sampling (DADO) and Diversity-based Instance-Wise Over-sampling (DIWO). We evaluate the proposed formulations' performance by designing experiments using both synthetic and real data with unbalanced classes. We examine the performance through area under curve (AUC), F1-score and g-mean measures in comparison with comparable synthetic over-sampling methods. We compare the methods using the obtained measures of the best performing classifier and statistical testing of all combinations over three imbalance levels using seven classifiers. The results show that both proposed formulations perform competitive to improve the performance of classifiers, and DIWO outperforms other comparable methods. Both perform robust by reducing the classifiers' variance. We discuss the strengths and limitations of these formulations using the real data examples, runtime complexity and sensitivity analysis. We also demonstrate the possibility of utilising DADO and DIWO for multi-class imbalanced data.
Mapless Navigation with Deep Reinforcement Learning based on The Convolutional Proximal Policy Optimization Network

In recent years, mapless navigation with a deep reinforcement learning approach in Autonomous Mobile Robot has shown a considerable benefit in improving robot behavior flexibility. Specifically, the Robot adapts to complex constraints and performs well in various environments without the need for a predetermined map and route plan. However, several previous studies show a lack of stability in the training of deep reinforcement learning networks for mapless navigation. Moreover, the exploration and exploitation in a specific work environment play an essential role in improving mapless navigation performance, which needs to be carefully considered. From the aforementioned issues, as well as inspired by the Proximal Policy optimization algorithm has shown that it does not only evaluates the advantages and disadvantages of policy but also prevents the policy from changing too much after each time weight update. In this paper, we propose to build a Convolutional Proximal Policy optimization network for the Mapless Navigation problem. Furthermore, the use of Boltzmann's policy to help balance exploration and exploitation also contributes to the Robot's ability to explore more deeply in complex environments, and the performance of the mapless navigation problem is also significantly improved.
Residual U-Network for Breast Tumor Segmentation from Magnetic Resonance Images

Breast cancer positions as the most well-known threat and the main source of malignant growth-related morbidity and mortality throughout the world.It is apical of all new cancer incidences analyzed among females.Two features substantially in uence the classi cation accuracy of malignancy and benignity in automated cancer diagnostics.These are the precision of tumor segmentation and appropriateness of extracted attributes required for the diagnosis.In this research, the authors have proposed a ResU-Net (Residual U-Network) model for breast tumor segmentation.The proposed methodology renders augmented, and precise identi cation of tumor regions and produces accurate breast tumor segmentation in contrast-enhanced MR images.Furthermore, the proposed framework also encompasses the residual network technique, which subsequently enhances the performance and displays the improved training process.Over and above, the performance of ResU-Net has experimentally been analyzed with conventional U-Net, FCN8, FCN32.Algorithm performance is evaluated in the form of dice coef cient and MIoU (Mean Intersection of Union), accuracy, loss, sensitivity, speci city, F1score.Experimental results show that ResU-Net achieved validation accuracy & dice coef cient value of 73.22% & 85.32% respectively on the Rider Breast MRI dataset and outperformed as compared to the other algorithms used in experimentation.
Blockchain-Enabled HMM Model for Sports Performance Prediction

The historical training or exam data of an athlete produced in the past sport exercise or test activities have provided a promising way to objectively and accurately evaluate the real-time sport performance of the athlete. However, the continuous generation of sport training or exam data has placed a heavy transmission and processing burden on the traditional centralized data processing paradigm (e.g., cloud platform). Considering this drawback, a decentralized blockchain-based athlete sport data transmission and utilization solution is proposed in this research work. Moreover, the available athlete sport data produced in past sport exercise or test activities is often sparse and time-related, which call for a robust and time-aware data fusion and processing solution. In this situation, HMM model is employed in this article to cope with the data sparsity and dynamics and further make accurate sports performance prediction for athletes accordingly. Finally, we design a set of experiments on a real-world dataset to validate the feasibility of our proposal in terms of effectiveness and efficiency.
A network-based sparse and multi-manifold regularized multiple non-negative matrix factorization for multi-view clustering

Multi-view clustering has attracted increasing attention in recent years since many real data sets are usually gathered from different sources or described by different feature types. Amongst various existing multi-view clustering algorithms, those that are based on non-negative matrix factorization (NMF) have exhibited superior performance. However, NMF decomposing original data directly fails to exploit global relationships between data samples and cannot be applied to datasets that are not strictly non-negative. In this paper, a network-based sparse and multi-manifold regularized multiple NMF (NSM_MNMF) for multi-view clustering is proposed, where multi-view data is transformed into multiple networks, and NMF is used to jointly factorize transformed multiple networks for capturing the shared cluster structure embedded in different views. Furthermore, sparse and multi-manifold regularization are incorporated into NMF to keep the intrinsic geometrical information of the multi-view network manifold space. Networks characterize intra-view similarity, and joint factorization reveals inter-view similarity across distinct views, while using NMF to decompose the networks instead of the original data means NSM_MNMF can be applied to datasets that are not strictly non-negative and the clustering results are interpretable. Extensive experiments are conducted on nine real data sets to assess the method proposed, and the results illustrate that NSM_MNMF outperforms other baseline approaches.
Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension

Compared to the traditional machine reading comprehension (MRC) with limitation to the information in a passage, knowledge-driven MRC tasks aim to enable models to answer the question according to text and related commonsense knowledge. Although pre-trained Transformer-based language models (TrLMs) such as BERT and Roberta, have shown powerful performance in MRC, external knowledge such as unspoken commonsense and world knowledge still cannot be used and explained explicitly. In this work, we present three simple yet effective injection methods integrated into the structure of TrLMs to fine-tune downstream knowledge-driven MRC tasks with off-the-shelf commonsense representations. Moreover, we introduce a mask mechanism for a token-level multi-hop relationship searching to filter external knowledge. We have conducted extensive experiments on DREAM and CosmosQA, two prevalent knowledge-driven datasets. Experimental results indicate that the incremental TrLMs have outperformed the baseline systems by 1%-4.1% with a fewer computational cost. Further analysis shows the effectiveness of the proposed methods and the robustness of the incremental model in the case of an incomplete training set.
Any-Shot Object Detection

Previous work on novel object detection considers zero or few-shot settings where none or few examples of each category are available for training. In real world scenarios, it is less practical to expect that 'all' the novel classes are either unseen or have few-examples. Here, we propose a more realistic setting termed 'Any-shot detection', where totally unseen and few-shot categories can simultaneously co-occur during inference. Any-shot detection offers unique challenges compared to conventional novel object detection such as, a high imbalance between unseen, few-shot and seen object classes, susceptibility to forget base-training while learning novel classes and distinguishing novel classes from the background. To address these challenges, we propose a unified any-shot detection model, that can concurrently learn to detect both zero-shot and few-shot object classes. Our core idea is to use class semantics as prototypes for object detection, a formulation that naturally minimizes knowledge forgetting and mitigates the class-imbalance in the label space. Besides, we propose a rebalanced loss function that emphasizes difficult few-shot cases but avoids overfitting on the novel classes to allow detection of totally unseen classes. Without bells and whistles, our framework can also be used solely for Zero-shot object detection and Few-shot object detection tasks. We report extensive experiments on Pascal VOC and MS-COCO datasets where our approach is shown to provide significant improvements.
Social Density Monitoring Toward Selective Cleaning by Human Support Robot With 3D Based Perception System

Monitoring the safe social distancing then conducting efficient sterilization in potentially crowded public places are necessary but challenging especially during the COVID-19 pandemic. This work presents the 3D human space-based surveillance system enabling selective cleaning framework. To this end, the proposed AI-assisted perception techniques is deployed on Toyota Human Support Robot (HSR) equipped with autonomous navigation, Lidar, and RGBD vision sensor. The human density mapping represented as heatmap was constructed to identify areas with the level being likely the risks for interactions. The surveillance framework adopts the 3D human joints tracking technique and the accumulated asymmetrical Gaussian distribution scheme modeling the human location, size, and direction to quantify human density. The HSR generates the human density map as a grid-based heatmap to perform the safe human distance monitoring task while navigating autonomously inside the pre-built map. Then, the cleaning robot uses the levels of the generated heatmap to sterilize by the selective scheme. The experiment was tested in public places, including food court and wet market. The proposed framework performance analyzed with standard performance metrics in various map sizes spares about 19 % of the disinfection time and 15 % of the disinfection liquid usage, respectively.
A Fringe Phase Extraction Method Based on Neural Network

In optical metrology, the output is usually in the form of a fringe pattern, from which a phase map can be generated and phase information can be converted into the desired parameters. This paper proposes an end-to-end method of fringe phase extraction based on the neural network. This method uses the U-net neural network to directly learn the correspondence between the gray level of a fringe pattern and the wrapped phase map, which is simpler than the exist deep learning methods. The results of simulation and experimental fringe patterns verify the accuracy and the robustness of this method. While it yields the same accuracy, the proposed method features easier operation and a simpler principle than the traditional phase-shifting method and has a faster speed than wavelet transform method.
Runtime Performance Prediction for Deep Learning Models with Graph Neural Network

Deep learning models have been widely adopted in many application domains. Predicting the runtime performance of deep learning models, such as GPU memory consumption and training time, is important for boosting development productivity and reducing resource waste. The reason is that improper configurations of hyperparameters and neural architectures can result in many failed training jobs or unsatisfactory models. However, the runtime performance prediction of deep learning models is challenging because of the hybrid programming paradigm, complicated hidden factors within the framework runtime, enormous model configuration space, and broad differences among models. In this paper, we propose DNNPerf, a novel ML-based tool for predicting the runtime performance of deep learning models using Graph Neural Network. DNNPerf represents a model as a directed acyclic computation graph and incorporates a rich set of performance-related features based on the computational semantics of both nodes and edges. We also propose a new Attention-based Node-Edge Encoder for the node and edge features. DNNPerf is evaluated on thousands of configurations of real-world and synthetic deep learning models to predict their GPU memory consumption and training time. The experimental results show that DNNPerf achieves accurate predictions, with an overall error of 7.4% for the training time prediction and an overall error of 13.7% for the GPU memory consumption prediction, confirming its effectiveness.
Deep saliency detection via spatial-wise dilated convolutional attention

Saliency detection aims to highlight the area which significantly attracts human attention and stands out in an image. In recent years, deep learning-based saliency detection has achieved fantastic performance over conventional works while still facing huge challenges in multi-features fusion and the enlargement of the receptive field. Current top-performing saliency detectors on the basis of FCNs benefit from their powerful feature representations but suffer from high computational costs due to the integration of multi-scale features without distinction. So in this paper, we propose a novel and simple network, the DCAM, based on attention mechanism with dilated convolutions (DAM), incorporating multi-scale features with enlarged receptive field. Specifically, we apply DAM to guide each side output respectively which selectively emphasizes the significant regions, thus efficiently enhancing the representation ability of each layer. Our spatial attention module helps us looking for areas in the image that have a greater impact and give them a higher weight.Besides, we adopt FPN to integrate features adjacent to each other layer and a CRF scheme for refining saliency results. Experiments on five benchmark datasets demonstrate that the proposed approach performs favorably against five state-of-the-art methods with a fast speed (56 FPS on a single GPU).
A Manifesto on Resource Re-Use in Interactive Information Retrieval

This perspective paper on resource re-use intends to draw the attention of the interactive information retrieval (IIR) community to the challenges of research documentation and archiving for future use. Resources are understood as encompassing research designs, research data and research infrastructures. It proposes eight principles for improving the re-use of resources in the IIR community and presents concrete steps on how to achieve them. A five-level system for data archiving and documentation envisions increasingly open and stable documentation and access infrastructures.