pwc-archive/papers-with-abstracts huggingface dataset:
CARE: Causality Reasoning for Empathetic Responses by Conditional Graph Generation
Authors: Jiashuo Wang, Yi Cheng, Wenjie Li
Date: 2022-11-01 00:00:00
Tasks: Decoder, Empathetic Response Generation, Graph Generation, Response Generation

Recent approaches to empathetic response generation incorporate emotion causalities to enhance comprehension of both the user's feelings and experiences. However, these approaches suffer from two critical issues. First, they only consider causalities between the user's emotion and the user's experiences, and ignore those between the user's experiences. Second, they neglect interdependence among causalities and reason them independently. To solve the above problems, we expect to reason all plausible causalities interdependently and simultaneously, given the user's emotion, dialogue history, and future dialogue content. Then, we infuse these causalities into response generation for empathetic responses. Specifically, we design a new model, i.e., the Conditional Variational Graph Auto-Encoder (CVGAE), for the causality reasoning, and adopt a multi-source attention mechanism in the decoder for the causality infusion. We name the whole framework as CARE, abbreviated for CAusality Reasoning for Empathetic conversation. Experimental results indicate that our method achieves state-of-the-art performance.
FADO: Feedback-Aware Double COntrolling Network for Emotional Support Conversation
Authors: Wei Peng, Ziyuan Qin, Yue Hu, Yuqiang Xie, Yunpeng Li
Date: 2022-11-01 00:00:00
Tasks: Response Generation

Emotional Support Conversation (ESConv) aims to reduce help-seekers'emotional distress with the supportive strategy and response. It is essential for the supporter to select an appropriate strategy with the feedback of the help-seeker (e.g., emotion change during dialog turns, etc) in ESConv. However, previous methods mainly focus on the dialog history to select the strategy and ignore the help-seeker's feedback, leading to the wrong and user-irrelevant strategy prediction. In addition, these approaches only model the context-to-strategy flow and pay less attention to the strategy-to-context flow that can focus on the strategy-related context for generating the strategy-constrain response. In this paper, we propose a Feedback-Aware Double COntrolling Network (FADO) to make a strategy schedule and generate the supportive response. The core module in FADO consists of a dual-level feedback strategy selector and a double control reader. Specifically, the dual-level feedback strategy selector leverages the turn-level and conversation-level feedback to encourage or penalize strategies. The double control reader constructs the novel strategy-to-context flow for generating the strategy-constrain response. Furthermore, a strategy dictionary is designed to enrich the semantic information of the strategy and improve the quality of strategy-constrain response. Experimental results on ESConv show that the proposed FADO has achieved the state-of-the-art performance in terms of both strategy selection and response generation. Our code is available at https://github.com/Thedatababbler/FADO.
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition
Authors: Suyoun Kim, Ke Li, Lucas Kabela, Rongqing Huang, Jiedan Zhu, Ozlem Kalinli, Duc Le
Date: 2022-10-31 00:00:00
Tasks: speech-recognition, Speech Recognition

Recently, there has been an increasing interest in two-pass streaming end-to-end speech recognition (ASR) that incorporates a 2nd-pass rescoring model on top of the conventional 1st-pass streaming ASR model to improve recognition accuracy while keeping latency low. One of the latest 2nd-pass rescoring model, Transformer Rescorer, takes the n-best initial outputs and audio embeddings from the 1st-pass model, and then choose the best output by re-scoring the n-best initial outputs. However, training this Transformer Rescorer requires expensive paired audio-text training data because the model uses audio embeddings as input. In this work, we present our Joint Audio/Text training method for Transformer Rescorer, to leverage unpaired text-only data which is relatively cheaper than paired audio-text data. We evaluate Transformer Rescorer with our Joint Audio/Text training on Librispeech dataset as well as our large-scale in-house dataset and show that our training method can improve word error rate (WER) significantly compared to standard Transformer Rescorer without requiring any extra model parameters or latency.
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation
Authors: Xinjian Li, Ye Jia, Chung-Cheng Chiu
Date: 2022-10-31 00:00:00
Tasks: Speech-to-Speech Translation, Translation

Research on speech-to-speech translation (S2ST) has progressed rapidly in recent years. Many end-to-end systems have been proposed and show advantages over conventional cascade systems, which are often composed of recognition, translation and synthesis sub-systems. However, most of the end-to-end systems still rely on intermediate textual supervision during training, which makes it infeasible to work for languages without written forms. In this work, we propose a novel model, Textless Translatotron, which is based on Translatotron 2, for training an end-to-end direct S2ST model without any textual supervision. Instead of jointly training with an auxiliary task predicting target phonemes as in Translatotron 2, the proposed model uses an auxiliary task predicting discrete speech representations which are obtained from learned or random speech quantizers. When a speech encoder pre-trained with unsupervised speech data is used for both models, the proposed model obtains translation quality nearly on-par with Translatotron 2 on the multilingual CVSS-C corpus as well as the bilingual Fisher Spanish-English corpus. On the latter, it outperforms the prior state-of-the-art textless model by +18.5 BLEU.
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
Authors: Rochelle Choenni, Dan Garrette, Ekaterina Shutova
Date: 2022-10-31 00:00:00
Tasks: Cross-Lingual Transfer, Meta-Learning

Large multilingual language models typically share their parameters across all languages, which enables cross-lingual task transfer, but learning can also be hindered when training updates from different languages are in conflict. In this paper, we propose novel methods for using language-specific subnetworks, which control cross-lingual parameter sharing, to reduce conflicts and increase positive transfer during fine-tuning. We introduce dynamic subnetworks, which are jointly updated with the model, and we combine our methods with meta-learning, an established, but complementary, technique for improving cross-lingual transfer. Finally, we provide extensive analyses of how each of our methods affects the models.
An analysis of degenerating speech due to progressive dysarthria on ASR performance
Authors: Katrin Tomanek, Katie Seaver, Pan-Pan Jiang, Richard Cave, Lauren Harrel, Jordan R. Green
Date: 2022-10-31 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Although personalized automatic speech recognition (ASR) models have recently been designed to recognize even severely impaired speech, model performance may degrade over time for persons with degenerating speech. The aims of this study were to (1) analyze the change of performance of ASR over time in individuals with degrading speech, and (2) explore mitigation strategies to optimize recognition throughout disease progression. Speech was recorded by four individuals with degrading speech due to amyotrophic lateral sclerosis (ALS). Word error rates (WER) across recording sessions were computed for three ASR models: Unadapted Speaker Independent (U-SI), Adapted Speaker Independent (A-SI), and Adapted Speaker Dependent (A-SD or personalized). The performance of all three models degraded significantly over time as speech became more impaired, but the performance of the A-SD model improved markedly when it was updated with recordings from the severe stages of speech progression. Recording additional utterances early in the disease before speech degraded significantly did not improve the performance of A-SD models. Overall, our findings emphasize the importance of continuous recording (and model retraining) when providing personalized models for individuals with progressive speech impairments.
Generating Sequences by Learning to Self-Correct
Authors: Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, Yejin Choi
Date: 2022-10-31 00:00:00
Tasks: Language Modeling, Language Modelling, Program Synthesis

Sequence generation applications require satisfying semantic constraints, such as ensuring that programs are correct, using certain keywords, or avoiding undesirable content. Language models, whether fine-tuned or prompted with few-shot demonstrations, frequently violate these constraints, and lack a mechanism to iteratively revise their outputs. Moreover, some powerful language models are of extreme scale or inaccessible, making it inefficient, if not infeasible, to update their parameters for task-specific adaptation. We present Self-Correction, an approach that decouples an imperfect base generator (an off-the-shelf language model or supervised sequence-to-sequence model) from a separate corrector that learns to iteratively correct imperfect generations. To train the corrector, we propose an online training procedure that can use either scalar or natural language feedback on intermediate imperfect generations. We show that Self-Correction improves upon the base generator in three diverse generation tasks - mathematical program synthesis, lexically-constrained generation, and toxicity control - even when the corrector is much smaller than the base generator.
Very Low Resource Sentence Alignment: Luhya and Swahili
Authors: Everlyn Asiko Chimoto, Bruce A. Bassett
Date: 2022-10-31 00:00:00
Tasks: Machine Translation, Sentence, Sentence Embedding, Sentence-Embedding, Sentence Embeddings

Language-agnostic sentence embeddings generated by pre-trained models such as LASER and LaBSE are attractive options for mining large datasets to produce parallel corpora for low-resource machine translation. We test LASER and LaBSE in extracting bitext for two related low-resource African languages: Luhya and Swahili. For this work, we created a new parallel set of nearly 8000 Luhya-English sentences which allows a new zero-shot test of LASER and LaBSE. We find that LaBSE significantly outperforms LASER on both languages. Both LASER and LaBSE however perform poorly at zero-shot alignment on Luhya, achieving just 1.5% and 22.0% successful alignments respectively (P@1 score). We fine-tune the embeddings on a small set of parallel Luhya sentences and show significant gains, improving the LaBSE alignment accuracy to 53.3%. Further, restricting the dataset to sentence embedding pairs with cosine similarity above 0.7 yielded alignments with over 85% accuracy.
Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillation
Authors: Liyong Guo, Xiaoyu Yang, Quandong Wang, Yuxiang Kong, Zengwei Yao, Fan Cui, Fangjun Kuang, Wei Kang, Long Lin, Mingshuang Luo, Piotr Zelasko, Daniel Povey
Date: 2022-10-31 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Knowledge Distillation, Quantization, speech-recognition, Speech Recognition

Knowledge distillation(KD) is a common approach to improve model performance in automatic speech recognition (ASR), where a student model is trained to imitate the output behaviour of a teacher model. However, traditional KD methods suffer from teacher label storage issue, especially when the training corpora are large. Although on-the-fly teacher label generation tackles this issue, the training speed is significantly slower as the teacher model has to be evaluated every batch. In this paper, we reformulate the generation of teacher label as a codec problem. We propose a novel Multi-codebook Vector Quantization (MVQ) approach that compresses teacher embeddings to codebook indexes (CI). Based on this, a KD training framework (MVQ-KD) is proposed where a student model predicts the CI generated from the embeddings of a self-supervised pre-trained teacher model. Experiments on the LibriSpeech clean-100 hour show that MVQ-KD framework achieves comparable performance as traditional KD methods (l1, l2), while requiring 256 times less storage. When the full LibriSpeech dataset is used, MVQ-KD framework results in 13.8% and 8.2% relative word error rate reductions (WERRs) for non -streaming transducer on test-clean and test-other and 4.0% and 4.9% for streaming transducer. The implementation of this work is already released as a part of the open-source project icefall.
Pi theorem formulation of flood mapping
Authors: Mark S. Bartlett, Jared Van Blitterswyk, Martha Farella, Jinshu Li, Curtis Smith, Anthony J. Parolari, Lalitha Krishnamoorthy, Assaad Mrad
Date: 2022-11-01 00:00:00
Tasks: Management, regression

Rapid delineation of flash flood extents is critical to mobilize emergency resources and to manage evacuations, thereby saving lives and property. Machine learning (ML) approaches enable rapid flood delineation with reduced computational demand compared to conventional high-resolution, 2D flood models. However, existing ML approaches are limited by a lack of generalization to never-before-seen conditions. Here, we propose a framework to improve ML model generalization based on dimensionless, multi-scale features that capture the similarity of the flooding process across regions. The dimensionless features are constrained with the Buckingham $\Pi$ theorem and used with a logistic regression model for a probabilistic determination of flood risk. The features were calculated at different scales by varying accumulation thresholds for stream delineation. The modeled flood maps compared well with the results of 2D hydraulic models that are the basis of the Federal Emergency Management Agency (FEMA) flood hazard maps. Dimensionless features outperformed dimensional features, with some of the largest gains (in the AUC) occurring when the model was trained in one region and tested in another. Dimensionless and multi-scale features in ML flood modeling have the potential to improve generalization, enabling mapping in unmapped areas and across a broader spectrum of landscapes, climates, and events.
Amplifying Membership Exposure via Data Poisoning
Authors: Yufei Chen, Chao Shen, Yun Shen, Cong Wang, Yang Zhang
Date: 2022-11-01 00:00:00
Tasks: Data Poisoning, Overall - Test, Transfer Learning

As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks. Such attacks typically lead to test-time accuracy degradation or controlled misprediction. In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples. To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class. We first propose a generic dirty-label attack for supervised classification algorithms. We then propose an optimization-based clean-label attack in the transfer learning scenario, whereby the poisoning samples are correctly labeled and look "natural" to evade human moderation. We extensively evaluate our attacks on computer vision benchmarks. Our results show that the proposed attacks can substantially increase the membership inference precision with minimum overall test-time model performance degradation. To mitigate the potential negative impacts of our attacks, we also investigate feasible countermeasures.
Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics
Authors: Yulong Lu, Dejan Slepčev, Lihan Wang
Date: 2022-11-01 00:00:00

Motivated by the challenge of sampling Gibbs measures with nonconvex potentials, we study a continuum birth-death dynamics. We improve results in previous works [51,57] and provide weaker hypotheses under which the probability density of the birth-death governed by Kullback-Leibler divergence or by $\chi^2$ divergence converge exponentially fast to the Gibbs equilibrium measure, with a universal rate that is independent of the potential barrier. To build a practical numerical sampler based on the pure birth-death dynamics, we consider an interacting particle system, which is inspired by the gradient flow structure and the classical Fokker-Planck equation and relies on kernel-based approximations of the measure. Using the technique of $\Gamma$-convergence of gradient flows, we show that on the torus, smooth and bounded positive solutions of the kernelized dynamics converge on finite time intervals, to the pure birth-death dynamics as the kernel bandwidth shrinks to zero. Moreover we provide quantitative estimates on the bias of minimizers of the energy corresponding to the kernelized dynamics. Finally we prove the long-time asymptotic results on the convergence of the asymptotic states of the kernelized dynamics towards the Gibbs measure.
Electroencephalography and mild cognitive impairment research: A scoping review and bibliometric analysis (ScoRBA)
Authors: Adi Wijaya, Noor Akhmad Setiawan, Asma Hayati Ahmad, Rahimah Zakaria, Zahiruddin Othman
Date: 2022-11-01 00:00:00
Tasks: Articles, EEG, Electroencephalogram (EEG), ERP

Background: Mild cognitive impairment (MCI) is often considered a precursor to Alzheimer's disease (AD) due to the high rate of progression from MCI to AD. Sensitive neural biomarkers may provide a tool for an accurate MCI diagnosis, enabling earlier and perhaps more effective treatment. Despite the availability of numerous neuroscience techniques, electroencephalography (EEG) is the most popular and frequently used tool among researchers due to its low cost and superior temporal resolution. Objective: We conducted a scoping review of EEG and MCI between 2012 and 2022 to track the progression of research in this field. Methods: In contrast to previous scoping reviews, the data charting was aided by co-occurrence analysis using VOSviewer, while data reporting adopted a Patterns, Advances, Gaps, Evidence of Practice, and Research Recommendations (PAGER) framework to increase the quality of the results. Results: Event-related potentials (ERPs) and EEG, epilepsy, quantitative EEG (QEEG), and EEG-based machine learning were the research themes addressed by 2310 peer-reviewed articles on EEG and MCI. Conclusion: Our review identified the main research themes in EEG and MCI with high-accuracy detection of seizure and MCI performed using ERP/EEG, QEEG and EEG-based machine learning frameworks.
Conservative Likelihood Ratio Estimator for Infrequent Data Slightly above a Frequency Threshold
Authors: Masato Kikuchi, Yuhi Kusakabe, Tadachika Ozono
Date: 2022-10-28 00:00:00

A naive likelihood ratio (LR) estimation using the observed frequencies of events can overestimate LRs for infrequent data. One approach to avoid this problem is to use a frequency threshold and set the estimates to zero for frequencies below the threshold. This approach eliminates the computation of some estimates, thereby making practical tasks using LRs more efficient. However, it still overestimates LRs for low frequencies near the threshold. This study proposes a conservative estimator for low frequencies, slightly above the threshold. Our experiment used LRs to predict the occurrence contexts of named entities from a corpus. The experimental results demonstrate that our estimator improves the prediction accuracy while maintaining efficiency in the context prediction task.
Asymmetric Hashing for Fast Ranking via Neural Network Measures
Authors: Khoa Doan, Shulong Tan, Weijie Zhao, Ping Li
Date: 2022-11-01 00:00:00
Tasks: Recommendation Systems

Fast item ranking is an important task in recommender systems. In previous works, graph-based Approximate Nearest Neighbor (ANN) approaches have demonstrated good performance on item ranking tasks with generic searching/matching measures (including complex measures such as neural network measures). However, since these ANN approaches must go through the neural measures several times during ranking, the computation is not practical if the neural measure is a large network. On the other hand, fast item ranking using existing hashing-based approaches, such as Locality Sensitive Hashing (LSH), only works with a limited set of measures. Previous learning-to-hash approaches are also not suitable to solve the fast item ranking problem since they can take a significant amount of time and computation to train the hash functions. Hashing approaches, however, are attractive because they provide a principle and efficient way to retrieve candidate items. In this paper, we propose a simple and effective learning-to-hash approach for the fast item ranking problem that can be used for any type of measure, including neural network measures. Specifically, we solve this problem with an asymmetric hashing framework based on discrete inner product fitting. We learn a pair of related hash functions that map heterogeneous objects (e.g., users and items) into a common discrete space where the inner product of their binary codes reveals their true similarity defined via the original searching measure. The fast ranking problem is reduced to an ANN search via this asymmetric hashing scheme. Then, we propose a sampling strategy to efficiently select relevant and contrastive samples to train the hashing model. We empirically validate the proposed method against the existing state-of-the-art fast item ranking methods in several combinations of non-linear searching functions and prominent datasets.
Secure Web-Based Student Information Management System
Authors: Oluwatosin Samuel Falebita
Date: 2022-10-25 00:00:00
Tasks: Management

The reliability and success of any organization such as academic institution rely on its ability to provide secure, accurate and timely data about its operations. Erstwhile managing student information in academic institution was done through paper-based information system, where academic records are documented in several files that are kept in shelves. Several problems are associated with paper-based information system. Managing information through the manual approach require physical exertion to retrieve, alter, and re-file the paper records. These are nonvalue added services results in data inconsistency and redundancy, currently institutions have migrated to web-based student information management system without considering the security architecture of the web portal. This project seeks to ameliorates and secure how information is being managed in Nigeria Police Academy through the development of a secured web-based student information management system, which has a friendly user interface that provides an easy and secure way to manage academic information such as students information, staff information, course registration, course materials and results. This project was developed using Laravel 5.5 PHP Framework to provide a robust secure web-based student information system that is not vulnerable to 2018 OWASP TOP 10 web vulnerabilities.