pwc-archive/papers-with-abstracts huggingface dataset:
How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis
Authors: Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu
Date: 2022-03-31 00:00:00
Proceeding: Findings (ACL) 2022 5

Recently, there has been a trend to investigate the factual knowledge captured by Pre-trained Language Models (PLMs). Many works show the PLMs' ability to fill in the missing factual words in cloze-style prompts such as "Dante was born in [MASK]." However, it is still a mystery how PLMs generate the results correctly: relying on effective clues or shortcut patterns? We try to answer this question by a causal-inspired analysis that quantitatively measures and evaluates the word-level patterns that PLMs depend on to generate the missing words. We check the words that have three typical associations with the missing words: knowledge-dependent, positionally close, and highly co-occurred. Our analysis shows: (1) PLMs generate the missing factual words more by the positionally close and highly co-occurred words than the knowledge-dependent words; (2) the dependence on the knowledge-dependent words is more effective than the positionally close and highly co-occurred words. Accordingly, we conclude that the PLMs capture the factual knowledge ineffectively because of depending on the inadequate associations.
Improving speaker de-identification with functional data analysis of f0 trajectories
Authors: Lauri Tavi, Tomi Kinnunen, Rosa González Hautamäki
Date: 2022-03-31 00:00:00
Tasks: De-identification

Due to a constantly increasing amount of speech data that is stored in different types of databases, voice privacy has become a major concern. To respond to such concern, speech researchers have developed various methods for speaker de-identification. The state-of-the-art solutions utilize deep learning solutions which can be effective but might be unavailable or impractical to apply for, for example, under-resourced languages. Formant modification is a simpler, yet effective method for speaker de-identification which requires no training data. Still, remaining intonational patterns in formant-anonymized speech may contain speaker-dependent cues. This study introduces a novel speaker de-identification method, which, in addition to simple formant shifts, manipulates f0 trajectories based on functional data analysis. The proposed speaker de-identification method will conceal plausibly identifying pitch characteristics in a phonetically controllable manner and improve formant-based speaker de-identification up to 25%.
End-to-End Table Question Answering via Retrieval-Augmented Generation
Authors: Feifei Pan, Mustafa Canim, Michael Glass, Alfio Gliozzo, James Hendler
Date: 2022-03-30 00:00:00
Tasks: Information Retrieval, Question Answering, RAG, Retrieval, Retrieval-augmented Generation, Table Retrieval

Most existing end-to-end Table Question Answering (Table QA) models consist of a two-stage framework with a retriever to select relevant table candidates from a corpus and a reader to locate the correct answers from table candidates. Even though the accuracy of the reader models is significantly improved with the recent transformer-based approaches, the overall performance of such frameworks still suffers from the poor accuracy of using traditional information retrieval techniques as retrievers. To alleviate this problem, we introduce T-RAG, an end-to-end Table QA model, where a non-parametric dense vector index is fine-tuned jointly with BART, a parametric sequence-to-sequence model to generate answer tokens. Given any natural language question, T-RAG utilizes a unified pipeline to automatically search through a table corpus to directly locate the correct answer from the table cells. We apply T-RAG to recent open-domain Table QA benchmarks and demonstrate that the fine-tuned T-RAG model is able to achieve state-of-the-art performance in both the end-to-end Table QA and the table retrieval tasks.
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Authors: Naoyuki Kanda, Jian Wu, Yu Wu, Xiong Xiao, Zhong Meng, Xiaofei Wang, Yashesh Gaur, Zhuo Chen, Jinyu Li, Takuya Yoshioka
Date: 2022-03-30 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, speaker-diarization, Speaker Diarization, Speaker Identification, speech-recognition, Speech Recognition

This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize ``who spoke what'' with low latency even when multiple people are speaking simultaneously. Our model is based on token-level serialized output training (t-SOT) which was recently proposed to transcribe multi-talker speech in a streaming fashion. To further recognize speaker identities, we propose an encoder-decoder based speaker embedding extractor that can estimate a speaker representation for each recognized token not only from non-overlapping speech but also from overlapping speech. The proposed speaker embedding, named t-vector, is extracted synchronously with the t-SOT ASR model, enabling joint execution of speaker identification (SID) or speaker diarization (SD) with the multi-talker transcription with low latency. We evaluate the proposed model for a joint task of ASR and SID/SD by using LibriSpeechMix and LibriCSS corpora. The proposed model achieves substantially better accuracy than a prior streaming model and shows comparable or sometimes even superior results to the state-of-the-art offline SA-ASR model.
Is Word Error Rate a good evaluation metric for Speech Recognition in Indic Languages?
Authors: Priyanshi Shah, Harveen Singh Chadha, Anirudh Gupta, Ankur Dhuriya, Neeraj Chhimwal, Rishabh Gaur, Vivek Raghavan
Date: 2022-03-30 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Benchmarking, speech-recognition, Speech Recognition

We propose a new method for the calculation of error rates in Automatic Speech Recognition (ASR). This new metric is for languages that contain half characters and where the same character can be written in different forms. We implement our methodology in Hindi which is one of the main languages from Indic context and we think this approach is scalable to other similar languages containing a large character set. We call our metrics Alternate Word Error Rate (AWER) and Alternate Character Error Rate (ACER). We train our ASR models using wav2vec 2.0\cite{baevski2020wav2vec} for Indic languages. Additionally we use language models to improve our model performance. Our results show a significant improvement in analyzing the error rates at word and character level and the interpretability of the ASR system is improved upto $3$\% in AWER and $7$\% in ACER for Hindi. Our experiments suggest that in languages which have complex pronunciation, there are multiple ways of writing words without changing their meaning. In such cases AWER and ACER will be more useful rather than WER and CER as metrics. Further, we open source a new benchmarking dataset of 21 hours for Hindi with the new metric scripts.
Improving Speech Recognition for Indic Languages using Language Model
Authors: Ankur Dhuriya, Harveen Singh Chadha, Anirudh Gupta, Priyanshi Shah, Neeraj Chhimwal, Rishabh Gaur, Vivek Raghavan
Date: 2022-03-30 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

We study the effect of applying a language model (LM) on the output of Automatic Speech Recognition (ASR) systems for Indic languages. We fine-tune wav2vec $2.0$ models for $18$ Indic languages and adjust the results with language models trained on text derived from a variety of sources. Our findings demonstrate that the average Character Error Rate (CER) decreases by over $28$ \% and the average Word Error Rate (WER) decreases by about $36$ \% after decoding with LM. We show that a large LM may not provide a substantial improvement as compared to a diverse one. We also demonstrate that high quality transcriptions can be obtained on domain-specific data without retraining the ASR model and show results on biomedical domain.
Code Switched and Code Mixed Speech Recognition for Indic languages
Authors: Harveen Singh Chadha, Priyanshi Shah, Ankur Dhuriya, Neeraj Chhimwal, Anirudh Gupta, Vivek Raghavan
Date: 2022-03-30 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Identification, speech-recognition, Speech Recognition

Training multilingual automatic speech recognition (ASR) systems is challenging because acoustic and lexical information is typically language specific. Training multilingual system for Indic languages is even more tougher due to lack of open source datasets and results on different approaches. We compare the performance of end to end multilingual speech recognition system to the performance of monolingual models conditioned on language identification (LID). The decoding information from a multilingual model is used for language identification and then combined with monolingual models to get an improvement of 50% WER across languages. We also propose a similar technique to solve the Code Switched problem and achieve a WER of 21.77 and 28.27 over Hindi-English and Bengali-English respectively. Our work talks on how transformer based ASR especially wav2vec 2.0 can be applied in developing multilingual ASR and code switched ASR for Indic languages.
A Cognitive Architecture for Machine Consciousness and Artificial Superintelligence: Thought Is Structured by the Iterative Updating of Working Memory
Authors: Jared Edward Reser
Date: 2022-03-29 00:00:00

This article provides an analytical framework for how to simulate human-like thought processes within a computer. It describes how attention and memory should be structured, updated, and utilized to search for associative additions to the stream of thought. The focus is on replicating the dynamics of the mammalian working memory system, which features two forms of persistent activity: sustained firing (preserving information on the order of seconds) and synaptic potentiation (preserving information from minutes to hours). The article uses a series of figures to systematically demonstrate how the iterative updating of these working memory stores provides functional organization to behavior, cognition, and awareness. In a machine learning implementation, these two memory stores should be updated continuously and in an iterative fashion. This means each state should preserve a proportion of the coactive representations from the state before it (where each representation is an ensemble of neural network nodes). This makes each state a revised iteration of the preceding state and causes successive configurations to overlap and blend with respect to the information they contain. Thus, the set of concepts in working memory will evolve gradually and incrementally over time. Transitions between states happen as persistent activity spreads activation energy throughout the hierarchical network, searching long-term memory for the most appropriate representation to be added to the global workspace. The result is a chain of associatively linked intermediate states capable of advancing toward a solution or goal. Iterative updating is conceptualized here as an information processing strategy, a model of working memory, a theory of consciousness, and an algorithm for designing and programming artificial intelligence (AI, AGI, and ASI).
An energy-based deep splitting method for the nonlinear filtering problem
Authors: Kasper Bågmark, Adam Andersson, Stig Larsson
Date: 2022-03-31 00:00:00

The purpose of this paper is to explore the use of deep learning for the solution of the nonlinear filtering problem. This is achieved by solving the Zakai equation by a deep splitting method, previously developed for approximate solution of (stochastic) partial differential equations. This is combined with an energy-based model for the approximation of functions by a deep neural network. This results in a computationally fast filter that takes observations as input and that does not require re-training when new observations are received. The method is tested on four examples, two linear in one and twenty dimensions and two nonlinear in one dimension. The method shows promising performance when benchmarked against the Kalman filter and the bootstrap particle filter.
Sequential Recommendation with User Evolving Preference Decomposition
Authors: Weiqi Shao, Xu Chen, Long Xia, Jiashu Zhao, Dawei Yin
Date: 2022-03-31 00:00:00
Tasks: Sequential Recommendation

Modeling user sequential behaviors has recently attracted increasing attention in the recommendation domain. Existing methods mostly assume coherent preference in the same sequence. However, user personalities are volatile and easily changed, and there can be multiple mixed preferences underlying user behaviors. To solve this problem, in this paper, we propose a novel sequential recommender model via decomposing and modeling user independent preferences. To achieve this goal, we highlight three practical challenges considering the inconsistent, evolving and uneven nature of the user behavior, which are seldom noticed by the previous work. For overcoming these challenges in a unified framework, we introduce a reinforcement learning module to simulate the evolution of user preference. More specifically, the action aims to allocate each item into a sub-sequence or create a new one according to how the previous items are decomposed as well as the time interval between successive behaviors. The reward is associated with the final loss of the learning objective, aiming to generate sub-sequences which can better fit the training data. We conduct extensive experiments based on six real-world datasets across different domains. Compared with the state-of-the-art methods, empirical studies manifest that our model can on average improve the performance by about 8.21%, 10.08%, 10.32%, and 9.82% on the metrics of Precision, Recall, NDCG and MRR, respectively.
Cross-Domain Recommendation to Cold-Start Users via Variational Information Bottleneck
Authors: Jiangxia Cao, Jiawei Sheng, Xin Cong, Tingwen Liu, Bin Wang
Date: 2022-03-31 00:00:00
Tasks: Recommendation Systems

Recommender systems have been widely deployed in many real-world applications, but usually suffer from the long-standing user cold-start problem. As a promising way, Cross-Domain Recommendation (CDR) has attracted a surge of interest, which aims to transfer the user preferences observed in the source domain to make recommendations in the target domain. Previous CDR approaches mostly achieve the goal by following the Embedding and Mapping (EMCDR) idea which attempts to learn a mapping function to transfer the pre-trained user representations (embeddings) from the source domain into the target domain. However, they pre-train the user/item representations independently for each domain, ignoring to consider both domain interactions simultaneously. Therefore, the biased pre-trained representations inevitably involve the domain-specific information which may lead to negative impact to transfer information across domains. In this work, we consider a key point of the CDR task: what information needs to be shared across domains? To achieve the above idea, this paper utilizes the information bottleneck (IB) principle, and proposes a novel approach termed as CDRIB to enforce the representations encoding the domain-shared information. To derive the unbiased representations, we devise two IB regularizers to model the cross-domain/in-domain user-item interactions simultaneously and thereby CDRIB could consider both domain interactions jointly for de-biasing.
Robust Reputation Independence in Ranking Systems for Multiple Sensitive Attributes
Authors: Guilherme Ramos, Ludovico Boratto, Mirko Marras
Date: 2022-03-30 00:00:00
Tasks: Attribute

Ranking systems have an unprecedented influence on how and what information people access, and their impact on our society is being analyzed from different perspectives, such as users' discrimination. A notable example is represented by reputation-based ranking systems, a class of systems that rely on users' reputation to generate a non-personalized item-ranking, proved to be biased against certain demographic classes. To safeguard that a given sensitive user's attribute does not systematically affect the reputation of that user, prior work has operationalized a reputation independence constraint on this class of systems. In this paper, we uncover that guaranteeing reputation independence for a single sensitive attribute is not enough. When mitigating biases based on one sensitive attribute (e.g., gender), the final ranking might still be biased against certain demographic groups formed based on another attribute (e.g., age). Hence, we propose a novel approach to introduce reputation independence for multiple sensitive attributes simultaneously. We then analyze the extent to which our approach impacts on discrimination and other important properties of the ranking system, such as its quality and robustness against attacks. Experiments on two real-world datasets show that our approach leads to less biased rankings with respect to multiple users' sensitive attributes, without affecting the system's quality and robustness.
Estimating the Effects of Educational System Consolidation: The Case of China's Rural School Closure Initiative
Authors: Emily Hannum, Xiaoying Liu, Fan Wang
Date: 2022-03-31 00:00:00

Global trends of fertility decline, population aging, and rural outmigration are creating pressures to consolidate school systems, with the rationale that economies of scale will enable higher quality education to be delivered in an efficient manner, despite longer travel distances for students. Yet, few studies have considered the implications of system consolidation for educational access and inequality, outside of the context of developed countries. We estimate the impact of educational infrastructure consolidation on educational attainment using the case of China's rural primary school closure policies in the early 2000s. We use data from a large household survey covering 728 villages in 7 provinces, and exploit variation in villages' year of school closure and children's ages at closure to identify the causal impact of school closure. For girls exposed to closure during their primary school ages, we find an average decrease of 0.60 years of schooling by 2011, when children's mean age was 17 years old. Negative effects strengthen with time since closure. For boys, there is no corresponding significant effect. Different effects by gender may be related to greater sensitivity of girls' enrollment to distance and greater responsiveness of boys' enrollment to quality.
VQF: Highly Accurate IMU Orientation Estimation with Bias Estimation and Magnetic Disturbance Rejection
Authors: Daniel Laidig, Thomas Seel
Date: 2022-03-31 00:00:00

The miniaturization of inertial measurement units (IMUs) facilitates their widespread use in a growing number of application domains. Orientation estimation is a prerequisite for most further data processing steps in inertial motion tracking, such as position/velocity estimation, joint angle estimation, and 3D visualization. Errors in the estimated orientations severely affect all further processing steps. Recent systematic comparisons of existing algorithms show that out-of-the-box accuracy is often low and that application-specific tuning is required to obtain high accuracy. In the present work, we propose and extensively evaluate a quaternion-based orientation estimation algorithm that is based on a novel approach of filtering the acceleration measurements in an almost-inertial frame and that includes extensions for gyroscope bias estimation and magnetic disturbance rejection, as well as a variant for offline data processing. In contrast to all existing work, we perform an extensive evaluation, using a large collection of publicly available datasets and eight literature methods for comparison. The proposed method consistently outperforms all literature methods and achieves an average RMSE of 2.9{\deg}, while the errors obtained with literature methods range from 5.3{\deg} to 16.7{\deg}. Since the evaluation was performed with one single fixed parametrization across a very diverse dataset collection, we conclude that the proposed method provides unprecedented out-of-the-box performance for a broad range of motions, sensor hardware, and environmental conditions. This gain in orientation estimation accuracy is expected to advance the field of IMU-based motion analysis and provide performance benefits in numerous applications. The provided open-source implementation makes it easy to employ the proposed method.
VGGIN-Net: Deep Transfer Network for Imbalanced Breast Cancer Dataset
Authors: Manisha Saini, Seba Susan
Date: 2022-03-29 00:00:00
Proceeding: IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2022 3
Tasks: Breast Cancer Detection, Breast Cancer Histology Image Classification, Data Augmentation, Image Classification, Transfer Learning

In this paper, we have presented a novel deep neural network architecture involving transfer learning approach, formed 
by freezing and concatenating all the layers till block4 pool layer of VGG16 pre-trained model (at the lower level) with the layers 
of a randomly initialized naïve Inception block module (at the higher level). Further, we have added the batch normalization, flatten, 
dropout and dense layers in the proposed architecture. Our transfer network, called VGGIN-Net, facilitates the transfer of domain 
knowledge from the larger ImageNet object dataset to the smaller imbalanced breast cancer dataset. To improve the performance of 
the proposed model, regularization was used in the form of dropout and data augmentation. A detailed block-wise fine tuning has 
been conducted on the proposed deep transfer network for images of different magnification factors. The results of extensive 
experiments indicate a significant improvement of classification performance after the application of fine-tuning. The proposed deep 
learning architecture with transfer learning and fine-tuning yields the highest accuracies in comparison to other state-of-the-art 
approaches for the classification of BreakHis breast cancer dataset. The articulated architecture is designed in a way that it can be 
effectively transfer learned on other breast cancer datasets.
Designing and Implementing an Emotion Analytic System (EAS) on Instagram Social Network Data
Authors: Seyed Faridoddin Kiaei, Mohammad Dehghan Rouzi, Saeed Farzi
Date: 2019-01-01 00:00:00
Proceeding: International Journal of Web Research 2019 1

Being aware of people's attitudes and emotions about a specific person or an event can have a high impact on the decisions of individuals and organizations. With the rise of social networks, specifically Instagram, many people are sharing their attitudes on this social network. Analyzing the emotions of users of this social network can help managers make organizational decisions and predict essential events such as elections. In this research, the EAS system designed and implemented to extract emotions and visualize them. As a practical example, the Instagram users' feelings about the two main candidates for the 12th Iranian presidential election also examined. The data were Instagram Persian comments collected using a developed crawler. The result shows a more positive feeling about Rouhani in comparison with Raeisi. Also, the lexicon-based analysis of Rouhani revealed a high level of trust emotion, along with anger and disgust. The crawled and preprocessed dataset is publicly available at https://github.com/sfdk74/EAS.