pwc-archive/papers-with-abstracts huggingface dataset:
AV2Wav: Diffusion-Based Re-synthesis from Continuous Self-supervised Features for Audio-Visual Speech Enhancement
Authors: Ju-chieh Chou, Chung-Ming Chien, Karen Livescu
Date: 2023-09-14 00:00:00
Tasks: Resynthesis, Speech Enhancement

Speech enhancement systems are typically trained using pairs of clean and noisy speech. In audio-visual speech enhancement (AVSE), there is not as much ground-truth clean data available; most audio-visual datasets are collected in real-world environments with background noise and reverberation, hampering the development of AVSE. In this work, we introduce AV2Wav, a resynthesis-based audio-visual speech enhancement approach that can generate clean speech despite the challenges of real-world training data. We obtain a subset of nearly clean speech from an audio-visual corpus using a neural quality estimator, and then train a diffusion model on this subset to generate waveforms conditioned on continuous speech representations from AV-HuBERT with noise-robust training. We use continuous rather than discrete representations to retain prosody and speaker information. With this vocoding task alone, the model can perform speech enhancement better than a masking-based baseline. We further fine-tune the diffusion model on clean/noisy utterance pairs to improve the performance. Our approach outperforms a masking-based baseline in terms of both automatic metrics and a human listening test and is close in quality to the target speech in the listening test. Audio samples can be found at https://home.ttic.edu/~jcchou/demo/avse/avse_demo.html.
DiariST: Streaming Speech Translation with Speaker Diarization
Authors: Mu Yang, Naoyuki Kanda, Xiaofei Wang, Junkun Chen, Peidong Wang, Jian Xue, Jinyu Li, Takuya Yoshioka
Date: 2023-09-14 00:00:00
Tasks: speaker-diarization, Speaker Diarization, speech-recognition, Speech Recognition, Translation

End-to-end speech translation (ST) for conversation recordings involves several under-explored challenges such as speaker diarization (SD) without accurate word time stamps and handling of overlapping speech in a streaming fashion. In this work, we propose DiariST, the first streaming ST and SD solution. It is built upon a neural transducer-based streaming ST system and integrates token-level serialized output training and t-vector, which were originally developed for multi-talker speech recognition. Due to the absence of evaluation benchmarks in this area, we develop a new evaluation dataset, DiariST-AliMeeting, by translating the reference Chinese transcriptions of the AliMeeting corpus into English. We also propose new metrics, called speaker-agnostic BLEU and speaker-attributed BLEU, to measure the ST quality while taking SD accuracy into account. Our system achieves a strong ST and SD capability compared to offline systems based on Whisper, while performing streaming inference for overlapping speech. To facilitate the research in this new direction, we release the evaluation data, the offline baseline systems, and the evaluation code.
Exploring the Impact of Human Evaluator Group on Chat-Oriented Dialogue Evaluation
Authors: Sarah E. Finch, James D. Finch, Jinho D. Choi
Date: 2023-09-14 00:00:00
Tasks: Chatbot, Dialogue Evaluation

Human evaluation has been widely accepted as the standard for evaluating chat-oriented dialogue systems. However, there is a significant variation in previous work regarding who gets recruited as evaluators. Evaluator groups such as domain experts, university students, and professional annotators have been used to assess and compare dialogue systems, although it is unclear to what extent the choice of an evaluator group can affect results. This paper analyzes the evaluator group impact on dialogue system evaluation by testing 4 state-of-the-art dialogue systems using 4 distinct evaluator groups. Our analysis reveals a robustness towards evaluator groups for Likert evaluations that is not seen for Pairwise, with only minor differences observed when changing evaluator groups. Furthermore, two notable limitations to this robustness are observed, which reveal discrepancies between evaluators with different levels of chatbot expertise and indicate that evaluator objectivity is beneficial for certain dialogue metrics.
Leveraging Contextual Information for Effective Entity Salience Detection
Authors: Rajarshi Bhowmik, Marco Ponza, Atharva Tendle, Anant Gupta, Rebecca Jiang, Xingyu Lu, Qian Zhao, Daniel Preotiuc-Pietro
Date: 2023-09-14 00:00:00
Tasks: Articles, Benchmarking, Feature Engineering, Language Modeling, Language Modelling

In text documents such as news articles, the content and key events usually revolve around a subset of all the entities mentioned in a document. These entities, often deemed as salient entities, provide useful cues of the aboutness of a document to a reader. Identifying the salience of entities was found helpful in several downstream applications such as search, ranking, and entity-centric summarization, among others. Prior work on salient entity detection mainly focused on machine learning models that require heavy feature engineering. We show that fine-tuning medium-sized language models with a cross-encoder style architecture yields substantial performance gains over feature engineering approaches. To this end, we conduct a comprehensive benchmarking of four publicly available datasets using models representative of the medium-sized pre-trained language model family. Additionally, we show that zero-shot prompting of instruction-tuned language models yields inferior results, indicating the task's uniqueness and complexity.
A Spiking Binary Neuron -- Detector of Causal Links
Authors: Mikhail Kiselev, Denis Larionov, Andrey Urusov
Date: 2023-09-15 00:00:00
Tasks: Reinforcement Learning (RL)

Causal relationship recognition is a fundamental operation in neural networks aimed at learning behavior, action planning, and inferring external world dynamics. This operation is particularly crucial for reinforcement learning (RL). In the context of spiking neural networks (SNNs), events are represented as spikes emitted by network neurons or input nodes. Detecting causal relationships within these events is essential for effective RL implementation. This research paper presents a novel approach to realize causal relationship recognition using a simple spiking binary neuron. The proposed method leverages specially designed synaptic plasticity rules, which are both straightforward and efficient. Notably, our approach accounts for the temporal aspects of detected causal links and accommodates the representation of spiking signals as single spikes or tight spike sequences (bursts), as observed in biological brains. Furthermore, this study places a strong emphasis on the hardware-friendliness of the proposed models, ensuring their efficient implementation on modern and future neuroprocessors. Being compared with precise machine learning techniques, such as decision tree algorithms and convolutional neural networks, our neuron demonstrates satisfactory accuracy despite its simplicity. In conclusion, we introduce a multi-neuron structure capable of operating in more complex environments with enhanced accuracy, making it a promising candidate for the advancement of RL applications in SNNs.
Do Random and Chaotic Sequences Really Cause Different PSO Performance? Further Results
Authors: {Paul Moritz Nörenberg, Hendrik Richter
Date: 2023-09-15 00:00:00

Empirical results show that PSO performance may be different if using either chaotic or random sequences to drive the algorithm's search dynamics. We analyze the phenomenon by evaluating the performance based on a benchmark of test functions and comparing random and chaotic sequences according to equality or difference in underlying distribution or density. Our results show that the underlying distribution is the main influential factor in performance and thus the assumption of general and systematic performance differences between chaos and random appears not plausible.
Neural Network Exemplar Parallelization with Go
Authors: Georg Wiesinger, Erich Schikuta
Date: 2023-09-15 00:00:00

This paper presents a case for exemplar parallelism of neural networks using Go as parallelization framework. Further it is shown that also limited multi-core hardware systems are feasible for these parallelization tasks, as notebooks and single board computer systems. The main question was how much speedup can be generated when using concurrent Go goroutines specifically. A simple concurrent feedforward network for MNIST digit recognition with the programming language Go was created to find the answer. The first findings when using a notebook (Lenovo Yoga 2) showed a speedup of 252% when utilizing 4 goroutines. Testing a single board computer (Banana Pi M3) delivered more convincing results: 320% with 4 goroutines, and 432% with 8 goroutines.
Astrocyte-Integrated Dynamic Function Exchange in Spiking Neural Networks
Authors: Murat Isik, Kayode Inadagbo
Date: 2023-09-15 00:00:00
Tasks: Computational Efficiency, CPU, GPU

This paper presents an innovative methodology for improving the robustness and computational efficiency of Spiking Neural Networks (SNNs), a critical component in neuromorphic computing. The proposed approach integrates astrocytes, a type of glial cell prevalent in the human brain, into SNNs, creating astrocyte-augmented networks. To achieve this, we designed and implemented an astrocyte model in two distinct platforms: CPU/GPU and FPGA. Our FPGA implementation notably utilizes Dynamic Function Exchange (DFX) technology, enabling real-time hardware reconfiguration and adaptive model creation based on current operating conditions. The novel approach of leveraging astrocytes significantly improves the fault tolerance of SNNs, thereby enhancing their robustness. Notably, our astrocyte-augmented SNN displays near-zero latency and theoretically infinite throughput, implying exceptional computational efficiency. Through comprehensive comparative analysis with prior works, it's established that our model surpasses others in terms of neuron and synapse count while maintaining an efficient power consumption profile. These results underscore the potential of our methodology in shaping the future of neuromorphic computing, by providing robust and energy-efficient systems.
Traveling Waves Encode the Recent Past and Enhance Sequence Learning
Authors: T. Anderson Keller, Lyle Muller, Terrence Sejnowski, Max Welling
Date: 2023-09-03 00:00:00
Tasks: image-classification, Image Classification, Sequential Image Classification

Traveling waves of neural activity have been observed throughout the brain at a diversity of regions and scales; however, their precise computational role is still debated. One physically inspired hypothesis suggests that the cortical sheet may act like a wave-propagating system capable of invertibly storing a short-term memory of sequential stimuli through induced waves traveling across the cortical surface, and indeed many experimental results from neuroscience correlate wave activity with memory tasks. To date, however, the computational implications of this idea have remained hypothetical due to the lack of a simple recurrent neural network architecture capable of exhibiting such waves. In this work, we introduce a model to fill this gap, which we denote the Wave-RNN (wRNN), and demonstrate how such an architecture indeed efficiently encodes the recent past through a suite of synthetic memory tasks where wRNNs learn faster and reach significantly lower error than wave-free counterparts. We further explore the implications of this memory storage system on more complex sequence modeling tasks such as sequential image classification and find that wave-based models not only again outperform comparable wave-free RNNs while using significantly fewer parameters, but additionally perform comparably to more complex gated architectures such as LSTMs and GRUs.
Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition
Authors: Mohammad Zeineldeen, Albert Zeyer, Ralf Schlüter, Hermann Ney
Date: 2023-09-15 00:00:00
Tasks: Decoder, Form, speech-recognition, Speech Recognition

We study a streamable attention-based encoder-decoder model in which either the decoder, or both the encoder and decoder, operate on pre-defined, fixed-size windows called chunks. A special end-of-chunk (EOC) symbol advances from one chunk to the next chunk, effectively replacing the conventional end-of-sequence symbol. This modification, while minor, situates our model as equivalent to a transducer model that operates on chunks instead of frames, where EOC corresponds to the blank symbol. We further explore the remaining differences between a standard transducer and our model. Additionally, we examine relevant aspects such as long-form speech generalization, beam size, and length normalization. Through experiments on Librispeech and TED-LIUM-v2, and by concatenating consecutive sequences for long-form trials, we find that our streamable model maintains competitive performance compared to the non-streamable variant and generalizes very well to long-form speech.
IHT-Inspired Neural Network for Single-Snapshot DOA Estimation with Sparse Linear Arrays
Authors: Yunqiao Hu, Shunqiao Sun
Date: 2023-09-15 00:00:00
Tasks: Matrix Completion, Missing Elements, Specificity

Single-snapshot direction-of-arrival (DOA) estimation using sparse linear arrays (SLAs) has gained significant attention in the field of automotive MIMO radars. This is due to the dynamic nature of automotive settings, where multiple snapshots aren't accessible, and the importance of minimizing hardware costs. Low-rank Hankel matrix completion has been proposed to interpolate the missing elements in SLAs. However, the solvers of matrix completion, such as iterative hard thresholding (IHT), heavily rely on expert knowledge of hyperparameter tuning and lack task-specificity. Besides, IHT involves truncated-singular value decomposition (t-SVD), which has high computational cost in each iteration. In this paper, we propose an IHT-inspired neural network for single-snapshot DOA estimation with SLAs, termed IHT-Net. We utilize a recurrent neural network structure to parameterize the IHT algorithm. Additionally, we integrate shallow-layer autoencoders to replace t-SVD, reducing computational overhead while generating a novel optimizer through supervised learning. IHT-Net maintains strong interpretability as its network layer operations align with the iterations of the IHT algorithm. The learned optimizer exhibits fast convergence and higher accuracy in the full array signal reconstruction followed by single-snapshot DOA estimation. Numerical results validate the effectiveness of the proposed method.
iHAS: Instance-wise Hierarchical Architecture Search for Deep Learning Recommendation Models
Authors: Yakun Yu, Shi-ang Qi, Jiuding Yang, Liyao Jiang, Di Niu
Date: 2023-09-14 00:00:00
Tasks: Clustering, feature selection, Neural Architecture Search, Recommendation Systems

Current recommender systems employ large-sized embedding tables with uniform dimensions for all features, leading to overfitting, high computational cost, and suboptimal generalizing performance. Many techniques aim to solve this issue by feature selection or embedding dimension search. However, these techniques typically select a fixed subset of features or embedding dimensions for all instances and feed all instances into one recommender model without considering heterogeneity between items or users. This paper proposes a novel instance-wise Hierarchical Architecture Search framework, iHAS, which automates neural architecture search at the instance level. Specifically, iHAS incorporates three stages: searching, clustering, and retraining. The searching stage identifies optimal instance-wise embedding dimensions across different field features via carefully designed Bernoulli gates with stochastic selection and regularizers. After obtaining these dimensions, the clustering stage divides samples into distinct groups via a deterministic selection approach of Bernoulli gates. The retraining stage then constructs different recommender models, each one designed with optimal dimensions for the corresponding group. We conduct extensive experiments to evaluate the proposed iHAS on two public benchmark datasets from a real-world recommender system. The experimental results demonstrate the effectiveness of iHAS and its outstanding transferability to widely-used deep recommendation models.
Trajectory Tracking Control of UAV Bicopter using Linear Quadratic Gaussian
Authors: Fahmizal, Hanung Adi Nugroho, Adha Imam Cahyadi, Igi Ardiyanto
Date: 2023-09-15 00:00:00
Tasks: Position

This paper describes the design of a linear quadratic gaussian (LQG) for trajectory tracking control of UAV Bicopter. In this work, disturbance in the form of payload significantly affects the trajectory tracking control process on the UAV Bicopter when using a linear quadratic regulator (LQR) controller. The use of a LQR control will be optimal in the case of a state regulator towards an equilibrium point in a system, but for the tracking case, the LQR controller is not capable of optimally, especially in systems that have high levels of nonlinearity and system dynamic changes such as inertial disturbances. Therefore, this paper proposes the design of a LQG control that is expected to overcome system dynamic changes, in this case in the form of inertial disturbances to the UAV Bicopter when carrying a payload. The success of LQG control was tested in two scenarios, the first trajectory tracking at a circular position and the second with the position of the trajectory number "8". The simulation results show that the proposed LQG controller successfully overcame inertial disturbances when the UAV Bicopter performs trajectory tracking. When given an inertial disturbance, the trajectory tracking test results show that the LQG control has a lower root mean square error (RMSE) value than the LQR control.
Comparison of machine learning algorithms for content based personality resolution of tweets
Authors: Shruti Garg, Ashwani Garg
Date: 2021-06-18 00:00:00
Proceeding: ScienceDirect 2021 6

The content of social media (SM) is expanding quickly with individuals sharing their feelings in a variety of ways, all of which depict their personalities to varying degrees. This study endeavored to build a system that could predict an individual's personality through SM conversation. Four BIG5 personality items (i.e. Extraversion (EXT), Consciousness (CON), Agreeable (AGR) and Openness to Experiences (OPN) equivalent to the Myers–Briggs Type Indicator (MBTI)) were predicted using six supervised machine learning (SML) algorithms. In order to handle unstructured and unbalanced SM conversations, three feature extraction methods (i.e. term frequency and inverse document frequency (TF-IDF), the bag of words (BOW) and the global vector for word representation (GloVe)) were used. The TF-IDF method of feature extraction produces 2–9% higher accuracy than word2vec representation. GloVe is advocated as a better feature extractor because it maintains the spatial information of words.
Personality Traits Classification on Twitter
Authors: Pavan Kumar K. N., Marina L. Gavrilova
Date: 2019-12-14 00:00:00
Proceeding: IEEE 2019 12
Tasks: Classification

Personality traits have been shown to have strong influences on important aspects of life such as success in the
workplace, political temperament, and general emotional
stability. Computer-based personality assessments using
information from social networking platforms have shown
to be more accurate than judgments made by people close
to the subject. This paper presents a personality traits
classification system that incorporates language-based features, based on count-based vectorization (TF-IDF) and the
GloVe word embedding technique, with an ensemble prediction system consisting of gradient-boosted decision trees
and an SVM classifier. This combination allows to reliably estimate certain personality traits using only the latest 50 tweets from a user’s profile. The performance of the
proposed system is validated on a large, publicly available
dataset and compares favourably with other state-of-the-art
methods.
Low-Rank Tensor Based Proximity Learning for Multi-View Clustering
Authors: Man-Sheng Chen, Chang-Dong Wang, and Jian-Huang Lai
Date: 2023-05-01 00:00:00
Proceeding: IEEE Transactions on Knowledge and Data Engineering 2023 5
Tasks: Clustering, graph construction

Graph-oriented multi-view clustering methods have achieved impressive performances by employing relationships and complex structures hidden in multi-view data. However, most of them still suffer from the following two common problems. (1) They target at studying a common representation or pairwise correlations between views, neglecting the comprehensiveness and deeper higher-order correlations among multiple views. (2) The prior knowledge of view-specific representation can not be taken into account to obtain the consensus indicator graph in a unified graph construction and clustering framework. To deal with these problems, we propose a novel Low-rank Tensor Based Proximity Learning (LTBPL) approach for multi-view clustering, where multiple low-rank probability affinity matrices and consensus indicator graph reflecting the final performances are jointly studied in a unified framework. Specifically, multiple affinity representations are stacked in a low-rank constrained tensor to recover their comprehensiveness and higher-order correlations. Meanwhile, view-specific representation carrying different adaptive confidences is jointly linked with the consensus indicator graph. Extensive experiments on nine real-world datasets indicate the superiority of LTBPL compared with the state-of-the-art methods.
It’s Time to Reason: Annotating Argumentation Structures in Financial Earnings Calls: the FinArg Dataset
Authors: Alaa Alhamzeh, Romain Fonck, Erwan Versmée, Elöd Egyed-Zsigmond, Harald Kosch and Lionel Brunie
Date: 2022-12-06 00:00:00
Proceeding: Fourth Workshop on Financial Technology and Natural Language Processing (FinNLP) 2022 12

With the goal of reasoning on the financial textual data, we present in this paper, a novel approach for annotating arguments, their components, and relations in the transcripts of earnings conference calls (ECCs). The proposed scheme is driven from the argumentation theory at the micro-structure level of discourse. We further conduct a manual annotation study with four annotators on 136 documents. We obtained inter-annotator agreement of $\alpha_{U}$ = 0.70 for argument components and $\alpha$ = 0.81 for argument relations. The final created corpus, with the size of 804 documents, as well as the annotation guidelines are publicly available for researchers in the domains of computational argumentation, finance and FinNLP.