pwc-archive/papers-with-abstracts huggingface dataset:
Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study
Authors: Zeping Min, Jinbo Wang
Date: 2023-07-13 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), In-Context Learning, Instruction Following, speech-recognition, Speech Recognition

This paper explores the integration of Large Language Models (LLMs) into Automatic Speech Recognition (ASR) systems to improve transcription accuracy. The increasing sophistication of LLMs, with their in-context learning capabilities and instruction-following behavior, has drawn significant attention in the field of Natural Language Processing (NLP). Our primary focus is to investigate the potential of using an LLM's in-context learning capabilities to enhance the performance of ASR systems, which currently face challenges such as ambient noise, speaker accents, and complex linguistic contexts. We designed a study using the Aishell-1 and LibriSpeech datasets, with ChatGPT and GPT-4 serving as benchmarks for LLM capabilities. Unfortunately, our initial experiments did not yield promising results, indicating the complexity of leveraging LLM's in-context learning for ASR applications. Despite further exploration with varied settings and models, the corrected sentences from the LLMs frequently resulted in higher Word Error Rates (WER), demonstrating the limitations of LLMs in speech applications. This paper provides a detailed overview of these experiments, their results, and implications, establishing that using LLMs' in-context learning capabilities to correct potential errors in speech recognition transcriptions is still a challenging task at the current stage.
Agreement Tracking for Multi-Issue Negotiation Dialogues
Authors: Amogh Mannekote, Bonnie J. Dorr, Kristy Elizabeth Boyer
Date: 2023-07-13 00:00:00
Tasks: Transfer Learning

Automated negotiation support systems aim to help human negotiators reach more favorable outcomes in multi-issue negotiations (e.g., an employer and a candidate negotiating over issues such as salary, hours, and promotions before a job offer). To be successful, these systems must accurately track agreements reached by participants in real-time. Existing approaches either focus on task-oriented dialogues or produce unstructured outputs, rendering them unsuitable for this objective. Our work introduces the novel task of agreement tracking for two-party multi-issue negotiations, which requires continuous monitoring of agreements within a structured state space. To address the scarcity of annotated corpora with realistic multi-issue negotiation dialogues, we use GPT-3 to build GPT-Negochat, a synthesized dataset that we make publicly available. We present a strong initial baseline for our task by transfer-learning a T5 model trained on the MultiWOZ 2.4 corpus. Pre-training T5-small and T5-base on MultiWOZ 2.4's DST task enhances results by 21% and 9% respectively over training solely on GPT-Negochat. We validate our method's sample-efficiency via smaller training subset experiments. By releasing GPT-Negochat and our baseline models, we aim to encourage further research in multi-issue negotiation dialogue agreement tracking.
Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews
Authors: Eugene Syriani, Istvan David, Gauransh Kumar
Date: 2023-07-12 00:00:00
Tasks: Articles

By organizing knowledge within a research field, Systematic Reviews (SR) provide valuable leads to steer research. Evidence suggests that SRs have become first-class artifacts in software engineering. However, the tedious manual effort associated with the screening phase of SRs renders these studies a costly and error-prone endeavor. While screening has traditionally been considered not amenable to automation, the advent of generative AI-driven chatbots, backed with large language models is set to disrupt the field. In this report, we propose an approach to leverage these novel technological developments for automating the screening of SRs. We assess the consistency, classification performance, and generalizability of ChatGPT in screening articles for SRs and compare these figures with those of traditional classifiers used in SR automation. Our results indicate that ChatGPT is a viable option to automate the SR processes, but requires careful considerations from developers when integrating ChatGPT into their SR tools.
A Comprehensive Overview of Large Language Models
Authors: Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, Ajmal Mian
Date: 2023-07-12 00:00:00
Tasks: Benchmarking

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the existing literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to not only provide a systematic survey but also a quick comprehensive reference for the researchers and practitioners to draw insights from extensive informative summaries of the existing works to advance the LLM research.
The Acquisition of Semantic Relationships between words
Authors: Mohamed Naamane
Date: 2023-07-12 00:00:00

The study of semantic relationships has revealed a close connection between these relationships and the morphological characteristics of a language. Morphology, as a subfield of linguistics, investigates the internal structure and formation of words. By delving into the relationship between semantic relationships and language morphology, we can gain deeper insights into how the underlying structure of words contributes to the interpretation and comprehension of language. This paper explores the dynamic interplay between semantic relationships and the morphological aspects of different languages, by examining the intricate relationship between language morphology and semantic relationships, valuable insights can be gained regarding how the structure of words influences language comprehension.
Incomplete Utterance Rewriting as Sequential Greedy Tagging
Authors: Yunshan Chen
Date: 2023-07-08 00:00:00

The task of incomplete utterance rewriting has recently gotten much attention. Previous models struggled to extract information from the dialogue context, as evidenced by the low restoration scores. To address this issue, we propose a novel sequence tagging-based model, which is more adept at extracting information from context. Meanwhile, we introduce speaker-aware embedding to model speaker variation. Experiments on multiple public datasets show that our model achieves optimal results on all nine restoration scores while having other metric scores comparable to previous state-of-the-art models. Furthermore, benefitting from the model's simplicity, our approach outperforms most previous models on inference speed.
A Novel Site-Agnostic Multimodal Deep Learning Model to Identify Pro-Eating Disorder Content on Social Media
Authors: Jonathan Feldman
Date: 2023-07-06 00:00:00
Tasks: Deep Learning, image-classification, Image Classification, Multimodal Deep Learning, Time Series Analysis

Over the last decade, there has been a vast increase in eating disorder diagnoses and eating disorder-attributed deaths, reaching their zenith during the Covid-19 pandemic. This immense growth derived in part from the stressors of the pandemic but also from increased exposure to social media, which is rife with content that promotes eating disorders. This study aimed to create a multimodal deep learning model that can determine if a given social media post promotes eating disorders based on a combination of visual and textual data. A labeled dataset of Tweets was collected from Twitter, recently rebranded as X, upon which twelve deep learning models were trained and evaluated. Based on model performance, the most effective deep learning model was the multimodal fusion of the RoBERTa natural language processing model and the MaxViT image classification model, attaining accuracy and F1 scores of 95.9% and 0.959, respectively. The RoBERTa and MaxViT fusion model, deployed to classify an unlabeled dataset of posts from the social media sites Tumblr and Reddit, generated results akin to those of previous research studies that did not employ artificial intelligence-based techniques, indicating that deep learning models can develop insights congruent to those of researchers. Additionally, the model was used to conduct a time-series analysis of yet unseen Tweets from eight Twitter hashtags, uncovering that, since 2014, the relative abundance of content that promotes eating disorders has decreased drastically within those communities. Despite this reduction, by 2018, content that promotes eating disorders had either stopped declining or increased in ampleness anew on those hashtags.
Words are not Wind -- How Public Joint Commitment and Reputation Solve the Prisoner's Dilemma
Authors: Marcus Krellner, The Anh Han
Date: 2023-07-13 00:00:00
Tasks: Philosophy

To achieve common goals, we often use joint commitments. Our commitment helps us to coordinate with our partners and assures them that their cooperative efforts will benefit themselves. However, if one of us can exploit the other's cooperation (as in the Prisoner's Dilemma), our commitment appears less useful. It cannot remove the temptation for our partners to exploit us. Using methods from evolutionary game theory, we study the function of joint commitments in the Prisoner's Dilemma. We propose a reputation system akin to indirect reciprocity, wherein agents observe interactions even when not directly involved. They judge cooperation as good and defection as bad, but, crucially, only if the parties involved had committed to cooperate. This results in stable cooperation even though judgments are made privately, which had been a weakness in previous models of indirect reciprocity. Our work shows that joint commitments have utility beyond coordination problems, which could explain their prevalence. The proposed link between joint commitments and reputation could also explain why some joint commitments are pointedly public, like wedding vows. A reputation-based mechanism might have been particularly relevant in our distant past, in which no institutions existed to enforce commitments.
Learning fixed points of recurrent neural networks by reparameterizing the network model
Authors: Vicky Zhu, Robert Rosenbaum
Date: 2023-07-13 00:00:00

In computational neuroscience, fixed points of recurrent neural networks are commonly used to model neural responses to static or slowly changing stimuli. These applications raise the question of how to train the weights in a recurrent neural network to minimize a loss function evaluated on fixed points. A natural approach is to use gradient descent on the Euclidean space of synaptic weights. We show that this approach can lead to poor learning performance due, in part, to singularities that arise in the loss surface. We use a reparameterization of the recurrent network model to derive two alternative learning rules that produces more robust learning dynamics. We show that these learning rules can be interpreted as steepest descent and gradient descent, respectively, under a non-Euclidean metric on the space of recurrent weights. Our results question the common, implicit assumption that learning in the brain should be expected to follow the negative Euclidean gradient of synaptic weights.
Investigating Normalization in Preference-based Evolutionary Multi-objective Optimization Using a Reference Point
Authors: Ryoji Tanabe
Date: 2023-07-13 00:00:00

Normalization of objectives plays a crucial role in evolutionary multi-objective optimization (EMO) to handle objective functions with different scales, which can be found in real-world problems. Although the effect of normalization methods on the performance of EMO algorithms has been investigated in the literature, that of preference-based EMO (PBEMO) algorithms is poorly understood. Since PBEMO aims to approximate a region of interest, its population generally does not cover the Pareto front in the objective space. This property may make normalization of objectives in PBEMO difficult. This paper investigates the effectiveness of three normalization methods in three representative PBEMO algorithms. We present a bounded archive-based method for approximating the nadir point. First, we demonstrate that the normalization methods in PBEMO perform significantly worse than that in conventional EMO in terms of approximating the ideal point, nadir point, and range of the PF. Then, we show that PBEMO requires normalization of objectives on problems with differently scaled objectives. Our results show that there is no clear "best normalization method" in PBEMO, but an external archive-based method performs relatively well.
Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations
Authors: Wyatt Bridgman, Reese Jones, Mohammad Khalil
Date: 2023-07-12 00:00:00
Tasks: Variational Inference

For predictive modeling relying on Bayesian inversion, fully independent, or ``mean-field'', Gaussian distributions are often used as approximate probability density functions in variational inference since the number of variational parameters is twice the number of unknown model parameters. The resulting diagonal covariance structure coupled with unimodal behavior can be too restrictive when dealing with highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors in the form of Gaussian mixtures can capture any distribution to an arbitrary degree of accuracy while maintaining some analytical tractability. Variational inference with Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. Coupled with the existence of multiple local minima due to nonconvex trends in the loss functions often associated with variational inference, these challenges motivate the need for robust initialization procedures to improve the performance and scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with an optimization stage in model parameter space in which local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace method. Finally, the mixture weights are determined through constrained least squares regression. Robustness and scalability are demonstrated using synthetic tests. The methodology is applied to an inversion problem in structural dynamics involving unknown viscous damping coefficients.
A robust feature reinforcement framework for heterogeneous graphs neural networks
Authors: Zehao Wang, Huifeng Wu, Jin Fan, Danfeng Sun, Jia Wu
Date: 2022-11-16 00:00:00
Proceeding: Future Generation Computer Systems 2022 11
Tasks: Graph Embedding, Knowledge Graphs, Node Classification

In the real world, various kinds of data are able to be represented as heterogeneous graph structures. Heterogeneous graphs with multi-typed nodes and edges contain rich messages of heterogeneity and complex semantic information. Recently, diverse heterogeneous graph neural networks (HGNNs) have emerged to solve a range of tasks in this advanced area, such as node classification, knowledge graphs, etc. Heterogeneous graph embedding is a crucial step in HGNNs. It aims to embed rich information from heterogeneous graphs into low-dimensional eigenspaces to improve the performance of downstream tasks. Yet existing methods only project high-dimensional node features into the same low-dimensional space and subsequently aggregate those heterogeneous features directly. This approach ignores the balance between the informative dimensions and the redundant dimensions in the hidden layers. Further, after the dimensionality has been reduced, all kinds of nodes features are projected into the same eigenspace but in a mixed up fashion. One final problem with HGNNs is that their experimental results are always unstable and not reproducible. To solve these issues, we design a general framework named Robust Feature Reinforcement (RFR) for HGNNs to optimize embedding performance. RFR consists of three mechanisms: separate mapping, co-segregating and population-based bandits. The separate mapping mechanism improves the ability to preserve the most informative dimensions when projecting high-dimensional vectors into a low-dimensional eigenspace. The co-segregating mechanism minimizes the contrastive loss to ensure there is a distinction between the features extracted from different types of nodes in the latent feature layers. The population-based bandits mechanism further assures the stability of the experimental results with classification tasks. Supported by rigorous experimentation on three datasets, we assessed the performance of the designed framework and can verify that our models outperform the current state-of-the-arts.
Multistage Relation Network With Dual-Metric for Few-Shot Hyperspectral Image Classification
Authors: Jun Zeng; Zhaohui Xue; Ling Zhang; Qiuping Lan; Mengxue Zhang
Date: 2023-04-28 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2023 4
Tasks: Classification, Few-Shot Image Classification, Few-Shot Learning, Hyperspectral Image Classification, image-classification, Image Classification, Relation, Relation Network

Recently, few-shot learning (FSL) has exhibited great potential in the hyperspectral image (HSI) classification due to its promising performance under a few training samples. Although existing FSL methods have achieved great success, some limitations can still be witnessed. On the one hand, current methods mainly rely on a single metric to identify, which cannot effectively represent the class distribution with few labeled samples. On the other hand, existing methods usually only use the last deep feature of the feature extractor, which may lead to the under-utilization of scarce labeled samples. To overcome the above issues, a novel multistage relation network with dual-metric (DM-MRN) is proposed for few-shot HSI classification. First, a sample recombination strategy is designed to increase the variety of classification tasks in the training period. Second, an embedding module is employed to extract deep features of the input image patches. Third, we propose two relation modules: image-to-class (I2C) block and image-to-image (I2I) block. The I2C block is designed to compute the I2C-level relation score between second-order features, and the I2I block is conceived to generate the I2I-level relation score between first-order features. Finally, DM-MRN is constructed by integrating one embedding module, two I2C blocks, and one I2I block. In addition, an adaptive weighting strategy is designed to fuse the obtained relation scores, and classification can be achieved by assigning each query sample to the class with the highest value of the fused relation score. Extensive experiments carried out on five popular HSI datasets demonstrate that the proposed method outperforms other traditional and advanced models under few training samples in terms of classification accuracy and generalization performance, i.e., the performance improvement in terms of OA is around 0.30%–27.98% under ten labeled samples per class.
Get Better 1 Pixel PCK: Ladder Scales Correspondence Flow Networks for Remote Sensing Image Matching in Higher Resolution
Authors: Weitao Chen, Zhibin Wang, Hao Li
Date: 2021-11-24 00:00:00
Proceeding: ICCV 2021 11

Recently, remote sensing image matching by deep learning reaches competitive performance evaluated by Probability of Correct Keypoints(PCK). Percentage of image size is often used as the threshold of PCK. Even though it can achieve a good 1% PCK in high resolution by regression of transformer parameters,the value will be reduced by using the absolute 1 pixel as threshold in the higher resolution. Inspired by the flow-based methods used in natural image matching tasks, we convert the transformer to correspondence flow and propose ladder scales correspondence flow networks(LSCFN) to get better 1 pixel PCK in higher resolution.Input images are resized to multi scales and then sent to network backbone to generate multi feature pyramids. These pyramids are linked and effectively pull up the highest resolution of original backbone just like a ladder when the global correlation scale is fixed.LSCFN regress correspondence flow in ladder scales by a dense cascade way.We build LSCFN-b and LSCFN-s based on the degree of semantic change between compared images. One with only global correlation is used for the big change, another with global and local correlation is used for the opposite one.The proposed LSCFN achieve state-of-the-art performance evaluated by 1% of image size PCK and absolute 1 pixel PCK on google earth dataset.
Intelligent Omni Surfaces assisted Integrated Multi Target Sensing and Multi User MIMO Communications
Authors: Ziheng Zhang, Wen Chen, Qingqing Wu, Zhendong Li, Xusheng Zhu, Jinhong Yuan
Date: 2023-07-13 00:00:00
Tasks: ISAC

Drawing inspiration from the advantages of intelligent reflecting surfaces (IRS) in wireless networks,this paper presents a novel design for intelligent omni surface (IOS) enabled integrated sensing and communications (ISAC). By harnessing the power of multi antennas and a multitude of elements, the dual-function base station (BS) and IOS collaborate to realize joint active and passive beamforming, enabling seamless 360-degree ISAC coverage. The objective is to maximize the minimum signal-tointerference-plus-noise ratio (SINR) of multi-target sensing, while ensuring the multi-user multi-stream communications. To achieve this, a comprehensive optimization approach is employed, encompassing the design of radar receive vector, transmit beamforming matrix, and IOS transmissive and reflective coefficients. Due to the non-convex nature of the formulated problem, an auxiliary variable is introduced to transform it into a more tractable form. Consequently, the problem is decomposed into three subproblems based on the block coordinate descent algorithm. Semidefinite relaxation and successive convex approximation methods are leveraged to convert the sub-problem into a convex problem, while the iterative rank minimization algorithm and penalty function method ensure the equivalence. Furthermore,the scenario is extended to mode switching and time switching protocols. Simulation results validate the convergence and superior performance of the proposed algorithm compared to other benchmark algorithms.