pwc-archive/papers-with-abstracts huggingface dataset:
Exploring the Relationship between In-Context Learning and Instruction Tuning
Authors: Hanyu Duan, Yixuan Tang, Yi Yang, Ahmed Abbasi, Kar Yan Tam
Date: 2023-11-17 00:00:00
Tasks: In-Context Learning

In-Context Learning (ICL) and Instruction Tuning (IT) are two primary paradigms of adopting Large Language Models (LLMs) to downstream applications. However, they are significantly different. In ICL, a set of demonstrations are provided at inference time but the LLM's parameters are not updated. In IT, a set of demonstrations are used to tune LLM's parameters in training time but no demonstrations are used at inference time. Although a growing body of literature has explored ICL and IT, studies on these topics have largely been conducted in isolation, leading to a disconnect between these two paradigms. In this work, we explore the relationship between ICL and IT by examining how the hidden states of LLMs change in these two paradigms. Through carefully designed experiments conducted with LLaMA-2 (7B and 13B), we find that ICL is implicit IT. In other words, ICL changes an LLM's hidden states as if the demonstrations were used to instructionally tune the model. Furthermore, the convergence between ICL and IT is largely contingent upon several factors related to the provided demonstrations. Overall, this work offers a unique perspective to explore the connection between ICL and IT and sheds light on understanding the behaviors of LLM.
Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehension
Authors: Tongquan Zhou, Yao Zhang, Siyi Cao, Yulu Li, Tao Wang
Date: 2023-11-17 00:00:00
Tasks: Causal Inference, Reading Comprehension

ChatGPT has shown its great power in text processing, including its reasoning ability from text reading. However, there has not been any direct comparison between human readers and ChatGPT in reasoning ability related to text reading. This study was undertaken to investigate how ChatGPTs (i.e., ChatGPT and ChatGPT Plus) and Chinese senior school students as ESL learners exhibited their reasoning ability from English narrative texts. Additionally, we compared the two ChatGPTs in the reasoning performances when commands were updated elaborately. The whole study was composed of three reasoning tests: Test 1 for commonsense inference, Test 2 for emotional inference, and Test 3 for causal inference. The results showed that in Test 1, the students outdid the two ChatGPT versions in local-culture-related inferences but performed worse than the chatbots in daily-life inferences. In Test 2, ChatGPT Plus excelled whereas ChatGPT lagged behind in accuracy. In association with both accuracy and frequency of correct responses, the students were inferior to the two chatbots. Compared with ChatGPTs' better performance in positive emotions, the students showed their superiority in inferring negative emotions. In Test 3, the students demonstrated better logical analysis, outdoing both chatbots. In updating command condition, ChatGPT Plus displayed good causal reasoning ability while ChatGPT kept unchanged. Our study reveals that human readers and ChatGPTs have their respective advantages and disadvantages in drawing inferences from text reading comprehension, unlocking a complementary relationship in text-based reasoning.
Prompt Pool based Class-Incremental Continual Learning for Dialog State Tracking
Authors: Hong Liu, Yucheng Cai, Yuan Zhou, Zhijian Ou, Yi Huang, Junlan Feng
Date: 2023-11-17 00:00:00
Tasks: Continual Learning, dialog state tracking

Continual learning is crucial for dialog state tracking (DST) in dialog systems, since requirements from users for new functionalities are often encountered. However, most of existing continual learning methods for DST require task identities during testing, which is a severe limit in real-world applications. In this paper, we aim to address continual learning of DST in the class-incremental scenario (namely the task identity is unknown in testing). Inspired by the recently emerging prompt tuning method that performs well on dialog systems, we propose to use the prompt pool method, where we maintain a pool of key-value paired prompts and select prompts from the pool according to the distance between the dialog history and the prompt keys. The proposed method can automatically identify tasks and select appropriate prompts during testing. We conduct experiments on Schema-Guided Dialog dataset (SGD) and another dataset collected from a real-world dialog application. Experiment results show that the prompt pool method achieves much higher joint goal accuracy than the baseline. After combining with a rehearsal buffer, the model performance can be further improved.
Diagnosing and Debiasing Corpus-Based Political Bias and Insults in GPT2
Authors: Ambri Ma, Arnav Kumar, Brett Zeligson
Date: 2023-11-17 00:00:00

The training of large language models (LLMs) on extensive, unfiltered corpora sourced from the internet is a common and advantageous practice. Consequently, LLMs have learned and inadvertently reproduced various types of biases, including violent, offensive, and toxic language. However, recent research shows that generative pretrained transformer (GPT) language models can recognize their own biases and detect toxicity in generated content, a process referred to as self-diagnosis. In response, researchers have developed a decoding algorithm that allows LLMs to self-debias, or reduce their likelihood of generating harmful text. This study investigates the efficacy of the diagnosing-debiasing approach in mitigating two additional types of biases: insults and political bias. These biases are often used interchangeably in discourse, despite exhibiting potentially dissimilar semantic and syntactic properties. We aim to contribute to the ongoing effort of investigating the ethical and social implications of human-AI interaction.
Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study
Authors: Maike Züfle, Verna Dankers, Ivan Titov
Date: 2023-11-16 00:00:00
Tasks: Hate Speech Detection

With the ever-growing presence of social media platforms comes the increased spread of harmful content and the need for robust hate speech detection systems. Such systems easily overfit to specific targets and keywords, and evaluating them without considering distribution shifts that might occur between train and test data overestimates their benefit. We challenge hate speech models via new train-test splits of existing datasets that rely on the clustering of models' hidden representations. We present two split variants (Subset-Sum-Split and Closest-Split) that, when applied to two datasets using four pretrained models, reveal how models catastrophically fail on blind spots in the latent space. This result generalises when developing a split with one model and evaluating it on another. Our analysis suggests that there is no clear surface-level property of the data split that correlates with the decreased performance, which underscores that task difficulty is not always humanly interpretable. We recommend incorporating latent feature-based splits in model development and release two splits via the GenBench benchmark.
The Impact of Familiarity on Naming Variation: A Study on Object Naming in Mandarin Chinese
Authors: Yunke He, Xixian Liao, Jialing Liang, Gemma Boleda
Date: 2023-11-16 00:00:00
Tasks: Object

Different speakers often produce different names for the same object or entity (e.g., "woman" vs. "tourist" for a female tourist). The reasons behind variation in naming are not well understood. We create a Language and Vision dataset for Mandarin Chinese that provides an average of 20 names for 1319 naturalistic images, and investigate how familiarity with a given kind of object relates to the degree of naming variation it triggers across subjects. We propose that familiarity influences naming variation in two competing ways: increasing familiarity can either expand vocabulary, leading to higher variation, or promote convergence on conventional names, thereby reducing variation. We find evidence for both factors being at play. Our study illustrates how computational resources can be used to address research questions in Cognitive Science.
JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing
Authors: Shester Gueuwou, Sophie Siake, Colin Leong, Mathias Müller
Date: 2023-11-16 00:00:00
Tasks: Diversity, Machine Translation, Sign Language Translation, Translation

Advancements in sign language processing have been hindered by a lack of sufficient data, impeding progress in recognition, translation, and production tasks. The absence of comprehensive sign language datasets across the world's sign languages has widened the gap in this field, resulting in a few sign languages being studied more than others, making this research area extremely skewed mostly towards sign languages from high-income countries. In this work we introduce a new large and highly multilingual dataset for sign language translation: JWSign. The dataset consists of 2,530 hours of Bible translations in 98 sign languages, featuring more than 1,500 individual signers. On this dataset, we report neural machine translation experiments. Apart from bilingual baseline systems, we also train multilingual systems, including some that take into account the typological relatedness of signed or spoken languages. Our experiments highlight that multilingual systems are superior to bilingual baselines, and that in higher-resource scenarios, clustering language pairs that are related improves translation quality.
Fast Estimations of Hitting Time of Elitist Evolutionary Algorithms from Fitness Levels
Authors: Jun He, Siang Yew Chong, Xin Yao
Date: 2023-11-17 00:00:00
Tasks: Evolutionary Algorithms

The fitness level method is an easy-to-use tool for estimating the hitting time of elitist evolutionary algorithms. Recently, linear lower and upper bounds by fitness levels have been constructed. But these bounds require recursive computation, which makes them difficult to use in practice. We address this shortcoming with a new directed graph (digraph) method that does not require recursive computation and significantly simplifies the calculation of coefficients in the lower bound. In the method, we select a sub-digraph and divide it into fitness levels, then construct an explicit formula for computing the linear lower bound coefficients using transition probabilities restricted to the subdigraph. A major advantage of the new method is the derivation of tight lower bounds on fitness functions with shortcuts, which are difficult to achieve using previous fitness methods. We use three examples (FullyDeceptive, TwoMax1 and Deceptive) to demonstrate that each new lower bound is tight, but previous lower bounds are not. Our work significantly extends the fitness level method from addressing simple fitness functions without shortcuts to more complex functions with shortcuts.
On exploiting the synaptic interaction properties to obtain frequency-specific neurons
Authors: Guillaume Marthe, Claire Goursaud, Romain Cazé, Laurent Clavier
Date: 2023-11-17 00:00:00
Tasks: Temporal Sequences

Energy consumption remains the main limiting factors in many IoT applications. In particular, micro-controllers consume far too much power. In order to overcome this problem, new circuit designs have been proposed and the use of spiking neurons and analog computing has emerged as it allows a very significant consumption reduction. However, working in the analog domain brings difficulty to handle the sequential processing of incoming signals as is needed in many use cases. In this paper, we use a bio-inspired phenomenon called Interacting Synapses to produce a time filter, without using non-biological techniques such as synaptic delays. We propose a model of neuron and synapses that fire for a specific range of delays between two incoming spikes, but do not react when this Inter-Spike Timing is not in that range. We study the parameters of the model to understand how to choose them and adapt the Inter-Spike Timing. The originality of the paper is to propose a new way, in the analog domain, to deal with temporal sequences.
Ex2Vec: Characterizing Users and Items from the Mere Exposure Effect
Authors: Bruno Sguerra, Viet-Anh Tran, Romain Hennequin
Date: 2023-11-17 00:00:00

The traditional recommendation framework seeks to connect user and content, by finding the best match possible based on users past interaction. However, a good content recommendation is not necessarily similar to what the user has chosen in the past. As humans, users naturally evolve, learn, forget, get bored, they change their perspective of the world and in consequence, of the recommendable content. One well known mechanism that affects user interest is the Mere Exposure Effect: when repeatedly exposed to stimuli, users' interest tends to rise with the initial exposures, reaching a peak, and gradually decreasing thereafter, resulting in an inverted-U shape. Since previous research has shown that the magnitude of the effect depends on a number of interesting factors such as stimulus complexity and familiarity, leveraging this effect is a way to not only improve repeated recommendation but to gain a more in-depth understanding of both users and stimuli. In this work we present (Mere) Exposure2Vec (Ex2Vec) our model that leverages the Mere Exposure Effect in repeat consumption to derive user and item characterization and track user interest evolution. We validate our model through predicting future music consumption based on repetition and discuss its implications for recommendation scenarios where repetition is common.
Cross-Modal Search and Exploration of Greek Painted Pottery
Authors: Elisabeth Trinkl, Stephan Karl, Stefan Lengauer, Reinhold Preiner, Tobias Schreck
Date: 2023-11-17 00:00:00
Tasks: Data Visualization, Retrieval

This paper focuses on digitally-supported research methods for an important group of cultural heritage objects, the Greek pottery, especially with figured decoration. The design, development and application of new digital methods for searching, comparing, and visually exploring these vases needs an interdisciplinary approach to effectively analyse the various features of the vases, like shape, decoration, and manufacturing techniques, and relationships between the vases. We motivate the need and opportunities by a multimodal representation of the objects, including 3D shape, material, and painting. We then illustrate a range of innovative methods for these representations, including quantified surface and capacity comparison, material analysis, image flattening from 3D objects, retrieval and comparison of shapes and paintings, and multidimensional data visualization. We also discuss challenges and future work in this area.
Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation
Authors: Shenghao Yang, Chenyang Wang, Yankai Liu, Kangping Xu, Weizhi Ma, Yiqun Liu, Min Zhang, Haitao Zeng, Junlan Feng, Chao Deng
Date: 2023-11-17 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems, Representation Learning, Sequential Recommendation

Item representation learning (IRL) plays an essential role in recommender systems, especially for sequential recommendation. Traditional sequential recommendation models usually utilize ID embeddings to represent items, which are not shared across different domains and lack the transferable ability. Recent studies use pre-trained language models (PLM) for item text embeddings (text-based IRL) that are universally applicable across domains. However, the existing text-based IRL is unaware of the important collaborative filtering (CF) information. In this paper, we propose CoWPiRec, an approach of Collaborative Word-based Pre-trained item representation for Recommendation. To effectively incorporate CF information into text-based IRL, we convert the item-level interaction data to a word graph containing word-level collaborations. Subsequently, we design a novel pre-training task to align the word-level semantic- and CF-related item representation. Extensive experimental results on multiple public datasets demonstrate that compared to state-of-the-art transferable sequential recommenders, CoWPiRec achieves significantly better performances in both fine-tuning and zero-shot settings for cross-scenario recommendation and effectively alleviates the cold-start issue. The code is available at: https://github.com/ysh-1998/CoWPiRec.
A Comparative Analysis of Retrievability and PageRank Measures
Authors: Aman Sinha, Priyanshu Raj Mall, Dwaipayan Roy
Date: 2023-11-17 00:00:00
Tasks: Information Retrieval, Retrieval

The accessibility of documents within a collection holds a pivotal role in Information Retrieval, signifying the ease of locating specific content in a collection of documents. This accessibility can be achieved via two distinct avenues. The first is through some retrieval model using a keyword or other feature-based search, and the other is where a document can be navigated using links associated with them, if available. Metrics such as PageRank, Hub, and Authority illuminate the pathways through which documents can be discovered within the network of content while the concept of Retrievability is used to quantify the ease with which a document can be found by a retrieval model. In this paper, we compare these two perspectives, PageRank and retrievability, as they quantify the importance and discoverability of content in a corpus. Through empirical experimentation on benchmark datasets, we demonstrate a subtle similarity between retrievability and PageRank particularly distinguishable for larger datasets.
Tensor Wiener Filter
Authors: Shih Yu Chang, Hsiao-Chun Wu
Date: 2022-01-07 00:00:00
Proceeding: IEEE Transactions on Signal Processing ( Volume: 70) 2022 1
Tasks: Image Restoration

In signal processing and data analytics, Wiener filter is a classical powerful tool to transform an input signal to match a desired or target signal by a linear time-invariant (LTI)filter. The input signal of a Wiener filter is one-dimensional while its associated least-squares solution, namely Wiener-Hopf equation, involves a two-dimensional data-array, or correlation matrix. However, the actual match should often be carried out between a multi-dimensional filtered signal-sequence, which is the output of a multi-channel filter characterized as a linear-time-invariant MIMO (multi-input and multi-output) system, and a multi-dimensional desired signal-sequence simultaneously. In the presence of such a multi-channel filter, the solution to the corresponding Wiener filter, which we call MIMO Wiener-Hopf equation now, involves a correlation tensor. Therefore, we call this optimal multi-channel filter Tensor Wiener Filter (TWF). Due to lack of the pertinent mathematical framework of needed tensor operations, TWF has never been investigated so far. Now we would like to make the first-ever attempt to establish a new mathematical framework for TWF, which relies on the inverse of the correlation tensor. We propose the new parallel block-Jacobi tensor-inversion algorithm for this tensor inversion. A typical application of the new TWF approach is illustrated as a multi-channel linear predictor (MCLP) built upon a multi-channel autoregressive (MCAR) filter with multi-dimensional input data. Numerical experiments pertaining to seismic data, optical images, and macroeconomic time-series are conducted in comparison with other existing methods. The memory- and computational complexities corresponding to our proposed parallel block-Jacobi tensor-inversion algorithm are also studied in this paper.
A Comparative Study of Static Code Analysis tools for Vulnerability Detection in C/C++ and JAVA Source Code
Authors: Prof. Arvinder Kaur, Ruchikaa Nayyar
Date: 2020-10-20 00:00:00
Proceeding: 2020 2020 10
Tasks: Vulnerability Detection

Software security has become an essential component of software development process. It is necessary for an
organisation to maintain software security in order to ensure integrity, authenticity and availability of the software
product. To ensure software security, one of the major task is to identify vulnerabilities present in the source code
before the software is being deployed. Detecting vulnerabilities in early phases of software development cycle,
makes the process of fixing those vulnerabilities much easier for software developers. The vulnerability detection
can be done either at the production phase, this means when the software is still being developed by statically
auditing the source code, or dynamically at run time. In this study, vulnerability detection was done through Static
code analysis process. Static code analysis can be done either manually or through automated tools. This paper
focuses on using automated source code scanning tools for vulnerabilities detection in a software. Automated static
Code Analysis tools audits the entire source code for its quality and identify any potential security vulnerability, if
present. Unlike dynamic source code analysis that evaluates the source code behaviour during code execution, which
is done quite late in the software development life cycle, Static Code Analysis leads to detection of security
vulnerabilities in a source code in early stages of software development process, when the software is still in
production phase because it does not require code to be in execution state. This paper firstly explains the importance
of incorporating static code analysis in software development life cycle process so as to facilitate early detection of
vulnerabilities in software product, and then present a comparative study of various static code analysis tools
available for vulnerability detection in C/C++ and JAVA source code. The comparative study of three C/C++ static
code analysis tools (flawfinder, RATS and CPPCheck) and two JAVA static code analysis tools (spotbugs and
PMD) is done using Juliet (version1.3) test suite and APACHE tomcat dataset respectively, on the basis of category of vulnerability detected by each of the selected tool and the likelihood of false positive reported by each
tool. Results showed that Flawfinder detected maximum categories of vulnerabilities and RATS and CPPCheck
were almost similar in types of vulnerabilities detected. Also, it was observed that CPPCheck reported maximum
number of false positives as compared to other two tools. Java static code analysis tools Spot bugs was able to detect more number of vulnerabilities than PMD.
User Dynamics-Aware Edge Caching and Computing for Mobile Virtual Reality
Authors: Mushu Li, Jie Gao, Conghao Zhou, Xuemin Shen, Weihua Zhuang
Date: 2023-11-17 00:00:00
Tasks: Scheduling

In this paper, we present a novel content caching and delivery approach for mobile virtual reality (VR) video streaming. The proposed approach aims to maximize VR video streaming performance, i.e., minimizing video frame missing rate, by proactively caching popular VR video chunks and adaptively scheduling computing resources at an edge server based on user and network dynamics. First, we design a scalable content placement scheme for deciding which video chunks to cache at the edge server based on tradeoffs between computing and caching resource consumption. Second, we propose a machine learning-assisted VR video delivery scheme, which allocates computing resources at the edge server to satisfy video delivery requests from multiple VR headsets. A Whittle index-based method is adopted to reduce the video frame missing rate by identifying network and user dynamics with low signaling overhead. Simulation results demonstrate that the proposed approach can significantly improve VR video streaming performance over conventional caching and computing resource scheduling strategies.