pwc-archive/papers-with-abstracts huggingface dataset:
FinGPT: Large Generative Models for a Small Language
Authors: Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna-Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus, Thomas Wang, Nouamane Tazi, Teven Le Scao, Thomas Wolf, Osma Suominen, Samuli Sairanen, Mikko Merioksa, Jyrki Heinonen, Aija Vahtola, Samuel Antao, Sampo Pyysalo
Date: 2023-11-03 00:00:00

Large language models (LLMs) excel in many tasks in NLP and beyond, but most open models have very limited coverage of smaller languages and LLM work tends to focus on languages where nearly unlimited data is available for pretraining. In this work, we study the challenges of creating LLMs for Finnish, a language spoken by less than 0.1% of the world population. We compile an extensive dataset of Finnish combining web crawls, news, social media and eBooks. We pursue two approaches to pretrain models: 1) we train seven monolingual models from scratch (186M to 13B parameters) dubbed FinGPT, 2) we continue the pretraining of the multilingual BLOOM model on a mix of its original training data and Finnish, resulting in a 176 billion parameter model we call BLUUMI. For model evaluation, we introduce FIN-bench, a version of BIG-bench with Finnish tasks. We also assess other model qualities such as toxicity and bias. Our models and tools are openly available at https://turkunlp.org/gpt3-finnish.
Experimental Evidence on Negative Impact of Generative AI on Scientific Learning Outcomes
Authors: Qirui Ju
Date: 2023-09-23 00:00:00

In this study, I explored the impact of Generative AI on learning efficacy in academic reading materials using experimental methods. College-educated participants engaged in three cycles of reading and writing tasks. After each cycle, they responded to comprehension questions related to the material. After adjusting for background knowledge and demographic factors, complete reliance on AI for writing tasks led to a 25.1% reduction in accuracy. In contrast, AI-assisted reading resulted in a 12% decline. Interestingly, using AI for summarization significantly improved both quality and output. Accuracy exhibited notable variance in the AI-assisted section. Further analysis revealed that individuals with a robust background in the reading topic and superior reading/writing skills benefitted the most. I conclude the research by discussing educational policy implications, emphasizing the need for educators to warn students about the dangers of over-dependence on AI and provide guidance on its optimal use in educational settings.
Face-StyleSpeech: Enhancing Zero-shot Speech Synthesis from Face Images with Improved Face-to-Speech Mapping
Authors: Minki Kang, Wooseok Han, Eunho Yang
Date: 2023-09-25 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech

Generating speech from a face image is crucial for developing virtual humans capable of interacting using their unique voices, without relying on pre-recorded human speech. In this paper, we propose Face-StyleSpeech, a zero-shot Text-To-Speech (TTS) synthesis model that generates natural speech conditioned on a face image rather than reference speech. We hypothesize that learning entire prosodic features from a face image poses a significant challenge. To address this, our TTS model incorporates both face and prosody encoders. The prosody encoder is specifically designed to model speech style characteristics that are not fully captured by the face image, allowing the face encoder to focus on extracting speaker-specific features such as timbre. Experimental results demonstrate that Face-StyleSpeech effectively generates more natural speech from a face image than baselines, even for unseen faces. Samples are available on our demo page.
The Impact of Performance Expectancy, Workload, Risk, and Satisfaction on Trust in ChatGPT: Cross-sectional Survey Analysis
Authors: Hamid Shamszare, Avishek Choudhury
Date: 2023-10-20 00:00:00
Tasks: Survey

This study investigated how perceived workload, satisfaction, performance expectancy, and risk-benefit perception influenced users' trust in Chat Generative Pre-Trained Transformer (ChatGPT). We aimed to understand the nuances of user engagement and provide insights to improve future design and adoption strategies for similar technologies. A semi-structured, web-based survey was conducted among adults in the United States who actively use ChatGPT at least once a month. The survey was conducted from 22nd February 2023 through 24th March 2023. We used structural equation modeling to understand the relationships among the constructs of perceived workload, satisfaction, performance expectancy, risk-benefit, and trust. The analysis of 607 survey responses revealed a significant negative relationship between perceived workload and user satisfaction, a negative but insignificant relationship between perceived workload and trust, and a positive relationship between user satisfaction and trust. Trust was also found to increase with performance expectancy. In contrast, the relationship between the benefit-to-risk ratio of using ChatGPT and trust was insignificant. The findings underscore the importance of ensuring user-friendly design and functionality in AI-based applications to reduce workload and enhance user satisfaction, thereby increasing user trust. Future research should further explore the relationship between the benefit-to-risk ratio and trust in the context of AI chatbots.
Towards Instance-Optimality in Online PAC Reinforcement Learning
Authors: Aymen Al-Marjani, Andrea Tirinzoni, Emilie Kaufmann
Date: 2023-10-31 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning

Several recent works have proposed instance-dependent upper bounds on the number of episodes needed to identify, with probability $1-\delta$, an $\varepsilon$-optimal policy in finite-horizon tabular Markov Decision Processes (MDPs). These upper bounds feature various complexity measures for the MDP, which are defined based on different notions of sub-optimality gaps. However, as of now, no lower bound has been established to assess the optimality of any of these complexity measures, except for the special case of MDPs with deterministic transitions. In this paper, we propose the first instance-dependent lower bound on the sample complexity required for the PAC identification of a near-optimal policy in any tabular episodic MDP. Additionally, we demonstrate that the sample complexity of the PEDEL algorithm of \cite{Wagenmaker22linearMDP} closely approaches this lower bound. Considering the intractability of PEDEL, we formulate an open question regarding the possibility of achieving our lower bound using a computationally-efficient algorithm.
OmniVec: Learning robust representations with cross modal sharing
Authors: Siddharth Srivastava, Gaurav Sharma
Date: 2023-11-07 00:00:00
Tasks: 3D Point Cloud Classification, Action Classification, Audio Classification, Fine-Grained Image Classification, Image Classification, Semantic Segmentation, Text Summarization

Majority of research in learning based methods has been towards designing and training networks for specific tasks. However, many of the learning based tasks, across modalities, share commonalities and could be potentially tackled in a joint framework. We present an approach in such direction, to learn multiple tasks, in multiple modalities, with a unified architecture. The proposed network is composed of task specific encoders, a common trunk in the middle, followed by task specific prediction heads. We first pre-train it by self-supervised masked training, followed by sequential training for the different tasks. We train the network on all major modalities, e.g.\ visual, audio, text and 3D, and report results on $22$ diverse and challenging public benchmarks. We demonstrate empirically that, using a joint network to train across modalities leads to meaningful information sharing and this allows us to achieve state-of-the-art results on most of the benchmarks. We also show generalization of the trained network on cross-modal tasks as well as unseen datasets and tasks.
Multi-Modal Gaze Following in Conversational Scenarios
Authors: Yuqi Hou, Zhongqun Zhang, Nora Horanyi, Jaewon Moon, Yihua Cheng, Hyung Jin Chang
Date: 2023-11-09 00:00:00

Gaze following estimates gaze targets of in-scene person by understanding human behavior and scene information. Existing methods usually analyze scene images for gaze following. However, compared with visual images, audio also provides crucial cues for determining human behavior.This suggests that we can further improve gaze following considering audio cues. In this paper, we explore gaze following tasks in conversational scenarios. We propose a novel multi-modal gaze following framework based on our observation ``audiences tend to focus on the speaker''. We first leverage the correlation between audio and lips, and classify speakers and listeners in a scene. We then use the identity information to enhance scene images and propose a gaze candidate estimation network. The network estimates gaze candidates from enhanced scene images and we use MLP to match subjects with candidates as classification tasks. Existing gaze following datasets focus on visual images while ignore audios.To evaluate our method, we collect a conversational dataset, VideoGazeSpeech (VGS), which is the first gaze following dataset including images and audio. Our method significantly outperforms existing methods in VGS datasets. The visualization result also prove the advantage of audio cues in gaze following tasks. Our work will inspire more researches in multi-modal gaze following estimation.
Vehicle Detection in Aerial Imagery (VEDAI) : a benchmark
Authors: Sebastien Razakarivony, Frederic Jurie
Date: 2015-03-01 00:00:00
Proceeding: Journal of Visual Communication and Image Representation 2015 3
Tasks: vehicle detection

VEDAI is a dataset for Vehicle Detection in Aerial Imagery, provided as a tool to benchmark automatic target recognition algorithms in unconstrained environments. The vehicles contained in the database, in addition of being small, exhibit different variabilities such as multiple orientations, lighting/shadowing changes, specularities or occlusions. Furthermore, each image is available in several spectral bands and resolutions. A precise experimental protocol is also given, ensuring that the experimental results obtained by different people can be properly reproduced and compared. We also give the performance of some baseline algorithms on this dataset, for different settings of these algorithms, to illustrate the difficulties of the task and provide baseline comparisons.
On the Discussion of Large Language Models: Symmetry of Agents and Interplay with Prompts
Authors: Qineng Wang, ZiHao Wang, Ying Su, Yangqiu Song
Date: 2023-11-13 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Prompt Engineering

Two ways has been discussed to unlock the reasoning capability of a large language model. The first one is prompt engineering and the second one is to combine the multiple inferences of large language models, or the multi-agent discussion. Theoretically, this paper justifies the multi-agent discussion mechanisms from the symmetry of agents. Empirically, this paper reports the empirical results of the interplay of prompts and discussion mechanisms, revealing the empirical state-of-the-art performance of complex multi-agent mechanisms can be approached by carefully developed prompt engineering. This paper also proposes a scalable discussion mechanism based on conquer and merge, providing a simple multi-agent discussion solution with simple prompts but state-of-the-art performance.
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models
Authors: Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, Juanzi Li
Date: 2023-11-13 00:00:00
Tasks: Benchmarking, Instruction Following

To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible trace for watermark detection. Due to the two-stage nature of the task, most studies evaluate the generation and detection separately, thereby presenting a challenge in unbiased, thorough, and applicable evaluations. In this paper, we introduce WaterBench, the first comprehensive benchmark for LLM watermarks, in which we design three crucial factors: (1) For benchmarking procedure, to ensure an apples-to-apples comparison, we first adjust each watermarking method's hyper-parameter to reach the same watermarking strength, then jointly evaluate their generation and detection performance. (2) For task selection, we diversify the input and output length to form a five-category taxonomy, covering $9$ tasks. (3) For evaluation metric, we adopt the GPT4-Judge for automatically evaluating the decline of instruction-following abilities after watermarking. We evaluate $4$ open-source watermarks on $2$ LLMs under $2$ watermarking strengths and observe the common struggles for current methods on maintaining the generation quality. The code and data are available at https://github.com/THU-KEG/WaterBench.
A probabilistic forecast methodology for volatile electricity prices in the Australian National Electricity Market
Authors: Cameron Cornell, Nam Trong Dinh, S. Ali Pourmousavi
Date: 2023-11-13 00:00:00
Tasks: quantile regression

The South Australia region of the Australian National Electricity Market (NEM) displays some of the highest levels of price volatility observed in modern electricity markets. This paper outlines an approach to probabilistic forecasting under these extreme conditions, including spike filtration and several post-processing steps. We propose using quantile regression as an ensemble tool for probabilistic forecasting, with our combined forecasts achieving superior results compared to all constituent models. Within our ensemble framework, we demonstrate that averaging models with varying training length periods leads to a more adaptive model and increased prediction accuracy. The applicability of the final model is evaluated by comparing our median forecasts with the point forecasts available from the Australian NEM operator, with our model outperforming these NEM forecasts by a significant margin.
Attention Weighted Local Descriptors
Authors: Changwei Wang, Rongtao Xu, Ke Lu, Shibiao Xu, Weiliang Meng, Yuyang Zhang, Bin Fan, Xiaopeng Zhang
Date: 2023-04-19 00:00:00
Proceeding: journal 2023 4
Tasks: 3D Reconstruction, Homography Estimation, Knowledge Distillation, Visual Localization

Local features detection and description are widely used in many vision applications with high industrial and commercial demands. With large-scale applications, these tasks raise high expectations for both the accuracy and speed of local features. Most existing studies on local features learning focus on the local descriptions of individual keypoints, which neglect their relationships established from global spatial awareness. In this paper, we present AWDesc with a consistent attention mechanism (CoAM) that opens up the possibility for local descriptors to embrace image-level spatial awareness in both the training and matching stages. For local features detection, we adopt local features detection with feature pyramid to obtain more stable and accurate keypoints localization. For local features description, we provide two versions of AWDesc to cope with different accuracy and speed requirements. On the one hand, we introduceContext Augmentation to address the inherent locality of convolutional neural networks by injecting non-local context information, so that local descriptors can “look wider to describe better”. Specifically, well-designed Adaptive Global Context Augmented Module (AGCA) and Diverse Surrounding Context Augmented Module (DSCA) are proposed to construct robust local descriptors with context information from global to surrounding. On the other hand, we design an extremely lightweight backbone network coupled with the proposed special knowledge distillation strategy to achieve the best trade-off in accuracy and speed. What is more, we perform thorough experiments on image matching, homography estimation, visual localization, and 3D reconstruction tasks, and the results demonstrate that our method surpasses the current state-of-the-art local descriptors. Code is available at: https://github.com/vignywang/AWDesc.
STATGRAPH: Effective In-vehicle Intrusion Detection via Multi-view Statistical Graph Learning
Authors: Kai Wang, Qiguang Jiang, Bailing Wang, Yulei Wu, Hongke Zhang
Date: 2023-11-13 00:00:00
Tasks: Graph Learning, Intrusion Detection

In-vehicle network (IVN) is facing complex external cyber-attacks, especially the emerging masquerade attacks with extremely high difficulty of detection while serious damaging effects. In this paper, we propose the STATGRAPH, which is an effective and fine-grained intrusion detection methodology for IVN security services via multi-view statistical graph learning on in-vehicle controller area network (CAN) messages with insight into their variations in periodicity, payload and signal combinations. Specifically, STATGRAPH generates two statistical graphs, timing correlation graph (TCG) and coupling relationship graph (CRG), in every CAN message detection window, where edge attributes in TCGs represent temporal correlation between different message IDs while edge attributes in CRGs denote the neighbour relationship and contextual similarity. Besides, a lightweight shallow layered graph convolution network is trained based on graph property of TCGs and CRGs, which learns the universal laws of various patterns more effectively and further enhance the performance of detection. To address the problem of insufficient attack types in previous intrusion detection, we select two real in-vehicle CAN datasets covering five new instances of sophisticated and stealthy masquerade attacks that are never investigated before. Experimental result shows STATGRAPH improves both detection granularity and detection performance over state-of-the-art intrusion detection methods. Code is available at https://github.com/wangkai-tech23/StatGraph.
Embarassingly Simple Dataset Distillation
Authors: Feng Yunzhen, Vedantam Ramakrishna, Kempe Julia
Date: 2023-11-13 00:00:00
Tasks: Bilevel Optimization, Dataset Distillation, Dataset Distillation - 1IPC

Dataset distillation extracts a small set of synthetic training samples from a large dataset with the goal of achieving competitive performance on test data when trained on this sample. In this work, we tackle dataset distillation at its core by treating it directly as a bilevel optimization problem. Re-examining the foundational back-propagation through time method, we study the pronounced variance in the gradients, computational burden, and long-term dependencies. We introduce an improved method: Random Truncated Backpropagation Through Time (RaT-BPTT) to address them. RaT-BPTT incorporates a truncation coupled with a random window, effectively stabilizing the gradients and speeding up the optimization while covering long dependencies. This allows us to establish new state-of-the-art for a variety of standard dataset benchmarks. A deeper dive into the nature of distilled data unveils pronounced intercorrelation. In particular, subsets of distilled datasets tend to exhibit much worse performance than directly distilled smaller datasets of the same size. Leveraging RaT-BPTT, we devise a boosting mechanism that generates distilled datasets that contain subsets with near optimal performance across different data budgets.
Feature Channel Adaptive Enhancement for Fine-Grained Visual Classification
Authors: Dingzhou Xie, Cheng Pang, Guanhua Wu, Rushi Lan
Date: 2023-05-11 00:00:00
Proceeding: The 7th Asian Conference on Pattern Recognition 2023 5
Tasks: Classification, Fine-Grained Image Classification, Fine-Grained Image Recognition, image-classification, Image Classification

Fine-grained classification poses greater challenges compared
to basic-level image classification due to the visually similar sub-species.
To distinguish between confusing species, we introduce a novel framework
based on feature channel adaptive enhancement and attention erasure.
On one hand, a lightweight module employing both channel attention
and spatial attention is designed, adaptively enhancing the feature expression of important areas and obtaining more discriminative feature
vectors. On the other hand, we incorporate attention erasure methods
that compel the network to concentrate on less prominent areas, thereby
enhancing the network’s robustness. Our method can be seamlessly integrated into various backbone networks. Finally, an evaluation of our
approach is conducted across diverse public datasets, accompanied by
a comprehensive comparative analysis against state-of-the-art methodologies. The experimental findings substantiate the efficacy and viability
of our method in real-world scenarios, exemplifying noteworthy breakthroughs in intricate fine-grained classification endeavors.
CASTER: A Computer-Vision-Assisted Wireless Channel Simulator for Gesture Recognition
Authors: Zhenyu Ren, Guoliang Li, Chenqing Ji, Chao Yu, Shuai Wang, Rui Wang
Date: 2023-11-13 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

In this paper, a computer-vision-assisted simulation method is proposed to address the issue of training dataset acquisition for wireless hand gesture recognition. In the existing literature, in order to classify gestures via the wireless channel estimation, massive training samples should be measured in a consistent environment, consuming significant efforts. In the proposed CASTER simulator, however, the training dataset can be simulated via existing videos. Particularly, a gesture is represented by a sequence of snapshots, and the channel impulse response of each snapshot is calculated via tracing the rays scattered off a primitive-based hand model. Moreover, CASTER simulator relies on the existing videos to extract the motion data of gestures. Thus, the massive measurements of wireless channel can be eliminated. The experiments demonstrate a 90.8% average classification accuracy of simulation-to-reality inference.