pwc-archive/papers-with-abstracts huggingface dataset:
CCC-wav2vec 2.0: Clustering aided Cross Contrastive Self-supervised learning of speech representations
Authors: Vasista Sai Lodagala, Sreyan Ghosh, S. Umesh
Date: 2022-10-05 00:00:00
Tasks: Automatic Speech Recognition (ASR), Clustering, Language Modeling, Robust Speech Recognition, Self-Supervised Learning

While Self-Supervised Learning has helped reap the benefit of the scale from the available unlabeled data, the learning paradigms are continuously being bettered. We present a new pre-training strategy named ccc-wav2vec 2.0, which uses clustering and an augmentation-based cross-contrastive loss as its self-supervised objective. Through the clustering module, we scale down the influence of those negative examples that are highly similar to the positive. The Cross-Contrastive loss is computed between the encoder output of the original sample and the quantizer output of its augmentation and vice-versa, bringing robustness to the pre-training strategy. ccc-wav2vec 2.0 achieves up to 15.6% and 12.7% relative WER improvement over the baseline wav2vec 2.0 on the test-clean and test-other sets, respectively, of LibriSpeech, without the use of any language model. The proposed method also achieves up to 14.9% relative WER improvement over the baseline wav2vec 2.0 when fine-tuned on Switchboard data. We make all our codes publicly available on GitHub.
Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryption
Authors: Garam Lee, Minsoo Kim, Jai Hyun Park, Seung-won Hwang, Jung Hee Cheon
Date: 2022-10-05 00:00:00
Proceeding: NAACL 2022 7
Tasks: Classification, GPU, Privacy Preserving, text-classification, Text Classification

Embeddings, which compress information in raw text into semantics-preserving low-dimensional vectors, have been widely adopted for their efficacy. However, recent research has shown that embeddings can potentially leak private information about sensitive attributes of the text, and in some cases, can be inverted to recover the original input text. To address these growing privacy challenges, we propose a privatization mechanism for embeddings based on homomorphic encryption, to prevent potential leakage of any piece of information in the process of text classification. In particular, our method performs text classification on the encryption of embeddings from state-of-the-art models like BERT, supported by an efficient GPU implementation of CKKS encryption scheme. We show that our method offers encrypted protection of BERT embeddings, while largely preserving their utility on downstream text classification tasks.
JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT
Authors: Mayumi Ohta, Julia Kreutzer, Stefan Riezler
Date: 2022-10-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, NMT, Philosophy, speech-recognition, Speech Recognition, Speech-to-Text, Translation

JoeyS2T is a JoeyNMT extension for speech-to-text tasks such as automatic speech recognition and end-to-end speech translation. It inherits the core philosophy of JoeyNMT, a minimalist NMT toolkit built on PyTorch, seeking simplicity and accessibility. JoeyS2T's workflow is self-contained, starting from data pre-processing, over model training and prediction to evaluation, and is seamlessly integrated into JoeyNMT's compact and simple code base. On top of JoeyNMT's state-of-the-art Transformer-based encoder-decoder architecture, JoeyS2T provides speech-oriented components such as convolutional layers, SpecAugment, CTC-loss, and WER evaluation. Despite its simplicity compared to prior implementations, JoeyS2T performs competitively on English speech recognition and English-to-German speech translation benchmarks. The implementation is accompanied by a walk-through tutorial and available on https://github.com/may-/joeys2t.
Attention-based Ingredient Phrase Parser
Authors: Zhengxiang Shi, Pin Ni, MeiHui Wang, To Eun Kim, Aldo Lipani
Date: 2022-10-05 00:00:00
Tasks: Movie Recommendation

As virtual personal assistants have now penetrated the consumer market, with products such as Siri and Alexa, the research community has produced several works on task-oriented dialogue tasks such as hotel booking, restaurant booking, and movie recommendation. Assisting users to cook is one of these tasks that are expected to be solved by intelligent assistants, where ingredients and their corresponding attributes, such as name, unit, and quantity, should be provided to users precisely and promptly. However, existing ingredient information scraped from the cooking website is in the unstructured form with huge variation in the lexical structure, for example, '1 garlic clove, crushed', and '1 (8 ounce) package cream cheese, softened', making it difficult to extract information exactly. To provide an engaged and successful conversational service to users for cooking tasks, we propose a new ingredient parsing model that can parse an ingredient phrase of recipes into the structure form with its corresponding attributes with over 0.93 F1-score. Experimental results show that our model achieves state-of-the-art performance on AllRecipes and Food.com datasets.
"No, they did not": Dialogue response dynamics in pre-trained language models
Authors: Sanghee J. Kim, Lang Yu, Allyson Ettinger
Date: 2022-10-05 00:00:00
Tasks: Sensitivity

A critical component of competence in language is being able to identify relevant components of an utterance and reply appropriately. In this paper we examine the extent of such dialogue response sensitivity in pre-trained language models, conducting a series of experiments with a particular focus on sensitivity to dynamics involving phenomena of at-issueness and ellipsis. We find that models show clear sensitivity to a distinctive role of embedded clauses, and a general preference for responses that target main clause content of prior utterances. However, the results indicate mixed and generally weak trends with respect to capturing the full range of dynamics involved in targeting at-issue versus not-at-issue content. Additionally, models show fundamental limitations in grasp of the dynamics governing ellipsis, and response selections show clear interference from superficial factors that outweigh the influence of principled discourse constraints.
Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation
Authors: Arturo Oncevay, Kervy Dante Rivas Rojas, Liz Karen Chavez Sanchez, Roberto Zariquiey
Date: 2022-10-05 00:00:00
Proceeding: COLING 2022 10
Tasks: Language Modelling, Machine Translation, Segmentation, Translation

Language modelling and machine translation tasks mostly use subword or character inputs, but syllables are seldom used. Syllables provide shorter sequences than characters, require less-specialised extracting rules than morphemes, and their segmentation is not impacted by the corpus size. In this study, we first explore the potential of syllables for open-vocabulary language modelling in 21 languages. We use rule-based syllabification methods for six languages and address the rest with hyphenation, which works as a syllabification proxy. With a comparable perplexity, we show that syllables outperform characters and other subwords. Moreover, we study the importance of syllables on neural machine translation for a non-related and low-resource language-pair (Spanish--Shipibo-Konibo). In pairwise and multilingual systems, syllables outperform unsupervised subwords, and further morphological segmentation methods, when translating into a highly synthetic language with a transparent orthography (Shipibo-Konibo). Finally, we perform some human evaluation, and discuss limitations and opportunities.
Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors
Authors: Mohammad Reza Taesiri, Finlay Macklon, Yihe Wang, Hengshuo Shen, Cor-Paul Bezemer
Date: 2022-10-05 00:00:00
Tasks: Common Sense Reasoning, Language Modelling, Large Language Model, Question Answering

Video game testing requires game-specific knowledge as well as common sense reasoning about the events in the game. While AI-driven agents can satisfy the first requirement, it is not yet possible to meet the second requirement automatically. Therefore, video game testing often still relies on manual testing, and human testers are required to play the game thoroughly to detect bugs. As a result, it is challenging to fully automate game testing. In this study, we explore the possibility of leveraging the zero-shot capabilities of large language models for video game bug detection. By formulating the bug detection problem as a question-answering task, we show that large language models can identify which event is buggy in a sequence of textual descriptions of events from a game. To this end, we introduce the GameBugDescriptions benchmark dataset, which consists of 167 buggy gameplay videos and a total of 334 question-answer pairs across 8 games. We extensively evaluate the performance of six models across the OPT and InstructGPT large language model families on our benchmark dataset. Our results show promising results for employing language models to detect video game bugs. With the proper prompting technique, we could achieve an accuracy of 70.66%, and on some video games, up to 78.94%. Our code, evaluation data and the benchmark can be found on https://asgaardlab.github.io/LLMxBugs
Stochastic Neuromorphic Circuits for Solving MAXCUT
Authors: Bradley H. Theilman, Yipu Wang, Ojas D. Parekh, William Severa, J. Darby Smith, James B. Aimone
Date: 2022-10-05 00:00:00

Finding the maximum cut of a graph (MAXCUT) is a classic optimization problem that has motivated parallel algorithm development. While approximate algorithms to MAXCUT offer attractive theoretical guarantees and demonstrate compelling empirical performance, such approximation approaches can shift the dominant computational cost to the stochastic sampling operations. Neuromorphic computing, which uses the organizing principles of the nervous system to inspire new parallel computing architectures, offers a possible solution. One ubiquitous feature of natural brains is stochasticity: the individual elements of biological neural networks possess an intrinsic randomness that serves as a resource enabling their unique computational capacities. By designing circuits and algorithms that make use of randomness similarly to natural brains, we hypothesize that the intrinsic randomness in microelectronics devices could be turned into a valuable component of a neuromorphic architecture enabling more efficient computations. Here, we present neuromorphic circuits that transform the stochastic behavior of a pool of random devices into useful correlations that drive stochastic solutions to MAXCUT. We show that these circuits perform favorably in comparison to software solvers and argue that this neuromorphic hardware implementation provides a path for scaling advantages. This work demonstrates the utility of combining neuromorphic principles with intrinsic randomness as a computational resource for new computational architectures.
Biological neurons act as generalization filters in reservoir computing
Authors: Takuma Sumi, Hideaki Yamamoto, Yuichi Katori, Satoshi Moriya, Tomohiro Konno, Shigeo Sato, Ayumi Hirano-Iwata
Date: 2022-10-06 00:00:00
Tasks: Decoder, Time Series Analysis, Transfer Learning

Reservoir computing is a machine learning paradigm that transforms the transient dynamics of high-dimensional nonlinear systems for processing time-series data. Although reservoir computing was initially proposed to model information processing in the mammalian cortex, it remains unclear how the non-random network architecture, such as the modular architecture, in the cortex integrates with the biophysics of living neurons to characterize the function of biological neuronal networks (BNNs). Here, we used optogenetics and fluorescent calcium imaging to record the multicellular responses of cultured BNNs and employed the reservoir computing framework to decode their computational capabilities. Micropatterned substrates were used to embed the modular architecture in the BNNs. We first show that modular BNNs can be used to classify static input patterns with a linear decoder and that the modularity of the BNNs positively correlates with the classification accuracy. We then used a timer task to verify that BNNs possess a short-term memory of ~1 s and finally show that this property can be exploited for spoken digit classification. Interestingly, BNN-based reservoirs allow transfer learning, wherein a network trained on one dataset can be used to classify separate datasets of the same category. Such classification was not possible when the input patterns were directly decoded by a linear decoder, suggesting that BNNs act as a generalization filter to improve reservoir computing performance. Our findings pave the way toward a mechanistic understanding of information processing within BNNs and, simultaneously, build future expectations toward the realization of physical reservoir computing systems based on BNNs.
Inference on Causal Effects of Interventions in Time using Gaussian Processes
Authors: Gianluca Giudice, Sara Geneletti, Konstantinos Kalogeropoulos
Date: 2022-10-06 00:00:00
Tasks: Gaussian Processes, Time Series, Time Series Analysis

This paper focuses on drawing inference on the causal impact of an intervention at a specific time point, as manifested in an outcome variable over time. We operate on the interrupted time series framework and expand on approaches such as the synthetic control (Abadie 2003) and Bayesian structural time series (Brodersen et al 2015), by replacing the underlying dynamic linear regression model with a non-parametric formulation based on Gaussian Processes. The developed models possess a high degree of flexibility posing very little limitations on the functional form and allow to incorporate uncertainty, stemming from its estimation, under the Bayesian framework. We introduce two families of non-parametric structural time series models either operating on the trajectory of the outcome variable alone, or in a multivariate setting using multiple output Gaussian processes. The paper engages closely with a case study focusing on the impact of the accelerated UK vaccination schedule, as contrasted with the rest of Europe, to illustrate the methodology and present the implementation procedure.
Heterogeneous Treatment Effect Estimation for Observational Data using Model-based Forests
Authors: Susanne Dandl, Andreas Bender, Torsten Hothorn
Date: 2022-10-06 00:00:00
Tasks: Heterogeneous Treatment Effect Estimation

The estimation of heterogeneous treatment effects (HTEs) has attracted considerable interest in many disciplines, most prominently in medicine and economics. Contemporary research has so far primarily focused on continuous and binary responses where HTEs are traditionally estimated by a linear model, which allows the estimation of constant or heterogeneous effects even under certain model misspecifications. More complex models for survival, count, or ordinal outcomes require stricter assumptions to reliably estimate the treatment effect. Most importantly, the noncollapsibility issue necessitates the joint estimation of treatment and prognostic effects. Model-based forests allow simultaneous estimation of covariate-dependent treatment and prognostic effects, but only for randomized trials. In this paper, we propose modifications to model-based forests to address the confounding issue in observational data. In particular, we evaluate an orthogonalization strategy originally proposed by Robinson (1988, Econometrica) in the context of model-based forests targeting HTE estimation in generalized linear models and transformation models. We found that this strategy reduces confounding effects in a simulated study with various outcome distributions. We demonstrate the practical aspects of HTE estimation for survival and ordinal outcomes by an assessment of the potentially heterogeneous effect of Riluzole on the progress of Amyotrophic Lateral Sclerosis.
Sample-and-Forward: Communication-Efficient Control of the False Discovery Rate in Networks
Authors: Mehrdad Pournaderi, Yu Xiang
Date: 2022-10-05 00:00:00

This work concerns controlling the false discovery rate (FDR) in networks under communication constraints. We present sample-and-forward, a flexible and communication-efficient version of the Benjamini-Hochberg (BH) procedure for multihop networks with general topologies. Our method evidences that the nodes in a network do not need to communicate p-values to each other to achieve a decent statistical power under the global FDR control constraint. Consider a network with a total of $m$ p-values, our method consists of first sampling the (empirical) CDF of the p-values at each node and then forwarding $\mathcal{O}(\log m)$ bits to its neighbors. Under the same assumptions as for the original BH procedure, our method has both the provable finite-sample FDR control as well as competitive empirical detection power, even with a few samples at each node. We provide an asymptotic analysis of power under a mixture model assumption on the p-values.
Learning with Limited Samples -- Meta-Learning and Applications to Communication Systems
Authors: Lisha Chen, Sharu Theresa Jose, Ivana Nikoloska, Sangwoo Park, Tianyi Chen, Osvaldo Simeone
Date: 2022-10-03 00:00:00
Tasks: Bilevel Optimization, image-classification, Image Classification, Meta-Learning, speech-recognition, Speech Recognition

Deep learning has achieved remarkable success in many machine learning tasks such as image classification, speech recognition, and game playing. However, these breakthroughs are often difficult to translate into real-world engineering systems because deep learning models require a massive number of training samples, which are costly to obtain in practice. To address labeled data scarcity, few-shot meta-learning optimizes learning algorithms that can efficiently adapt to new tasks quickly. While meta-learning is gaining significant interest in the machine learning literature, its working principles and theoretic fundamentals are not as well understood in the engineering community. This review monograph provides an introduction to meta-learning by covering principles, algorithms, theory, and engineering applications. After introducing meta-learning in comparison with conventional and joint learning, we describe the main meta-learning algorithms, as well as a general bilevel optimization framework for the definition of meta-learning techniques. Then, we summarize known results on the generalization capabilities of meta-learning from a statistical learning viewpoint. Applications to communication systems, including decoding and power allocation, are discussed next, followed by an introduction to aspects related to the integration of meta-learning with emerging computing technologies, namely neuromorphic and quantum computing. The monograph is concluded with an overview of open research challenges.
DBkWik++ -- Multi Source Matching of Knowledge Graphs
Authors: Sven Hertling, Heiko Paulheim
Date: 2022-10-06 00:00:00
Tasks: Knowledge Graphs

Large knowledge graphs like DBpedia and YAGO are always based on the same source, i.e., Wikipedia. But there are more wikis that contain information about long-tail entities such as wiki hosting platforms like Fandom. In this paper, we present the approach and analysis of DBkWik++, a fused Knowledge Graph from thousands of wikis. A modified version of the DBpedia framework is applied to each wiki which results in many isolated Knowledge Graphs. With an incremental merge based approach, we reuse one-to-one matching systems to solve the multi source KG matching task. Based on this alignment we create a consolidated knowledge graph with more than 15 million instances.
Cyber-Resilient Privacy Preservation and Secure Billing Approach for Smart Energy Metering Devices
Authors: Venkatesh Kumar M
Date: 2022-10-06 00:00:00

Most of the smart applications, such as smart energy metering devices, demand strong privacy preservation to strengthen data privacy. However, it is difficult to protect the privacy of the smart device data, especially on the client side. It is mainly because payment for billing is computed by the server deployed at the client's side, and it is highly challenging to prevent the leakage of client's information to unauthorised users. Various researchers have discussed this problem and have proposed different privacy preservation techniques. Conventional techniques suffer from the problem of high computational and communication overload on the client side. In addition, the performance of these techniques deteriorates due to computational complexity and their inability to handle the security of large-scale data. Due to these limitations, it becomes easy for the attackers to introduce malicious attacks on the server, posing a significant threat to data security. In this context, this proposal intends to design novel privacy preservation and secure billing framework using deep learning techniques to ensure data security in smart energy metering devices. This research aims to overcome the limitations of the existing techniques to achieve robust privacy preservation in smart devices and increase the cyber resilience of these devices.
ForestQB: An Adaptive Query Builder to Support Wildlife Research
Authors: Omar Mussa, Omer Rana, Benoît Goossens, Pablo Orozco-terWengel, Charith Perera
Date: 2022-10-06 00:00:00

This paper presents ForestQB, a SPARQL query builder, to assist Bioscience and Wildlife Researchers in accessing Linked-Data. As they are unfamiliar with the Semantic Web and the data ontologies, ForestQB aims to empower them to benefit from using Linked-Data to extract valuable information without having to grasp the nature of the data and its underlying technologies. ForestQB is integrating Form-Based Query builders with Natural Language to simplify query construction to match the user requirements. Demo available at https://iotgarage.net/demo/forestQB
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
Authors: Anonymous
Date: 2022-09-22 00:00:00
Proceeding: ICLR2023 submitted 2022 9
Tasks: Action Classification, Action Recognition, Video Understanding

Learning discriminative spatiotemporal representation is the key problem of video understanding. Recently, Vision Transformers (ViTs) have shown their power in learning long-term video dependency with self-attention. Unfortunately, they exhibit limitations in tackling local video redundancy, due to the blind global comparison among tokens. UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format. However, this model has to require a tiresome and complicated image-pretraining phrase, before being finetuned on videos. This blocks its wide usage in practice. On the contrary, open-sourced ViTs are readily available and well-pretrained with rich image supervision. Based on these observations, we propose a generic paradigm to build a powerful family of video networks, by arming the pretrained ViTs with efficient UniFormer designs. We call this family UniFormerV2, since it inherits the concise style of the UniFormer block. But it contains brand-new local and global relation aggregators, which allow for preferable accuracy-computation balance by seamlessly integrating advantages from both ViTs and UniFormer. Without any bells and whistles, our UniFormerV2 gets the state-of-the-art recognition performance on 8 popular video benchmarks, including scene-related Kinetics-400/600/700 and Moments in Time, temporal-related Something-Something V1/V2, untrimmed ActivityNet and HACS. In particular, it is the first model to achieve 90% top-1 accuracy on Kinetics-400, to our best knowledge. The models will be released afterward.