pwc-archive/papers-with-abstracts huggingface dataset:
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
Authors: Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, Daxin Jiang
Date: 2022-03-04 00:00:00
Proceeding: ACL 2022 5
Tasks: counterfactual, Few-Shot Learning

Generating new events given context with correlated ones plays a crucial role in many event-centric reasoning tasks. Existing works either limit their scope to specific scenarios or overlook event-level correlations. In this paper, we propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. To achieve this, we propose three novel event-centric objectives, i.e., whole event recovering, contrastive event-correlation encoding and prompt-based event locating, which highlight event-level correlations with effective training. The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of (i) event-correlation types (e.g., causal, temporal, contrast), (ii) application formulations (i.e., generation and classification), and (iii) reasoning types (e.g., abductive, counterfactual and ending reasoning). Empirical fine-tuning results, as well as zero- and few-shot learning, on 9 benchmarks (5 generation and 4 classification tasks covering 4 reasoning types with diverse event correlations), verify its effectiveness and generalization ability.
SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models
Authors: Liang Wang, Wei Zhao, Zhuoyu Wei, Jingming Liu
Date: 2022-03-04 00:00:00
Proceeding: ACL 2022 5
Tasks: Contrastive Learning, Graph Embedding, Knowledge Graph Completion, Link Prediction

Knowledge graph completion (KGC) aims to reason over known facts and infer the missing links. Text-based methods such as KGBERT (Yao et al., 2019) learn entity representations from natural language descriptions, and have the potential for inductive KGC. However, the performance of text-based methods still largely lag behind graph embedding-based methods like TransE (Bordes et al., 2013) and RotatE (Sun et al., 2019b). In this paper, we identify that the key issue is efficient contrastive learning. To improve the learning efficiency, we introduce three types of negatives: in-batch negatives, pre-batch negatives, and self-negatives which act as a simple form of hard negatives. Combined with InfoNCE loss, our proposed model SimKGC can substantially outperform embedding-based methods on several benchmark datasets. In terms of mean reciprocal rank (MRR), we advance the state-of-the-art by +19% on WN18RR, +6.8% on the Wikidata5M transductive setting, and +22% on the Wikidata5M inductive setting. Thorough analyses are conducted to gain insights into each component. Our code is available at https://github.com/intfloat/SimKGC .
Deep Lexical Hypothesis: Identifying personality structure in natural language
Authors: Andrew Cutler, David M. Condon
Date: 2022-03-04 00:00:00
Tasks: Language Modelling, Survey

Recent advances in natural language processing (NLP) have produced general models that can perform complex tasks such as summarizing long passages and translating across languages. Here, we introduce a method to extract adjective similarities from language models as done with survey-based ratings in traditional psycholexical studies but using millions of times more text in a natural setting. The correlational structure produced through this method is highly similar to that of self- and other-ratings of 435 terms reported by Saucier and Goldberg (1996a). The first three unrotated factors produced using NLP are congruent with those in survey data, with coefficients of 0.89, 0.79, and 0.79. This structure is robust to many modeling decisions: adjective set, including those with 1,710 terms (Goldberg, 1982) and 18,000 terms (Allport & Odbert, 1936); the query used to extract correlations; and language model. Notably, Neuroticism and Openness are only weakly and inconsistently recovered. This is a new source of signal that is closer to the original (semantic) vision of the Lexical Hypothesis. The method can be applied where surveys cannot: in dozens of languages simultaneously, with tens of thousands of items, on historical text, and at extremely large scale for little cost. The code is made public to facilitate reproduction and fast iteration in new directions of research.
Deep Latent-Variable Models for Text Generation
Authors: Xiaoyu Shen
Date: 2022-03-03 00:00:00
Tasks: Decoder, Dialogue Generation, Diversity, Document Summarization, Machine Translation, Text Generation

Text generation aims to produce human-like natural language output for down-stream tasks. It covers a wide range of applications like machine translation, document summarization, dialogue generation and so on. Recently deep neural network-based end-to-end architectures have been widely adopted. The end-to-end approach conflates all sub-modules, which used to be designed by complex handcrafted rules, into a holistic encode-decode architecture. Given enough training data, it is able to achieve state-of-the-art performance yet avoiding the need of language/domain-dependent knowledge. Nonetheless, deep learning models are known to be extremely data-hungry, and text generated from them usually suffer from low diversity, interpretability and controllability. As a result, it is difficult to trust the output from them in real-life applications. Deep latent-variable models, by specifying the probabilistic distribution over an intermediate latent process, provide a potential way of addressing these problems while maintaining the expressive power of deep neural networks. This dissertation presents how deep latent-variable models can improve over the standard encoder-decoder model for text generation.
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages
Authors: Vaidehi Patil, Partha Talukdar, Sunita Sarawagi
Date: 2022-03-03 00:00:00
Proceeding: ACL 2022 5
Tasks: Cross-Lingual Transfer, XLM-R, Zero-Shot Cross-Lingual Transfer

Pre-trained multilingual language models such as mBERT and XLM-R have demonstrated great potential for zero-shot cross-lingual transfer to low web-resource languages (LRL). However, due to limited model capacity, the large difference in the sizes of available monolingual corpora between high web-resource languages (HRL) and LRLs does not provide enough scope of co-embedding the LRL with the HRL, thereby affecting downstream task performance of LRLs. In this paper, we argue that relatedness among languages in a language family along the dimension of lexical overlap may be leveraged to overcome some of the corpora limitations of LRLs. We propose Overlap BPE (OBPE), a simple yet effective modification to the BPE vocabulary generation algorithm which enhances overlap across related languages. Through extensive experiments on multiple NLP tasks and datasets, we observe that OBPE generates a vocabulary that increases the representation of LRLs via tokens shared with HRLs. This results in improved zero-shot transfer from related HRLs to LRLs without reducing HRL representation and accuracy. Unlike previous studies that dismissed the importance of token-overlap, we show that in the low-resource related language setting, token overlap matters. Synthetically reducing the overlap to zero can cause as much as a four-fold drop in zero-shot transfer accuracy.
HV-Net: Hypervolume Approximation based on DeepSets
Authors: Ke Shang, WeiYu Chen, Weiduo Liao, Hisao Ishibuchi
Date: 2022-03-04 00:00:00

In this letter, we propose HV-Net, a new method for hypervolume approximation in evolutionary multi-objective optimization. The basic idea of HV-Net is to use DeepSets, a deep neural network with permutation invariant property, to approximate the hypervolume of a non-dominated solution set. The input of HV-Net is a non-dominated solution set in the objective space, and the output is an approximated hypervolume value of this solution set. The performance of HV-Net is evaluated through computational experiments by comparing it with two commonly-used hypervolume approximation methods (i.e., point-based method and line-based method). Our experimental results show that HV-Net outperforms the other two methods in terms of both the approximation error and the runtime, which shows the potential of using deep learning technique for hypervolume approximation.
Adaptive Semi-Supervised Inference for Optimal Treatment Decisions with Electronic Medical Record Data
Authors: Kevin Gunn, Wenbin Lu, Rui Song
Date: 2022-03-04 00:00:00
Tasks: Imputation

A treatment regime is a rule that assigns a treatment to patients based on their covariate information. Recently, estimation of the optimal treatment regime that yields the greatest overall expected clinical outcome of interest has attracted a lot of attention. In this work, we consider estimation of the optimal treatment regime with electronic medical record data under a semi-supervised setting. Here, data consist of two parts: a set of `labeled' patients for whom we have the covariate, treatment and outcome information, and a much larger set of `unlabeled' patients for whom we only have the covariate information. We proposes an imputation-based semi-supervised method, utilizing `unlabeled' individuals to obtain a more efficient estimator of the optimal treatment regime. The asymptotic properties of the proposed estimators and their associated inference procedure are provided. Simulation studies are conducted to assess the empirical performance of the proposed method and to compare with a fully supervised method using only the labeled data. An application to an electronic medical record data set on the treatment of hypotensive episodes during intensive care unit (ICU) stays is also given for further illustration.
Multi-objective robust optimization using adaptive surrogate models for problems with mixed continuous-categorical parameters
Authors: M. Moustapha, A. Galimshina, G. Habert, B. Sudret
Date: 2022-03-03 00:00:00
Tasks: Robust Design

Explicitly accounting for uncertainties is paramount to the safety of engineering structures. Optimization which is often carried out at the early stage of the structural design offers an ideal framework for this task. When the uncertainties are mainly affecting the objective function, robust design optimization is traditionally considered. This work further assumes the existence of multiple and competing objective functions that need to be dealt with simultaneously. The optimization problem is formulated by considering quantiles of the objective functions which allows for the combination of both optimality and robustness in a single metric. By introducing the concept of common random numbers, the resulting nested optimization problem may be solved using a general-purpose solver, herein the non-dominated sorting genetic algorithm (NSGA-II). The computational cost of such an approach is however a serious hurdle to its application in real-world problems. We therefore propose a surrogate-assisted approach using Kriging as an inexpensive approximation of the associated computational model. The proposed approach consists of sequentially carrying out NSGA-II while using an adaptively built Kriging model to estimate the quantiles. Finally, the methodology is adapted to account for mixed categorical-continuous parameters as the applications involve the selection of qualitative design parameters as well. The methodology is first applied to two analytical examples showing its efficiency. The third application relates to the selection of optimal renovation scenarios of a building considering both its life cycle cost and environmental impact. It shows that when it comes to renovation, the heating system replacement should be the priority.
Hypernets -- Good (G)news for Gnutella
Authors: N. J. Gunther
Date: 2002-02-16 00:00:00

Criticism of Gnutella network scalability has rested on the bandwidth attributes of the original interconnection topology: a Cayley tree. Trees, in general, are known to have lower aggregate bandwidth than higher dimensional topologies e.g., hypercubes, meshes and tori. Gnutella was intended to support thousands to millions of peers. Studies of interconnection topologies in the literature, however, have focused on hardware implementations which are limited by cost to a few thousand nodes. Since the Gnutella network is virtual, hyper-topologies are relatively unfettered by such constraints. We present performance models for several plausible hyper-topologies and compare their query throughput up to millions of peers. The virtual hypercube and the virtual hypertorus are shown to offer near linear scalability subject to the number of peer TCP/IP connections that can be simultaneously kept open.
Sound-to-imagination: Unsupervised cross- modal translation using deep dense network architecture.
Authors: Leonardo A. Fanzeres · Climent Nadeu
Date: 2021-06-02 00:00:00
Proceeding: 2021 2021 6
Tasks: Diversity, Translation

The motivation of our research is to develop a sound-to-image (S2I) translation system for enabling a human receiver
to visually infer the occurrence of sound related events. We expect the computer to ‘imagine’ the scene from the captured
sound, generating original images that picture the sound emitting source. Previous studies on similar topics opted for simplified approaches using data with low content diversity and/or strong supervision. Differently, we propose to perform
unsupervised S2I translation using thousands of distinct and unknown scenes, with slightly pre-cleaned data, just enough to
guarantee aural-visual semantic coherence. To that end, we employ conditional generative adversarial networks (GANs) with a
deep densely connected generator. Besides, we implemented a moving-average adversarial loss to address GANs training
instability. Though the specified S2I translation problem is quite challenging, we were able to generalize the translator model
enough to obtain more than 14%, in average, of interpretable and semantically coherent images translated from unknown
sounds. Additionally, we present a solution using informativity classifiers to perform quantitative evaluation of S2I translation.
Consistent Depth Prediction under Various Illuminations using Dilated Cross Attention
Authors: Zitian Zhang, Chuhua Xian
Date: 2021-12-15 00:00:00
Tasks: Depth Estimation, Depth Prediction

In this paper, we aim to solve the problem of consistent depth prediction in complex scenes under various illumination conditions. The existing indoor datasets based on RGB-D sensors or virtual rendering have two critical limitations - sparse depth maps (NYU Depth V2) and non-realistic illumination (SUN CG, SceneNet RGB-D). We propose to use internet 3D indoor scenes and manually tune their illuminations to render photo-realistic RGB photos and their corresponding depth and BRDF maps, obtaining a new indoor depth dataset called Vari dataset. We propose a simple convolutional block named DCA by applying depthwise separable dilated convolution on encoded features to process global information and reduce parameters. We perform cross attention on these dilated features to retain the consistency of depth prediction under different illuminations. Our method is evaluated by comparing it with current state-of-the-art methods on Vari dataset and a significant improvement is observed in our experiments. We also conduct the ablation study, finetune our model on NYU Depth V2 and also evaluate on real-world data to further validate the effectiveness of our DCA block. The code, pre-trained weights and Vari dataset are open-sourced.
Malware Images:Visualization and Automatic Classification
Authors: Nataraj L., Karthikeyan S., Jacob G., Manjunath B. S.
Date: 2011-07-20 00:00:00
Proceeding: Proceedings of the 8th International Symposium on Visualization for Cyber Security 2011 7
Tasks: Classification

We propose a simple yet effective method for visualizing and classifying malware using image processing techniques. Malware binaries are visualized as gray-scale images, with the observation that for many malware families, the images belonging to the same family appear very similar in layout and texture. Motivated by this visual similarity, a classification method using standard image features is proposed. Neither disassembly nor code execution is required for classification. Preliminary experimental results are quite promising with 98% classification accuracy on a malware database of 9,458 samples with 25 different malware families. Our technique also exhibits interesting resilience to popular obfuscation techniques such as section encryption.
Fast Unsupervised Anomaly Detection in Traffic Videos
Authors: Keval Doshi, Yasin Yilmaz
Date: 2020-07-28 00:00:00
Proceeding: CVPR 2020 7
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

Anomaly detection in traffic videos has been recently gaining attention due to its importance in intelligent transportation systems. Due to several factors such as weather, viewpoint, lighting conditions, etc. affecting the video quality of a real time traffic feed, it still remains a challenging problem. Even though the performance of state-of-the-art methods on the available benchmark dataset has been competitive, they demand a massive amount of external training data combined with significant computational resources. In this paper, we propose a fast unsupervised anomaly detection system comprising of three modules: preprocessing module, candidate selection module and backtracking anomaly detection module. The preprocessing module outputs stationary objects detected in a video. Then, the candidate selection module removes the misclassified stationary objects using a nearest neighbor approach and then uses K-means clustering to identify potential anomalous regions. Finally, the backtracking anomaly detection algorithm computes a similarity statistic and decides on the onset time of the anomaly. Experimental results on the Track 4 test set of the NVIDIA AI CITY 2020 challenge show the efficacy of the proposed framework as we achieve an F1-score of 0.5926 along with 8.2386 root mean square error (RMSE) and are ranked second in the competition
The Speechtransformer for Large-scale Mandarin Chinese Speech Recognition
Authors: YuanYuan Zhao, Jie Li, Xiaorui Wang, Yan Li
Date: 2019-04-17 00:00:00
Proceeding: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2019 4
Tasks: Decoder, speech-recognition, Speech Recognition

Attention-based sequence-to-sequence architectures have made great progress in the speech recognition task. The SpeechTransformer, a no-recurrence encoder-decoder architecture, has shown promising results on small-scale speech recognition data sets in previous works. In this paper, we focus on a large-scale Mandarin Chinese speech recognition task and propose three optimization strategies to further improve the performance and efficiency of the SpeechTransformer. Our first improvement is to use a much lower frame rate, which is shown very beneficial to not only the computation efficiency but also the model performance. The other two strategies are scheduled sampling and focal loss, which are both very effective to reduce the character error rate (CER). On a 8,000 hours task, the proposed improvements yield 10.8%-26.1% relative gain in CER on four different test sets. Compared to a strong hybrid TDNN-LSTM system, which is trained with LF-MMI criterion and decoded with a large 4-gram LM, the final optimized Speech-Transformer gives 12.2%-19.1% relative CER reduction without any explicit language models.
Speech watermarking: an approach for the forensic analysis of digital telephonic recordings
Authors: Marcos Faundez-Zanuy, Jose Juan Lucena-Molina, Martin Hagmueller
Date: 2022-02-23 00:00:00
Tasks: Articles, Speaker Identification, valid

In this article, the authors discuss the problem of forensic authentication of digital audio recordings. Although forensic audio has been addressed in several articles, the existing approaches are focused on analog magnetic recordings, which are less prevalent because of the large amount of digital recorders available on the market (optical, solid state, hard disks, etc.). An approach based on digital signal processing that consists of spread spectrum techniques for speech watermarking is presented. This approach presents the advantage that the authentication is based on the signal itself rather than the recording format. Thus, it is valid for usual recording devices in police-controlled telephone intercepts. In addition, our proposal allows for the introduction of relevant information such as the recording date and time and all the relevant data (this is not always possible with classical systems). Our experimental results reveal that the speech watermarking procedure does not interfere in a significant way with the posterior forensic speaker identification.
Social Opinion Formation and Decision Making Under Communication Trends
Authors: Mert Kayaalp, Virginia Bordignon, Ali H. Sayed
Date: 2022-03-04 00:00:00
Tasks: Decision Making

This work studies the learning process over social networks under partial and random information sharing. In traditional social learning models, agents exchange full belief information with each other while trying to infer the true state of nature. We study the case where agents share information about only one hypothesis, namely, the trending topic, which can be randomly changing at every iteration. We show that agents can learn the true hypothesis even if they do not discuss it, at rates comparable to traditional social learning. We also show that using one's own belief as a prior for estimating the neighbors' non-transmitted beliefs might create opinion clusters that prevent learning with full confidence. This phenomenon occurs when a single hypothesis corresponding to the truth is exchanged exclusively during all times. Such a practice, however, avoids the complete rejection of the truth under any information exchange procedure -- something that could happen if priors were uniform.
Solution of integrals with fractional Brownian motion for different Hurst indices
Authors: Fei Gao, Shuaiqiang Liu, Cornelis W. Oosterlee, Nico M. Temme
Date: 2022-03-04 00:00:00

In this paper, we will evaluate integrals that define the conditional expectation, variance and characteristic function of stochastic processes with respect to fractional Brownian motion (fBm) for all relevant Hurst indices, i.e. $H \in (0,1)$. The fractional Ornstein-Uhlenbeck (fOU) process, for example, gives rise to highly nontrivial integration formulas that need careful analysis when considering the whole range of Hurst indices. We will show that the classical technique of analytic continuation, from complex analysis, provides a way of extending the domain of validity of an integral, from $H\in(1/2,1)$, to the larger domain, $H\in(0,1)$. Numerical experiments for different Hurst indices confirm the robustness and efficiency of the integral formulations presented here. Moreover, we provide accurate and highly efficient financial option pricing results for processes that are related to the fOU process, with the help of Fourier cosine expansions.