pwc-archive/papers-with-abstracts huggingface dataset:
A Relation-Specific Attention Network for Joint Entity and Relation Extraction
Authors: Yue Yuan, Xiaofei Zhou, Shirui Pan, Qiannan Zhu, Zeliang Song, Li Guo
Date: 2020-07-01 00:00:00
Tasks: Joint Entity and Relation Extraction, Relation, Relation Extraction, Sentence

Joint extraction of entities and relations is an important task in natural language processing (NLP), which aims to capture all relational triplets from plain texts. This is a big challenge due to some of the triplets extracted from one sentence may have overlapping entities. Most existing methods perform entity recognition followed by relation detection between every possible entity pairs, which usually suffers from numerous redundant operations. In this paper, we propose a relation-specific attention network (RSAN) to handle the issue. Our RSAN utilizes relation-aware attention mechanism to construct specific sentence representations for each relation, and then performs sequence labeling to extract its corresponding head and tail entities. Experiments on two public datasets show that our model can effectively extract overlapping triplets and achieve state-of-the-art performance. Our code is available at https://github.com/Anery/RSAN
Modeling Dense Cross-Modal Interactions for Joint Entity-Relation Extraction
Authors: Shan Zhao, Minghao Hu, Zhiping Cai, Fang Liu
Date: 2020-07-01 00:00:00
Tasks: Joint Entity and Relation Extraction, Relation, Relation Classification, Relation Extraction

Joint extraction of entities and their relations benefits from the close interaction between named entities and their relation information. Therefore, how to effectively model such cross-modal interactions is critical for the final performance. Previous works have used simple methods such as label-feature concatenation to perform coarse-grained semantic fusion among cross-modal instances, but fail to capture fine-grained correlations over token and label spaces, resulting in insufficient interactions. In this paper, we propose a deep Cross-Modal Attention Network (CMAN) for joint entity and relation extraction. The network is carefully constructed by stacking multiple attention units in depth to fully model dense interactions over token-label spaces, in which two basic attention units are proposed to explicitly capture fine-grained correlations across different modalities (e.g., token-to-token and labelto-token). Experiment results on CoNLL04 dataset show that our model obtains state-of-the-art results by achieving 90.62% F1 on entity recognition and 72.97% F1 on relation classification. In ADE dataset, our model surpasses existing approaches by more than 1.9% F1 on relation classification. Extensive analyses further confirm the effectiveness of our approach.
Neuro-Adaptive Formation Control and Target Tracking for Nonlinear Multi-Agent Systems with Time-Delay
Authors: Kiarash Aryankia, Rastko R. Selmic
Date: 2020-06-01 00:00:00

This paper proposes an adaptive neural network-based backstepping controller that uses rigid graph theory to address the distance-based formation control problem and target tracking for nonlinear multi-agent systems with bounded time-delay and disturbance. The radial basis function neural network (RBFNN) is used to overcome and compensate for the unknown nonlinearity and disturbance in the system dynamics. The effect of the state time-delay of the agents is alleviated by using an appropriate control signal that is designed based on specific Lyapunov function and Young's inequality. The adaptive neural network (NN) weights tuning law is derived using this Lyapunov function. An upper bound for the singular value of the normalized rigidity matrix is introduced, and uniform ultimate boundedness (UUB) of the formation distance error is rigorously proven based on the Lyapunov stability theory. Finally, the performance and effectiveness of the proposed method are validated through the simulation results on nonlinear multi-agent systems. Comparisons between the proposed distance-based method and an existing, displacement-based method are provided to evaluate the performance of the suggested method.
Contextual Combinatorial Volatile Multi-armed Bandit with Adaptive Discretization
Authors: Andi Nika, Sepehr Elahi, Cem Tekin
Date: 2020-08-28 00:00:00

We consider contextual combinatorial volatile multi-armed bandit (CCV-MAB), in which at each round, the learner observes a set of available base arms and their contexts, and then, selects a super arm that contains $K$ base arms in order to maximize its cumulative reward. Under the semi-bandit feedback setting and assuming that the contexts lie in a space ${\cal X}$ endowed with the Euclidean norm and that the expected base arm outcomes (expected rewards) are Lipschitz continuous in the contexts (expected base arm outcomes), we propose an algorithm called Adaptive Contextual Combinatorial Upper Confidence Bound (ACC-UCB). This algorithm, which adaptively discretizes ${\cal X}$ to form estimates of base arm outcomes and uses an $\alpha$-approximation oracle as a subroutine to select a super arm in each round, achieves $\tilde{O} ( T^{(\bar{D}+1)/(\bar{D}+2) + \epsilon} )$ regret for any $\epsilon>0$, where $\bar{D}$ represents the approximate optimality dimension related to ${\cal X}$. This dimension captures both the benignness of the base arm arrivals and the structure of the expected reward. In addition, we provide a recipe for obtaining more optimistic regret bounds by taking into account the volatility of the base arms and show that ACC-UCB achieves significant performance gains compared to the state-of-the-art for worker selection in mobile crowdsourcing.
Fourier instantaneous estimators and the Epps effect
Authors: Patrick Chang
Date: 2020-07-06 00:00:00

We compare the Malliavin-Mancino and Cuchiero-Teichmann Fourier instantaneous estimators to investigate the impact of the Epps effect arising from asynchrony in the instantaneous estimates. We demonstrate the instantaneous Epps effect under a simulation setting and provide a simple method to ameliorate the effect. We find that using the previous tick interpolation in the Cuchiero-Teichmann estimator results in unstable estimates when dealing with asynchrony, while the ability to bypass the time domain with the Malliavin-Mancino estimator allows it to produce stable estimates and is therefore better suited for ultra-high frequency finance. An empirical analysis using Trade and Quote data from the Johannesburg Stock Exchange illustrates the instantaneous Epps effect and how the intraday correlation dynamics can vary between days for the same equity pair.
Stochastic Model Predictive Control with a Safety Guarantee for Automated Driving: Extended Version
Authors: Tim Brüdigam, Michael Olbrich, Dirk Wollherr, Marion Leibold
Date: 2020-09-20 00:00:00
Tasks: Model Predictive Control, Motion Planning, Trajectory Planning

Automated vehicles require efficient and safe planning to maneuver in uncertain environments. Largely this uncertainty is caused by other traffic participants, e.g., surrounding vehicles. Future motion of surrounding vehicles is often difficult to predict. Whereas robust control approaches achieve safe, yet conservative motion planning for automated vehicles, Stochastic Model Predictive Control (SMPC) provides efficient planning in the presence of uncertainty. Probabilistic constraints are applied to ensure that the maximal risk remains below a predefined level. However, safety cannot be ensured as probabilistic constraints may be violated, which is not acceptable for automated vehicles. Here, we propose an efficient trajectory planning framework with safety guarantees for automated vehicles. SMPC is applied to obtain efficient vehicle trajectories for a finite horizon. Based on the first optimized SMPC input, a guaranteed safe backup trajectory is planned using reachable sets. This backup is used to overwrite the SMPC input if necessary for safety. Recursive feasibility of the safe SMPC algorithm is proved. Highway simulations show the effectiveness of the proposed method regarding performance and safety.
Query-Key Normalization for Transformers
Authors: Alex Henry, Prudhvi Raj Dachapally, Shubham Pawar, Yuxuan Chen
Date: 2020-10-08 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Translation

Low-resource language translation is a challenging but socially valuable NLP task. Building on recent work adapting the Transformer's normalization to this setting, we propose QKNorm, a normalization technique that modifies the attention mechanism to make the softmax function less prone to arbitrary saturation without sacrificing expressivity. Specifically, we apply $\ell_2$ normalization along the head dimension of each query and key matrix prior to multiplying them and then scale up by a learnable parameter instead of dividing by the square root of the embedding dimension. We show improvements averaging 0.928 BLEU over state-of-the-art bilingual benchmarks for 5 low-resource translation pairs from the TED Talks corpus and IWSLT'15.
Weaponizing Unicodes with Deep Learning -- Identifying Homoglyphs with Weakly Labeled Data
Authors: Perry Deng, Cooper Linsky, Matthew Wright
Date: 2020-10-09 00:00:00
Tasks: Clustering, Transfer Learning

Visually similar characters, or homoglyphs, can be used to perform social engineering attacks or to evade spam and plagiarism detectors. It is thus important to understand the capabilities of an attacker to identify homoglyphs -- particularly ones that have not been previously spotted -- and leverage them in attacks. We investigate a deep-learning model using embedding learning, transfer learning, and augmentation to determine the visual similarity of characters and thereby identify potential homoglyphs. Our approach uniquely takes advantage of weak labels that arise from the fact that most characters are not homoglyphs. Our model drastically outperforms the Normalized Compression Distance approach on pairwise homoglyph identification, for which we achieve an average precision of 0.97. We also present the first attempt at clustering homoglyphs into sets of equivalence classes, which is more efficient than pairwise information for security practitioners to quickly lookup homoglyphs or to normalize confusable string encodings. To measure clustering performance, we propose a metric (mBIOU) building on the classic Intersection-Over-Union (IOU) metric. Our clustering method achieves 0.592 mBIOU, compared to 0.430 for the naive baseline. We also use our model to predict over 8,000 previously unknown homoglyphs, and find good early indications that many of these may be true positives. Source code and list of predicted homoglyphs are uploaded to Github: https://github.com/PerryXDeng/weaponizing_unicode
A stacking model using URL and HTML features for phishing webpage detection
Authors: Yukun Li a, Zhenguo Yang b, C, ∗, Xu Chen a, Huaping Yuan b, Wenyin Liu b, ∗∗
Date: 2019-05-01 00:00:00

In this paper, we present a stacking model to detect phishing webpages using URL and HTML features. In
terms of features, we design lightweight URL and HTML features and introduce HTML string embedding
without using the third-party services, making it possible to develop real-time detection applications.
Furthermore, we devise a stacking model by combining GBDT, XGBoost and LightGBM in multiple layers,
which enables different models to be complementary, thus improving the performance on phishing
webpage detection. In particular, we collect two real-world datasets for evaluations, named as 50KPD and 50K-IPD, respectively. 50K-PD contains 49,947 webpages with URLs and HTML codes. 50K-IPD
contains 53,103 webpages with screenshots in addition to URLs and HTML codes. The proposed approach
outperforms quite a few machine learning models on multiple metrics, achieving 97.30% on accuracy,
4.46% on missing alarm rate, and 1.61% on false alarm rate on 50K-PD dataset. On 50K-IPD dataset, the
proposed approach achieves 98.60% on accuracy, 1.28% on missing alarm rate, and 1.54% on false alarm
rate.
GRF: Learning a General Radiance Field for 3D Representation and Rendering
Authors: Alex Trevithick, Bo Yang
Date: 2020-10-09 00:00:00
Proceeding: ICCV 2021 10
Tasks: 3D Scene Reconstruction, Position

We present a simple yet powerful neural network that implicitly represents and renders 3D objects and scenes only from 2D observations. The network models 3D geometries as a general radiance field, which takes a set of 2D images with camera poses and intrinsics as input, constructs an internal representation for each point of the 3D space, and then renders the corresponding appearance and geometry of that point viewed from an arbitrary position. The key to our approach is to learn local features for each pixel in 2D images and to then project these features to 3D points, thus yielding general and rich point representations. We additionally integrate an attention mechanism to aggregate pixel features from multiple 2D views, such that visual occlusions are implicitly taken into account. Extensive experiments demonstrate that our method can generate high-quality and realistic novel views for novel objects, unseen categories and challenging real-world scenes.
NutCracker at WNUT-2020 Task 2: Robustly Identifying Informative COVID-19 Tweets using Ensembling and Adversarial Training
Authors: Priyanshu Kumar, Aadarsh Singh
Date: 2020-10-09 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Task 2, Text Classification

We experiment with COVID-Twitter-BERT and RoBERTa models to identify informative COVID-19 tweets. We further experiment with adversarial training to make our models robust. The ensemble of COVID-Twitter-BERT and RoBERTa obtains a F1-score of 0.9096 (on the positive class) on the test data of WNUT-2020 Task 2 and ranks 1st on the leaderboard. The ensemble of the models trained using adversarial training also produces similar result.
Self-Paced Learning for Neural Machine Translation
Authors: Yu Wan, Baosong Yang, Derek F. Wong, Yikai Zhou, Lidia S. Chao, Haibo Zhang, Boxing Chen
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Machine Translation, NMT, Sentence, Translation

Recent studies have proven that the training of neural machine translation (NMT) can be facilitated by mimicking the learning process of humans. Nevertheless, achievements of such kind of curriculum learning rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. We ameliorate this procedure with a more flexible manner by proposing self-paced learning, where NMT model is allowed to 1) automatically quantify the learning confidence over training examples; and 2) flexibly govern its learning via regulating the loss in each iteration step. Experimental results over multiple translation tasks demonstrate that the proposed model yields better performance than strong baselines and those models trained with human-designed curricula on both translation quality and convergence speed.
Generalized Autoregressive Score asymmetric Laplace Distribution and Extreme Downward Risk Prediction
Authors: Hong Shaopeng
Date: 2020-08-04 00:00:00

Due to the skessed distribution, high peak and thick tail and asymmetry of financial return data, it is difficult to describe the traditional distribution. In recent years, generalized autoregressive score (GAS) has been used in many fields and achieved good results. In this paper, under the framework of generalized autoregressive score (GAS), the asymmetric Laplace distribution (ALD) is improved, and the GAS-ALD model is proposed, which has the characteristics of time-varying parameters, can describe the peak thick tail, biased and asymmetric distribution. The model is used to study the Shanghai index, Shenzhen index and SME board index. It is found that: 1) the distribution parameters and moments of the three indexes have obvious time-varying characteristics and aggregation characteristics. 2) Compared with the commonly used models for calculating VaR and ES, the GAS-ALD model has a high prediction effect.
Unsupervised 3D Brain Anomaly Detection
Authors: Jaime Simarro, Ezequiel de la Rosa, Thijs Vande Vyvere, David Robben, Diana M. Sima
Date: 2020-10-09 00:00:00
Tasks: Anomaly Detection

Anomaly detection (AD) is the identification of data samples that do not fit a learned data distribution. As such, AD systems can help physicians to determine the presence, severity, and extension of a pathology. Deep generative models, such as Generative Adversarial Networks (GANs), can be exploited to capture anatomical variability. Consequently, any outlier (i.e., sample falling outside of the learned distribution) can be detected as an abnormality in an unsupervised fashion. By using this method, we can not only detect expected or known lesions, but we can even unveil previously unrecognized biomarkers. To the best of our knowledge, this study exemplifies the first AD approach that can efficiently handle volumetric data and detect 3D brain anomalies in one single model. Our proposal is a volumetric and high-detail extension of the 2D f-AnoGAN model obtained by combining a state-of-the-art 3D GAN with refinement training steps. In experiments using non-contrast computed tomography images from traumatic brain injury (TBI) patients, the model detects and localizes TBI abnormalities with an area under the ROC curve of ~75%. Moreover, we test the potential of the method for detecting other anomalies such as low quality images, preprocessing inaccuracies, artifacts, and even the presence of post-operative signs (such as a craniectomy or a brain shunt). The method has potential for rapidly labeling abnormalities in massive imaging datasets, as well as identifying new biomarkers.
Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models
Authors: Eric Zelikman, Sharon Zhou, Jeremy Irvin, Cooper Raterink, Hao Sheng, Anand Avati, Jack Kelly, Ram Rajagopal, Andrew Y. Ng, David Gagne
Date: 2020-10-09 00:00:00
Tasks: Solar Irradiance Forecasting

Advancing probabilistic solar forecasting methods is essential to supporting the integration of solar energy into the electricity grid. In this work, we develop a variety of state-of-the-art probabilistic models for forecasting solar irradiance. We investigate the use of post-hoc calibration techniques for ensuring well-calibrated probabilistic predictions. We train and evaluate the models using public data from seven stations in the SURFRAD network, and demonstrate that the best model, NGBoost, achieves higher performance at an intra-hourly resolution than the best benchmark solar irradiance forecasting model across all stations. Further, we show that NGBoost with CRUDE post-hoc calibration achieves comparable performance to a numerical weather prediction model on hourly-resolution forecasting.
PathoNet: Deep learning assisted evaluation of Ki-67 and tumor infiltrating lymphocytes (TILs) as prognostic factors in breast cancer; A large dataset and baseline
Authors: Farzin Negahbani, Rasool Sabzi, Bita Pakniyat Jahromi, Fateme Movahedi, Mahsa Kohandel Shirazi, Shayan Majidi, Dena Firouzabadi, Amirreza Dehghanian
Date: 2020-10-09 00:00:00
Tasks: Cell Detection

The nuclear protein Ki-67 and Tumor infiltrating lymphocytes (TILs) have been introduced as prognostic factors in predicting tumor progression and its treatment response. The value of the Ki-67 index and TILs in approach to heterogeneous tumors such as Breast cancer (BC), known as the most common cancer in women worldwide, has been highlighted in the literature. Due to the indeterminable and subjective nature of Ki-67 as well as TILs scoring, automated methods using machine learning, specifically approaches based on deep learning, have attracted attention. Yet, deep learning methods need considerable annotated data. In the absence of publicly available benchmarks for BC Ki-67 stained cell detection and further annotated classification of cells, we propose SHIDC-BC-Ki-67 as a dataset for the aforementioned purpose. We also introduce a novel pipeline and a backend, namely PathoNet for Ki-67 immunostained cell detection and classification and simultaneous determination of intratumoral TILs score. Further, we show that despite facing challenges, our proposed backend, PathoNet, outperforms the state of the art methods proposed to date in the harmonic mean measure.
Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty
Authors: Katherine A. Keith, Christoph Teichmann, Brendan O'Connor, Edgar Meij
Date: 2020-10-09 00:00:00
Proceeding: EMNLP (NLP+CSS) 2020 11

Methods and applications are inextricably linked in science, and in particular in the domain of text-as-data. In this paper, we examine one such text-as-data application, an established economic index that measures economic policy uncertainty from keyword occurrences in news. This index, which is shown to correlate with firm investment, employment, and excess market returns, has had substantive impact in both the private sector and academia. Yet, as we revisit and extend the original authors' annotations and text measurements we find interesting text-as-data methodological research questions: (1) Are annotator disagreements a reflection of ambiguity in language? (2) Do alternative text measurements correlate with one another and with measures of external predictive validity? We find for this application (1) some annotator disagreements of economic policy uncertainty can be attributed to ambiguity in language, and (2) switching measurements from keyword-matching to supervised machine learning classifiers results in low correlation, a concerning implication for the validity of the index.
Algorithmic Frameworks for the Detection of High Density Anomalies
Authors: Ralph Foorthuis
Date: 2020-10-09 00:00:00
Tasks: Anomaly Detection, Vocal Bursts Intensity Prediction

This study explores the concept of high-density anomalies. As opposed to the traditional concept of anomalies as isolated occurrences, high-density anomalies are deviant cases positioned in the most normal regions of the data space. Such anomalies are relevant for various practical use cases, such as misbehavior detection and data quality analysis. Effective methods for identifying them are particularly important when analyzing very large or noisy sets, for which traditional anomaly detection algorithms will return many false positives. In order to be able to identify high-density anomalies, this study introduces several non-parametric algorithmic frameworks for unsupervised detection. These frameworks are able to leverage existing underlying anomaly detection algorithms and offer different solutions for the balancing problem inherent in this detection task. The frameworks are evaluated with both synthetic and real-world datasets, and are compared with existing baseline algorithms for detecting traditional anomalies. The Iterative Partial Push (IPP) framework proves to yield the best detection results.
Recursive Top-Down Production for Sentence Generation with Latent Trees
Authors: Shawn Tan, Yikang Shen, Timothy J. O'Donnell, Alessandro Sordoni, Aaron Courville
Date: 2020-10-09 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Sentence, Translation

We model the recursive production property of context-free grammars for natural and synthetic languages. To this end, we present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves, allowing us to compute the likelihood of a sequence of $N$ tokens under a latent tree model, which we maximise to train a recursive neural function. We demonstrate performance on two synthetic tasks: SCAN (Lake and Baroni, 2017), where it outperforms previous models on the LENGTH split, and English question formation (McCoy et al., 2020), where it performs comparably to decoders with the ground-truth tree structure. We also present experimental results on German-English translation on the Multi30k dataset (Elliott et al., 2016), and qualitatively analyse the induced tree structures our model learns for the SCAN tasks and the German-English translation task.