pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion
Authors: Anonymous
Date: 2020-05-27 00:00:00
Tasks: Language Modeling, Language Modelling, Relation, Relation Extraction

We show that state-of-the-art self-supervised language models can be readily used to extract relations from a corpus without the need to  train a fine-tuned extractive head. We introduce RE-Flex, a simple framework that performs constrained cloze completion over pre-trained  language  models  to  perform  unsupervised  relation  extraction. RE-Flex uses contextual matching to ensure that language model predictions matches supporting evidence from the input corpus that is relevant to a target relation. We perform an extensive experimental study over multiple relation extraction benchmarks and demonstrate that RE-Flex outperforms competing unsupervised  relation  extraction  methods based on pretrained language models by up to 27.8 F1 points  compared  to  the  next best  method. Our results show that  constrained inference queries against a language model can enable accurate unsupervised relation extraction.
A Simple Imitation Learning Method via Contrastive Regularization
Authors: Anonymous
Date: 2020-05-16 00:00:00
Tasks: Atari Games, Imitation Learning, Reinforcement Learning (RL), Representation Learning

Learning to imitate expert behavior from demonstrations is a challenging problem, especially in environments with high-dimensional, continuous observations and unknown dynamics. The simplest methods are behavioral cloning (BC), but they suffer from the problem of distribution shift: it can shift away from demonstrated states due to accumulated errors, since the agent greedily imitates demonstrated actions. Recent methods using reinforcement learning (RL), such as generative adversarial imitation learning (GAIL) and its variants, overcome this issue by training an RL agent to match the demonstrations over a long horizon. However, they all require a brittle adversarial training process with unstable rewards. And in order to augment RL process, some other papers build a specific generative model for the expert demonstrations, which increase the model and implementation complexity significantly. In this paper, we propose to train the policy as a classifier over states in expert dataset, and attenuate distribution shift by RL with fixed rewards. Here we calculate fixed rewards, based on an energy-based model (EBM) hidden in the policy. Moreover, we train this EBM by contrastive divergence method, further regularized by contrastive representation learning. Different from adversarial learning-based methods, we use fixed rewards obtained in a simple manner. There are no extra models needed here for distribution estimation or rewards modeling, reducing the model and implementation complexity significantly. The experiments on various Atari games show its performance improvement over many previous methods.
HistNet: Histopathology Segmentation Using Context Aggregation
Authors: Anonymous
Date: 2020-05-14 00:00:00
Tasks: Decoder, Segmentation, whole slide images

 Digital histopathology has seen a significant improvement towards aiding pathologists diagnosis by the use of deep learning segmentation models. The segmentation task allows for segregating benign and malignant cells from digitized tissue whole slide images (WSI). However, for reliable deployment, it is essential to produce accurate object boundaries. Pathologists performs this challenging task using context to extract complex features from a WSI. We leverage this idea and propose a model that aggregates context through use of parallel dilated and standard convolutions, embedded within each encoder-decoder block of the segmentation network. We show that, the proposed architecture not only beats the baseline by a large margin but also achieves an accuracy improvement over state-of-the-art segmentation architecture by upto 1.95% DICE Score.
Collecting Entailment Data for Pretraining: New Protocols and Negative Results
Authors: Anonymous
Date: 2019-12-10 00:00:00
Tasks: Diversity, Natural Language Inference

Textual entailment (or NLI) data has proven useful as pretraining data for tasks requiring language understanding, even when building on an already-pretrained model like RoBERTa. The standard protocol for collecting NLI was not designed for the creation of pretraining data, and it is likely far from ideal for this purpose. With this application in mind, we propose four alternative protocols, each aimed at improving either the ease with which annotators can produce sound training examples or the quality and diversity of those examples. Using these alternatives and a simple MNLI-based baseline, we collect and compare five new 8.5k-example training sets. Our primary results are solidly negative, with our baseline MNLI-style dataset yielding good transfer performance, but none of our four new methods (nor the recent ANLI) showing any improvements on that baseline. However, we do observe that all four of these interventions, especially the use of seed sentences for inspiration, reduce previously observed issues with annotation artifacts.
VPDroid System Design
Authors: Anonymous
Date: 2020-04-12 00:00:00

Detailed VPDroid system design includes: VPDroid’s updates to Cells’s virtualization; VPDroid’s new additions to Cells’s
virtualization; virtual phone customization; VPDroid implementation.
List of top-venue papers related to binary diffing in the last decade
Authors: Anonymous
Date: 2020-04-05 00:00:00

List of top-venue papers related to binary diffing in the last decade.
Device-agnostic Firmware Execution is Possible
Authors: Anonymous
Date: 2019-09-23 00:00:00

With the rapid proliferation of IoT devices, our cyberspace is nowadays dominated by billions of low-cost computing nodes, which expose an unprecedented heterogeneity to our computing systems. Dynamic analysis, one of the most effective approaches to finding software bugs, has become paralyzed due to the lack of a generic emulator capable of running diverse previously-unseen firmware. In recent years, we have witnessed devastating security breaches targeting IoT devices. These security concerns have significantly hamstrung further evolution of IoT technology. In this work, we present Laelaps, a device emulator specifically designed to run diverse software on low-cost IoT devices. We do not encode into our emulator any specific information about a device. Instead, Laelaps infers the expected behavior of firmware via symbolic-execution-assisted peripheral emulation and generates proper inputs to steer concrete execution on the fly. This unique design feature makes Laelaps the first generic device emulator capable of running diverse firmware with no a priori knowledge about the target device. To demonstrate the capabilities of Laelaps, we deployed two popular dynamic analysis techniques---fuzzing testing and dynamic symbolic execution---on top of our emulator. We successfully identified both self-injected and real-world vulnerabilities.
Wasserstein is all you need
Authors: Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, Martin Jaggi
Date: 2018-06-05 00:00:00
Tasks: All, Sentence

We propose a unified framework for building unsupervised representations of individual objects or entities (and their compositions), by associating with each object both a distributional as well as a point estimate (vector embedding). This is made possible by the use of optimal transport, which allows us to build these associated estimates while harnessing the underlying geometry of the ground space. Our method gives a novel perspective for building rich and powerful feature representations that simultaneously capture uncertainty (via a distributional estimate) and interpretability (with the optimal transport map). As a guiding example, we formulate unsupervised representations for text, in particular for sentence representation and entailment detection. Empirical results show strong advantages gained through the proposed framework. This approach can be used for any unsupervised or supervised problem (on text or other modalities) with a co-occurrence structure, such as any sequence data. The key tools underlying the framework are Wasserstein distances and Wasserstein barycenters (and, hence the title!).

Please refer to https://arxiv.org/abs/1808.09663 for the latest version.
NA
Authors: Anonymous
Date: 2019-11-09 00:00:00

NA
Machine Learning from Archives Gauquelin
Authors: Anonymous
Date: 2020-01-14 00:00:00
Tasks: BIG-bench Machine Learning

We apply machine learning methods to the data from Archives Gauquelin in an attempt to build a binary classifier able to distinguish between outstanding scientists and sports champions using only astronomical factors derived from their natal data. We apply a special splitting into training, validation and testing sets, and a set of three combined features, each of which combines dozens of elementary astronomical features. Our null hypothesis is that accuracy on Testing sets must be 0.5 if the Training sets contain the same number of group A and group B representatives born on each year, that is, if yearly frequencies are equal. Our external Testing sets contain only persons born later than those in Archives Gauquelin. Logistic Regression is our primary method, and Random Forest an alternative. All data and implementations of our algorithms are available from a public repository on GitLab.com.
Language Modeling Teaches You More than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Authors: Kelly W. Zhang, Samuel R. Bowman
Date: 2018-05-24 00:00:00
Tasks: Language Modeling, Language Modelling, Transfer Learning, Translation

There is mounting evidence that pretraining can be valuable for neural network language understanding models, but we do not yet have a clear understanding of how the choice of pretraining objective affects the type of linguistic information that models learn. With this in mind, we compare four objectives---language modeling, translation, skip-thought, and autoencoding---on their ability to induce syntactic and part-of-speech information, holding constant the genre and quantity of training data. We find that representations from language models consistently perform best on our syntactic auxiliary prediction tasks, even when trained on relatively small amounts of data, which suggests that language modeling may be the best data-rich pretraining task for transfer learning applications requiring syntactic information. We also find that a randomly-initialized, frozen model can perform strikingly well on our auxiliary tasks, but that this effect disappears when the amount of training data for the auxiliary tasks is reduced.
Holographic volume and closed surface integal
Authors: Anonymous
Date: 2019-12-06 00:00:00

Holographic volume and closed surface integal
holographic vacuum closed function
Authors: Anonymous
Date: 2019-12-06 00:00:00

holographic vacuum closed function
Surface focusing, holographic geometry and surface holographic integral
Authors: Anonymous
Date: 2019-12-06 00:00:00

Surface focusing, holographic geometry and surface holographic integral
Holographic geometry and equation the Schwinger-Tomonaga
Authors: Anonymous
Date: 2019-12-06 00:00:00

Holographic geometry and equation the Schwinger-Tomonaga
Gravity as the curvature of the wave function of the universe.
Authors: Anonymous
Date: 2019-12-06 00:00:00

Gravity as the curvature of the wave function of the universe.
The gravitational condensate of the higgs field
Authors: Anonymous
Date: 2019-12-06 00:00:00

The gravitational condensate of the higgs field
A Unified Framework for Lifelong Learning in Deep Neural Networks
Authors: Anonymous
Date: 2019-11-28 00:00:00
Tasks: Few-Shot Learning, Lifelong learning

Humans can learn a variety of concepts and skills incrementally over the course of their lives while exhibiting an array of desirable properties, such as non-forgetting, concept rehearsal, forward transfer and backward transfer of knowledge, few-shot learning, and selective forgetting. Previous approaches to lifelong machine learning can only demonstrate subsets of these properties, often by combining multiple complex mechanisms.  In this Perspective, we propose a powerful unified framework that can demonstrate all of the properties by utilizing a small number of weight consolidation parameters in deep neural networks. In addition, we are able to draw many parallels between the behaviours and mechanisms of our proposed framework and those surrounding human learning, such as memory loss or sleep deprivation. This Perspective serves as a conduit for two-way inspiration to further understand lifelong learning in machines and humans.
Using Noisy Self-Reports to Predict Twitter User Demographics
Authors: Anonymous
Date: 2019-11-18 00:00:00

Computational social science studies often contextualize content analysis within standard demographics.  Since demographic attributes are unavailable on many social media platforms, such as Twitter, numerous studies have inferred demographic traits automatically.  Despite many studies presenting proof of concept inference of race and ethnicity, training of practical systems remains elusive since there are few annotated datasets.  Existing datasets are small, errorful, or fail to cover the four most common racial and ethnic groups in the United States.  We present a method to identify self-reports of race and ethnicity from Twitter profile descriptions.  Despite errors inherent in automated supervision, we train models sufficiently accurate to identify demographics when measured on a gold standard self-report survey.  The result is a reproducible method for creating large-scale training resources for race and ethnicity.
Finding a human-like classifier
Authors: Anonymous
Date: 2019-11-13 00:00:00
Tasks: Adversarial Robustness, Continual Learning

There were many attempts to explain the trade-off between accuracy and adversarial robustness. However,  there  was  no  clear  understanding  of  the  behaviors  of  a  robust  classifier  which  has human-like robustness.
We  argue  (1)  why  we  need  to  consider  adversarial  robustness  against  varying  magnitudes  of perturbations not only focusing on a fixed perturbation threshold, (2) why we need to use different method to generate adversarially perturbed samples that can be used to train a robust classifier and measure the robustness of classifiers and (3) why we need to prioritize adversarial accuracies with different magnitudes.
We introduce Lexicographical Genuine Robustness (LGR) of classifiers that combines the above requirements.  We also suggest a candidate oracle classifier called "Optimal Lexicographically Genuinely  Robust  Classifier  (OLGRC)"  that  prioritizes  accuracy  on  meaningful  adversarially perturbed  examples  generated  by  smaller  magnitude  perturbations.   The  training  algorithm  for estimating OLGRC requires lexicographical optimization unlike existing adversarial training methods. To apply lexicographical optimization to neural network, we utilize Gradient Episodic Memory (GEM) which was originally developed for continual learning by preventing catastrophic forgetting.
Neural Architecture Search for Natural Language Understanding
Authors: Anonymous
Date: 2019-11-11 00:00:00
Tasks: Avg, Language Modeling, Language Modelling, Natural Language Understanding, Neural Architecture Search

Neural architecture search (NAS) has made rapid progress incomputervision,wherebynewstate-of-the-artresultshave beenachievedinaseriesoftaskswithautomaticallysearched neural network (NN) architectures. In contrast, NAS has not made comparable advances in natural language understanding (NLU). Corresponding to encoder-aggregator meta architecture of typical neural networks models for NLU tasks (Gong et al. 2018), we re-deﬁne the search space, by splittingitinto twoparts:encodersearchspace,andaggregator search space. Encoder search space contains basic operations such as convolutions, RNNs, multi-head attention and its sparse variants, star-transformers. Dynamic routing is included in the aggregator search space, along with max (avg) pooling and self-attention pooling. Our search algorithm is then fulﬁlled via DARTS, a differentiable neural architecture search framework. We progressively reduce the search space every few epochs, which further reduces the search time and resource costs. Experiments on ﬁve benchmark data-sets show that, the new neural networks we generate can achieve performances comparable to the state-of-the-art models that does not involve language model pre-training.
LIPS: Lightweight Intra-Mode Privilege Separation against New Control Hijacking Attacks on RTOS Task Sandboxing
Authors: Anonymous
Date: 2019-11-11 00:00:00

Lightweight Intra-Mode Privilege Separation against New Control Hijacking Attacks on RTOS Task Sandboxing
Toward Understanding The Effect of Loss Function on The Performance of Knowledge Graph Embedding
Authors: Anonymous
Date: 2019-05-22 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Translation

Knowledge graphs (KGs) represent world's facts in structured forms.  KG completion exploits the existing facts in a KG to discover new ones. Translation-based embedding model (TransE) is a prominent formulation to do KG completion. 
Despite the efficiency of TransE in memory and time, it suffers from several limitations in encoding relation patterns such as  symmetric, reflexive etc. To resolve this problem, most of the attempts have circled around the revision of the score function of TransE i.e., proposing a more complicated score function such as Trans(A, D, G, H, R, etc) to mitigate the limitations.  In this paper, we tackle this problem from a different perspective. We show that existing theories corresponding to the limitations of TransE are inaccurate because they ignore the effect of loss function. Accordingly, we pose theoretical investigations of the main limitations of TransE in the light of loss function.  To the best of our knowledge, this has not been investigated so far comprehensively. We show that by a proper selection of the loss function for training the TransE model, the main limitations of the model are mitigated. This is explained by setting upper-bound for the scores of positive samples, showing the region of truth (i.e., the region that a triple is considered positive by the model).
Our theoretical proofs with experimental results fill the gap between the capability of translation-based class of embedding models and the loss function. The theories emphasis the importance of the selection of the loss functions for training the models.  Our experimental evaluations on different loss functions used for training the models justify our theoretical proofs and confirm the importance of the loss functions on the performance.
The blood diamond effect in neural art: On ethically troublesome images of the imagenet dataset
Authors: Anonymous
Date: 2019-09-26 00:00:00

The main goal of this short paper is to inform the neural art community at large on the ethical ramifications of using models trained on the imagenet dataset, or using seed images from classes 445 -n02892767- [’bikini, two-piece’] and 459- n02837789- [’brassiere, bra, bandeau’] of the same. We discovered that many of the images belong to these classes were verifiably pornographic, shot in a non-consensual setting, voyeuristic and also entailed underage nudity. Akin to the \textit{ivory carving-illegal poaching} and \textit{diamond jewelry art-blood diamond} nexuses, we posit there is a similar moral conundrum at play here and would like to instigate a conversation amongst the neural artists in the community.
Efficient Inference and Exploration for Reinforcement Learning
Authors: Anonymous
Date: 2019-09-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference. In this paper, we investigate the large-sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances. This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors. This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates. Numerical experiments show superior performances of our exploration strategy than other benchmark approaches.
Improved Text-Image Matching by Mitigating Visual Semantic Hubs
Authors: Anonymous
Date: 2019-05-22 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval

The hubness problem widely exists in high-dimensional embedding space and is a fundamental source of error for cross-modal matching tasks. In this work, we study the emergence of hubs in Visual Semantic Embeddings (VSE) with application to text-image matching. We introduce novel methods that mitigate hubs during both training and inference. For training, we analyze the pros and cons of two widely adopted optimization objectives and propose a novel hubness-aware loss function. The loss is self-adaptive in the sense that it utilizes local statistics to scale up the weights of ``hubs'' within a mini-batch. For inference, we propose a heuristic algorithm that imposes hard constraints on the existence of hubs in the predicted graph. It can be combined with previously proposed cross-modal retrieval criterion which together achieve even better performance. We experiment our methods with various configurations of model architectures and datasets. Both the loss function and the heuristic algorithm exhibit surprisingly good robustness and bring consistent improvement on the task of text-image matching across all settings. Specifically, we report results on Flickr30k and MS-COCO datasets that are above the state-of-the-art.
LogicENN: A Neural Based Knowledge Graphs Embedding Model with Logical Rules
Authors: Anonymous
Date: 2019-05-21 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Negation

Knowledge graph embedding models have gained significant attention in AI and NLP research. Recent works have shown that the inclusion of background knowledge, such as logical rules, can improve the performance of embeddings in downstream machine learning tasks. However, so far, most existing models do not allow the inclusion of rules. We address the challenge of including rules and present a new neural based embedding model (LogicENN). We prove that LogicENN can learn every ground truth of encoded rules in a knowledge graph. To the best of our knowledge, this has not been proved so far for the neural based family of embedding models. Moreover, we derive formulae for the inclusion of various rules, including (anti-)symmetric, inverse, irreflexive and transitive, implication, composition, equivalence and negation. Our formulation allows to avoid grounding for implication and equivalence relations. Our experiments show that LogicENN outperforms the state-of-the-art models in link prediction.