pwc-archive/papers-with-abstracts huggingface dataset:
An Empirical Study of the Application of Machine Learning and Keyword Terms Methodologies to Privilege-Document Review Projects in Legal Matters
Authors: Peter Gronvall, Nathaniel Huber-Fliflet, Jianping Zhang, Robert Keeling, Robert Neary, Haozhen Zhao
Date: 2019-04-03 00:00:00

Protecting privileged communications and data from disclosure is paramount
for legal teams. Unrestricted legal advice, such as attorney-client
communications or litigation strategy. are vital to the legal process and are
exempt from disclosure in litigations or regulatory events. To protect this
information from being disclosed, companies and outside counsel must review
vast amounts of documents to determine those that contain privileged material.
This process is extremely costly and time consuming. As data volumes increase,
legal counsel employ methods to reduce the number of documents requiring review
while balancing the need to ensure the protection of privileged information.
Keyword searching is relied upon as a method to target privileged information
and reduce document review populations. Keyword searches are effective at
casting a wide net but return over inclusive results -- most of which do not
contain privileged information -- and without detailed knowledge of the data,
keyword lists cannot be crafted to find all privilege material.
Overly-inclusive keyword searching can also be problematic, because even while
it drives up costs, it also can cast `too far of a net' and thus produce
unreliable results.To overcome these weaknesses of keyword searching, legal
teams are using a new method to target privileged information called predictive
modeling. Predictive modeling can successfully identify privileged material but
little research has been published to confirm its effectiveness when compared
to keyword searching. This paper summarizes a study of the effectiveness of
keyword searching and predictive modeling when applied to real-world data. With
this study, this group of collaborators wanted to examine and understand the
benefits and weaknesses of both approaches to legal teams with identifying
privilege material in document populations.
Explainable Text Classification in Legal Document Review A Case Study of Explainable Predictive Coding
Authors: Rishi Chhatwal, Peter Gronvall, Nathaniel Huber-Fliflet, Robert Keeling, Jianping Zhang, Haozhen Zhao
Date: 2019-04-03 00:00:00
Tasks: Document Classification, General Classification, text-classification, Text Classification

In today's legal environment, lawsuits and regulatory investigations require
companies to embark upon increasingly intensive data-focused engagements to
identify, collect and analyze large quantities of data. When documents are
staged for review the process can require companies to dedicate an
extraordinary level of resources, both with respect to human resources, but
also with respect to the use of technology-based techniques to intelligently
sift through data. For several years, attorneys have been using a variety of
tools to conduct this exercise, and most recently, they are accepting the use
of machine learning techniques like text classification to efficiently cull
massive volumes of data to identify responsive documents for use in these
matters. In recent years, a group of AI and Machine Learning researchers have
been actively researching Explainable AI. In an explainable AI system, actions
or decisions are human understandable. In typical legal `document review'
scenarios, a document can be identified as responsive, as long as one or more
of the text snippets in a document are deemed responsive. In these scenarios,
if predictive coding can be used to locate these responsive snippets, then
attorneys could easily evaluate the model's document classification decision.
When deployed with defined and explainable results, predictive coding can
drastically enhance the overall quality and speed of the document review
process by reducing the time it takes to review documents. The authors of this
paper propose the concept of explainable predictive coding and simple
explainable predictive coding methods to locate responsive snippets within
responsive documents. We also report our preliminary experimental results using
the data from an actual legal matter that entailed this type of document
review.
Neural Program Repair by Jointly Learning to Localize and Repair
Authors: Marko Vasic, Aditya Kanade, Petros Maniatis, David Bieber, Rishabh Singh
Date: 2019-04-03 00:00:00
Proceeding: ICLR 2019 5
Tasks: Program Repair, Variable misuse

Due to its potential to improve programmer productivity and software quality,
automated program repair has been an active topic of research. Newer techniques
harness neural networks to learn directly from examples of buggy programs and
their fixes. In this work, we consider a recently identified class of bugs
called variable-misuse bugs. The state-of-the-art solution for variable misuse
enumerates potential fixes for all possible bug locations in a program, before
selecting the best prediction. We show that it is beneficial to train a model
that jointly and directly localizes and repairs variable-misuse bugs. We
present multi-headed pointer networks for this purpose, with one head each for
localization and repair. The experimental results show that the joint model
significantly outperforms an enumerative solution that uses a pointer based
model for repair alone.
Empirical Evaluations of Active Learning Strategies in Legal Document Review
Authors: Rishi Chhatwal, Nathaniel Huber-Fliflet, Robert Keeling, Jianping Zhang, Haozhen Zhao
Date: 2019-04-03 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning, text-classification, Text Classification

One type of machine learning, text classification, is now regularly applied
in the legal matters involving voluminous document populations because it can
reduce the time and expense associated with the review of those documents. One
form of machine learning - Active Learning - has drawn attention from the legal
community because it offers the potential to make the machine learning process
even more effective. Active Learning, applied to legal documents, is considered
a new technology in the legal domain and is continuously applied to all
documents in a legal matter until an insignificant number of relevant documents
are left for review. This implementation is slightly different than traditional
implementations of Active Learning where the process stops once achieving
acceptable model performance. The purpose of this paper is twofold: (i) to
question whether Active Learning actually is a superior learning methodology
and (ii) to highlight the ways that Active Learning can be most effectively
applied to real legal industry data. Unlike other studies, our experiments were
performed against large data sets taken from recent, real-world legal matters
covering a variety of areas. We conclude that, although these experiments show
the Active Learning strategy popularly used in legal document review can
quickly identify informative training documents, it becomes less effective over
time. In particular, our findings suggest this most popular form of Active
Learning in the legal arena, where the highest-scoring documents are selected
as training examples, is in fact not the most efficient approach in most
instances. Ultimately, a different Active Learning strategy may be best suited
to initiate the predictive modeling process but not to continue through the
entire document review.
Empirical Evaluations of Preprocessing Parameters' Impact on Predictive Coding's Effectiveness
Authors: Rishi Chhatwal, Nathaniel Huber-Fliflet, Robert Keeling, Jianping Zhang, Haozhen Zhao
Date: 2019-04-03 00:00:00

Predictive coding, once used in only a small fraction of legal and business
matters, is now widely deployed to quickly cull through increasingly vast
amounts of data and reduce the need for costly and inefficient human document
review. Previously, the sole front-end input used to create a predictive model
was the exemplar documents (training data) chosen by subject-matter experts.
Many predictive coding tools require users to rely on static preprocessing
parameters and a single machine learning algorithm to develop the predictive
model. Little research has been published discussing the impact preprocessing
parameters and learning algorithms have on the effectiveness of the technology.
A deeper dive into the generation of a predictive model shows that the settings
and algorithm can have a strong effect on the accuracy and efficacy of a
predictive coding tool. Understanding how these input parameters affect the
output will empower legal teams with the information they need to implement
predictive coding as efficiently and effectively as possible. This paper
outlines different preprocessing parameters and algorithms as applied to
multiple real-world data sets to understand the influence of various
approaches.
Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions
Authors: Anil Yaman, Giovanni Iacca, Decebal Constantin Mocanu, Matt Coler, George Fletcher, Mykola Pechenizkiy
Date: 2019-04-02 00:00:00

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.
Improving Noise Tolerance of Mixed-Signal Neural Networks
Authors: Michael Klachko, Mohammad Reza Mahmoodi, Dmitri B. Strukov
Date: 2019-04-02 00:00:00

Mixed-signal hardware accelerators for deep learning achieve orders of
magnitude better power efficiency than their digital counterparts. In the
ultra-low power consumption regime, limited signal precision inherent to analog
computation becomes a challenge. We perform a case study of a 6-layer
convolutional neural network running on a mixed-signal accelerator and evaluate
its sensitivity to hardware specific noise. We apply various methods to improve
noise robustness of the network and demonstrate an effective way to optimize
useful signal ranges through adaptive signal clipping. The resulting model is
robust enough to achieve 80.2% classification accuracy on CIFAR-10 dataset with
just 1.4 mW power budget, while 6 mW budget allows us to achieve 87.1%
accuracy, which is within 1% of the software baseline. For comparison, the
unoptimized version of the same model achieves only 67.7% accuracy at 1.4 mW
and 78.6% at 6 mW.
3DRegNet: A Deep Neural Network for 3D Point Registration
Authors: G. Dias Pais, Srikumar Ramalingam, Venu Madhav Govindu, Jacinto C. Nascimento, Rama Chellappa, Pedro Miraldo
Date: 2019-04-02 00:00:00
Proceeding: CVPR 2020 6
Tasks: regression

We present 3DRegNet, a novel deep learning architecture for the registration of 3D scans. Given a set of 3D point correspondences, we build a deep neural network to address the following two challenges: (i) classification of the point correspondences into inliers/outliers, and (ii) regression of the motion parameters that align the scans into a common reference frame. With regard to regression, we present two alternative approaches: (i) a Deep Neural Network (DNN) registration and (ii) a Procrustes approach using SVD to estimate the transformation. Our correspondence-based approach achieves a higher speedup compared to competing baselines. We further propose the use of a refinement network, which consists of a smaller 3DRegNet as a refinement to improve the accuracy of the registration. Extensive experiments on two challenging datasets demonstrate that we outperform other methods and achieve state-of-the-art results. The code is available.
Multigrid Predictive Filter Flow for Unsupervised Learning on Videos
Authors: Shu Kong, Charless Fowlkes
Date: 2019-04-02 00:00:00
Tasks: Optical Flow Estimation, Pose Tracking, Semantic Segmentation, Skeleton Based Action Recognition, Video Semantic Segmentation

We introduce multigrid Predictive Filter Flow (mgPFF), a framework for
unsupervised learning on videos. The mgPFF takes as input a pair of frames and
outputs per-pixel filters to warp one frame to the other. Compared to optical
flow used for warping frames, mgPFF is more powerful in modeling sub-pixel
movement and dealing with corruption (e.g., motion blur). We develop a
multigrid coarse-to-fine modeling strategy that avoids the requirement of
learning large filters to capture large displacement. This allows us to train
an extremely compact model (4.6MB) which operates in a progressive way over
multiple resolutions with shared weights. We train mgPFF on unsupervised,
free-form videos and show that mgPFF is able to not only estimate long-range
flow for frame reconstruction and detect video shot transitions, but also
readily amendable for video object segmentation and pose tracking, where it
substantially outperforms the published state-of-the-art without bells and
whistles. Moreover, owing to mgPFF's nature of per-pixel filter prediction, we
have the unique opportunity to visualize how each pixel is evolving during
solving these tasks, thus gaining better interpretability.
DeLTA: GPU Performance Model for Deep Learning Applications with In-depth Memory System Traffic Analysis
Authors: Sangkug Lym, Donghyuk Lee, Mike O'Connor, Niladrish Chatterjee, Mattan Erez
Date: 2019-04-02 00:00:00
Tasks: GPU

Training convolutional neural networks (CNNs) requires intense compute
throughput and high memory bandwidth. Especially, convolution layers account
for the majority of the execution time of CNN training, and GPUs are commonly
used to accelerate these layer workloads. GPU design optimization for efficient
CNN training acceleration requires the accurate modeling of how their
performance improves when computing and memory resources are increased. We
present DeLTA, the first analytical model that accurately estimates the traffic
at each GPU memory hierarchy level, while accounting for the complex reuse
patterns of a parallel convolution algorithm. We demonstrate that our model is
both accurate and robust for different CNNs and GPU architectures. We then show
how this model can be used to carefully balance the scaling of different GPU
resources for efficient CNN performance improvement.
The Tower of Babel Meets Web 2.0: User-Generated Content and its Applications in a Multilingual Context
Authors: B. Hecht, D. Gergle
Date: 2019-04-02 00:00:00
Tasks: Diversity, World Knowledge

This study explores language's fragmenting effect on user-generated content
by examining the diversity of knowledge representations across 25 different
Wikipedia language editions. This diversity is measured at two levels: the
concepts that are included in each edition and the ways in which these concepts
are described. We demonstrate that the diversity present is greater than has
been presumed in the literature and has a significant influence on applications
that use Wikipedia as a source of world knowledge. We close by explicating how
knowledge diversity can be beneficially leveraged to create "culturally-aware
applications" and "hyperlingual applications".
Measuring Calibration in Deep Learning
Authors: Jeremy Nixon, Mike Dusenberry, Ghassen Jerfel, Timothy Nguyen, Jeremiah Liu, Linchuan Zhang, Dustin Tran
Date: 2019-04-02 00:00:00
Tasks: Deep Learning

Overconfidence and underconfidence in machine learning classifiers is measured by calibration: the degree to which the probabilities predicted for each class match the accuracy of the classifier on that prediction. How one measures calibration remains a challenge: expected calibration error, the most popular metric, has numerous flaws which we outline, and there is no clear empirical understanding of how its choices affect conclusions in practice, and what recommendations there are to counteract its flaws. In this paper, we perform a comprehensive empirical study of choices in calibration measures including measuring all probabilities rather than just the maximum prediction, thresholding probability values, class conditionality, number of bins, bins that are adaptive to the datapoint density, and the norm used to compare accuracies to confidences. To analyze the sensitivity of calibration measures, we study the impact of optimizing directly for each variant with recalibration techniques. Across MNIST, Fashion MNIST, CIFAR-10/100, and ImageNet, we find that conclusions on the rank ordering of recalibration methods is drastically impacted by the choice of calibration measure. We find that conditioning on the class leads to more effective calibration evaluations, and that using the L2 norm rather than the L1 norm improves both optimization for calibration metrics and the rank correlation measuring metric consistency. Adaptive binning schemes lead to more stablity of metric rank ordering when the number of bins vary, and is also recommended. We open source a library for the use of our calibration measures.
Impact of ASR on Alzheimer's Disease Detection: All Errors are Equal, but Deletions are More Equal than Others
Authors: Aparna Balagopalan, Ksenia Shkaruta, Jekaterina Novikova
Date: 2019-04-02 00:00:00
Tasks: All, Alzheimer's Disease Detection, Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Automatic Speech Recognition (ASR) is a critical component of any fully-automated speech-based dementia detection model. However, despite years of speech recognition research, little is known about the impact of ASR accuracy on dementia detection. In this paper, we experiment with controlled amounts of artificially generated ASR errors and investigate their influence on dementia detection. We find that deletion errors affect detection performance the most, due to their impact on the features of syntactic complexity and discourse representation in speech. We show the trend to be generalisable across two different datasets for cognitive impairment detection. As a conclusion, we propose optimising the ASR to reflect a higher penalty for deletion errors in order to improve dementia detection performance.
Augmented Neural ODEs
Authors: Emilien Dupont, Arnaud Doucet, Yee Whye Teh
Date: 2019-04-02 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Image Classification

We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive models, are empirically more stable, generalize better and have a lower computational cost than Neural ODEs.
Hammering Mizar by Learning Clause Guidance
Authors: Jan Jakubův, Josef Urban
Date: 2019-04-02 00:00:00
Tasks: Automated Theorem Proving

We describe a very large improvement of existing hammer-style proof
automation over large ITP libraries by combining learning and theorem proving.
In particular, we have integrated state-of-the-art machine learners into the E
automated theorem prover, and developed methods that allow learning and
efficient internal guidance of E over the whole Mizar library. The resulting
trained system improves the real-time performance of E on the Mizar library by
70% in a single-strategy setting.
Lautum Regularization for Semi-supervised Transfer Learning
Authors: Daniel Jakubovitz, Miguel R. D. Rodrigues, Raja Giryes
Date: 2019-04-02 00:00:00
Tasks: Transfer Learning

Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are commonplace and are known to have a substantial impact on algorithm performance. In this work we suggest a novel information theoretic approach for the analysis of the performance of deep neural networks in the context of transfer learning. We focus on the task of semi-supervised transfer learning, in which unlabeled samples from the target dataset are available during the network training on the source dataset. Our theory suggests that one may improve the transferability of a deep neural network by imposing a Lautum information based regularization that relates the network weights to the target data. We demonstrate the effectiveness of the proposed approach in various transfer learning experiments.
Mirroring to Build Trust in Digital Assistants
Authors: Katherine Metcalf, Barry-John Theobald, Garrett Weinberg, Robert Lee, Ing-Marie Jonsson, Russ Webb, Nicholas Apostoloff
Date: 2019-04-02 00:00:00

We describe experiments towards building a conversational digital assistant
that considers the preferred conversational style of the user. In particular,
these experiments are designed to measure whether users prefer and trust an
assistant whose conversational style matches their own. To this end we
conducted a user study where subjects interacted with a digital assistant that
responded in a way that either matched their conversational style, or did not.
Using self-reported personality attributes and subjects' feedback on the
interactions, we built models that can reliably predict a user's preferred
conversational style.