OpenRaiser/Intern-Atlas huggingface dataset:
VR with Older Adults: Participatory Design of a Virtual ATM Training Simulation
Venue: arXiv cs.HC 2019
Year: 2019

In this paper we report on a study conducted with a group of older adults in which they engaged in participatory design workshops to create a VR ATM training simulation. Based on observation, recordings and the developed VR application we present the results of the workshops and offer considerations and recommendations for organizing opportunities for end users, in this case older adults, to directly engage in co-creation of cutting-edge ICT solutions. These include co-designing interfaces and interaction schemes for emerging technologies like VR and AR. We discuss such aspects as user engagement and hardware and software tools suitable for participatory prototyping of VR applications. Finally, we present ideas for further research in the area of VR participatory prototyping with users of various proficiency levels, taking steps towards developing a unified framework for co-design in AR and VR.
BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding
Venue: arXiv cs.CL 2019
Year: 2019

Fine-tuning language models, such as BERT, on domain specific corpora has proven to be valuable in domains like scientific papers and biomedical text. In this paper, we show that fine-tuning BERT on legal documents similarly provides valuable improvements on NLP tasks in the legal domain. Demonstrating this outcome is significant for analyzing commercial agreements, because obtaining large legal corpora is challenging due to their confidential nature. As such, we show that having access to large legal corpora is a competitive advantage for commercial applications, and academic research on analyzing contracts.
Learning Without Loss
Venue: arXiv cs.LG 2019
Year: 2019

We explore a new approach for training neural networks where all loss functions are replaced by hard constraints. The same approach is very successful in phase retrieval, where signals are reconstructed from magnitude constraints and general characteristics (sparsity, support, etc.). Instead of taking gradient steps, the optimizer in the constraint based approach, called relaxed-reflect-reflect (RRR), derives its steps from projections to local constraints. In neural networks one such projection makes the minimal modification to the inputs $x$, the associated weights $w$, and the pre-activation value $y$ at each neuron, to satisfy the equation $x\cdot w=y$. These projections, along with a host of other local projections (constraining pre- and post-activations, etc.) can be partitioned into two sets such that all the projections in each set can be applied concurrently, across the network and across all data in the training batch. This partitioning into two sets is analogous to the situation in phase retrieval and the setting for which the general purpose RRR optimizer was designed. Owing to the novelty of the method, this paper also serves as a self-contained tutorial. Starting with a single-layer network that performs non-negative matrix factorization, and concluding with a generative model comprising an autoencoder and classifier, all applications and their implementations by projections are described in complete detail. Although the new approach has the potential to extend the scope of neural networks (e.g. by defining activation not through functions but constraint sets), most of the featured models are standard to allow comparison with stochastic gradient descent.
GDPR Anti-Patterns: How Design and Operation of Modern Cloud-scale Systems Conflict with GDPR
Venue: arXiv cs.CY 2019
Year: 2019

In recent years, our society is being plagued by unprecedented levels of privacy and security breaches. To rein in this trend, the European Union, in 2018, introduced a comprehensive legislation called the General Data Protection Regulation (GDPR). In this article, we review GDPR from a systems perspective, and identify how the design and operation of modern cloud-scale systems conflict with this regulation. We illustrate these conflicts via six GDPR anti-patterns: storing data without a clear timeline for deletion; reusing data indiscriminately; creating walled gardens and black markets; risk-agnostic data processing; hiding data breaches; making unexplainable decisions. Our findings reveal deep-rooted tussle between GDPR requirements and how cloud-scale systems that process personal data have evolved in the modern era. While it is imperative to avoid these anti-patterns, we believe that achieving compliance requires comprehensive, grounds up solutions; anything short would amount to fixing a leaky faucet in a sinking ship.
Contributions to the Formalization and Extraction of Generic Bases of Association Rules
Venue: arXiv cs.DB 2019
Year: 2019

In this thesis, a detailed study shows that closed itemsets and minimal generators play a key role for concisely representing both frequent itemsets and association rules. These itemsets structure the search space into equivalence classes such that each class gathers the itemsets appearing in the same subset aka objects or transactions of the given data. In this respect, we proposed lossless reductions of the minimal generator set thanks to a new substitution-based process. Our theoretical results are extended to the association rule framework in order to reduce as much as possible the number of retained rules without information loss. We then give a thorough formal study of the related inference mechanism allowing to derive all redundant association rules, starting from the retained ones. We also lead a thorough exploration of the disjunctive search space, where itemsets are characterized by their respective disjunctive supports, instead of the conjunctive ones. This exploration is motivated by the fact that, in some applications, such information brings richer knowledge to the end-users. To obtain a redundancy free representation of the disjunctive search space, an interesting solution consists in selecting a unique element to represent itemsets covering the same set of data. Two itemsets are equivalent if their respective items cover the same set of data. In this regard, we introduced a new operator dedicated to this task. In each induced equivalence class, minimal elements are called essential itemsets, while the largest one is called disjunctive closed itemset. The introduced operator is then at the roots of new concise representations of frequent itemsets. We also exploit the disjunctive search space to derive generalized association rules. These latter rules generalize classic ones to also offer disjunction and negation connectors between items, in addition to the conjunctive one.
Goals, Process, and Challenges of Exploratory Data Analysis: An Interview Study
Venue: arXiv cs.HC 2019
Year: 2019

How do analysis goals and context affect exploratory data analysis (EDA)? To investigate this question, we conducted semi-structured interviews with 18 data analysts. We characterize common exploration goals: profiling (assessing data quality) and discovery (gaining new insights). Though the EDA literature primarily emphasizes discovery, we observe that discovery only reliably occurs in the context of open-ended analyses, whereas all participants engage in profiling across all of their analyses. We describe the process and challenges of EDA highlighted by our interviews. We find that analysts must perform repetitive tasks (e.g., examine numerous variables), yet they may have limited time or lack domain knowledge to explore data. Analysts also often have to consult other stakeholders and oscillate between exploration and other tasks, such as acquiring and wrangling additional data. Based on these observations, we identify design opportunities for exploratory analysis tools, such as augmenting exploration with automation and guidance.
Credibility-based Fake News Detection
Venue: arXiv cs.CL 2019
Year: 2019

Fake news can significantly misinform people who often rely on online sources and social media for their information. Current research on fake news detection has mostly focused on analyzing fake news content and how it propagates on a network of users. In this paper, we emphasize the detection of fake news by assessing its credibility. By analyzing public fake news data, we show that information on news sources (and authors) can be a strong indicator of credibility. Our findings suggest that an author's history of association with fake news, and the number of authors of a news article, can play a significant role in detecting fake news. Our approach can help improve traditional fake news detection methods, wherein content features are often used to detect fake news.
Topological Feature Search in Time-Varying Multifield Data
Venue: arXiv cs.CG 2019
Year: 2019

A wide range of data that appear in scientific experiments and simulations are multivariate or multifield in nature, consisting of multiple scalar fields. Topological feature search of such data aims to reveal important properties useful to the domain scientists. It has been shown in recent works that a single scalar field is insufficient to capture many important topological features in the data, instead one needs to consider topological relationships between multiple scalar fields. In the current paper, we propose a novel method of finding similarity between two multifield data by comparing their respective fiber component distributions. Given a time-varying multifield data, the method computes a metric plot for each pair of histograms at consecutive time stamps to understand the topological changes in the data over time. We validate the method using real and synthetic data. The effectiveness of the proposed method is shown by its ability to capture important topological features that are not always possible to detect using the individual component scalar fields.
How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and fine-tuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training models based on reading comprehension and open domain question answering models and determine the performance when fine-tuned and tested over BIOASQ question answering dataset. We find open domain question answering model to be a better fit for this task rather than reading comprehension model.
Do Chinese Internet Users Exist Heterogeneity in Search Behavior?
Venue: arXiv cs.CY 2019
Year: 2019

Investor attention is an important concept in behavioral finance. Many articles have conducted cross-disciplinary research leading by this concept. In this paper, we use data extraction technology to collect a large number of Baidu Index keyword search volume data. After analyzing the data, we draw a conclusion that has not been paid attention to in all the past research. We find heterogeneity in searching by internet users in China. Firstly, in terms of search behavior, internet users are more inclined to use the PC end to obtain information when facing areas which need to be taken seriously by them. Secondly, attention is heterogeneous while searching. When Internet users search for information in mobile end, their attention is divergent, and search for seemingly unrelated keywords at the same time which limits their attention to information.
On secure communication in sensor networks under q-composite key predistribution with unreliable links
Venue: arXiv cs.IT 2019
Year: 2019

Many applications of wireless sensor networks (WSNs) require deploying sensors in hostile environments, where an adversary may eavesdrop communications. To secure communications in WSNs, the q-composite key predistribution scheme has been proposed in the literature. In this paper, we investigate secure k-connectivity in WSNs operating under the q-composite scheme, in consideration of the unreliability of wireless links. Secure k-connectivity ensures that any two sensors can find a path in between for secure communication, even when k - 1 sensors fail. We present conditions on how to set the network parameters such that the network has secure k-connectivity asymptotically almost surely. The result is given in the form of a sharp zero-one law.
Beta DVBF: Learning State-Space Models for Control from High Dimensional Observations
Venue: arXiv cs.LG 2019
Year: 2019

Learning a model of dynamics from high-dimensional images can be a core ingredient for success in many applications across different domains, especially in sequential decision making. However, currently prevailing methods based on latent-variable models are limited to working with low resolution images only. In this work, we show that some of the issues with using high-dimensional observations arise from the discrepancy between the dimensionality of the latent and observable space, and propose solutions to overcome them.
Adaptive Statistical Learning with Bayesian Differential Privacy
Venue: arXiv cs.LG 2019
Year: 2019

In statistical learning, a dataset is often partitioned into two parts: the training set and the holdout (i.e., testing) set. For instance, the training set is used to learn a predictor, and then the holdout set is used for estimating the accuracy of the predictor on the true distribution. However, often in practice, the holdout dataset is reused and the estimates tested on the holdout dataset are chosen adaptively based on the results of prior estimates, leading to that the predictor may become dependent of the holdout set. Hence, overfitting may occur, and the learned models may not generalize well to the unseen datasets. Prior studies have established connections between the stability of a learning algorithm and its ability to generalize, but the traditional generalization is not robust to adaptive composition. Recently, Dwork et al. in NIPS, STOC, and Science 2015 show that the holdout dataset from i.i.d. data samples can be reused in adaptive statistical learning, if the estimates are perturbed and coordinated using techniques developed for differential privacy, which is a widely used notion to quantify privacy. Yet, the results of Dwork et al. are applicable to only the case of i.i.d. samples. In contrast, correlations between data samples exist because of various behavioral, social, and genetic relationships between users. Our results in adaptive statistical learning generalize the results of Dwork et al. for i.i.d. data samples to arbitrarily correlated data. Specifically, we show that the holdout dataset from correlated samples can be reused in adaptive statistical learning, if the estimates are perturbed and coordinated using techniques developed for Bayesian differential privacy, which is a privacy notion recently introduced by Yang et al. in SIGMOD 2015 to broaden the application scenarios of differential privacy when data records are correlated.
Automating Ethernet VPN Deployment in SDN-based Data Centers
Venue: arXiv cs.NI 2019
Year: 2019

Layer 2 Virtual Private Network (L2VPN) is widely deployed in both service provider networks and enterprises. However, legacy L2VPN solutions have scalability limitations in the context of Data Center (DC) interconnection and networking which require new approaches that address the requirements of service providers for virtual private cloud services. Recently, Ethernet VPN (EVPN) has been proposed to address many of those concerns and vendors started to deploy EVPN based solutions in DC edge routers. However, manual configuration leads to a time-consuming, error-prone configuration and high operational costs. Automating the EVPN deployment from cloud platforms such as OpenStack enhances both the deployment and flexibility of EVPN Instances (EVIs). This paper proposes a Software Defined Network (SDN) based framework that automates the EVPN deployment and management inside SDN-based DCs using OpenStack and OpenDaylight (ODL). We implemented and extended several modules inside ODL controller to manage and interact with EVIs and an interface to OpenStack that allows the deployment and configuration of EVIs. We conclude with scalability analysis of our solution.
Adaptive Rate Allocation for View-Aware Point-Cloud Streaming
Venue: arXiv cs.MM 2019
Year: 2019

In the context of view-dependent point-cloud streaming in a scene, our rate allocation is "adaptive" in the sense that it priorities the point-cloud models depending on the camera view and the visibility of the objects and their distance as described. The algorithm delivers higher bitrate to the point-cloud models which are inside user's viewport, more likely for the user to look at, or are closer to the view camera or, while delivers lower quality level to the point-cloud models outside of a user's immediate viewport or farther away from the camera. For that purpose, we hereby explain the rate allocation problem within the context of multi-point-cloud streaming where multiple point-cloud models are aimed to be streamed to the target device, and propose a rate allocation heuristic algorithm to enable the adaptations within this context. To the best of our knowledge, this is the first work to mathematically model, and propose a rate allocation heuristic algorithm within the context of point-cloud streaming.
Computationally efficient versions of conformal predictive distributions
Venue: arXiv cs.LG 2019
Year: 2019

Conformal predictive systems are a recent modification of conformal predictors that output, in regression problems, probability distributions for labels of test observations rather than set predictions. The extra information provided by conformal predictive systems may be useful, e.g., in decision making problems. Conformal predictive systems inherit the relative computational inefficiency of conformal predictors. In this paper we discuss two computationally efficient versions of conformal predictive systems, which we call split conformal predictive systems and cross-conformal predictive systems. The main advantage of split conformal predictive systems is their guaranteed validity, whereas for cross-conformal predictive systems validity only holds empirically and in the absence of excessive randomization. The main advantage of cross-conformal predictive systems is their greater predictive efficiency.
Attributed Sequence Embedding
Venue: arXiv cs.LG 2019
Year: 2019

Mining tasks over sequential data, such as clickstreams and gene sequences, require a careful design of embeddings usable by learning algorithms. Recent research in feature learning has been extended to sequential data, where each instance consists of a sequence of heterogeneous items with a variable length. However, many real-world applications often involve attributed sequences, where each instance is composed of both a sequence of categorical items and a set of attributes. In this paper, we study this new problem of attributed sequence embedding, where the goal is to learn the representations of attributed sequences in an unsupervised fashion. This problem is core to many important data mining tasks ranging from user behavior analysis to the clustering of gene sequences. This problem is challenging due to the dependencies between sequences and their associated attributes. We propose a deep multimodal learning framework, called NAS, to produce embeddings of attributed sequences. The embeddings are task independent and can be used on various mining tasks of attributed sequences. We demonstrate the effectiveness of our embeddings of attributed sequences in various unsupervised learning tasks on real-world datasets.
Zeroth Order Non-convex optimization with Dueling-Choice Bandits
Venue: arXiv cs.LG 2019
Year: 2019

We consider a novel setting of zeroth order non-convex optimization, where in addition to querying the function value at a given point, we can also duel two points and get the point with the larger function value. We refer to this setting as optimization with dueling-choice bandits since both direct queries and duels are available for optimization. We give the COMP-GP-UCB algorithm based on GP-UCB (Srinivas et al., 2009), where instead of directly querying the point with the maximum Upper Confidence Bound (UCB), we perform a constrained optimization and use comparisons to filter out suboptimal points. COMP-GP-UCB comes with theoretical guarantee of $O(\fracΦ{\sqrt{T}})$ on simple regret where $T$ is the number of direct queries and $Φ$ is an improved information gain corresponding to a comparison based constraint set that restricts the search space for the optimum. In contrast, in the direct query only setting, $Φ$ depends on the entire domain. Finally, we present experimental results to show the efficacy of our algorithm.
Metric Learning for Dynamic Text Classification
Venue: arXiv cs.LG 2019
Year: 2019

Traditional text classifiers are limited to predicting over a fixed set of labels. However, in many real-world applications the label set is frequently changing. For example, in intent classification, new intents may be added over time while others are removed. We propose to address the problem of dynamic text classification by replacing the traditional, fixed-size output layer with a learned, semantically meaningful metric space. Here the distances between textual inputs are optimized to perform nearest-neighbor classification across overlapping label sets. Changing the label set does not involve removing parameters, but rather simply adding or removing support points in the metric space. Then the learned metric can be fine-tuned with only a few additional training examples. We demonstrate that this simple strategy is robust to changes in the label space. Furthermore, our results show that learning a non-Euclidean metric can improve performance in the low data regime, suggesting that further work on metric spaces may benefit low-resource research.
Human eye inspired log-polar pre-processing for neural networks
Venue: arXiv cs.NE 2019
Year: 2019

In this paper we draw inspiration from the human visual system, and present a bio-inspired pre-processing stage for neural networks. We implement this by applying a log-polar transformation as a pre-processing step, and to demonstrate, we have used a naive convolutional neural network (CNN). We demonstrate that a bio-inspired pre-processing stage can achieve rotation and scale robustness in CNNs. A key point in this paper is that the CNN does not need to be trained to identify rotation or scaling permutations; rather it is the log-polar pre-processing step that converts the image into a format that allows the CNN to handle rotation and scaling permutations. In addition we demonstrate how adding a log-polar transformation as a pre-processing step can reduce the image size to ~20\% of the Euclidean image size, without significantly compromising classification accuracy of the CNN. The pre-processing stage presented in this paper is modelled after the retina and therefore is only tested against an image dataset. Note: This paper has been submitted for SAUPEC/RobMech/PRASA 2020.