OpenRaiser/Intern-Atlas huggingface dataset:
Blocking Transferability of Adversarial Examples in Black-Box Learning Systems
Venue: arXiv cs.LG 2017
Year: 2017

Advances in Machine Learning (ML) have led to its adoption as an integral component in many applications, including banking, medical diagnosis, and driverless cars. To further broaden the use of ML models, cloud-based services offered by Microsoft, Amazon, Google, and others have developed ML-as-a-service tools as black-box systems. However, ML classifiers are vulnerable to adversarial examples: inputs that are maliciously modified can cause the classifier to provide adversary-desired outputs. Moreover, it is known that adversarial examples generated on one classifier are likely to cause another classifier to make the same mistake, even if the classifiers have different architectures or are trained on disjoint datasets. This property, which is known as transferability, opens up the possibility of attacking black-box systems by generating adversarial examples on a substitute classifier and transferring the examples to the target classifier. Therefore, the key to protect black-box learning systems against the adversarial examples is to block their transferability. To this end, we propose a training method that, as the input is more perturbed, the classifier smoothly outputs lower confidence on the original label and instead predicts that the input is "invalid". In essence, we augment the output class set with a NULL label and train the classifier to reject the adversarial examples by classifying them as NULL. In experiments, we apply a wide range of attacks based on adversarial examples on the black-box systems. We show that a classifier trained with the proposed method effectively resists against the adversarial examples, while maintaining the accuracy on clean data.
Story Cloze Ending Selection Baselines and Data Examination
Venue: arXiv cs.CL 2017
Year: 2017

This paper describes two supervised baseline systems for the Story Cloze Test Shared Task (Mostafazadeh et al., 2016a). We first build a classifier using features based on word embeddings and semantic similarity computation. We further implement a neural LSTM system with different encoding strategies that try to model the relation between the story and the provided endings. Our experiments show that a model using representation features based on average word embedding vectors over the given story words and the candidate ending sentences words, joint with similarity features between the story and candidate ending representations performed better than the neural models. Our best model achieves an accuracy of 72.42, ranking 3rd in the official evaluation.
A Visual Representation of Wittgenstein's Tractatus Logico-Philosophicus
Venue: arXiv cs.IR 2017
Year: 2017

In this paper we present a data visualization method together with its potential usefulness in digital humanities and philosophy of language. We compile a multilingual parallel corpus from different versions of Wittgenstein's Tractatus Logico-Philosophicus, including the original in German and translations into English, Spanish, French, and Russian. Using this corpus, we compute a similarity measure between propositions and render a visual network of relations for different languages.
Nematus: a Toolkit for Neural Machine Translation
Venue: arXiv cs.CL 2017
Year: 2017

We present Nematus, a toolkit for Neural Machine Translation. The toolkit prioritizes high translation accuracy, usability, and extensibility. Nematus has been used to build top-performing submissions to shared translation tasks at WMT and IWSLT, and has been used to train systems for production environments.
El Lenguaje Natural como Lenguaje Formal
Venue: arXiv cs.CL 2017
Year: 2017

Formal languages theory is useful for the study of natural language. In particular, it is of interest to study the adequacy of the grammatical formalisms to express syntactic phenomena present in natural language. First, it helps to draw hypothesis about the nature and complexity of the speaker-hearer linguistic competence, a fundamental question in linguistics and other cognitive sciences. Moreover, from an engineering point of view, it allows the knowledge of practical limitations of applications based on those formalisms. In this article I introduce the adequacy problem of grammatical formalisms for natural language, also introducing some formal language theory concepts required for this discussion. Then, I review the formalisms that have been proposed in history, and the arguments that have been given to support or reject their adequacy. ----- La teoría de lenguajes formales es útil para el estudio de los lenguajes naturales. En particular, resulta de interés estudiar la adecuación de los formalismos gramaticales para expresar los fenómenos sintácticos presentes en el lenguaje natural. Primero, ayuda a trazar hipótesis acerca de la naturaleza y complejidad de las competencias lingüísticas de los hablantes-oyentes del lenguaje, un interrogante fundamental de la lingüística y otras ciencias cognitivas. Además, desde el punto de vista de la ingeniería, permite conocer limitaciones prácticas de las aplicaciones basadas en dichos formalismos. En este artículo hago una introducción al problema de la adecuación de los formalismos gramaticales para el lenguaje natural, introduciendo también algunos conceptos de la teoría de lenguajes formales necesarios para esta discusión. Luego, hago un repaso de los formalismos que han sido propuestos a lo largo de la historia, y de los argumentos que se han dado para sostener o refutar su adecuación.
Improving LBP and its variants using anisotropic diffusion
Venue: arXiv cs.CV 2017
Year: 2017

The main purpose of this paper is to propose a new preprocessing step in order to improve local feature descriptors and texture classification. Preprocessing is implemented by using transformations which help highlight salient features that play a significant role in texture recognition. We evaluate and compare four different competing methods: three different anisotropic diffusion methods including the classical anisotropic Perona-Malik diffusion and two subsequent regularizations of it and the application of a Gaussian kernel, which is the classical multiscale approach in texture analysis. The combination of the transformed images and the original ones are analyzed. The results show that the use of the preprocessing step does lead to improved texture recognition.
Modeling and Dimensioning of a Virtualized MME for 5G Mobile Networks
Venue: arXiv cs.NI 2017
Year: 2017

Network Function Virtualization is considered one of the key technologies for developing the future mobile networks. In this paper, we propose a theoretical framework to evaluate the performance of an LTE virtualized Mobility Management Entity (vMME) hosted in a data center. This theoretical framework consists of i) a queuing network to model the vMME in a data center, and ii) analytic expressions to estimate the overall mean system delay and the signaling workload to be processed by the vMME. We validate our mathematical model by simulation. One direct use of the proposed model is vMME dimensioning, i.e., to compute the number of vMME processing instances to provide a target system delay given the number of users in the system. Additionally, the paper includes a scalability analysis of the system. In our study, we consider the billing model and a data center setup of Amazon Elastic Compute Cloud service, and estimate experimentally the processing time of MME processing instances for different LTE control procedures. For the considered setup, our results show that a vMME is scalable for signaling workloads up to 37000 LTE control procedures per second for a target mean system delay of 1 ms. The database performance assumed imposes this limit in the system scalability.
Information geometry, simulation and complexity in Gaussian random fields
Venue: arXiv cs.IT 2017
Year: 2017

Random fields are useful mathematical objects in the characterization of non-deterministic complex systems. A fundamental issue in the evolution of dynamical systems is how intrinsic properties of such structures change in time. In this paper, we propose to quantify how changes in the spatial dependence structure affect the Riemannian metric tensor that equips the model's parametric space. Defining Fisher curves, we measure the variations in each component of the metric tensor when visiting different entropic states of the system. Simulations show that the geometric deformations induced by the metric tensor in case of a decrease in the inverse temperature are not reversible for an increase of the same amount, provided there is significant variation in the system entropy: the process of taking a system from a lower entropy state A to a higher entropy state B and then bringing it back to A, induces a natural intrinsic one-way direction of evolution. In this context, Fisher curves resemble mathematical models of hysteresis in which the natural orientation is pointed by an arrow of time.
DRAGNN: A Transition-based Framework for Dynamically Connected Neural Networks
Venue: arXiv cs.CL 2017
Year: 2017

In this work, we present a compact, modular framework for constructing novel recurrent neural architectures. Our basic module is a new generic unit, the Transition Based Recurrent Unit (TBRU). In addition to hidden layer activations, TBRUs have discrete state dynamics that allow network connections to be built dynamically as a function of intermediate activations. By connecting multiple TBRUs, we can extend and combine commonly used architectures such as sequence-to-sequence, attention mechanisms, and re-cursive tree-structured models. A TBRU can also serve as both an encoder for downstream tasks and as a decoder for its own task simultaneously, resulting in more accurate multi-task learning. We call our approach Dynamic Recurrent Acyclic Graphical Neural Networks, or DRAGNN. We show that DRAGNN is significantly more accurate and efficient than seq2seq with attention for syntactic dependency parsing and yields more accurate multi-task learning for extractive summarization tasks.
High-Throughput and Language-Agnostic Entity Disambiguation and Linking on User Generated Data
Venue: arXiv cs.IR 2017
Year: 2017

The Entity Disambiguation and Linking (EDL) task matches entity mentions in text to a unique Knowledge Base (KB) identifier such as a Wikipedia or Freebase id. It plays a critical role in the construction of a high quality information network, and can be further leveraged for a variety of information retrieval and NLP tasks such as text categorization and document tagging. EDL is a complex and challenging problem due to ambiguity of the mentions and real world text being multi-lingual. Moreover, EDL systems need to have high throughput and should be lightweight in order to scale to large datasets and run on off-the-shelf machines. More importantly, these systems need to be able to extract and disambiguate dense annotations from the data in order to enable an Information Retrieval or Extraction task running on the data to be more efficient and accurate. In order to address all these challenges, we present the Lithium EDL system and algorithm - a high-throughput, lightweight, language-agnostic EDL system that extracts and correctly disambiguates 75% more entities than state-of-the-art EDL systems and is significantly faster than them.
Healthcare
Venue: arXiv cs.CY 2017
Year: 2017

The complexity of the healthcare ecosystem and the trans-disciplinary convergence which is essential for its function, makes it difficult to address healthcare as one domain. Data curation and analysis of the information may boost our health related knowledge. Increasing connectivity and improving infrastructure may help, among other things, to uncover facts and observations which may influence the future of global health.
Software stage-effort estimation based on association rule mining and fuzzy set theory
Venue: arXiv cs.SE 2017
Year: 2017

Relaying on early effort estimation to predict the required number of resources is not often sufficient, and could lead to under or over estimation. It is widely acknowledge that that software development process should be refined regularly and that software prediction made at early stage of software development is yet kind of guesses. Even good predictions are not sufficient with inherent uncertainty and risks. The stage-effort estimation allows project manager to re-allocate correct number of resources, re-schedule project and control project progress to finish on time and within budget. In this paper we propose an approach to utilize prior effort records to predict stage effort. The proposed model combines concepts of Fuzzy set theory and association rule mining. The results were good in terms of prediction accuracy and have potential to deliver good stage-effort estimation.
Analogy-based effort estimation: a new method to discover set of analogies from dataset characteristics
Venue: arXiv cs.SE 2017
Year: 2017

Analogy-based effort estimation (ABE) is one of the efficient methods for software effort estimation because of its outstanding performance and capability of handling noisy datasets. Conventional ABE models usually use the same number of analogies for all projects in the datasets in order to make good estimates. The authors' claim is that using same number of analogies may produce overall best performance for the whole dataset but not necessarily best performance for each individual project. Therefore there is a need to better understand the dataset characteristics in order to discover the optimum set of analogies for each project rather than using a static k nearest projects. Method: We propose a new technique based on Bisecting k-medoids clustering algorithm to come up with the best set of analogies for each individual project before making the prediction. Results & Conclusions: With Bisecting k-medoids it is possible to better understand the dataset characteristic, and automatically find best set of analogies for each test project. Performance figures of the proposed estimation method are promising and better than those of other regular ABE models
An empirical evaluation of ensemble adjustment methods for analogy-based effort estimation
Venue: arXiv cs.SE 2017
Year: 2017

Objective: This paper investigates the potential of ensemble learning for variants of adjustment methods used in analogy-based effort estimation. The number k of analogies to be used is also investigated. Method We perform a large scale comparison study where many ensembles constructed from n out of 40 possible valid variants of adjustment methods are applied to eight datasets. The performance of each method was evaluated based on standardized accuracy and effect size. Results: The results have been subjected to statistical significance testing, and show reasonable significant improvements on the predictive performance where ensemble methods are applied. Conclusion: Our conclusions suggest that ensembles of adjustment methods can work well and achieve good performance, even though they are not always superior to single methods. We also recommend constructing ensembles from only linear adjustment methods, as they have shown better performance and were frequently ranked higher.
Optimization of Lattice Boltzmann Simulations on Heterogeneous Computers
Venue: arXiv cs.DC 2017
Year: 2017

High-performance computing systems are more and more often based on accelerators. Computing applications targeting those systems often follow a host-driven approach in which hosts offload almost all compute-intensive sections of the code onto accelerators; this approach only marginally exploits the computational resources available on the host CPUs, limiting performance and energy efficiency. The obvious step forward is to run compute-intensive kernels in a concurrent and balanced way on both hosts and accelerators. In this paper we consider exactly this problem for a class of applications based on Lattice Boltzmann Methods, widely used in computational fluid-dynamics. Our goal is to develop just one program, portable and able to run efficiently on several different combinations of hosts and accelerators. To reach this goal, we define common data layouts enabling the code to exploit efficiently the different parallel and vector options of the various accelerators, and matching the possibly different requirements of the compute-bound and memory-bound kernels of the application. We also define models and metrics that predict the best partitioning of workloads among host and accelerator, and the optimally achievable overall performance level. We test the performance of our codes and their scaling properties using as testbeds HPC clusters incorporating different accelerators: Intel Xeon-Phi many-core processors, NVIDIA GPUs and AMD GPUs.
Locality and Singularity for Store-Atomic Memory Models
Venue: arXiv cs.DC 2017
Year: 2017

Robustness is a correctness notion for concurrent programs running under relaxed consistency models. The task is to check that the relaxed behavior coincides (up to traces) with sequential consistency (SC). Although computationally simple on paper (robustness has been shown to be PSPACE-complete for TSO, PGAS, and Power), building a practical robustness checker remains a challenge. The problem is that the various relaxations lead to a dramatic number of computations, only few of which violate robustness. In the present paper, we set out to reduce the search space for robustness checkers. We focus on store-atomic consistency models and establish two completeness results. The first result, called locality, states that a non-robust program always contains a violating computation where only one thread delays commands. The second result, called singularity, is even stronger but restricted to programs without lightweight fences. It states that there is a violating computation where a single store is delayed. As an application of the results, we derive a linear-size source-to-source translation of robustness to SC-reachability. It applies to general programs, regardless of the data domain and potentially with an unbounded number of threads and with unbounded buffers. We have implemented the translation and verified, for the first time, PGAS algorithms in a fully automated fashion. For TSO, our analysis outperforms existing tools.
Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection
Venue: arXiv cs.CV 2017
Year: 2017

Local descriptors based on the image noise residual have proven extremely effective for a number of forensic applications, like forgery detection and localization. Nonetheless, motivated by promising results in computer vision, the focus of the research community is now shifting on deep learning. In this paper we show that a class of residual-based descriptors can be actually regarded as a simple constrained convolutional neural network (CNN). Then, by relaxing the constraints, and fine-tuning the net on a relatively small training set, we obtain a significant performance improvement with respect to the conventional detector.
HyPPI NoC: Bringing Hybrid Plasmonics to an Opto-Electronic Network-on-Chip
Venue: arXiv cs.ET 2017
Year: 2017

As we move towards an era of hundreds of cores, the research community has witnessed the emergence of opto-electronic network on-chip designs based on nanophotonics, in order to achieve higher network throughput, lower latencies, and lower dynamic power. However, traditional nanophotonics options face limitations such as large device footprints compared with electronics, higher static power due to continuous laser operation, and an upper limit on achievable data rates due to large device capacitances. Nanoplasmonics is an emerging technology that has the potential for providing transformative gains on multiple metrics due to its potential to increase the light-matter interaction. In this paper, we propose and analyze a hybrid opto-electric NoC that incorporates Hybrid Plasmonics Photonics Interconnect (HyPPI), an optical interconnect that combines photonics with plasmonics. We explore various opto-electronic network hybridization options by augmenting a mesh network with HyPPI links, and compare them with the equivalent options afforded by conventional nanophotonics as well as pure electronics. Our design space exploration indicates that augmenting an electronic NoC with HyPPI gives a performance to cost ratio improvement of up to 1.8x. To further validate our estimates, we conduct trace based simulations using the NAS Parallel Benchmark suite. These benchmarks show latency improvements up to 1.64x, with negligible energy increase. We then further carry out performance and cost projections for fully optical NoCs, using HyPPI as well as conventional nanophotonics. These futuristic projections indicate that all-HyPPI NoCs would be two orders more energy efficient than electronics, and two orders more area efficient than all-photonic NoCs.
A computational investigation of sources of variability in sentence comprehension difficulty in aphasia
Venue: arXiv cs.CL 2017
Year: 2017

We present a computational evaluation of three hypotheses about sources of deficit in sentence comprehension in aphasia: slowed processing, intermittent deficiency, and resource reduction. The ACT-R based Lewis and Vasishth (2005) model is used to implement these three proposals. Slowed processing is implemented as slowed default production-rule firing time; intermittent deficiency as increased random noise in activation of chunks in memory; and resource reduction as reduced goal activation. As data, we considered subject vs. object rela- tives whose matrix clause contained either an NP or a reflexive, presented in a self-paced listening modality to 56 individuals with aphasia (IWA) and 46 matched controls. The participants heard the sentences and carried out a picture verification task to decide on an interpretation of the sentence. These response accuracies are used to identify the best parameters (for each participant) that correspond to the three hypotheses mentioned above. We show that controls have more tightly clustered (less variable) parameter values than IWA; specifically, compared to controls, among IWA there are more individuals with low goal activations, high noise, and slow default action times. This suggests that (i) individual patients show differential amounts of deficit along the three dimensions of slowed processing, intermittent deficient, and resource reduction, (ii) overall, there is evidence for all three sources of deficit playing a role, and (iii) IWA have a more variable range of parameter values than controls. In sum, this study contributes a proof of concept of a quantitative implementation of, and evidence for, these three accounts of comprehension deficits in aphasia.