OpenRaiser/Intern-Atlas huggingface dataset:
Bringing Network Coding into SDN: A Case-study for Highly Meshed Heterogeneous Communications
Venue: arXiv cs.NI 2020
Year: 2020

Modern communications have moved away from point-to-point models to increasingly heterogeneous network models. In this article, we propose a novel controller-based protocol to deploy adaptive causal network coding in heterogeneous and highly-meshed communication networks. Specifically, we consider using Software-Defined-Network (SDN) as the main controller. We first present an architecture for the highly-meshed heterogeneous multi-source multi-destination networks that represents the practical communication networks encountered in the fifth generation of wireless networks (5G) and beyond. Next, we present a promising solution to deploy network coding over the new architecture. In fact, we investigate how to generalize adaptive and causal random linear network coding (AC-RLNC), proposed for multipath multi-hop (MP-MH) communication channels, to a protocol for the new multi-source multi-destination network architecture using controller. To this end, we present a modularized implementation of AC-RLNC solution where the modules work together in a distributed fashion and perform the AC-RLNC technology. We also present a new controller-based setting through which the network coding modules can communicate and can attain their required information. Finally, we briefly discuss how the proposed architecture and network coding solution provide a good opportunity for future technologies, e.g., distributed coded computation and storage, mmWave communication environments, and innovative and efficient security features.
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text
Venue: arXiv cs.CL 2020
Year: 2020

Long Short-Term Memory recurrent neural network (LSTM) is widely used and known to capture informative long-term syntactic dependencies. However, how such information are reflected in its internal vectors for natural text has not yet been sufficiently investigated. We analyze them by learning a language model where syntactic structures are implicitly given. We empirically show that the context update vectors, i.e. outputs of internal gates, are approximately quantized to binary or ternary values to help the language model to count the depth of nesting accurately, as Suzgun et al. (2019) recently show for synthetic Dyck languages. For some dimensions in the context vector, we show that their activations are highly correlated with the depth of phrase structures, such as VP and NP. Moreover, with an $L_1$ regularization, we also found that it can accurately predict whether a word is inside a phrase structure or not from a small number of components of the context vector. Even for the case of learning from raw text, context vectors are shown to still correlate well with the phrase structures. Finally, we show that natural clusters of the functional words and the part of speeches that trigger phrases are represented in a small but principal subspace of the context-update vector of LSTM.
Strategy for Boosting Pair Comparison and Improving Quality Assessment Accuracy
Venue: arXiv cs.AI 2020
Year: 2020

The development of rigorous quality assessment model relies on the collection of reliable subjective data, where the perceived quality of visual multimedia is rated by the human observers. Different subjective assessment protocols can be used according to the objectives, which determine the discriminability and accuracy of the subjective data. Single stimulus methodology, e.g., the Absolute Category Rating (ACR) has been widely adopted due to its simplicity and efficiency. However, Pair Comparison (PC) is of significant advantage over ACR in terms of discriminability. In addition, PC avoids the influence of observers' bias regarding their understanding of the quality scale. Nevertheless, full pair comparison is much more time-consuming. In this study, we therefore 1) employ a generic model to bridge the pair comparison data and ACR data, where the variance term could be recovered and the obtained information is more complete; 2) propose a fusion strategy to boost pair comparisons by utilizing the ACR results as initialization information; 3) develop a novel active batch sampling strategy based on Minimum Spanning Tree (MST) for PC. In such a way, the proposed methodology could achieve the same accuracy of pair comparison but with the compelxity as low as ACR. Extensive experimental results demonstrate the efficiency and accuracy of the proposed approach, which outperforms the state of the art approaches.
LUMÁWIG: An Efficient Algorithm for Dimension Zero Bottleneck Distance Computation in Topological Data Analysis
Venue: arXiv cs.CG 2020
Year: 2020

Stability of persistence diagrams under slight perturbations is a key characteristic behind the validity and growing popularity of topological data analysis in exploring real-world data. Central to this stability is the use of Bottleneck distance which entails matching points between diagrams. Use of this metric in practical studies has, however, been few and sparingly because of the computational obstruction, especially in dimension zero where the computational cost explodes with the growth of data size. We present LUMÁWIG, a novel efficient algorithm to compute dimension zero bottleneck distance between two persistent diagrams which runs significantly faster and provides significantly sharper approximates with respect to the output of the original algorithm than any other available algorithm. We bypass the overwhelming matching problem in previous implementations of the bottleneck distance, and prove that the zero dimensional bottleneck distance can be recovered from a very small number of matching cases. We show that LUMÁWIG generally enjoys linear complexity as shown by empirical tests. We also present an application that leverages dimension zero persistence diagrams and the bottleneck distance to produce features for classification tasks.
Mini-DDSM: Mammography-based Automatic Age Estimation
Venue: arXiv cs.CV 2020
Year: 2020

Age estimation has attracted attention for its various medical applications. There are many studies on human age estimation from biomedical images. However, there is no research done on mammograms for age estimation, as far as we know. The purpose of this study is to devise an AI-based model for estimating age from mammogram images. Due to lack of public mammography data sets that have the age attribute, we resort to using a web crawler to download thumbnail mammographic images and their age fields from the public data set; the Digital Database for Screening Mammography. The original images in this data set unfortunately can only be retrieved by a software which is broken. Subsequently, we extracted deep learning features from the collected data set, by which we built a model using Random Forests regressor to estimate the age automatically. The performance assessment was measured using the mean absolute error values. The average error value out of 10 tests on random selection of samples was around 8 years. In this paper, we show the merits of this approach to fill up missing age values. We ran logistic and linear regression models on another independent data set to further validate the advantage of our proposed work. This paper also introduces the free-access Mini-DDSM data set.
Cardea: An Open Automated Machine Learning Framework for Electronic Health Records
Venue: arXiv cs.LG 2020
Year: 2020

An estimated 180 papers focusing on deep learning and EHR were published between 2010 and 2018. Despite the common workflow structure appearing in these publications, no trusted and verified software framework exists, forcing researchers to arduously repeat previous work. In this paper, we propose Cardea, an extensible open-source automated machine learning framework encapsulating common prediction problems in the health domain and allows users to build predictive models with their own data. This system relies on two components: Fast Healthcare Interoperability Resources (FHIR) -- a standardized data structure for electronic health systems -- and several AUTOML frameworks for automated feature engineering, model selection, and tuning. We augment these components with an adaptive data assembler and comprehensive data- and model- auditing capabilities. We demonstrate our framework via 5 prediction tasks on MIMIC-III and Kaggle datasets, which highlight Cardea's human competitiveness, flexibility in problem definition, extensive feature generation capability, adaptable automatic data assembler, and its usability.
Designing Indicators to Combat Fake Media
Venue: arXiv cs.HC 2020
Year: 2020

The growth of misinformation technology necessitates the need to identify fake videos. One approach to preventing the consumption of these fake videos is provenance which allows the user to authenticate media content to its original source. This research designs and investigates the use of provenance indicators to help users identify fake videos. We first interview users regarding their experiences with different misinformation modes (text, image, video) to guide the design of indicators within users' existing perspectives. Then, we conduct a participatory design study to develop and design fake video indicators. Finally, we evaluate participant-designed indicators via both expert evaluations and quantitative surveys with a large group of end-users. Our results provide concrete design guidelines for the emerging issue of fake videos. Our findings also raise concerns regarding users' tendency to overgeneralize from misinformation warning messages, suggesting the need for further research on warning design in the ongoing fight against misinformation.
Deep learning for time series classification
Venue: arXiv cs.LG 2020
Year: 2020

Time series analysis is a field of data science which is interested in analyzing sequences of numerical values ordered in time. Time series are particularly interesting because they allow us to visualize and understand the evolution of a process over time. Their analysis can reveal trends, relationships and similarities across the data. There exists numerous fields containing data in the form of time series: health care (electrocardiogram, blood sugar, etc.), activity recognition, remote sensing, finance (stock market price), industry (sensors), etc. Time series classification consists of constructing algorithms dedicated to automatically label time series data. The sequential aspect of time series data requires the development of algorithms that are able to harness this temporal property, thus making the existing off-the-shelf machine learning models for traditional tabular data suboptimal for solving the underlying task. In this context, deep learning has emerged in recent years as one of the most effective methods for tackling the supervised classification task, particularly in the field of computer vision. The main objective of this thesis was to study and develop deep neural networks specifically constructed for the classification of time series data. We thus carried out the first large scale experimental study allowing us to compare the existing deep methods and to position them compared other non-deep learning based state-of-the-art methods. Subsequently, we made numerous contributions in this area, notably in the context of transfer learning, data augmentation, ensembling and adversarial attacks. Finally, we have also proposed a novel architecture, based on the famous Inception network (Google), which ranks among the most efficient to date.
Biocybersecurity -- A Converging Threat as an Auxiliary to War
Venue: arXiv cs.CY 2020
Year: 2020

Biodefense is the discipline of ensuring biosecurity with respect to select groups of organisms and limiting their spread. This field has increasingly been challenged by novel threats from nature that have been weaponized such as SARS, Anthrax, and similar pathogens, but has emerged victorious through collaboration of national and world health groups. However, it may come under additional stress in the 21st century as the field intersects with the cyberworld -- a world where governments have already been struggling to keep up with cyber attacks from small to state-level actors as cyberthreats have been relied on to level the playing field in international disputes. Disruptions to military logistics and economies through cyberattacks have been able to be done at a mere fraction of economic and moral costs through conventional military means, making it an increasingly tempting means of disruption. In the field of biocybersecurity (BCS), the strengths within biotechnology and cybersecurity merge, along with many of their vulnerabilities, and this could spell increased trouble for biodefense, as novel threats can be synthesized and disseminated in ways that fuse the routes of attacks seen in biosecurity and cybersecurity. Herein, we offer an exploration of how threats in the domain of biocybersecurity may emerge through less foreseen routes as it might be an attractive auxiliary to conventional war. This is done through an analysis of potential payload and delivery methods to develop notional threat vectorizations. We conclude with several paradigms through which to view BCS-based threats.
Tabular GANs for uneven distribution
Venue: arXiv cs.LG 2020
Year: 2020

GANs are well known for success in the realistic image generation. However, they can be applied in tabular data generation as well. We will review and examine some recent papers about tabular GANs in action. We will generate data to make train distribution bring closer to the test. Then compare model performance trained on the initial train dataset, with trained on the train with GAN generated data, also we train the model by sampling train by adversarial training. We show that using GAN might be an option in case of uneven data distribution between train and test data.
The h-index and the Harmonic Mean
Venue: arXiv cs.DL 2020
Year: 2020

The Harmonic Mean between the number of papers and the citation number per paper is proposed as a simple single-value index to quantify an individual's research output. Two simple comparisons with the Hirsch h-index are performed.
Traffic Behavior in Cloud Data Centers: A Survey
Venue: arXiv cs.NI 2020
Year: 2020

Data centers (DCs) nowadays house tens of thousands of servers and switches, interconnected by high-speed communication links. With the rapid growth of cloud DCs, in both size and number, tremendous efforts have been undertaken to efficiently design the network and manage the traffic within these DCs. However, little effort has been made toward measuring, understanding and chattelizing how the network-level traffic of these DCs behave. In this paper, we aim to present a systematic taxonomy and survey of these DC studies. Specifically, our survey first decomposes DC network traffic behavior into two main stages, namely (1) data collection methodologies and (2) research findings, and then classifies and discusses the recent research studies in each stage. Finally, the survey highlights few research challenges related to DC network traffic that require further research investigation.
Evaluating a Generative Adversarial Framework for Information Retrieval
Venue: arXiv cs.LG 2020
Year: 2020

Recent advances in Generative Adversarial Networks (GANs) have resulted in its widespread applications to multiple domains. A recent model, IRGAN, applies this framework to Information Retrieval (IR) and has gained significant attention over the last few years. In this focused work, we critically analyze multiple components of IRGAN, while providing experimental and theoretical evidence of some of its shortcomings. Specifically, we identify issues with the constant baseline term in the policy gradients optimization and show that the generator harms IRGAN's performance. Motivated by our findings, we propose two models influenced by self-contrastive estimation and co-training which outperform IRGAN on two out of the three tasks considered.
Fast fully-reproducible serial/parallel Monte Carlo and MCMC simulations and visualizations via ParaMonte::Python library
Venue: arXiv cs.MS 2020
Year: 2020

ParaMonte::Python (standing for Parallel Monte Carlo in Python) is a serial and MPI-parallelized library of (Markov Chain) Monte Carlo (MCMC) routines for sampling mathematical objective functions, in particular, the posterior distributions of parameters in Bayesian modeling and analysis in data science, Machine Learning, and scientific inference in general. In addition to providing access to fast high-performance serial/parallel Monte Carlo and MCMC sampling routines, the ParaMonte::Python library provides extensive post-processing and visualization tools that aim to automate and streamline the process of model calibration and uncertainty quantification in Bayesian data analysis. Furthermore, the automatically-enabled restart functionality of ParaMonte::Python samplers ensure seamless fully-deterministic into-the-future restart of Monte Carlo simulations, should any interruptions happen. The ParaMonte::Python library is MIT-licensed and is permanently maintained on GitHub at https://github.com/cdslaborg/paramonte/tree/master/src/interface/Python.
XDA: Accurate, Robust Disassembly with Transfer Learning
Venue: arXiv cs.CR 2020
Year: 2020

Accurate and robust disassembly of stripped binaries is challenging. The root of the difficulty is that high-level structures, such as instruction and function boundaries, are absent in stripped binaries and must be recovered based on incomplete information. Current disassembly approaches rely on heuristics or simple pattern matching to approximate the recovery, but these methods are often inaccurate and brittle, especially across different compiler optimizations. We present XDA, a transfer-learning-based disassembly framework that learns different contextual dependencies present in machine code and transfers this knowledge for accurate and robust disassembly. We design a self-supervised learning task motivated by masked Language Modeling to learn interactions among byte sequences in binaries. The outputs from this task are byte embeddings that encode sophisticated contextual dependencies between input binaries' byte tokens, which can then be finetuned for downstream disassembly tasks. We evaluate XDA's performance on two disassembly tasks, recovering function boundaries and assembly instructions, on a collection of 3,121 binaries taken from SPEC CPU2017, SPEC CPU2006, and the BAP corpus. The binaries are compiled by GCC, ICC, and MSVC on x86/x64 Windows and Linux platforms over 4 optimization levels. XDA achieves 99.0% and 99.7% F1 score at recovering function boundaries and instructions, respectively, surpassing the previous state-of-the-art on both tasks. It also maintains speed on par with the fastest ML-based approach and is up to 38x faster than hand-written disassemblers like IDA Pro. We release the code of XDA at https://github.com/CUMLSec/XDA.
SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces
Venue: arXiv cs.CL 2020
Year: 2020

Lexical semantic change detection (also known as semantic shift tracing) is a task of identifying words that have changed their meaning over time. Unsupervised semantic shift tracing, focal point of SemEval2020, is particularly challenging. Given the unsupervised setup, in this work, we propose to identify clusters among different occurrences of each target word, considering these as representatives of different word meanings. As such, disagreements in obtained clusters naturally allow to quantify the level of semantic shift per each target word in four target languages. To leverage this idea, clustering is performed on contextualized (BERT-based) embeddings of word occurrences. The obtained results show that our approach performs well both measured separately (per language) and overall, where we surpass all provided SemEval baselines.
An Evaluation of Classification Methods for 3D Printing Time-Series Data
Venue: arXiv cs.LG 2020
Year: 2020

Additive Manufacturing presents a great application area for Machine Learning because of the vast volume of data generated and the potential to mine this data to control outcomes. In this paper we present preliminary work on classifying infrared time-series data representing melt-pool temperature in a metal 3D printing process. Our ultimate objective is to use this data to predict process outcomes (e.g. hardness, porosity, surface roughness). In the work presented here we simply show that there is a signal in this data that can be used for the classification of different components and stages of the AM process. In line with other Machine Learning research on time-series classification we use k-Nearest Neighbour classifiers. The results we present suggests that Dynamic Time Warping is an effective distance measure compared with alternatives for 3D printing data of this type.
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems
Venue: arXiv cs.CL 2020
Year: 2020

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applications where new data come in a stream, we study NLG in a "continual learning" setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to forget the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on ElasticWeight Consolidation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outperforms other methods by effectively mitigating the detrimental catastrophic forgetting issue.
Splay trees on trees
Venue: arXiv cs.DS 2020
Year: 2020

Search trees on trees (STTs) are a far-reaching generalization of binary search trees (BSTs), allowing the efficient exploration of tree-structured domains. (BSTs are the special case in which the underlying domain is a path.) Trees on trees have been extensively studied under various guises in computer science and discrete mathematics. Recently Bose, Cardinal, Iacono, Koumoutsos, and Langerman (SODA 2020) considered adaptive STTs and observed that, apart from notable exceptions, the machinery developed for BSTs in the past decades does not readily transfer to STTs. In particular, they asked whether the optimal STT can be efficiently computed or approximated (by analogy to Knuth's algorithm for optimal BSTs), and whether natural self-adjusting BSTs such as Splay trees (Sleator, Tarjan, 1983) can be extended to this more general setting. We answer both questions affirmatively. First, we show that a $(1 + \frac{1}{t})$-approximation of an optimal size-$n$ STT for a given search distribution can be computed in time $O(n^{2t + 1})$ for all integers $t \geq 1$. Second, we identify a broad family of STTs with linear rotation-distance, allowing the generalization of Splay trees to the STT setting. We show that our generalized Splay satisfies a static optimality theorem, asymptotically matching the cost of the optimal STT in an online fashion, i.e. without knowledge of the search distribution. Our results suggest an extension of the dynamic optimality conjecture for Splay trees to the broader setting of trees on trees.