OpenRaiser/Intern-Atlas huggingface dataset:
Flight Delay Prediction using Airport Situational Awareness Map
Venue: arXiv cs.CY 2019
Year: 2019

The prediction of flight delays plays a significantly important role for airlines and travelers because flight delays cause not only tremendous economic loss but also potential security risks. In this work, we aim to integrate multiple data sources to predict the departure delay of a scheduled flight. Different from previous work, we are the first group, to our best knowledge, to take advantage of airport situational awareness map, which is defined as airport traffic complexity (ATC), and combine the proposed ATC factors with weather conditions and flight information. Features engineering methods and most state-of-the-art machine learning algorithms are applied to a large real-world data sources. We reveal a couple of factors at the airport which has a significant impact on flight departure delay time. The prediction results show that the proposed factors are the main reasons behind the flight delays. Using our proposed framework, an improvement in accuracy for flight departure delay prediction is obtained.
AReN: Assured ReLU NN Architecture for Model Predictive Control of LTI Systems
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we consider the problem of automatically designing a Rectified Linear Unit (ReLU) Neural Network (NN) architecture that is sufficient to implement the optimal Model Predictive Control (MPC) strategy for an LTI system with quadratic cost. Specifically, we propose AReN, an algorithm to generate Assured ReLU Architectures. AReN takes as input an LTI system with quadratic cost specification, and outputs a ReLU NN architecture with the assurance that there exist network weights that exactly implement the associated MPC controller. AReN thus offers new insight into the design of ReLU NN architectures for the control of LTI systems: instead of training a heuristically chosen NN architecture on data -- or iterating over many architectures until a suitable one is found -- AReN can suggest an adequate NN architecture before training begins. While several previous works were inspired by the fact that both ReLU NN controllers and optimal MPC controller are both Continuous, Piecewise-Linear (CPWL) functions, exploiting this similarity to design NN architectures with correctness guarantees has remained elusive. AReN achieves this using two novel features. First, we reinterpret a recent result about the implementation of CPWL functions via ReLU NNs to show that a CPWL function may be implemented by a ReLU architecture that is determined by the number of distinct affine regions in the function. Second, we show that we can efficiently over-approximate the number of affine regions in the optimal MPC controller without solving the MPC problem exactly. Together, these results connect the MPC problem to a ReLU NN implementation without explicitly solving the MPC and directly translates this feature to a ReLU NN architecture that comes with the assurance that it can implement the MPC controller. We show through numerical results the effectiveness of AReN in designing an NN architecture.
Fast Multiple Pattern Cartesian Tree Matching
Venue: arXiv cs.DS 2019
Year: 2019

Cartesian tree matching is the problem of finding all substrings in a given text which have the same Cartesian trees as that of a given pattern. In this paper, we deal with Cartesian tree matching for the case of multiple patterns. We present two fingerprinting methods, i.e., the parent-distance encoding and the binary encoding. By combining an efficient fingerprinting method and a conventional multiple string matching algorithm, we can efficiently solve multiple pattern Cartesian tree matching. We propose three practical algorithms for multiple pattern Cartesian tree matching based on the Wu-Manber algorithm, the Rabin-Karp algorithm, and the Alpha Skip Search algorithm, respectively. In the experiments we compare our solutions against the previous algorithm [18]. Our solutions run faster than the previous algorithm as the pattern lengths increase. Especially, our algorithm based on Wu-Manber runs up to 33 times faster.
ColluEagle: Collusive review spammer detection using Markov random fields
Venue: arXiv cs.IR 2019
Year: 2019

Product reviews are extremely valuable for online shoppers in providing purchase decisions. Driven by immense profit incentives, fraudsters deliberately fabricate untruthful reviews to distort the reputation of online products. As online reviews become more and more important, group spamming, i.e., a team of fraudsters working collaboratively to attack a set of target products, becomes a new fashion. Previous works use review network effects, i.e. the relationships among reviewers, reviews, and products, to detect fake reviews or review spammers, but ignore time effects, which are critical in characterizing group spamming. In this paper, we propose a novel Markov random field (MRF)-based method (ColluEagle) to detect collusive review spammers, as well as review spam campaigns, considering both network effects and time effects. First we identify co-review pairs, a review phenomenon that happens between two reviewers who review a common product in a similar way, and then model reviewers and their co-review pairs as a pairwise-MRF, and use loopy belief propagation to evaluate the suspiciousness of reviewers. We further design a high quality yet easy-to-compute node prior for ColluEagle, through which the review spammer groups can also be subsequently identified. Experiments show that ColluEagle can not only detect collusive spammers with high precision, significantly outperforming state-of-the-art baselines --- FraudEagle and SpEagle, but also identify highly suspicious review spammer campaigns.
A GMM based algorithm to generate point-cloud and its application to neuroimaging
Venue: arXiv cs.LG 2019
Year: 2019

Recent years have witnessed the emergence of 3D medical imaging techniques with the development of 3D sensors and technology. Due to the presence of noise in image acquisition, registration researchers focused on an alternative way to represent medical images. An alternative way to analyze medical imaging is by understanding the 3D shapes represented in terms of point-cloud. Though in the medical imaging community, 3D point-cloud processing is not a ``go-to'' choice, it is a ``natural'' way to capture 3D shapes. However, as the number of samples for medical images are small, researchers have used pre-trained models to fine-tune on medical images. Furthermore, due to different modality in medical images, standard generative models can not be used to generate new samples of medical images. In this work, we use the advantage of point-cloud representation of 3D structures of medical images and propose a Gaussian mixture model-based generation scheme. Our proposed method is robust to outliers. Experimental validation has been performed to show that the proposed scheme can generate new 3D structures using interpolation techniques, i.e., given two 3D structures represented as point-clouds, we can generate point-clouds in between. We have also generated new point-clouds for subjects with and without dementia and show that the generated samples are indeed closely matched to the respective training samples from the same class.
Numerical methods for antiferromagnetics
Venue: arXiv cs.CE 2019
Year: 2019

Compared with ferromagnetic counterparts, antiferromagnetic materials are considered as the future of spintronic applications since these materials are robust against the magnetic perturbation, produce no stray field, and display ultrafast dynamics. There are (at least) two sets of magnetic moments in antiferromagnets (with magnetization of the same magnitude but antiparallel directions) and ferrimagnets (with magnetization of the different magnitude). The coupled dynamics for the bipartite collinear antiferromagnets is modeled by a coupled system of Landau-Lifshitz-Gilbert equations with an additional term originated from the antiferromagnetic exchange, which leads to femtosecond magnetization dynamics. In this paper, we develop three Gauss-Seidel projection methods for micromagnetics simulation in antiferromagnets and ferrimagnets. They are first-order accurate in time and second-order in space, and only solve linear systems of equations with constant coefficients at each step. Femtosecond dynamics, Néel wall structure, and phase transition in presence of an external magnetic field for antiferromagnets are provided with the femtosecond stepsize.
Deep Flow Collaborative Network for Online Visual Tracking
Venue: arXiv cs.CV 2019
Year: 2019

The deep learning-based visual tracking algorithms such as MDNet achieve high performance leveraging to the feature extraction ability of a deep neural network. However, the tracking efficiency of these trackers is not very high due to the slow feature extraction for each frame in a video. In this paper, we propose an effective tracking algorithm to alleviate the time-consuming problem. Specifically, we design a deep flow collaborative network, which executes the expensive feature network only on sparse keyframes and transfers the feature maps to other frames via optical flow. Moreover, we raise an effective adaptive keyframe scheduling mechanism to select the most appropriate keyframe. We evaluate the proposed approach on large-scale datasets: OTB2013 and OTB2015. The experiment results show that our algorithm achieves considerable speedup and high precision as well.
Video Captioning with Text-based Dynamic Attention and Step-by-Step Learning
Venue: arXiv cs.CV 2019
Year: 2019

Automatically describing video content with natural language has been attracting much attention in CV and NLP communities. Most existing methods predict one word at a time, and by feeding the last generated word back as input at the next time, while the other generated words are not fully exploited. Furthermore, traditional methods optimize the model using all the training samples in each epoch without considering their learning situations, which leads to a lot of unnecessary training and can not target the difficult samples. To address these issues, we propose a text-based dynamic attention model named TDAM, which imposes a dynamic attention mechanism on all the generated words with the motivation to improve the context semantic information and enhance the overall control of the whole sentence. Moreover, the text-based dynamic attention mechanism and the visual attention mechanism are linked together to focus on the important words. They can benefit from each other during training. Accordingly, the model is trained through two steps: "starting from scratch" and "checking for gaps". The former uses all the samples to optimize the model, while the latter only trains for samples with poor control. Experimental results on the popular datasets MSVD and MSR-VTT demonstrate that our non-ensemble model outperforms the state-of-the-art video captioning benchmarks.
A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Venue: arXiv cs.LG 2019
Year: 2019

Stochastic gradient descent (SGD) is the workhorse of modern machine learning. Sometimes, there are many different potential gradient estimators that can be used. When so, choosing the one with the best tradeoff between cost and variance is important. This paper analyzes the convergence rates of SGD as a function of time, rather than iterations. This results in a simple rule to select the estimator that leads to the best optimization convergence guarantee. This choice is the same for different variants of SGD, and with different assumptions about the objective (e.g. convexity or smoothness). Inspired by this principle, we propose a technique to automatically select an estimator when a finite pool of estimators is given. Then, we extend to infinite pools of estimators, where each one is indexed by control variate weights. This is enabled by a reduction to a mixed-integer quadratic program. Empirically, automatically choosing an estimator performs comparably to the best estimator chosen with hindsight.
Non-idempotent intersection types in logical form
Venue: arXiv cs.LO 2019
Year: 2019

Intersection types are an essential tool in the analysis of operational and denotational properties of lambda-terms and functional programs. Among them, non-idempotent intersection types provide precise quantitative information about the evaluation of terms and programs. However, unlike simple or second-order types, intersection types cannot be considered as a logical system because the application rule (or the intersection rule, depending on the presentation of the system) involves a condition expressing that the proofs of premises satisfy a very strong uniformity condition: the underlying lambda-terms must be the same. Using earlier work introducing an indexed version of Linear Logic, we show that non-idempotent typing can be given a logical form in a system where formulas represent hereditarily indexed families of intersection types.
Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)
Venue: arXiv cs.LG 2019
Year: 2019

We implement a Tensor Train layer in the TensorFlow Neural Machine Translation (NMT) model using the t3f library. We perform training runs on the IWSLT English-Vietnamese '15 and WMT German-English '16 datasets with learning rates $\in \{0.0004,0.0008,0.0012\}$, maximum ranks $\in \{2,4,8,16\}$ and a range of core dimensions. We compare against a target BLEU test score of 24.0, obtained by our benchmark run. For the IWSLT English-Vietnamese training, we obtain BLEU test/dev scores of 24.0/21.9 and 24.2/21.9 using core dimensions $(2, 2, 256) \times (2, 2, 512)$ with learning rate 0.0012 and rank distributions $(1,4,4,1)$ and $(1,4,16,1)$ respectively. These runs use 113\% and 397\% of the flops of the benchmark run respectively. We find that, of the parameters surveyed, a higher learning rate and more `rectangular' core dimensions generally produce higher BLEU scores. For the WMT German-English dataset, we obtain BLEU scores of 24.0/23.8 using core dimensions $(4, 4, 128) \times (4, 4, 256)$ with learning rate 0.0012 and rank distribution $(1,2,2,1)$. We discuss the potential for future optimization and application of Tensor Train decomposition to other NMT models.
Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks
Venue: arXiv cs.CL 2019
Year: 2019

Textual entailment is a fundamental task in natural language processing. Most approaches for solving the problem use only the textual content present in training data. A few approaches have shown that information from external knowledge sources like knowledge graphs (KGs) can add value, in addition to the textual content, by providing background knowledge that may be critical for a task. However, the proposed models do not fully exploit the information in the usually large and noisy KGs, and it is not clear how it can be effectively encoded to be useful for entailment. We present an approach that complements text-based entailment models with information from KGs by (1) using Personalized PageR- ank to generate contextual subgraphs with reduced noise and (2) encoding these subgraphs using graph convolutional networks to capture KG structure. Our technique extends the capability of text models exploiting structural and semantic information found in KGs. We evaluate our approach on multiple textual entailment datasets and show that the use of external knowledge helps improve prediction accuracy. This is particularly evident in the challenging BreakingNLI dataset, where we see an absolute improvement of 5-20% over multiple text-based entailment models.
Failure Analysis and Quantification for Contemporary and Future Supercomputers
Venue: arXiv cs.DC 2019
Year: 2019

Large-scale computing systems today are assembled by numerous computing units for massive computational capability needed to solve problems at scale, which enables failures common events in supercomputing scenarios. Considering the demanding resilience requirements of supercomputers today, we present a quantitative study on fine-grained failure modeling for contemporary and future large-scale computing systems. We integrate various types of failures from different system hierarchical levels and system components, and summarize the overall system failure rates formally. Given that nowadays system-wise failure rate needs to be capped under a threshold value for reliability and cost-efficiency purposes, we quantitatively discuss different scenarios of system resilience, and analyze the impacts of resilience to different error types on the variation of system failure rates, and the correlation of hierarchical failure rates. Moreover, we formalize and showcase the resilience efficiency of failure-bounded supercomputers today.
Are highly connected 1-planar graphs Hamiltonian?
Venue: arXiv cs.DM 2019
Year: 2019

It is well-known that every planar 4-connected graph has a Hamiltonian cycle. In this paper, we study the question whether every 1-planar 4-connected graph has a Hamiltonian cycle. We show that this is false in general, even for 5-connected graphs, but true if the graph has a 1-planar drawing where every region is a triangle.
Spatially regularized active diffusion learning for high-dimensional images
Venue: arXiv cs.LG 2019
Year: 2019

An active learning algorithm for the classification of high-dimensional images is proposed in which spatially-regularized nonlinear diffusion geometry is used to characterize cluster cores. The proposed method samples from estimated cluster cores in order to generate a small but potent set of training labels which propagate to the remainder of the dataset via the underlying diffusion process. By spatially regularizing the rich, high-dimensional spectral information of the image to efficiently estimate the most significant and influential points in the data, our approach avoids redundancy in the training dataset. This allows it to produce high-accuracy labelings with a very small number of training labels. The proposed algorithm admits an efficient numerical implementation that scales essentially linearly in the number of data points under a suitable data model and enjoys state-of-the-art performance on real hyperspectral images.
Channel Estimation for Wireless Communication Systems Assisted by Large Intelligent Surfaces
Venue: arXiv cs.IT 2019
Year: 2019

In this letter, the channel estimation problem is studied for wireless communication systems assisted by large intelligent surface. Due to features of assistant channel, channel estimation (CE) problem for the investigated system is shown as a constrained estimation error minimization problem, which differs from traditional CE problems. A Lagrange multiplier and dual ascent-based estimation scheme is then designed to obtain a closed-form solution for the estimator iteratively. Moreover, the Cramer-Rao lower bounds are deduced for performance evaluation. Simulation results show that the designed scheme could improve estimation accuracy up to 18%, compared with least square method in low signal-to-noise ratio regime.
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
Venue: arXiv cs.CL 2019
Year: 2019

Multi-paragraph reasoning is indispensable for open-domain question answering (OpenQA), which receives less attention in the current OpenQA systems. In this work, we propose a knowledge-enhanced graph neural network (KGNN), which performs reasoning over multiple paragraphs with entities. To explicitly capture the entities' relatedness, KGNN utilizes relational facts in knowledge graph to build the entity graph. The experimental results show that KGNN outperforms in both distractor and full wiki settings than baselines methods on HotpotQA dataset. And our further analysis illustrates KGNN is effective and robust with more retrieved paragraphs.
Analysis and Optimization of Tail-Biting Spatially Coupled Protograph LDPC Codes for BICM-ID Systems
Venue: arXiv cs.IT 2019
Year: 2019

As a typical example of bandwidth-efficient techniques, bit-interleaved coded modulation with iterative decoding (BICM-ID) provides desirable spectral efficiencies in various wireless communication scenarios. In this paper, we carry out a comprehensive investigation on tail-biting (TB) spatially coupled protograph (SCP) low-density parity-check (LDPC) codes in BICM-ID systems. Specifically, we first develop a two-step design method to formulate a novel type of constellation mappers, referred to as labeling-bit-partial-match (LBPM) constellation mappers, for SC-P-based BICM-ID systems. The LBPM constellation mappers can be seamlessly combined with high-order modulations, such as M-ary phase-shift keying (PSK) and M-ary quadrature amplitude modulation (QAM). Furthermore, we conceive a new bit-level interleaving scheme, referred to as variable node matched mapping (VNMM) scheme, which can substantially exploit the structure feature of SC-P codes and the unequal protection-degree property of labeling bits to trigger the wave-like convergence for TB-SC-P codes. In addition, we propose a hierarchical extrinsic information transfer (EXIT) algorithm to predict the convergence performance (i.e., decoding thresholds) of the proposed SC-P-based BICM-ID systems. Theoretical analyses and simulation results illustrate that the LBPM-mapped SC-P-based BICM-ID systems are remarkably superior to the state-of-the-art mapped counterparts. Moreover, the proposed SC-P-based BICM-ID systems can achieve even better error performance with the aid of the VNMM scheme. As a consequence, the proposed LBPM constellation mappers and VNMM scheme make the SC-P-based BICM-ID systems a favorable choice for the future-generation wireless communication systems.
Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds
Venue: arXiv cs.CL 2019
Year: 2019

Automatic question generation aims at the generation of questions from a context, with the corresponding answers being sub-spans of the given passage. Whereas, most of the methods mostly rely on heuristic rules to generate questions, more recently also neural network approaches have been proposed. In this work, we propose a variant of the self-attention Transformer network architectures model to generate meaningful and diverse questions. To this end, we propose an easy to use model consisting of the conjunction of the Transformer decoder GPT-2 model with Transformer encoder BERT for the downstream task for question answering. The model is trained in an end-to-end fashion, where the language model is trained to produce a question-answer-aware input representation that facilitates to generate an answer focused question. Our result of neural question generation from text on the SQuAD 1.1 dataset suggests that our method can produce semantically correct and diverse questions. Additionally, we assessed the performance of our proposed method for the downstream task of question answering. The analysis shows that our proposed generation & answering collaboration framework relatively improves both tasks and is particularly powerful in the semi-supervised setup. The results further suggest a robust and comparably lean pipeline facilitating question generation in the small-data regime.
Towards Human Centered AutoML
Venue: arXiv cs.HC 2019
Year: 2019

Building models from data is an integral part of the majority of data science workflows. While data scientists are often forced to spend the majority of the time available for a given project on data cleaning and exploratory analysis, the time available to practitioners to build actual models from data is often rather short due to time constraints for a given project. AutoML systems are currently rising in popularity, as they can build powerful models without human oversight. In this position paper, we aim to discuss the impact of the rising popularity of such systems and how a user-centered interface for such systems could look like. More importantly, we also want to point out features that are currently missing in those systems and start to explore better usability of such systems from a data-scientists perspective.