OpenRaiser/Intern-Atlas huggingface dataset:
Dealing with Stochasticity in Biological ODE Models
Venue: arXiv cs.LG 2019
Year: 2019

Mathematical modeling with Ordinary Differential Equations (ODEs) has proven to be extremely successful in a variety of fields, including biology. However, these models are completely deterministic given a certain set of initial conditions. We convert mathematical ODE models of three benchmark biological systems to Dynamic Bayesian Networks (DBNs). The DBN model can handle model uncertainty and data uncertainty in a principled manner. They can be used for temporal data mining for noisy and missing variables. We apply Particle Filtering algorithm to infer the model variables by re-estimating the models parameters of various biological ODE models. The model parameters are automatically re-estimated using temporal evidence in the form of data streams. The results show that DBNs are capable of inferring the model variables of the ODE model with high accuracy in situations where data is missing, incomplete, sparse and irregular and true values of model parameters are not known.
Truly Subcubic Min-Plus Product for Less Structured Matrices, with Applications
Venue: arXiv cs.DS 2019
Year: 2019

The goal of this paper is to get truly subcubic algorithms for Min-Plus product for less structured inputs than what was previously known, and to apply them to versions of All-Pairs Shortest Paths (APSP) and other problems. The results are as follows: (1) Our main result is the first truly subcubic algorithm for the Min-Plus product of two $n\times n$ matrices $A$ and $B$ with $\text{polylog}(n)$ bit integer entries, where $B$ has a partitioning into $n^ε\times n^ε$ blocks (for any $ε>0$) where each block is at most $n^δ$-far (for $δ<3-ω$, where $2\leq ω<2.373$) in $\ell_\infty$ norm from a constant rank integer matrix. This result presents the most general case to date of Min-Plus product that is solvable in truly subcubic time. (2) The first application of our main result is a truly subcubic algorithm for APSP in a new type of geometric graph. Our result extends the result of Chan'10 in the case of integer edge weights by allowing the weights to differ from a function of the end-point identities by at most $n^δ$ for small $δ$. (3) In the second application we consider a batch version of the range mode problem in which one is given a length $n$ sequence and $n$ contiguous subsequences, and one is asked to compute the range mode of each subsequence. We give the first $O(n^{1.5-ε})$ time for $ε>0$ algorithm for this batch range mode problem. (4) Our final application is to the Maximum Subarray problem: given an $n\times n$ integer matrix, find the contiguous subarray of maximum entry sum. We show that Maximum Subarray can be solved in truly subcubic, $O(n^{3-ε})$ (for $ε>0$) time, as long as the entries are no larger than $O(n^{0.62})$ in absolute value. We also improve all the known conditional hardness results for the $d$-dimensional variant of Maximum Subarray.
Enabling Failure-resilient Intermittent Systems Without Runtime Checkpointing
Venue: arXiv cs.OS 2019
Year: 2019

Self-powered intermittent systems typically adopt runtime checkpointing as a means to accumulate computation progress across power cycles and recover system status from power failures. However, existing approaches based on the checkpointing paradigm normally require system suspension and/or logging at runtime. This paper presents a design which overcomes the drawbacks of checkpointing-based approaches, to enable failure-resilient intermittent systems. Our design allows accumulative execution and instant system recovery under frequent power failures while enforcing the serializability of concurrent task execution to improve computation progress and ensuring data consistency without system suspension during runtime, by leveraging the characteristics of data accessed in hybrid memory. We integrated the design into FreeRTOS running on a Texas Instruments device. Experimental results show that our design can still accumulate progress when the power source is too weak for checkpointing-based approaches to make progress, and improves the computation progress by up to 43% under a relatively strong power source, while reducing the recovery time by at least 90%.
Inundation Modeling in Data Scarce Regions
Venue: arXiv cs.LG 2019
Year: 2019

Flood forecasts are crucial for effective individual and governmental protective action. The vast majority of flood-related casualties occur in developing countries, where providing spatially accurate forecasts is a challenge due to scarcity of data and lack of funding. This paper describes an operational system providing flood extent forecast maps covering several flood-prone regions in India, with the goal of being sufficiently scalable and cost-efficient to facilitate the establishment of effective flood forecasting systems globally.
Hardware Security Evaluation of MAX 10 FPGA
Venue: arXiv cs.CR 2019
Year: 2019

With the ubiquity of IoT devices there is a growing demand for confidentiality and integrity of data. Solutions based on reconfigurable logic (CPLD or FPGA) have certain advantages over ASIC and MCU/SoC alternatives. Programmable logic devices are ideal for both confidentiality and upgradability purposes. In this context the hardware security aspects of CPLD/FPGA devices are paramount. This paper shows preliminary evaluation of hardware security in Intel MAX 10 devices. These FPGAs are one of the most suitable candidates for applications demanding extensive features and high level of security. Their strong and week security aspects are revealed and some recommendations are suggested to counter possible security vulnerabilities in real designs. This is a feasibility study paper. Its purpose is to highlight the most vulnerable areas to attacks aimed at data extraction and reverse engineering. That way further investigations could be performed on specific areas of concern.
Data-driven discovery of free-form governing differential equations
Venue: arXiv cs.CE 2019
Year: 2019

We present a method of discovering governing differential equations from data without the need to specify a priori the terms to appear in the equation. The input to our method is a dataset (or ensemble of datasets) corresponding to a particular solution (or ensemble of particular solutions) of a differential equation. The output is a human-readable differential equation with parameters calibrated to the individual particular solutions provided. The key to our method is to learn differentiable models of the data that subsequently serve as inputs to a genetic programming algorithm in which graphs specify computation over arbitrary compositions of functions, parameters, and (potentially differential) operators on functions. Differential operators are composed and evaluated using recursive application of automatic differentiation, allowing our algorithm to explore arbitrary compositions of operators without the need for human intervention. We also demonstrate an active learning process to identify and remedy deficiencies in the proposed governing equations.
Developing an ANFIS PSO Model to Estimate Mercury Emission in Combustion Flue Gases
Venue: arXiv cs.LG 2019
Year: 2019

Accurate prediction of mercury content emitted from fossil fueled power stations is of utmost important for environmental pollution assessment and hazard mitigation. In this paper, mercury content in the output gas of power stations boilers was predicted using adaptive neuro fuzzy inference system method integrated with particle swarm optimization. The input parameters of the model include coal characteristics and the operational parameters of the boilers. The dataset has been collected from a number of power plants and employed to educate and examine the proposed model. To evaluate the performance of the proposed ANFIS PSO model the statistical meter of MARE was implemented. Furthermore, relative errors between acquired data and predicted values presented, which confirm the accuracy of the model to deal nonlinearity and representing the dependency of flue gas mercury content into the specifications of coal and the boiler type.
Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval
Venue: arXiv cs.CV 2019
Year: 2019

Image-text retrieval of natural scenes has been a popular research topic. Since image and text are heterogeneous cross-modal data, one of the key challenges is how to learn comprehensive yet unified representations to express the multi-modal data. A natural scene image mainly involves two kinds of visual concepts, objects and their relationships, which are equally essential to image-text retrieval. Therefore, a good representation should account for both of them. In the light of recent success of scene graph in many CV and NLP tasks for describing complex natural scenes, we propose to represent image and text with two kinds of scene graphs: visual scene graph (VSG) and textual scene graph (TSG), each of which is exploited to jointly characterize objects and relationships in the corresponding modality. The image-text retrieval task is then naturally formulated as cross-modal scene graph matching. Specifically, we design two particular scene graph encoders in our model for VSG and TSG, which can refine the representation of each node on the graph by aggregating neighborhood information. As a result, both object-level and relationship-level cross-modal features can be obtained, which favorably enables us to evaluate the similarity of image and text in the two levels in a more plausible way. We achieve state-of-the-art results on Flickr30k and MSCOCO, which verifies the advantages of our graph matching based approach for image-text retrieval.
Efficient and Adaptive Kernelization for Nonlinear Max-margin Multi-view Learning
Venue: arXiv cs.LG 2019
Year: 2019

Existing multi-view learning methods based on kernel function either require the user to select and tune a single predefined kernel or have to compute and store many Gram matrices to perform multiple kernel learning. Apart from the huge consumption of manpower, computation and memory resources, most of these models seek point estimation of their parameters, and are prone to overfitting to small training data. This paper presents an adaptive kernel nonlinear max-margin multi-view learning model under the Bayesian framework. Specifically, we regularize the posterior of an efficient multi-view latent variable model by explicitly mapping the latent representations extracted from multiple data views to a random Fourier feature space where max-margin classification constraints are imposed. Assuming these random features are drawn from Dirichlet process Gaussian mixtures, we can adaptively learn shift-invariant kernels from data according to Bochners theorem. For inference, we employ the data augmentation idea for hinge loss, and design an efficient gradient-based MCMC sampler in the augmented space. Having no need to compute the Gram matrix, our algorithm scales linearly with the size of training set. Extensive experiments on real-world datasets demonstrate that our method has superior performance.
Improving Gradient Estimation in Evolutionary Strategies With Past Descent Directions
Venue: arXiv cs.NE 2019
Year: 2019

Evolutionary Strategies (ES) are known to be an effective black-box optimization technique for deep neural networks when the true gradients cannot be computed, such as in Reinforcement Learning. We continue a recent line of research that uses surrogate gradients to improve the gradient estimation of ES. We propose a novel method to optimally incorporate surrogate gradient information. Our approach, unlike previous work, needs no information about the quality of the surrogate gradients and is always guaranteed to find a descent direction that is better than the surrogate gradient. This allows to iteratively use the previous gradient estimate as surrogate gradient for the current search point. We theoretically prove that this yields fast convergence to the true gradient for linear functions and show under simplifying assumptions that it significantly improves gradient estimates for general functions. Finally, we evaluate our approach empirically on MNIST and reinforcement learning tasks and show that it considerably improves the gradient estimation of ES at no extra computational cost.
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models
Venue: arXiv cs.CL 2019
Year: 2019

Large language models can produce powerful contextual representations that lead to improvements across many NLP tasks. Since these models are typically guided by a sequence of learned self attention mechanisms and may comprise undesired inductive biases, it is paramount to be able to explore what the attention has learned. While static analyses of these models lead to targeted insights, interactive tools are more dynamic and can help humans better gain an intuition for the model-internal reasoning process. We present exBERT, an interactive tool named after the popular BERT language model, that provides insights into the meaning of the contextual representations by matching a human-specified input to similar contexts in a large annotated dataset. By aggregating the annotations of the matching similar contexts, exBERT helps intuitively explain what each attention-head has learned.
A Software-Defined Opto-Acoustic Network Architecture for Internet of Underwater Things
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, we envision a hybrid opto-acoustic network design for the internet of underwater things (IoUT). Software-defined underwater networking (SDUN) is presented as an enabler of hybridizing benefits of optic and acoustic systems and adapting IoUT nodes to the challenging and dynamically changing underwater environment. We explain inextricably interwoven relations among functionalities of different layers and analyze their impacts on key network attributes. Network function virtualization (NFV) concept is then introduced to realize application specific cross-layer protocol suites through an NFV management and orchestration system. We finally discuss how SDUN and NFV can slice available network resources as per the diverging service demands of different underwater applications. Such a revolutionary architectural paradigm shift is not only a cure for chronicle underwater networking problems but also a way of smoothly integrating IoUT and IoT ecosystems.
Deep Reinforcement Learning Based Power control for Wireless Multicast Systems
Venue: arXiv cs.NI 2019
Year: 2019

We consider a multicast scheme recently proposed for a wireless downlink in [1]. It was shown earlier that power control can significantly improve its performance. However for this system, obtaining optimal power control is intractable because of a very large state space. Therefore in this paper we use deep reinforcement learning where we use function approximation of the Q-function via a deep neural network. We show that optimal power control can be learnt for reasonably large systems via this approach. The average power constraint is ensured via a Lagrange multiplier, which is also learnt. Finally, we demonstrate that a slight modification of the learning algorithm allows the optimal control to track the time varying system statistics.
Optimized sensor placement for dependable roadside infrastructures
Venue: arXiv cs.NI 2019
Year: 2019

We present a multi-stage optimization method for efficient sensor deployment in traffic surveillance scenarios. Based on a genetic optimization scheme, our algorithm places an optimal number of roadside sensors to obtain full road coverage in the presence of obstacles and dynamic occlusions. The efficiency of the procedure is demonstrated for selected, realistic road sections. Our analysis helps to leverage the economic feasibility of distributed infrastructure sensor networks with high perception quality.
Learning Analogy-Preserving Sentence Embeddings for Answer Selection
Venue: arXiv cs.CL 2019
Year: 2019

Answer selection aims at identifying the correct answer for a given question from a set of potentially correct answers. Contrary to previous works, which typically focus on the semantic similarity between a question and its answer, our hypothesis is that question-answer pairs are often in analogical relation to each other. Using analogical inference as our use case, we propose a framework and a neural network architecture for learning dedicated sentence embeddings that preserve analogical properties in the semantic space. We evaluate the proposed method on benchmark datasets for answer selection and demonstrate that our sentence embeddings indeed capture analogical properties better than conventional embeddings, and that analogy-based question answering outperforms a comparable similarity-based technique.
Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing
Venue: arXiv cs.NI 2019
Year: 2019

As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment, Edgentis properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.
Aff-Wild Database and AffWildNet
Venue: arXiv cs.LG 2019
Year: 2019

In the context of HCI, building an automatic system to recognize affect of human facial expression in real-world condition is very crucial to make machine interact naturallisticaly with a man. However, existing facial emotion databases usually contain expression in the limited scenario under well-controlled condition. Aff-Wild is currently the largest database consisting of spontaneous facial expression in the wild annotated with valence and arousal. The first contribution of this project is the completion of extending Aff-Wild database which is fulfilled by collecting videos from YouTube on which the videos have spontaneous facial expressions in the wild, annotating videos with valence and arousal ranging in [-1,1], detecting faces in frames using FFLD2 detector and partitioning the whole data set into train, validate and test set, with 527056, 94223 and 135145 frames. The diversity is guaranteed regarding age, ethnicity and values of valence and arousal. The ratio of male to female is close to 1. Regarding the techniques used to build the automatic system, deep learning is outstanding since almost all winning methods in emotion challenges adopt DNN techniques. The second contribution of this project is that an end-to-end DNN is constructed to have joint CNN and RNN block and gives the estimation on valence and arousal for each frame in sequential data. VGGFace, ResNet, DenseNet with the corresponding pre-trained model for CNN block and LSTM, GRU, IndRNN, Attention mechanism for RNN block are experimented aiming to find the best combination. Fine tuning and transfer learning techniques are also tried out. By comparing the CCC evaluation value on test data, the best model is found to be pre-trained VGGFace connected with 2 layers GRU with attention mechanism. The models test performance is 0.555 CCC for valence with sequence length 80 and 0.499 CCC for arousal with sequence length 70.
Persistence and Big Data Analytics Architectures for Smart Connected Vehicles
Venue: arXiv cs.DC 2019
Year: 2019

Up until recently, relational databases were considered as the de-facto technology for persisting and managing large volumes of data. This came to change with the emergence of enterprises producing extremely large datasets and having unprecedentedly high availability requirements. The need for levels of availability beyond those supported by relational databases and the challenges involved in scaling such databases horizontally led to the emergence of a new generation of purpose-specific databases grouped under the term NoSQL. The NoSQL databases despite being designed with horizontal scalability as a primary concern and deliver increased availability and fault-tolerance, they have the cost of temporary inconsistency and reduced durability of data. The present paper presents a concept and application of how TYPHON aims to face the challenges emerging from the attempts to balance the best world of relational and NoSQL databases, by supporting the migration towards hybrid data persistence architectures.
Classification of Resting-State fMRI using Evolutionary Algorithms: Towards a Brain Imaging Biomarker for Parkinson's Disease
Venue: arXiv cs.NE 2019
Year: 2019

Accurate early diagnosis and monitoring of neurodegenerative conditions is essential for effective disease management and delivery of medication and treatment. This research develops automatic methods for detecting brain imaging preclinical biomarkers for Parkinson's disease (PD) by considering the novel application of evolutionary algorithms. A fundamental novel element of this work is the use of evolutionary algorithms to both map and predict the functional connectivity in patients using resting state functional MRI data taken from the PPMI to identify PD progression biomarkers. Specifically, Cartesian Genetic Programming was used to classify DCM data as well as time-series data. The findings were validated using two other commonly used classification methods (Artificial Neural Networks and Support Vector Machines) and by employing k-fold cross-validation. Across DCM and time-series analyses, findings revealed maximum accuracies of 75.21% for early stage (prodromal) PD patients versus healthy controls, 85.87% for PD patients versus prodromal PD patients, and 92.09% for PD patients versus healthy controls. Prodromal PD patients were classified from healthy controls with high accuracy - this is notable and represents the key finding of this research since current methods of diagnosing prodromal PD have both low reliability and low accuracy. Furthermore, Cartesian Genetic Programming provided comparable performance accuracy relative to ANN and SVM. Evolutionary algorithms enable us to decode the classifier in terms of understanding the data inputs that are used, more easily than in ANN and SVM. Hence, these findings underscore the relevance of both DCM analyses for classification and CGP as a novel classification tool for brain imaging data with medical implications for disease diagnosis, particularly in early and asymptomatic stages.
A Proof of Levi's Extension Lemma
Venue: arXiv cs.CG 2019
Year: 2019

We give a short and self-contained proof of Levi's Extension Lemma for pseudoline arrangements.