OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised Learning for Surgical Motion by Learning to Predict the Future
Venue: arXiv cs.CV 2018
Year: 2018

We show that it is possible to learn meaningful representations of surgical motion, without supervision, by learning to predict the future. An architecture that combines an RNN encoder-decoder and mixture density networks (MDNs) is developed to model the conditional distribution over future motion given past motion. We show that the learned encodings naturally cluster according to high-level activities, and we demonstrate the usefulness of these learned encodings in the context of information retrieval, where a database of surgical motion is searched for suturing activity using a motion-based query. Future prediction with MDNs is found to significantly outperform simpler baselines as well as the best previously-published result for this task, advancing state-of-the-art performance from an F1 score of 0.60 +- 0.14 to 0.77 +- 0.05.
Discovering Signals from Web Sources to Predict Cyber Attacks
Venue: arXiv cs.SI 2018
Year: 2018

Cyber attacks are growing in frequency and severity. Over the past year alone we have witnessed massive data breaches that stole personal information of millions of people and wide-scale ransomware attacks that paralyzed critical infrastructure of several countries. Combating the rising cyber threat calls for a multi-pronged strategy, which includes predicting when these attacks will occur. The intuition driving our approach is this: during the planning and preparation stages, hackers leave digital traces of their activities on both the surface web and dark web in the form of discussions on platforms like hacker forums, social media, blogs and the like. These data provide predictive signals that allow anticipating cyber attacks. In this paper, we describe machine learning techniques based on deep neural networks and autoregressive time series models that leverage external signals from publicly available Web sources to forecast cyber attacks. Performance of our framework across ground truth data over real-world forecasting tasks shows that our methods yield a significant lift or increase of F1 for the top signals on predicted cyber attacks. Our results suggest that, when deployed, our system will be able to provide an effective line of defense against various types of targeted cyber attacks.
Measuring Conversational Productivity in Child Forensic Interviews
Venue: arXiv cs.CL 2018
Year: 2018

Child Forensic Interviewing (FI) presents a challenge for effective information retrieval and decision making. The high stakes associated with the process demand that expert legal interviewers are able to effectively establish a channel of communication and elicit substantive knowledge from the child-client while minimizing potential for experiencing trauma. As a first step toward computationally modeling and producing quality spoken interviewing strategies and a generalized understanding of interview dynamics, we propose a novel methodology to computationally model effectiveness criteria, by applying summarization and topic modeling techniques to objectively measure and rank the responsiveness and conversational productivity of a child during FI. We score information retrieval by constructing an agenda to represent general topics of interest and measuring alignment with a given response and leveraging lexical entrainment for responsiveness. For comparison, we present our methods along with traditional metrics of evaluation and discuss the use of prior information for generating situational awareness.
Data Synthesis based on Generative Adversarial Networks
Venue: arXiv cs.DB 2018
Year: 2018

Privacy is an important concern for our society where sharing data with partners or releasing data to the public is a frequent occurrence. Some of the techniques that are being used to achieve privacy are to remove identifiers, alter quasi-identifiers, and perturb values. Unfortunately, these approaches suffer from two limitations. First, it has been shown that private information can still be leaked if attackers possess some background knowledge or other information sources. Second, they do not take into account the adverse impact these methods will have on the utility of the released data. In this paper, we propose a method that meets both requirements. Our method, called table-GAN, uses generative adversarial networks (GANs) to synthesize fake tables that are statistically similar to the original table yet do not incur information leakage. We show that the machine learning models trained using our synthetic tables exhibit performance that is similar to that of models trained using the original table for unknown testing cases. We call this property model compatibility. We believe that anonymization/perturbation/synthesis methods without model compatibility are of little value. We used four real-world datasets from four different domains for our experiments and conducted in-depth comparisons with state-of-the-art anonymization, perturbation, and generation techniques. Throughout our experiments, only our method consistently shows a balance between privacy level and model compatibility.
Robust Semantic Segmentation with Ladder-DenseNet Models
Venue: arXiv cs.CV 2018
Year: 2018

We present semantic segmentation experiments with a model capable to perform predictions on four benchmark datasets: Cityscapes, ScanNet, WildDash and KITTI. We employ a ladder-style convolutional architecture featuring a modified DenseNet-169 model in the downsampling datapath, and only one convolution in each stage of the upsampling datapath. Due to limited computing resources, we perform the training only on Cityscapes Fine train+val, ScanNet train, WildDash val and KITTI train. We evaluate the trained model on the test subsets of the four benchmarks in concordance with the guidelines of the Robust Vision Challenge ROB 2018. The performed experiments reveal several interesting findings which we describe and discuss.
Real-Time Equality-Constrained Hybrid State Estimation in Complex Variables
Venue: arXiv cs.SY 2018
Year: 2018

The hybrid power system state estimation problem requires computing the state of the power network using data from both legacy and phasor measurements. Recent research has shown that the normal equations approach in complex variables is computationally advantageous, particularly in the presence of phasor measurement values, and that its software implementation is best suited to modern processors that employ single instruction multiple data (SIMD) processor extensions. The complex normal equations approach is however not ideal for handling zero injection measurements, as it requires their modeling as pseudo-measurements with high weights. This paper employs Wirtinger calculus for extending the complex normal equations approach to include equality constraints, and contrasts it with two previously published implementations: the normal equations approach in complex variables and the hybrid equality constrained state estimator in real variables. Numerical results are reported on transmission networks having up to 9241 nodes; they show that the complex variable equality constrained hybrid state estimator exhibits superior performance as compared to the above two techniques in terms of both computational time and accuracy. Moreover, the execution time on the largest network is less than 300 ms, which makes the proposed implementation commensurate with the requirements of real-time applications.
Engaging Millennials into Learning Formal Methods
Venue: arXiv cs.SE 2018
Year: 2018

This paper summarizes our experience in teaching courses on formal methods (FM) to Computer Science (CS) and Software Engineering (SE) students at various universities around the world, including University of Madeira (UMa) in Portugal, Pontificia Universidad Javeriana (PUJ) and University of Los Andes (Uniandes) in Colombia, Carnegie Mellon University (CMU) in the USA, and at Innopolis University (INNO) in the Russian Federation. We report challenges faced during the past 10 to 15 years to teach FM to millennials undergradu- ate and graduate students and describe how we have coped with those challenges. We formulate a characterization of millennials, based on our experience, and show how this characterization has shaped our decisions in terms of course structure and content. We show how these decisions are reflected on the current structure of the MSS (Models of Software Systems) course that currently runs as part of the MSIT-SE (Master of Science in Information Technology - Software Engineering) programme offered at INNO. We have conducted two surveys among students, the first one at CMU and the second one at INNO that we have used to document and justify our decisions. The first survey is about the choice of Event-B as mathematical formalism and the second one is about the organization of teams of students within the classroom to work on software projects based on Event-B.
Application of Correlation Indices on Intrusion Detection Systems: Protecting the Power Grid Against Coordinated Attacks
Venue: arXiv cs.CR 2018
Year: 2018

The future power grid will be characterized by the pervasive use of heterogeneous and non-proprietary information and communication technology, which exposes the power grid to a broad scope of cyber-attacks. In particular, Monitoring-Control Attacks (MCA) --i.e., attacks in which adversaries manipulate control decisions by fabricating measurement signals in the feedback loop-- are highly threatening. This is because, MCAs are (i) more likely to happen with greater attack surface and lower cost, (ii) difficult to detect by hiding in measurement signals, and (iii) capable of inflicting severe consequences by coordinating attack resources. To defend against MCAs, we have developed a semantic analysis framework for Intrusion Detection Systems (IDS) in power grids. The framework consists of two parts running in parallel: a Correlation Index Generator (CIG), which indexes correlated MCAs, and a Correlation Knowledge-Base~(CKB), which is updated aperiodically with attacks' Correlation Indices (CI). The framework has the advantage of detecting MCAs and estimating attack consequences with promising runtime and detection accuracy. To evaluate the performance of the framework, we computed its false alarm rates under different attack scenarios.
On closeness to k-wise uniformity
Venue: arXiv cs.DS 2018
Year: 2018

A probability distribution over {-1, 1}^n is (eps, k)-wise uniform if, roughly, it is eps-close to the uniform distribution when restricted to any k coordinates. We consider the problem of how far an (eps, k)-wise uniform distribution can be from any globally k-wise uniform distribution. We show that every (eps, k)-wise uniform distribution is O(n^(k/2) eps)-close to a k-wise uniform distribution in total variation distance. In addition, we show that this bound is optimal for all even k: we find an (eps, k)-wise uniform distribution that is Omega(n^(k/2) eps)-far from any k-wise uniform distribution in total variation distance. For k = 1, we get a better upper bound of O(eps), which is also optimal. One application of our closeness result is to the sample complexity of testing whether a distribution is k-wise uniform or delta-far from k-wise uniform. We give an upper bound of O(n^k/delta^2) (or O(log n/delta^2) when k = 1) on the required samples. We show an improved upper bound of O~(n^(k/2)/delta^2) for the special case of testing fully uniform vs. delta-far from k-wise uniform. Finally, we complement this with a matching lower bound of Omega(n/delta^2) when k = 2. Our results improve upon the best known bounds from [AAK+07], and have simpler proofs.
EREL Selection using Morphological Relation
Venue: arXiv cs.CV 2018
Year: 2018

This work concentrates on Extremal Regions of Extremum Level (EREL) selection. EREL is a recently proposed feature detector aiming at detecting regions from a set of extremal regions. This is a branching problem derived from segmentation of arterial wall boundaries from Intravascular Ultrasound (IVUS) images. For each IVUS frame, a set of EREL regions is generated to describe the luminal area of human coronary. Each EREL is then fitted by an ellipse to represent the luminal border. The goal is to assign the most appropriate EREL as the lumen. In this work, EREL selection carries out in two rounds. In the first round, the pattern in a set of EREL regions is analyzed and used to generate an approximate luminal region. Then, the two-dimensional (2D) correlation coefficients are computed between this approximate region and each EREL to keep the ones with tightest relevance. In the second round, a compactness measure is calculated for each EREL and its fitted ellipse to guarantee that the resulting EREL has not affected by the common artifacts such as bifurcations, shadows, and side branches. We evaluated the selected ERELs in terms of Hausdorff Distance (HD) and Jaccard Measure (JM) on the train and test set of a publicly available dataset. The results show that our selection strategy outperforms the current state-of-the-art.
Cross-Lingual Task-Specific Representation Learning for Text Classification in Resource Poor Languages
Venue: arXiv cs.CL 2018
Year: 2018

Neural network models have shown promising results for text classification. However, these solutions are limited by their dependence on the availability of annotated data. The prospect of leveraging resource-rich languages to enhance the text classification of resource-poor languages is fascinating. The performance on resource-poor languages can significantly improve if the resource availability constraints can be offset. To this end, we present a twin Bidirectional Long Short Term Memory (Bi-LSTM) network with shared parameters consolidated by a contrastive loss function (based on a similarity metric). The model learns the representation of resource-poor and resource-rich sentences in a common space by using the similarity between their assigned annotation tags. Hence, the model projects sentences with similar tags closer and those with different tags farther from each other. We evaluated our model on the classification tasks of sentiment analysis and emoji prediction for resource-poor languages - Hindi and Telugu and resource-rich languages - English and Spanish. Our model significantly outperforms the state-of-the-art approaches in both the tasks across all metrics.
ML + FV = $\heartsuit$? A Survey on the Application of Machine Learning to Formal Verification
Venue: arXiv cs.SE 2018
Year: 2018

Formal Verification (FV) and Machine Learning (ML) can seem incompatible due to their opposite mathematical foundations and their use in real-life problems: FV mostly relies on discrete mathematics and aims at ensuring correctness; ML often relies on probabilistic models and consists of learning patterns from training data. In this paper, we postulate that they are complementary in practice, and explore how ML helps FV in its classical approaches: static analysis, model-checking, theorem-proving, and SAT solving. We draw a landscape of the current practice and catalog some of the most prominent uses of ML inside FV tools, thus offering a new perspective on FV techniques that can help researchers and practitioners to better locate the possible synergies. We discuss lessons learned from our work, point to possible improvements and offer visions for the future of the domain in the light of the science of software and systems modeling.
VoxelAtlasGAN: 3D Left Ventricle Segmentation on Echocardiography with Atlas Guided Generation and Voxel-to-voxel Discrimination
Venue: arXiv cs.CV 2018
Year: 2018

3D left ventricle (LV) segmentation on echocardiography is very important for diagnosis and treatment of cardiac disease. It is not only because of that echocardiography is a real-time imaging technology and widespread in clinical application, but also because of that LV segmentation on 3D echocardiography can provide more full volume information of heart than LV segmentation on 2D echocardiography. However, 3D LV segmentation on echocardiography is still an open and challenging task owing to the lower contrast, higher noise and data dimensionality, limited annotation of 3D echocardiography. In this paper, we proposed a novel real-time framework, i.e., VoxelAtlasGAN, for 3D LV segmentation on 3D echocardiography. This framework has three contributions: 1) It is based on voxel-to-voxel conditional generative adversarial nets (cGAN). For the first time, cGAN is used for 3D LV segmentation on echocardiography. And cGAN advantageously fuses substantial 3D spatial context information from 3D echocardiography by self-learning structured loss; 2) For the first time, it embeds the atlas into an end-to-end optimization framework, which uses 3D LV atlas as a powerful prior knowledge to improve the inference speed, address the lower contrast and the limited annotation problems of 3D echocardiography; 3) It combines traditional discrimination loss and the new proposed consistent constraint, which further improves the generalization of the proposed framework. VoxelAtlasGAN was validated on 60 subjects on 3D echocardiography and it achieved satisfactory segmentation results and high inference speed. The mean surface distance is 1.85 mm, the mean hausdorff surface distance is 7.26 mm, mean dice is 0.953, the correlation of EF is 0.918, and the mean inference speed is 0.1s. These results have demonstrated that our proposed method has great potential for clinical application
Directional Spatial Channel Estimation For Massive FD-MIMO in Next Generation 5G Networks
Venue: arXiv cs.IT 2018
Year: 2018

Full-dimensional (FD) channel state information at transmitter (CSIT) has always been a major limitation of the spectral efficiency of cellular multi-input multi-output (MIMO) networks. This letter proposes an FD-directional spatial channel estimation algorithm for frequency division duplex massive FD-MIMO systems. The proposed algorithm uses the statistical spatial correlation between the uplink (UL) and downlink (DL) channels of each user equipment. It spatially decomposes the UL channel into azimuthal and elevation dimensions to estimate the array principal receive responses. An FD spatial rotation matrix is constructed to estimate the corresponding transmit responses of the DL channel, in terms of the frequency band gap between the UL and DL channels. The proposed algorithm shows significantly promising performance, approaching the ideal perfect-CSIT case without UL feedback overhead.
LexNLP: Natural language processing and information extraction for legal and regulatory texts
Venue: arXiv cs.CL 2018
Year: 2018

LexNLP is an open source Python package focused on natural language processing and machine learning for legal and regulatory text. The package includes functionality to (i) segment documents, (ii) identify key text such as titles and section headings, (iii) extract over eighteen types of structured information like distances and dates, (iv) extract named entities such as companies and geopolitical entities, (v) transform text into features for model training, and (vi) build unsupervised and supervised models such as word embedding or tagging models. LexNLP includes pre-trained models based on thousands of unit tests drawn from real documents available from the SEC EDGAR database as well as various judicial and regulatory proceedings. LexNLP is designed for use in both academic research and industrial applications, and is distributed at https://github.com/LexPredict/lexpredict-lexnlp.
Grayscale-based Block Scrambling Image Encryption for Social Networking Services
Venue: arXiv cs.CR 2018
Year: 2018

This paper proposes a new block scrambling encryption scheme that enhances the security of encryption-then-compression (EtC) systems for JPEG images, which are used, for example, to securely transmit images through an untrusted channel provider. The proposed method allows the use of a smaller block size and a larger number of blocks than the conventional ones. Moreover, images encrypted using proposed scheme include less color information due to the use of grayscale even when the original image has three color channels. These features enhance security against various attacks such as jigsaw puzzle solver and brute-force attacks. The results of an experiment in which encrypted images were uploaded to and then downloaded from Twitter and Facebook demonstrated the effectiveness of the proposed scheme for EtC systems.
Enhancing PHY Security of MISO NOMA SWIPT Systems With a Practical Non-Linear EH Model
Venue: arXiv cs.IT 2018
Year: 2018

Non-orthogonal multiple-access (NOMA) and simultaneous wireless information and power transfer (SWIPT) are promising techniques to improve spectral efficiency and energy efficiency. However, the security of NOMA SWIPT systems has not received much attention in the literature. In this paper, an artificial noise-aided beamforming design problem is studied to enhance the security of a multiple-input single-output NOMA SWIPT system where a practical non-linear energy harvesting model is adopted. The problem is non-convex and challenging to solve. Two algorithms are proposed to tackle this problem based on semidefinite relaxation (SDR) and successive convex approximation. Simulation results show that a performance gain can be obtained by using NOMA compared to the conventional orthogonal multiple access. It is also shown that the performance of the algorithm using a cost function is better than the algorithm using SDR at the cost of a higher computation complexity.
Know What You Don't Know: Unanswerable Questions for SQuAD
Venue: arXiv cs.CL 2018
Year: 2018

Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in the context. Existing datasets either focus exclusively on answerable questions, or use automatically generated unanswerable questions that are easy to identify. To address these weaknesses, we present SQuAD 2.0, the latest version of the Stanford Question Answering Dataset (SQuAD). SQuAD 2.0 combines existing SQuAD data with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD 2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. SQuAD 2.0 is a challenging natural language understanding task for existing models: a strong neural system that gets 86% F1 on SQuAD 1.1 achieves only 66% F1 on SQuAD 2.0.
A Multimodal Classifier Generative Adversarial Network for Carry and Place Tasks from Ambiguous Language Instructions
Venue: arXiv cs.RO 2018
Year: 2018

This paper focuses on a multimodal language understanding method for carry-and-place tasks with domestic service robots. We address the case of ambiguous instructions, that is, when the target area is not specified. For instance "put away the milk and cereal" is a natural instruction where there is ambiguity regarding the target area, considering environments in daily life. Conventionally, this instruction can be disambiguated from a dialogue system, but at the cost of time and cumbersome interaction. Instead, we propose a multimodal approach, in which the instructions are disambiguated using the robot's state and environment context. We develop the Multi-Modal Classifier Generative Adversarial Network (MMC-GAN) to predict the likelihood of different target areas considering the robot's physical limitation and the target clutter. Our approach, MMC-GAN, significantly improves accuracy compared with baseline methods that use instructions only or simple deep neural networks.