OpenRaiser/Intern-Atlas huggingface dataset:
Multimodal Social Media Analysis for Gang Violence Prevention
Venue: arXiv cs.LG 2018
Year: 2018

Gang violence is a severe issue in major cities across the U.S. and recent studies [Patton et al. 2017] have found evidence of social media communications that can be linked to such violence in communities with high rates of exposure to gang activity. In this paper we partnered computer scientists with social work researchers, who have domain expertise in gang violence, to analyze how public tweets with images posted by youth who mention gang associations on Twitter can be leveraged to automatically detect psychosocial factors and conditions that could potentially assist social workers and violence outreach workers in prevention and early intervention programs. To this end, we developed a rigorous methodology for collecting and annotating tweets. We gathered 1,851 tweets and accompanying annotations related to visual concepts and the psychosocial codes: aggression, loss, and substance use. These codes are relevant to social work interventions, as they represent possible pathways to violence on social media. We compare various methods for classifying tweets into these three classes, using only the text of the tweet, only the image of the tweet, or both modalities as input to the classifier. In particular, we analyze the usefulness of mid-level visual concepts and the role of different modalities for this tweet classification task. Our experiments show that individually, text information dominates classification performance of the loss class, while image information dominates the aggression and substance use classes. Our multimodal approach provides a very promising improvement (18% relative in mean average precision) over the best single modality approach. Finally, we also illustrate the complexity of understanding social media data and elaborate on open challenges.
A Multi-Level Deep Ensemble Model for Skin Lesion Classification in Dermoscopy Images
Venue: arXiv cs.CV 2018
Year: 2018

A multi-level deep ensemble (MLDE) model that can be trained in an 'end to end' manner is proposed for skin lesion classification in dermoscopy images. In this model, four pre-trained ResNet-50 networks are used to characterize the multiscale information of skin lesions and are combined by using an adaptive weighting scheme that can be learned during the error back propagation. The proposed MLDE model achieved an average AUC value of 86.5% on the ISIC-skin 2018 official validation dataset, which is substantially higher than the average AUC values achieved by each of four ResNet-50 networks.
Implementing Neural Turing Machines
Venue: arXiv cs.LG 2018
Year: 2018

Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.
Multisensor Management Algorithm for Airborne Sensors Using Frank-Wolfe Method
Venue: arXiv cs.MA 2018
Year: 2018

This study proposes an airborne multisensor management algorithm for target tracking, taking each of multiple unmanned aircraft as a sensor. The purpose of the algorithm is to determine the configuration of the sensor deployment and to guide the mobile sensors to track moving targets in an optimal way. The cost function as a performance metric is defined as a combination of the D-optimality criterion of the Fisher information matrix. The convexity of the cost function is proved and the optimal solution for deployment and guidance problem is derived by the Frank-Wolfe method, also known as the conditional gradient descent method. An intuitive optimal approach to deal with the problem is to direct the sensor to the optimal position obtained by solving a nonlinear optimization problem. On the other hand, the proposed method takes the conditional gradient of the cost function as the command to the deployed sensors, so that the sensors are guaranteed to be in the feasible points and they achieve the current best performance. Simulation results demonstrate that the proposed algorithm provides better performance than directing each sensor to its optimal position.
MVDepthNet: Real-time Multiview Depth Estimation Neural Network
Venue: arXiv cs.RO 2018
Year: 2018

Although deep neural networks have been widely applied to computer vision problems, extending them into multiview depth estimation is non-trivial. In this paper, we present MVDepthNet, a convolutional network to solve the depth estimation problem given several image-pose pairs from a localized monocular camera in neighbor viewpoints. Multiview observations are encoded in a cost volume and then combined with the reference image to estimate the depth map using an encoder-decoder network. By encoding the information from multiview observations into the cost volume, our method achieves real-time performance and the flexibility of traditional methods that can be applied regardless of the camera intrinsic parameters and the number of images. Geometric data augmentation is used to train MVDepthNet. We further apply MVDepthNet in a monocular dense mapping system that continuously estimates depth maps using a single localized moving camera. Experiments show that our method can generate depth maps efficiently and precisely.
A refined mean field approximation of synchronous discrete-time population models
Venue: arXiv cs.PF 2018
Year: 2018

Mean field approximation is a popular method to study the behaviour of stochastic models composed of a large number of interacting objects. When the objects are asynchronous, the mean field approximation of a population model can be expressed as an ordinary differential equation. When the objects are (clock-) synchronous the mean field approximation is a discrete time dynamical system. We focus on the latter.We study the accuracy of mean field approximation when this approximation is a discrete-time dynamical system. We extend a result that was shown for the continuous time case and we prove that expected performance indicators estimated by mean field approximation are $O(1/N)$-accurate. We provide simple expressions to effectively compute the asymptotic error of mean field approximation, for finite time-horizon and steady-state, and we use this computed error to propose what we call a \emph{refined} mean field approximation. We show, by using a few numerical examples, that this technique improves the quality of approximation compared to the classical mean field approximation, especially for relatively small population sizes.
Automating Verification of State Machines with Reactive Designs and Isabelle/UTP
Venue: arXiv cs.LO 2018
Year: 2018

State-machine based notations are ubiquitous in the description of component systems, particularly in the robotic domain. To ensure these systems are safe and predictable, formal verification techniques are important, and can be cost-effective if they are both automated and scalable. In this paper, we present a verification approach for a diagrammatic state machine language that utilises theorem proving and a denotational semantics based on Unifying Theories of Programming (UTP). We provide the necessary theory to underpin state machines (including induction theorems for iterative processes), mechanise an action language for states and transitions, and use these to formalise the semantics. We then describe the verification approach, which supports infinite state systems, and exemplify it with a fully automated deadlock-freedom check. The work has been mechanised in our proof tool, Isabelle/UTP, and so also illustrates the use of UTP to build practical verification tools.
Recent Advances in Convolutional Neural Network Acceleration
Venue: arXiv cs.LG 2018
Year: 2018

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed during training and inference. However, as the dimension of data becomes higher and the CNN architecture becomes more complicated, the end-to-end approach or the combined manner of CNN is computationally intensive, which becomes limitation to CNN's further implementation. Therefore, it is necessary and urgent to implement CNN in a faster way. In this paper, we first summarize the acceleration methods that contribute to but not limited to CNN by reviewing a broad variety of research papers. We propose a taxonomy in terms of three levels, i.e.~structure level, algorithm level, and implementation level, for acceleration methods. We also analyze the acceleration methods in terms of CNN architecture compression, algorithm optimization, and hardware-based improvement. At last, we give a discussion on different perspectives of these acceleration and optimization methods within each level. The discussion shows that the methods in each level still have large exploration space. By incorporating such a wide range of disciplines, we expect to provide a comprehensive reference for researchers who are interested in CNN acceleration.
3D Convolutional Neural Networks for Tumor Segmentation using Long-range 2D Context
Venue: arXiv cs.CV 2018
Year: 2018

We present an efficient deep learning approach for the challenging task of tumor segmentation in multisequence MR images. In recent years, Convolutional Neural Networks (CNN) have achieved state-of-the-art performances in a large variety of recognition tasks in medical imaging. Because of the considerable computational cost of CNNs, large volumes such as MRI are typically processed by subvolumes, for instance slices (axial, coronal, sagittal) or small 3D patches. In this paper we introduce a CNN-based model which efficiently combines the advantages of the short-range 3D context and the long-range 2D context. To overcome the limitations of specific choices of neural network architectures, we also propose to merge outputs of several cascaded 2D-3D models by a voxelwise voting strategy. Furthermore, we propose a network architecture in which the different MR sequences are processed by separate subnetworks in order to be more robust to the problem of missing MR sequences. Finally, a simple and efficient algorithm for training large CNN models is introduced. We evaluate our method on the public benchmark of the BRATS 2017 challenge on the task of multiclass segmentation of malignant brain tumors. Our method achieves good performances and produces accurate segmentations with median Dice scores of 0.918 (whole tumor), 0.883 (tumor core) and 0.854 (enhancing core). Our approach can be naturally applied to various tasks involving segmentation of lesions or organs.
A Frequency-Domain Characterization of Optimal Error Covariance for the Kalman-Bucy Filter
Venue: arXiv cs.SY 2018
Year: 2018

In this paper, we discover that the trace of the division of the optimal output estimation error covariance over the noise covariance attained by the Kalman-Bucy filter can be explicitly expressed in terms of the plant dynamics and noise statistics in a frequency-domain integral characterization. Towards this end, we examine the algebraic Riccati equation associated with Kalman-Bucy filtering using analytic function theory and relate it to the Bode integral. Our approach features an alternative, frequency-domain framework for analyzing algebraic Riccati equations and reduces to various existing related results.
Differential quadrature element for second strain gradient beam theory
Venue: arXiv cs.CE 2018
Year: 2018

In this paper, first we present the variational formulation for a second strain gradient Euler-Bernoulli beam theory for the first time. The governing equation and associated classical and non-classical boundary conditions are obtained. Later, we propose a novel and efficient differential quadrature element based on Lagrange interpolation to solve the eight order partial differential equation associated with the second strain gradient Euler-Bernoulli beam theory. The second strain gradient theory has displacement, slope, curvature and triple displacement derivative as degrees of freedom. A generalize scheme is proposed herein to implement these multi-degrees of freedom in a simplified and efficient way. The proposed element is based on the strong form of governing equation and has displacement as the only degree of freedom in the domain, whereas, at the boundaries it has displacement, slope, curvature and triple derivative of displacement. A novel DQ framework is presented to incorporate the classical and non-classical boundary conditions by modifying the conventional weighting coefficients. The accuracy and efficiency of the proposed element is demonstrated through numerical examples on static, free vibration and stability analysis of second strain gradient elastic beams for different boundary conditions and intrinsic length scale values.
Which US and European Higher Education Institutions are visible in ResearchGate and what affects their RG Score?
Venue: arXiv cs.DL 2018
Year: 2018

While ResearchGate has become the most popular academic social networking site in terms of regular users, not all institutions have joined and the scores it assigns to academics and institutions are controversial. This paper assesses the presence in ResearchGate of higher education institutions in Europe and the US in 2017, and the extent to which institutional ResearchGate Scores reflect institutional academic impact. Most of the 2258 European and 4355 US higher educational institutions included in the sample had an institutional ResearchGate profile, with near universal coverage for PhD-awarding institutions found in the Web of Science (WoS). For non-PhD awarding institutions that did not publish, size (number of staff members) was most associated with presence in ResearchGate. For PhD-awarding institutions in WoS, presence in RG was strongly related to the number of WoS publications. In conclusion, a) institutional RG scores reflect research volume more than visibility and b) this indicator is highly correlated to the number of WoS publications. Hence, the value of RG Scores for institutional comparisons is limited.
Hybrid Diffusion: Spectral-Temporal Graph Filtering for Manifold Ranking
Venue: arXiv cs.CV 2018
Year: 2018

State of the art image retrieval performance is achieved with CNN features and manifold ranking using a k-NN similarity graph that is pre-computed off-line. The two most successful existing approaches are temporal filtering, where manifold ranking amounts to solving a sparse linear system online, and spectral filtering, where eigen-decomposition of the adjacency matrix is performed off-line and then manifold ranking amounts to dot-product search online. The former suffers from expensive queries and the latter from significant space overhead. Here we introduce a novel, theoretically well-founded hybrid filtering approach allowing full control of the space-time trade-off between these two extremes. Experimentally, we verify that our hybrid method delivers results on par with the state of the art, with lower memory demands compared to spectral filtering approaches and faster compared to temporal filtering.
Constructive Galois Connections
Venue: arXiv cs.PL 2018
Year: 2018

Galois connections are a foundational tool for structuring abstraction in semantics and their use lies at the heart of the theory of abstract interpretation. Yet, mechanization of Galois connections using proof assistants remains limited to restricted modes of use, preventing their general application in mechanized metatheory and certified programming. This paper presents constructive Galois connections, a variant of Galois connections that is effective both on paper and in proof assistants; is complete with respect to a large subset of classical Galois connections; and enables more general reasoning principles, including the "calculational" style advocated by Cousot. To design constructive Galois connections we identify a restricted mode of use of classical ones which is both general and amenable to mechanization in dependently-typed functional programming languages. Crucial to our metatheory is the addition of monadic structure to Galois connections to control a "specification effect." Effectful calculations may reason classically, while pure calculations have extractable computational content. Explicitly moving between the worlds of specification and implementation is enabled by our metatheory. To validate our approach, we provide two case studies in mechanizing existing proofs from the literature: the first uses calculational abstract interpretation to design a static analyzer; the second forms a semantic basis for gradual typing. Both mechanized proofs closely follow their original paper-and-pencil counterparts, employ reasoning principles not captured by previous mechanization approaches, support the extraction of verified algorithms, and are novel.
Influence of Selective Exposure to Viewing Contents Diversity
Venue: arXiv cs.CY 2018
Year: 2018

Personalization, including both self-selected and pre-selected, is inevitable when tremendous amounts of media content are available. Personalization, which is believed to cause people to consume fewer diverse contents, can lead to fragmentation and polarization in society. Therefore, it is important to investigate the diversity of consumed contents over time. In this paper, first, we propose a framework to measure and analyze how the diversity of the consumed contents of users changes over time. In our framework, we introduce a new metric to measure content diversity based on our redefinition of diversity. Then, we investigate the relationship between selective exposure and content diversity changes using our framework and examine what factors encourage people to consume contents that are more diverse. We find that people autonomously consume more diverse contents from a macro-perspective without an external influence, suggesting that people are less likely to be fragmented and polarized, although from a micro-perspective they consume limited contents. We also obtain evidence that users who consume highly ambiguous contents tend to increase the diversity of their consumed contents.
CNN-based Facial Affect Analysis on Mobile Devices
Venue: arXiv cs.HC 2018
Year: 2018

This paper focuses on the design, deployment and evaluation of Convolutional Neural Network (CNN) architectures for facial affect analysis on mobile devices. Unlike traditional CNN approaches, models deployed to mobile devices must minimise storage requirements while retaining high performance. We therefore propose three variants of established CNN architectures and comparatively evaluate them on a large, in-the-wild benchmark dataset of facial images. Our results show that the proposed architectures retain similar performance to the dataset baseline while minimising storage requirements: achieving 58% accuracy for eight-class emotion classification and average RMSE of 0.39 for valence/arousal prediction. To demonstrate the feasibility of deploying these models for real-world applications, we implement a music recommendation interface based on predicted user affect. Although the CNN models were not trained in the context of music recommendation, our case study shows that: (i) the trained models achieve similar prediction performance to the benchmark dataset, and (ii) users tend to positively rate the song recommendations provided by the interface. Average runtime of the deployed models on an iPhone 6S equates to ~45 fps, suggesting that the proposed architectures are also well suited for real-time deployment on video streams.
Fast Vessel Segmentation and Tracking in Ultra High-Frequency Ultrasound Images
Venue: arXiv cs.CV 2018
Year: 2018

Ultra High Frequency Ultrasound (UHFUS) enables the visualization of highly deformable small and medium vessels in the hand. Intricate vessel-based measurements, such as intimal wall thickness and vessel wall compliance, require sub-millimeter vessel tracking between B-scans. Our fast GPU-based approach combines the advantages of local phase analysis, a distance-regularized level set, and an Extended Kalman Filter (EKF), to rapidly segment and track the deforming vessel contour. We validated on 35 UHFUS sequences of vessels in the hand, and we show the transferability of the approach to 5 more diverse datasets acquired by a traditional High Frequency Ultrasound (HFUS) machine. To the best of our knowledge, this is the first algorithm capable of rapidly segmenting and tracking deformable vessel contours in 2D UHFUS images. It is also the fastest and most accurate system for 2D HFUS images.
PCNNA: A Photonic Convolutional Neural Network Accelerator
Venue: arXiv cs.ET 2018
Year: 2018

Convolutional Neural Networks (CNN) have been the centerpiece of many applications including but not limited to computer vision, speech processing, and Natural Language Processing (NLP). However, the computationally expensive convolution operations impose many challenges to the performance and scalability of CNNs. In parallel, photonic systems, which are traditionally employed for data communication, have enjoyed recent popularity for data processing due to their high bandwidth, low power consumption, and reconfigurability. Here we propose a Photonic Convolutional Neural Network Accelerator (PCNNA) as a proof of concept design to speedup the convolution operation for CNNs. Our design is based on the recently introduced silicon photonic microring weight banks, which use broadcast-and-weight protocol to perform Multiply And Accumulate (MAC) operation and move data through layers of a neural network. Here, we aim to exploit the synergy between the inherent parallelism of photonics in the form of Wavelength Division Multiplexing (WDM) and sparsity of connections between input feature maps and kernels in CNNs. While our full system design offers up to more than 3 orders of magnitude speedup in execution time, its optical core potentially offers more than 5 order of magnitude speedup compared to state-of-the-art electronic counterparts.
GPU-based Commonsense Paradigms Reasoning for Real-Time Query Answering and Multimodal Analysis
Venue: arXiv cs.DB 2018
Year: 2018

We utilize commonsense knowledge bases to address the problem of real- time multimodal analysis. In particular, we focus on the problem of multimodal sentiment analysis, which consists in the simultaneous analysis of different modali- ties, e.g., speech and video, for emotion and polarity detection. Our approach takes advantages of the massively parallel processing power of modern GPUs to enhance the performance of feature extraction from different modalities. In addition, in order to ex- tract important textual features from multimodal sources we generate domain-specific graphs based on commonsense knowledge and apply GPU-based graph traversal for fast feature detection. Then, powerful ELM classifiers are applied to build the senti- ment analysis model based on the extracted features. We conduct our experiments on the YouTube dataset and achieve an accuracy of 78% which outperforms all previous systems. In term of processing speed, our method shows improvements of several orders of magnitude for feature extraction compared to CPU-based counterparts.