OpenRaiser/Intern-Atlas huggingface dataset:
FuseVis: Interpreting neural networks for image fusion using per-pixel saliency visualization
Venue: arXiv cs.CV 2020
Year: 2020

Image fusion helps in merging two or more images to construct a more informative single fused image. Recently, unsupervised learning based convolutional neural networks (CNN) have been utilized for different types of image fusion tasks such as medical image fusion, infrared-visible image fusion for autonomous driving as well as multi-focus and multi-exposure image fusion for satellite imagery. However, it is challenging to analyze the reliability of these CNNs for the image fusion tasks since no groundtruth is available. This led to the use of a wide variety of model architectures and optimization functions yielding quite different fusion results. Additionally, due to the highly opaque nature of such neural networks, it is difficult to explain the internal mechanics behind its fusion results. To overcome these challenges, we present a novel real-time visualization tool, named FuseVis, with which the end-user can compute per-pixel saliency maps that examine the influence of the input image pixels on each pixel of the fused image. We trained several image fusion based CNNs on medical image pairs and then using our FuseVis tool, we performed case studies on a specific clinical application by interpreting the saliency maps from each of the fusion methods. We specifically visualized the relative influence of each input image on the predictions of the fused image and showed that some of the evaluated image fusion methods are better suited for the specific clinical application. To the best of our knowledge, currently, there is no approach for visual analysis of neural networks for image fusion. Therefore, this work opens up a new research direction to improve the interpretability of deep fusion networks. The FuseVis tool can also be adapted in other deep neural network based image processing applications to make them interpretable.
Next Generation Opportunistic Networking in Beyond 5G Networks
Venue: arXiv cs.NI 2020
Year: 2020

Beyond 5G networks are expected to support massive traffic through decentralized solutions and advanced networking mechanisms. This paper aims at contributing towards this vision through the integration of device-centric wireless networks, including Device-to-Device (D2D) communications, and the Next Generation of Opportunistic networking (NGO). This integration offers multiple communication modes such as opportunistic cellular and opportunistic D2D-aided communications. Previous studies have demonstrated the potential and benefits of this integration in terms of energy efficiency, spectral efficiency and traffic offloading. We propose an integration of device-centric wireless networks and NGO that is not driven by a precise knowledge of the presence of the links. The proposed technique utilizes a novel concept of graph to model the evolution of the networking conditions and network connectivity. Uncertainties and future conditions are included in the proposed graph model through anticipatory mobile networking to estimate the transmission energy cost of the different communication modes. Based on these estimates, the devices schedule their transmissions using the most efficient communication mode. These decisions are later revisited in real-time using more precise knowledge about the network state. The conducted evaluation shows that the proposed technique significantly reduces the energy consumption (from 60% to 90% depending on the scenario) compared to traditional single-hop cellular communications and performs closely to an ideal oracle based system with full knowledge of present and future events. The transmission and computational overheads of the proposed technique show small impact on such energy gains.
Beyond kinetic harm and towards a dynamic conceptualization of cyberterrorism
Venue: arXiv cs.CY 2020
Year: 2020

After more than two decades of discussion, the concept of cyberterrorism remains plagued by confusion. This article presents the result of an integrative review which maps the development of the term and situates the epistemic communities that have shaped the debate. After critically assessing existing accounts and highlighting the key ethical, social, and legal dimensions at stake in preventing cyberterrorist attacks, it calls for a more dynamic conceptualization that views cyberterrorism as more abstract, difficult to predict, and hard to isolate; and which embraces a different conception of sufficient harm. In concluding it proposes a novel definition of cyberterrorism, intended to catalyse a new research programme, and sketches a roadmap for further research.
Resolving Implicit Coordination in Multi-Agent Deep Reinforcement Learning with Deep Q-Networks & Game Theory
Venue: arXiv cs.MA 2020
Year: 2020

We address two major challenges of implicit coordination in multi-agent deep reinforcement learning: non-stationarity and exponential growth of state-action space, by combining Deep-Q Networks for policy learning with Nash equilibrium for action selection. Q-values proxy as payoffs in Nash settings, and mutual best responses define joint action selection. Coordination is implicit because multiple/no Nash equilibria are resolved deterministically. We demonstrate that knowledge of game type leads to an assumption of mirrored best responses and faster convergence than Nash-Q. Specifically, the Friend-or-Foe algorithm demonstrates signs of convergence to a Set Controller which jointly chooses actions for two agents. This encouraging given the highly unstable nature of decentralized coordination over joint actions. Inspired by the dueling network architecture, which decouples the Q-function into state and advantage streams, as well as residual networks, we learn both a single and joint agent representation, and merge them via element-wise addition. This simplifies coordination by recasting it is as learning a residual function. We also draw high level comparative insights on key MADRL and game theoretic variables: competitive vs. cooperative, asynchronous vs. parallel learning, greedy versus socially optimal Nash equilibria tie breaking, and strategies for the no Nash equilibrium case. We evaluate on 3 custom environments written in Python using OpenAI Gym: a Predator Prey environment, an alternating Warehouse environment, and a Synchronization environment. Each environment requires successively more coordination to achieve positive rewards.
On the Generation of Disassembly Ground Truth and the Evaluation of Disassemblers
Venue: arXiv cs.PL 2020
Year: 2020

When a software transformation or software security task needs to analyze a given program binary, the first step is often disassembly. Since many modern disassemblers have become highly accurate on many binaries, we believe reliable disassembler benchmarking requires standardizing the set of binaries used and the disassembly ground truth about these binaries. This paper presents (i) a first version of our work-in-progress disassembly benchmark suite, which comprises 879 binaries from diverse projects compiled with multiple compilers and optimization settings, and (ii) a novel disassembly ground truth generator leveraging the notion of "listing files", which has broad support by Clang, GCC, ICC, and MSVC. In additional, it presents our evaluation of four prominent open-source disassemblers using this benchmark suite and a custom evaluation system. Our entire system and all generated data are maintained openly on GitHub to encourage community adoption.
MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
Venue: arXiv cs.CL 2020
Year: 2020

We introduce a new dataset, MELINDA, for Multimodal biomEdicaL experImeNt methoD clAssification. The dataset is collected in a fully automated distant supervision manner, where the labels are obtained from an existing curated database, and the actual contents are extracted from papers associated with each of the records in the database. We benchmark various state-of-the-art NLP and computer vision models, including unimodal models which only take either caption texts or images as inputs, and multimodal models. Extensive experiments and analysis show that multimodal models, despite outperforming unimodal ones, still need improvements especially on a less-supervised way of grounding visual concepts with languages, and better transferability to low resource domains. We release our dataset and the benchmarks to facilitate future research in multimodal learning, especially to motivate targeted improvements for applications in scientific domains.
Data optimization for large batch distributed training of deep neural networks
Venue: arXiv cs.LG 2020
Year: 2020

Distributed training in deep learning (DL) is common practice as data and models grow. The current practice for distributed training of deep neural networks faces the challenges of communication bottlenecks when operating at scale, and model accuracy deterioration with an increase in global batch size. Present solutions focus on improving message exchange efficiency as well as implementing techniques to tweak batch sizes and models in the training process. The loss of training accuracy typically happens because the loss function gets trapped in a local minima. We observe that the loss landscape minimization is shaped by both the model and training data and propose a data optimization approach that utilizes machine learning to implicitly smooth out the loss landscape resulting in fewer local minima. Our approach filters out data points which are less important to feature learning, enabling us to speed up the training of models on larger batch sizes to improved accuracy.
Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization
Venue: arXiv cs.CL 2020
Year: 2020

Information retrieval (IR) for precision medicine (PM) often involves looking for multiple pieces of evidence that characterize a patient case. This typically includes at least the name of a condition and a genetic variation that applies to the patient. Other factors such as demographic attributes, comorbidities, and social determinants may also be pertinent. As such, the retrieval problem is often formulated as ad hoc search but with multiple facets (e.g., disease, mutation) that may need to be incorporated. In this paper, we present a document reranking approach that combines neural query-document matching and text summarization toward such retrieval scenarios. Our architecture builds on the basic BERT model with three specific components for reranking: (a). document-query matching (b). keyword extraction and (c). facet-conditioned abstractive summarization. The outcomes of (b) and (c) are used to essentially transform a candidate document into a concise summary that can be compared with the query at hand to compute a relevance score. Component (a) directly generates a matching score of a candidate document for a query. The full architecture benefits from the complementary potential of document-query matching and the novel document transformation approach based on summarization along PM facets. Evaluations using NIST's TREC-PM track datasets (2017--2019) show that our model achieves state-of-the-art performance. To foster reproducibility, our code is made available here: https://github.com/bionlproc/text-summ-for-doc-retrieval.
Semi-Global Shape-aware Network
Venue: arXiv cs.CV 2020
Year: 2020

Non-local operations are usually used to capture long-range dependencies via aggregating global context to each position recently. However, most of the methods cannot preserve object shapes since they only focus on feature similarity but ignore proximity between central and other positions for capturing long-range dependencies, while shape-awareness is beneficial to many computer vision tasks. In this paper, we propose a Semi-Global Shape-aware Network (SGSNet) considering both feature similarity and proximity for preserving object shapes when modeling long-range dependencies. A hierarchical way is taken to aggregate global context. In the first level, each position in the whole feature map only aggregates contextual information in vertical and horizontal directions according to both similarity and proximity. And then the result is input into the second level to do the same operations. By this hierarchical way, each central position gains supports from all other positions, and the combination of similarity and proximity makes each position gain supports mostly from the same semantic object. Moreover, we also propose a linear time algorithm for the aggregation of contextual information, where each of rows and columns in the feature map is treated as a binary tree to reduce similarity computation cost. Experiments on semantic segmentation and image retrieval show that adding SGSNet to existing networks gains solid improvements on both accuracy and efficiency.
Formalization of PAL$\cdot$S5 in Proof Assistant
Venue: arXiv cs.LO 2020
Year: 2020

As an experiment to the application of proof assistant for logic research, we formalize the model and proof system for multi-agent modal logic S5 with PAL-style dynamic modality in Lean theorem prover. We provide a formal proof for the reduction axiom of public announcement, and the soundness and completeness of modal logic S5, which can be typechecked with Lean 3.19.0. The complete proof is now available at Github.
MASKER: Masked Keyword Regularization for Reliable Text Classification
Venue: arXiv cs.LG 2020
Year: 2020

Pre-trained language models have achieved state-of-the-art accuracies on various text classification tasks, e.g., sentiment analysis, natural language inference, and semantic textual similarity. However, the reliability of the fine-tuned text classifiers is an often underlooked performance criterion. For instance, one may desire a model that can detect out-of-distribution (OOD) samples (drawn far from training distribution) or be robust against domain shifts. We claim that one central obstacle to the reliability is the over-reliance of the model on a limited number of keywords, instead of looking at the whole context. In particular, we find that (a) OOD samples often contain in-distribution keywords, while (b) cross-domain samples may not always contain keywords; over-relying on the keywords can be problematic for both cases. In light of this observation, we propose a simple yet effective fine-tuning method, coined masked keyword regularization (MASKER), that facilitates context-based prediction. MASKER regularizes the model to reconstruct the keywords from the rest of the words and make low-confidence predictions without enough context. When applied to various pre-trained language models (e.g., BERT, RoBERTa, and ALBERT), we demonstrate that MASKER improves OOD detection and cross-domain generalization without degrading classification accuracy. Code is available at https://github.com/alinlab/MASKER.
Helping Reduce Environmental Impact of Aviation with Machine Learning
Venue: arXiv cs.AI 2020
Year: 2020

Commercial aviation is one of the biggest contributors towards climate change. We propose to reduce environmental impact of aviation by considering solutions that would reduce the flight time. Specifically, we first consider improving winds aloft forecast so that flight planners could use better information to find routes that are efficient. Secondly, we propose an aircraft routing method that seeks to find the fastest route to the destination by considering uncertainty in the wind forecasts and then optimally trading-off between exploration and exploitation.
Exploiting Learnable Joint Groups for Hand Pose Estimation
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose to estimate 3D hand pose by recovering the 3D coordinates of joints in a group-wise manner, where less-related joints are automatically categorized into different groups and exhibit different features. This is different from the previous methods where all the joints are considered holistically and share the same feature. The benefits of our method are illustrated by the principle of multi-task learning (MTL), i.e., by separating less-related joints into different groups (as different tasks), our method learns different features for each of them, therefore efficiently avoids the negative transfer (among less related tasks/groups of joints). The key of our method is a novel binary selector that automatically selects related joints into the same group. We implement such a selector with binary values stochastically sampled from a Concrete distribution, which is constructed using Gumbel softmax on trainable parameters. This enables us to preserve the differentiable property of the whole network. We further exploit features from those less-related groups by carrying out an additional feature fusing scheme among them, to learn more discriminative features. This is realized by implementing multiple 1x1 convolutions on the concatenated features, where each joint group contains a unique 1x1 convolution for feature fusion. The detailed ablation analysis and the extensive experiments on several benchmark datasets demonstrate the promising performance of the proposed method over the state-of-the-art (SOTA) methods. Besides, our method achieves top-1 among all the methods that do not exploit the dense 3D shape labels on the most recently released FreiHAND competition at the submission date. The source code and models are available at https://github.com/ moranli-aca/LearnableGroups-Hand.
PURE: A Framework for Analyzing Proximity-based Contact Tracing Protocols
Venue: arXiv cs.CY 2020
Year: 2020

Many proximity-based tracing (PCT) protocols have been proposed and deployed to combat the spreading of COVID-19. In this paper, we take a systematic approach to analyze PCT protocols. We identify a list of desired properties of a contact tracing design from the four aspects of Privacy, Utility, Resiliency, and Efficiency (PURE). We also identify two main design choices for PCT protocols: what information patients report to the server, and which party performs the matching. These two choices determine most of the PURE properties and enable us to conduct a comprehensive analysis and comparison of the existing protocols.
Online Shielding for Stochastic Systems
Venue: arXiv cs.LO 2020
Year: 2020

In this paper, we propose a method to develop trustworthy reinforcement learning systems. To ensure safety especially during exploration, we automatically synthesize a correct-by-construction runtime enforcer, called a shield, that blocks all actions that are unsafe with respect to a temporal logic specification from the agent. Our main contribution is a new synthesis algorithm for computing the shield online. Existing offline shielding approaches compute exhaustively the safety of all states-action combinations ahead-of-time, resulting in huge offline computation times, large memory consumption, and significant delays at run-time due to the look-ups in a huge database. The intuition behind online shielding is to compute during run-time the set of all states that could be reached in the near future. For each of these states, the safety of all available actions is analysed and used for shielding as soon as one of the considered states is reached. Our proposed method is general and can be applied to a wide range of planning problems with stochastic behavior. For our evaluation, we selected a 2-player version of the classical computer game SNAKE. The game requires fast decisions and the multiplayer setting induces a large state space, computationally expensive to analyze exhaustively. The safety objective of collision avoidance is easily transferable to a variety of planning tasks.
A framework to semantify BPMN models using DEMO business transaction pattern
Venue: arXiv cs.SE 2020
Year: 2020

BPMN is a specification language widely used by industry and researchers for business process modeling and execution. It defines clearly how to articulate its concepts, but do not provide mechanism to represent the semantics of the produced models. This paper addresses the problem of how to improve the expressiveness of BPMN models, proposing a definition for the semantics of a business process within a BPMN model, and improving the completeness of the models in a systematic manner, so that models can describe far more situations with few extra managed complexity. We conceive a framework based on the business transaction patterns available in the enterprise ontology body of knowledge to prescribe the foundations of semantic BPMN models. A tool has been developed to automate the framework. Then, two industrial proof-of-concepts are used to measure its coverage, both positive and negative, and to argue about our proposal's usefulness. After that, the proposal is compared with others using a systematic literature review. A full BPMN pattern is proposed encompassing the happy flow, the declinations, the rejections and the revocations, without adding any new element to the BPMN specification. A software tool has been developed, and made publicly available, to support the automatic generation of the BPMN models from the proposed patterns. Our semantified BPMN pattern allowed the identification of a large amount of implicit, and other not implemented, situations in both proof-of-concepts. It is concluded that the usage of a semantic solution, grounded on a sound pattern, allows the systematic enrichment of the BPMN models with a bounded effort. Moreover, to simplify the BPMN executable models' implementation, its elements could be classified as implicit, explicit, or not implemented yet. Finally, related work indicates that this work is demanded, but no full solutions are available.
Concurrent Encryption and Authentication for Wireless Networks using Compressed Sensing
Venue: arXiv cs.CR 2020
Year: 2020

Authentication and encryption are traditionally treated as two separate processes in wireless networks, this paper integrates user authentication into the process of solving eavesdropping attacks. A compressed sensing (CS)-based framework is proposed which manipulates the measurement matrix of CS to safeguard secure computationally. The framework is also capable of continuous authentication and transmission error correction and is robust to data loss. In detail, this paper first proposes an algorithm to generate a 2D key which depends on the physical property of communication channels. The 2D key is further used to generate authentication information and signal structure as well as encrypt original data. Then an encrypted message which contains both data and authentication information is formed for anonymous transmission. The legal receiver can split authentication information and data, and performing a data loss-robust and transmission error-robust authentication and recovery strategy. The framework is evaluated quantitatively using Monte Carlo simulation with simulated sparse signal. The secure transmission performance and authentication performance as well as data loss robustness are investigated. This framework provides an integrated security solution that efficiently safeguards the confidential, privacy and robust communication in cyber-physical systems, especially in resource-limited and safety-critical wireless networks.