OpenRaiser/Intern-Atlas huggingface dataset:
COVID-19 Knowledge Graph: Accelerating Information Retrieval and Discovery for Scientific Literature
Venue: arXiv cs.IR 2020
Year: 2020

The coronavirus disease (COVID-19) has claimed the lives of over 350,000 people and infected more than 6 million people worldwide. Several search engines have surfaced to provide researchers with additional tools to find and retrieve information from the rapidly growing corpora on COVID-19. These engines lack extraction and visualization tools necessary to retrieve and interpret complex relations inherent to scientific literature. Moreover, because these engines mainly rely upon semantic information, their ability to capture complex global relationships across documents is limited, which reduces the quality of similarity-based article recommendations for users. In this work, we present the COVID-19 Knowledge Graph (CKG), a heterogeneous graph for extracting and visualizing complex relationships between COVID-19 scientific articles. The CKG combines semantic information with document topological information for the application of similar document retrieval. The CKG is constructed using the latent schema of the data, and then enriched with biomedical entity information extracted from the unstructured text of articles using scalable AWS technologies to form relations in the graph. Finally, we propose a document similarity engine that leverages low-dimensional graph embeddings from the CKG with semantic embeddings for similar article retrieval. Analysis demonstrates the quality of relationships in the CKG and shows that it can be used to uncover meaningful information in COVID-19 scientific articles. The CKG helps power www.cord19.aws and is publicly available.
Detecting Dynamic States of Temporal Networks Using Connection Series Tensors
Venue: arXiv cs.SI 2020
Year: 2020

Many temporal networks exhibit multiple system states, such as weekday and weekend patterns in social contact networks. The detection of such distinct states in temporal network data has recently been explored as it helps reveal underlying dynamical processes. A commonly used method is network aggregation over a time window, which aggregates a subsequence of multiple network snapshots into one static network. This method, however, necessarily discards temporal dynamics within the time window. Here we develop a new method for detecting dynamic states in temporal networks using information regarding the timeline of contacts between each pair of nodes. We apply a similarity measure informed by the techniques of processing time series and community detection to sequentially discompose a given temporal network into multiple dynamic states (including repeated ones). Experiments with empirical temporal network data demonstrated that our method outperformed the conventional approach using simple network aggregation in revealing interpretable system states. In addition, our method allows users to analyze hierarchical temporal structures and to uncover dynamic state at different spatial/temporal resolutions.
Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models
Venue: arXiv cs.LG 2020
Year: 2020

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions ($\geq 1024 \times 1024$). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are $2-3\times$ faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.
Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wild
Venue: arXiv cs.CV 2020
Year: 2020

Bundle adjustment jointly optimizes camera intrinsics and extrinsics and 3D point triangulation to reconstruct a static scene. The triangulation constraint, however, is invalid for moving points captured in multiple unsynchronized videos and bundle adjustment is not designed to estimate the temporal alignment between cameras. We present a spatiotemporal bundle adjustment framework that jointly optimizes four coupled sub-problems: estimating camera intrinsics and extrinsics, triangulating static 3D points, as well as sub-frame temporal alignment between cameras and computing 3D trajectories of dynamic points. Key to our joint optimization is the careful integration of physics-based motion priors within the reconstruction pipeline, validated on a large motion capture corpus of human subjects. We devise an incremental reconstruction and alignment algorithm to strictly enforce the motion prior during the spatiotemporal bundle adjustment. This algorithm is further made more efficient by a divide and conquer scheme while still maintaining high accuracy. We apply this algorithm to reconstruct 3D motion trajectories of human bodies in dynamic events captured by multiple uncalibrated and unsynchronized video cameras in the wild. To make the reconstruction visually more interpretable, we fit a statistical 3D human body model to the asynchronous video streams.Compared to the baseline, the fitting significantly benefits from the proposed spatiotemporal bundle adjustment procedure. Because the videos are aligned with sub-frame precision, we reconstruct 3D motion at much higher temporal resolution than the input videos.
Counting Fish and Dolphins in Sonar Images Using Deep Learning
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning provides the opportunity to improve upon conflicting reports considering the relationship between the Amazon river's fish and dolphin abundance and reduced canopy cover as a result of deforestation. Current methods of fish and dolphin abundance estimates are performed by on-site sampling using visual and capture/release strategies. We propose a novel approach to calculating fish abundance using deep learning for fish and dolphin estimates from sonar images taken from the back of a trolling boat. We consider a data set of 143 images ranging from 0-34 fish, and 0-3 dolphins provided by the Fund Amazonia research group. To overcome the data limitation, we test the capabilities of data augmentation on an unconventional 15/85 training/testing split. Using 20 training images, we simulate a gradient of data up to 25,000 images using augmented backgrounds and randomly placed/rotation cropped fish and dolphin taken from the training set. We then train four multitask network architectures: DenseNet201, InceptionNetV2, Xception, and MobileNetV2 to predict fish and dolphin numbers using two function approximation methods: regression and classification. For regression, Densenet201 performed best for fish and Xception best for dolphin with mean squared errors of 2.11 and 0.133 respectively. For classification, InceptionResNetV2 performed best for fish and MobileNetV2 best for dolphins with a mean error of 2.07 and 0.245 respectively. Considering the 123 testing images, our results show the success of data simulation for limited sonar data sets. We find DenseNet201 is able to identify dolphins after approximately 5000 training images, while fish required the full 25,000. Our method can be used to lower costs and expedite the data analysis of fish and dolphin abundance to real-time along the Amazon river and river systems worldwide.
Open Access all you wanted to know and never dared to ask
Venue: arXiv cs.DL 2020
Year: 2020

This editorial presents the various forms of open access, discusses their pros and cons from the perspective of the Journal of Object Technology and its editors in chiefs, and illustrates how JOT implements a platinum open access model. The regular reader will also notice that this editorial features a new template for the journal that will be used from now on.
On Fast Retrial for Two-Step Random Access in MTC
Venue: arXiv cs.IT 2020
Year: 2020

In machine-type communication (MTC), a group of devices or sensors may need to send their data packets with certain access delay limits for delay-sensitive applications or real-time Internet-of-Things (IoT) applications. In this case, 2-step random access approaches would be preferable to 4-step random access approaches that are employed for most MTC standards in cellular systems. While 2-step approaches are efficient in terms of access delay, their access delay is still dependent on retransmission strategies. Thus, for a low access delay, fast retrial that allows immediate re-transmissions can be employed as a re-transmission strategy. In this paper, we study 2-step random access approaches with fast retrial as a buffered multichannel ALOHA with fast retrial, and derive an analytical way to obtain the quality-of-service (QoS) exponent for the distribution of queue length so that key parameters can be decided to meet QoS requirements in terms of access delay. Simulation results confirm that the derived analytical approach can provide a good approximation of QoS exponent.
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
Venue: arXiv cs.DC 2020
Year: 2020

We present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks. Deep learning-based emerging scientific workflows often require model training with large, high-dimensional samples, which can make training much more costly and even infeasible due to excessive memory usage. We solve these challenges by extensively applying hybrid parallelism throughout the end-to-end training pipeline, including both computations and I/O. Our hybrid-parallel algorithm extends the standard data parallelism with spatial parallelism, which partitions a single sample in the spatial domain, realizing strong scaling beyond the mini-batch dimension with a larger aggregated memory capacity. We evaluate our proposed training algorithms with two challenging 3D CNNs, CosmoFlow and 3D U-Net. Our comprehensive performance studies show that good weak and strong scaling can be achieved for both networks using up 2K GPUs. More importantly, we enable training of CosmoFlow with much larger samples than previously possible, realizing an order-of-magnitude improvement in prediction accuracy.
DD-CNN: Depthwise Disout Convolutional Neural Network for Low-complexity Acoustic Scene Classification
Venue: arXiv cs.SD 2020
Year: 2020

This paper presents a Depthwise Disout Convolutional Neural Network (DD-CNN) for the detection and classification of urban acoustic scenes. Specifically, we use log-mel as feature representations of acoustic signals for the inputs of our network. In the proposed DD-CNN, depthwise separable convolution is used to reduce the network complexity. Besides, SpecAugment and Disout are used for further performance boosting. Experimental results demonstrate that our DD-CNN can learn discriminative acoustic characteristics from audio fragments and effectively reduce the network complexity. Our DD-CNN was used for the low-complexity acoustic scene classification task of the DCASE2020 Challenge, which achieves 92.04% accuracy on the validation set.
EAGP: An Energy-Aware Gossip Protocol for Wireless Sensor Networks
Venue: arXiv cs.NI 2020
Year: 2020

In Wireless Sensor Networks (WSN), typically composed of nodes with resource constraints, leveraging efficient processes is crucial to enhance the network lifetime and, consequently, the sustainability in ultra-dense and heterogeneous environments, such as smart cities. Particularly, balancing the energy required to transport data efficiently across such dynamic environments poses significant challenges to routing protocol design and operation, being the trade-off of reducing data redundancy while achieving an acceptable delivery rate a fundamental research topic. In this way, this work proposes a new energy-aware epidemic protocol that uses the current state of the network energy to create a dynamic distribution topology by self-adjusting each node forwarding behavior as eager or lazy according to the local residual battery. Simulated evaluations demonstrate its efficiency in energy consumption, delivery rate, and reduced computational burden when compared with classical gossip protocols as well as with a directional protocol.
NoPropaganda at SemEval-2020 Task 11: A Borrowed Approach to Sequence Tagging and Text Classification
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes our contribution to SemEval-2020 Task 11: Detection Of Propaganda Techniques In News Articles. We start with simple LSTM baselines and move to an autoregressive transformer decoder to predict long continuous propaganda spans for the first subtask. We also adopt an approach from relation extraction by enveloping spans mentioned above with special tokens for the second subtask of propaganda technique classification. Our models report an F-score of 44.6% and a micro-averaged F-score of 58.2% for those tasks accordingly.
Multi-theorem (Malicious) Designated-Verifier NIZK for QMA
Venue: arXiv cs.CR 2020
Year: 2020

We present the first non-interactive zero-knowledge argument system for QMA with multi-theorem security. Our protocol setup constitutes an additional improvement and is constructed in the malicious designated-verifier (MDV-NIZK) model (Quach, Rothblum, and Wichs, EUROCRYPT 2019), where the setup consists of a trusted part that includes only a common uniformly random string and an untrusted part of classical public and secret verification keys, which even if sampled maliciously by the verifier, the zero knowledge property still holds. The security of our protocol is established under the Learning with Errors Assumption. Our main technical contribution is showing a general transformation that compiles any sigma protocol into a reusable MDV-NIZK protocol, using NIZK for NP. Our technique is classical but works for quantum protocols and allows the construction of a reusable MDV-NIZK for QMA.
Insightful Assistant: AI-compatible Operation Graph Representations for Enhancing Industrial Conversational Agents
Venue: arXiv cs.CL 2020
Year: 2020

Advances in voice-controlled assistants paved the way into the consumer market. For professional or industrial use, the capabilities of such assistants are too limited or too time-consuming to implement due to the higher complexity of data, possible AI-based operations, and requests. In the light of these deficits, this paper presents Insightful Assistant---a pipeline concept based on a novel operation graph representation resulting from the intents detected. Using a predefined set of semantically annotated (executable) functions, each node of the operation graph is assigned to a function for execution. Besides basic operations, such functions can contain artificial intelligence (AI) based operations (e.g., anomaly detection). The result is then visualized to the user according to type and extracted user preferences in an automated way. We further collected a unique crowd-sourced set of 869 requests, each with four different variants expected visualization, for an industrial dataset. The evaluation of our proof-of-concept prototype on this dataset shows its feasibility: it achieves an accuracy of up to 95.0% (74.5%) for simple (complex) request detection with different variants and a top3-accuracy up to 95.4% for data-/user-adaptive visualization.
SOTERIA: In Search of Efficient Neural Networks for Private Inference
Venue: arXiv cs.CR 2020
Year: 2020

ML-as-a-service is gaining popularity where a cloud server hosts a trained model and offers prediction (inference) service to users. In this setting, our objective is to protect the confidentiality of both the users' input queries as well as the model parameters at the server, with modest computation and communication overhead. Prior solutions primarily propose fine-tuning cryptographic methods to make them efficient for known fixed model architectures. The drawback with this line of approach is that the model itself is never designed to operate with existing efficient cryptographic computations. We observe that the network architecture, internal functions, and parameters of a model, which are all chosen during training, significantly influence the computation and communication overhead of a cryptographic method, during inference. Based on this observation, we propose SOTERIA -- a training method to construct model architectures that are by-design efficient for private inference. We use neural architecture search algorithms with the dual objective of optimizing the accuracy of the model and the overhead of using cryptographic primitives for secure inference. Given the flexibility of modifying a model during training, we find accurate models that are also efficient for private computation. We select garbled circuits as our underlying cryptographic primitive, due to their expressiveness and efficiency, but this approach can be extended to hybrid multi-party computation settings. We empirically evaluate SOTERIA on MNIST and CIFAR10 datasets, to compare with the prior work. Our results confirm that SOTERIA is indeed effective in balancing performance and accuracy.
Gradient Regularized Contrastive Learning for Continual Domain Adaptation
Venue: arXiv cs.CV 2020
Year: 2020

Human beings can quickly adapt to environmental changes by leveraging learning experience. However, the poor ability of adapting to dynamic environments remains a major challenge for AI models. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labeled source domain and a sequence of unlabeled target domains. There are two major obstacles in this problem: domain shifts and catastrophic forgetting. In this work, we propose Gradient Regularized Contrastive Learning to solve the above obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforces the gradient of contrastive loss not to increase the supervised training loss on the source domain, which maintains the discriminative power of learned features; (2) regularizes the gradient update on the new domain not to increase the classification loss on the old target domains, which enables the model to adapt to an in-coming target domain while preserving the performance of previously observed domains. Hence our method can jointly learn both semantically discriminative and domain-invariant features with labeled source domain and unlabeled target domains. The experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the state-of-the-art.
Sequential Multi-hypothesis Testing in Multi-armed Bandit Problems:An Approach for Asymptotic Optimality
Venue: arXiv cs.IT 2020
Year: 2020

We consider a multi-hypothesis testing problem involving a K-armed bandit. Each arm's signal follows a distribution from a vector exponential family. The actual parameters of the arms are unknown to the decision maker. The decision maker incurs a delay cost for delay until a decision and a switching cost whenever he switches from one arm to another. His goal is to minimise the overall cost until a decision is reached on the true hypothesis. Of interest are policies that satisfy a given constraint on the probability of false detection. This is a sequential decision making problem where the decision maker gets only a limited view of the true state of nature at each stage, but can control his view by choosing the arm to observe at each stage. An information-theoretic lower bound on the total cost (expected time for a reliable decision plus total switching cost) is first identified, and a variation on a sequential policy based on the generalised likelihood ratio statistic is then studied. Due to the vector exponential family assumption, the signal processing at each stage is simple; the associated conjugate prior distribution on the unknown model parameters enables easy updates of the posterior distribution. The proposed policy, with a suitable threshold for stopping, is shown to satisfy the given constraint on the probability of false detection. Under a continuous selection assumption, the policy is also shown to be asymptotically optimal in terms of the total cost among all policies that satisfy the constraint on the probability of false detection.
A Novel Approach to the Diagnosis of Heart Disease using Machine Learning and Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Heart disease is the leading cause of death worldwide. Currently, 33% of cases are misdiagnosed, and approximately half of myocardial infarctions occur in people who are not predicted to be at risk. The use of Artificial Intelligence could reduce the chance of error, leading to possible earlier diagnoses, which could be the difference between life and death for some. The objective of this project was to develop an application for assisted heart disease diagnosis using Machine Learning (ML) and Deep Neural Network (DNN) algorithms. The dataset was provided from the Cleveland Clinic Foundation, and the models were built based on various optimization and hyper parametrization techniques including a Grid Search algorithm. The application, running on Flask, and utilizing Bootstrap was developed using the DNN, as it performed higher than the Random Forest ML model with a total accuracy rate of 92%.