OpenRaiser/Intern-Atlas huggingface dataset:
Novel Deep Learning Architecture for Heart Disease Prediction using Convolutional Neural Network
Venue: arXiv cs.LG 2021
Year: 2021

Healthcare is one of the most important aspects of human life. Heart disease is known to be one of the deadliest diseases which is hampering the lives of many people around the world. Heart disease must be detected early so the loss of lives can be prevented. The availability of large-scale data for medical diagnosis has helped developed complex machine learning and deep learning-based models for automated early diagnosis of heart diseases. The classical approaches have been limited in terms of not generalizing well to new data which have not been seen in the training set. This is indicated by a large gap in training and test accuracies. This paper proposes a novel deep learning architecture using a 1D convolutional neural network for classification between healthy and non-healthy persons to overcome the limitations of classical approaches. Various clinical parameters are used for assessing the risk profile in the patients which helps in early diagnosis. Various techniques are used to avoid overfitting in the proposed network. The proposed network achieves over 97% training accuracy and 96% test accuracy on the dataset. The accuracy of the model is compared in detail with other classification algorithms using various performance parameters which proves the effectiveness of the proposed architecture.
Choosing is Losing: How to combine the benefits of shallow and deep embeddings through reflection
Venue: arXiv cs.PL 2021
Year: 2021

Dependently-typed host languages empower users to verify a wide range of properties of embedded languages and programs written in them. Designers of such embedded languages are faced with a difficult choice between using a shallow or a deep embedding. The former is easier to use because the entire infrastructure of the host langauge is immediately available. Meanwhile, the latter gives full access to the structure of embedded programs, but is difficult to use in practice, especially when the embedded language is itself dependently typed. The main insight presented in this paper is that the choice between shallow and deep embedding can be eliminated by working in a host language with reflection capabilities: we start from a shallow embedding that can use all libraries and tools of the host language, and later use reflection to expose the deep structure of the embedded programs. Concretely, we apply this technique to embed three programming languages -- Kaleidoscope, SaC, and (a subset of) APL -- into the dependently typed theorem prover Agda, using dependent types to statically enforce several properties of interest. We then use Agda's reflection capabilities to extract the embedded programs back into the original language, so that the existing toolchain can be leveraged. In this process, statically verified properties of the host language are mapped onto runtime checks in the target language, allowing extracted programs to interact safely with existing code. Finally, we demonstrate the feasibility of our approach with the implementation and extraction of a convolutional neural network in our embedding of APL.\@
Toolbox Spotter: A Computer Vision System for Real World Situational Awareness in Heavy Industries
Venue: arXiv cs.RO 2021
Year: 2021

The majority of fatalities and traumatic injuries in heavy industries involve mobile plant and vehicles, often resulting from a lapse of attention or communication. Existing approaches to hazard identification include the use of human spotters, passive reversing cameras, non-differentiating proximity sensors and tag based systems. These approaches either suffer from problems of worker attention or require the use of additional devices on all workers and obstacles. Whilst computer vision detection systems have previously been deployed in structured applications such as manufacturing and on-road vehicles, there does not yet exist a robust and portable solution for use in unstructured environments like construction that effectively communicates risks to relevant workers. To address these limitations, our solution, the Toolbox Spotter (TBS), acts to improve worker safety and reduce preventable incidents by employing an embedded robotic perception and distributed HMI alert system to augment both detection and communication of hazards in safety critical environments. In this paper we outline the TBS safety system and evaluate its performance based on data from real world implementations, demonstrating the suitability of the Toolbox Spotter for applications in heavy industries.
Experimental Evaluation of a Hierarchical Operating Framework for Ground Robots in Agriculture
Venue: arXiv cs.RO 2021
Year: 2021

For mobile robots to be effectively applied to real world unstructured environments -- such as large scale farming -- they require the ability to generate adaptive plans that account both for limited onboard resources, and the presence of dynamic changes, including nearby moving individuals. This work provides a real world empirical evaluation of our proposed hierarchical framework for long-term autonomy of field robots, conducted on University of Sydney's Swagbot agricultural robot platform. We demonstrate the ability of the framework to navigate an unstructured and dynamic environment in an effective manner, validating its use for long-term deployment in large scale farming, for tasks such as autonomous weeding in the presence of moving individuals.
Projection-Based Reduced Order Model and Machine Learning Closure for Transient Simulations of High-Re Flows
Venue: arXiv cs.CE 2021
Year: 2021

The paper presents a Projection-Based Reduced-Order Model for simulations of high Reynolds turbulent flows. The PBROM are enhanced by incorporating various models of turbulent viscosity and residual closures to model the effects of interactions among the modes and energy dissipations. Remarkable improvements in prediction accuracies are achieved with a suitable turbulent viscosity model and a residual closure. The enhanced PBROM models are demonstrated for high-Re flows past a cylinder in two- and three- dimensions. These enhancements have shown capable of capturing complex flow features and removing unnecessary ones, while not affecting the efficiency of the overall model.
RST Parsing from Scratch
Venue: arXiv cs.CL 2021
Year: 2021

We introduce a novel top-down end-to-end formulation of document-level discourse parsing in the Rhetorical Structure Theory (RST) framework. In this formulation, we consider discourse parsing as a sequence of splitting decisions at token boundaries and use a seq2seq network to model the splitting decisions. Our framework facilitates discourse parsing from scratch without requiring discourse segmentation as a prerequisite; rather, it yields segmentation as part of the parsing process. Our unified parsing model adopts a beam search to decode the best tree structure by searching through a space of high-scoring trees. With extensive experiments on the standard English RST discourse treebank, we demonstrate that our parser outperforms existing methods by a good margin in both end-to-end parsing and parsing with gold segmentation. More importantly, it does so without using any handcrafted features, making it faster and easily adaptable to new languages and domains.
DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media
Venue: arXiv cs.LG 2021
Year: 2021

Twitter is currently a popular online social media platform which allows users to share their user-generated content. This publicly-generated user data is also crucial to healthcare technologies because the discovered patterns would hugely benefit them in several ways. One of the applications is in automatically discovering mental health problems, e.g., depression. Previous studies to automatically detect a depressed user on online social media have largely relied upon the user behaviour and their linguistic patterns including user's social interactions. The downside is that these models are trained on several irrelevant content which might not be crucial towards detecting a depressed user. Besides, these content have a negative impact on the overall efficiency and effectiveness of the model. To overcome the shortcomings in the existing automatic depression detection methods, we propose a novel computational framework for automatic depression detection that initially selects relevant content through a hybrid extractive and abstractive summarization strategy on the sequence of all user tweets leading to a more fine-grained and relevant content. The content then goes to our novel deep learning framework comprising of a unified learning machinery comprising of Convolutional Neural Network (CNN) coupled with attention-enhanced Gated Recurrent Units (GRU) models leading to better empirical performance than existing strong baselines.
Who Watches the New Watchmen? The Challenges for Drone Digital Forensics Investigations
Venue: arXiv cs.CR 2021
Year: 2021

The technological advance of drone technology has augmented the existing capabilities of flying vehicles rendering them a valuable asset of the modern society. As more drones are expected to occupy the airspace in the near future, security-related incidents, either malicious acts or accidents, will increase as well. The forensics analysis of a security incident is essential, as drones are flying above populated areas and have also been weaponised from radical forces and perpetrators. Thus, it is an imperative need to establish a Drone Digital Forensics Investigation Framework and standardise the processes of collecting and processing such evidence. Although there are numerous drone platforms in the market, the same principles apply to all of them; just like mobile phones. Nevertheless, due to the nature of drones, standardised forensics procedures to date do not manage to address the required processes and challenges that such investigations pose. Acknowledging this need, we detail the unique characteristics of drones and the gaps in existing methodologies and standards, showcasing that there are fundamental issues in terms of their forensics analysis from various perspectives, ranging from operational and procedural ones, and escalate to manufacturers, as well as legal restrictions. The above creates a very complex environment where coordinated actions must be made among the key stakeholders. Therefore, this work paves the way to address these challenges by identifying the main issues, their origins, and the needs in the field by performing a thorough review of the literature and a gap analysis.
End-to-End Video Object Detection with Spatial-Temporal Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Recently, DETR and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors. However, their performance on Video Object Detection (VOD) has not been well explored. In this paper, we present TransVOD, an end-to-end video object detection model based on a spatial-temporal Transformer architecture. The goal of this paper is to streamline the pipeline of VOD, effectively removing the need for many hand-crafted components for feature aggregation, e.g., optical flow, recurrent neural networks, relation networks. Besides, benefited from the object query design in DETR, our method does not need complicated post-processing methods such as Seq-NMS or Tubelet rescoring, which keeps the pipeline simple and clean. In particular, we present temporal Transformer to aggregate both the spatial object queries and the feature memories of each frame. Our temporal Transformer consists of three components: Temporal Deformable Transformer Encoder (TDTE) to encode the multiple frame spatial details, Temporal Query Encoder (TQE) to fuse object queries, and Temporal Deformable Transformer Decoder to obtain current frame detection results. These designs boost the strong baseline deformable DETR by a significant margin (3%-4% mAP) on the ImageNet VID dataset. TransVOD yields comparable results performance on the benchmark of ImageNet VID. We hope our TransVOD can provide a new perspective for video object detection. Code will be made publicly available at https://github.com/SJTU-LuHe/TransVOD.
COTR: Convolution in Transformer Network for End to End Polyp Detection
Venue: arXiv cs.CV 2021
Year: 2021

Purpose: Colorectal cancer (CRC) is the second most common cause of cancer mortality worldwide. Colonoscopy is a widely used technique for colon screening and polyp lesions diagnosis. Nevertheless, manual screening using colonoscopy suffers from a substantial miss rate of polyps and is an overwhelming burden for endoscopists. Computer-aided diagnosis (CAD) for polyp detection has the potential to reduce human error and human burden. However, current polyp detection methods based on object detection framework need many handcrafted pre-processing and post-processing operations or user guidance that require domain-specific knowledge. Methods: In this paper, we propose a convolution in transformer (COTR) network for end-to-end polyp detection. Motivated by the detection transformer (DETR), COTR is constituted by a CNN for feature extraction, transformer encoder layers interleaved with convolutional layers for feature encoding and recalibration, transformer decoder layers for object querying, and a feed-forward network for detection prediction. Considering the slow convergence of DETR, COTR embeds convolution layers into transformer encoder for feature reconstruction and convergence acceleration. Results: Experimental results on two public polyp datasets show that COTR achieved 91.49\% precision, 82.69% sensitivity, and 86.87% F1-score on the ETIS-LARIB, and 91.67% precision, 93.54% sensitivity, and 92.60% F1-score on the CVC-ColonDB. Conclusion: This study proposed an end to end detection method based on detection transformer for colorectal polyp detection. Experimental results on ETIS-LARIB and CVC-ColonDB dataset demonstrated that the proposed model achieved comparable performance against state-of-the-art methods.
Uniform interpolation via nested sequents and hypersequents
Venue: arXiv cs.LO 2021
Year: 2021

A modular proof-theoretic framework was recently developed to prove Craig interpolation for normal modal logics based on generalizations of sequent calculi (e.g., nested sequents, hypersequents, and labelled sequents). In this paper, we turn to uniform interpolation, which is stronger than Craig interpolation. We develop a constructive method for proving uniform interpolation via nested sequents and apply it to reprove the uniform interpolation property for normal modal logics $\mathsf{K}$, $\mathsf{D}$, and $\mathsf{T}$. We then use the know-how developed for nested sequents to apply the same method to hypersequents and obtain the first direct proof of uniform interpolation for $\mathsf{S5}$ via a cut-free sequent-like calculus. While our method is proof-theoretic, the definition of uniform interpolation for nested sequents and hypersequents also uses semantic notions, including bisimulation modulo an atomic proposition.
Weakly Supervised Instance Attention for Multisource Fine-Grained Object Recognition with an Application to Tree Species Classification
Venue: arXiv cs.CV 2021
Year: 2021

Multisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborhoods around the expected object locations where an object with a given class label is present in the neighborhood without any knowledge of its exact location. The proposed method uses a single-source deep instance attention model with parallel branches for joint localization and classification of objects, and extends this model into a multisource setting where a reference source that is assumed to have no location uncertainty is used to aid the fusion of multiple sources in four different levels: probability level, logit level, feature level, and pixel level. We show that all levels of fusion provide higher accuracies compared to the state-of-the-art, with the best performing method of feature-level fusion resulting in 53% accuracy for the recognition of 40 different types of trees, corresponding to an improvement of 5.7% over the best performing baseline when RGB, multispectral, and LiDAR data are used. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement of 6.3% over the default capacity setting.
Revisiting 2D Convolutional Neural Networks for Graph-based Applications
Venue: arXiv cs.CV 2021
Year: 2021

Graph convolutional networks (GCNs) are widely used in graph-based applications such as graph classification and segmentation. However, current GCNs have limitations on implementation such as network architectures due to their irregular inputs. In contrast, convolutional neural networks (CNNs) are capable of extracting rich features from large-scale input data, but they do not support general graph inputs. To bridge the gap between GCNs and CNNs, in this paper we study the problem of how to effectively and efficiently map general graphs to 2D grids that CNNs can be directly applied to, while preserving graph topology as much as possible. We therefore propose two novel graph-to-grid mapping schemes, namely, {\em graph-preserving grid layout (GPGL)} and its extension {\em Hierarchical GPGL (H-GPGL)} for computational efficiency. We formulate the GPGL problem as integer programming and further propose an approximate yet efficient solver based on a penalized Kamada-Kawai method, a well-known optimization algorithm in 2D graph drawing. We propose a novel vertex separation penalty that encourages graph vertices to lay on the grid without any overlap. Along with this image representation, even extra 2D maxpooling layers contribute to the PointNet, a widely applied point-based neural network. We demonstrate the empirical success of GPGL on general graph classification with small graphs and H-GPGL on 3D point cloud segmentation with large graphs, based on 2D CNNs including VGG16, ResNet50 and multi-scale maxout (MSM) CNN.
Compressing Heavy-Tailed Weight Matrices for Non-Vacuous Generalization Bounds
Venue: arXiv cs.LG 2021
Year: 2021

Heavy-tailed distributions have been studied in statistics, random matrix theory, physics, and econometrics as models of correlated systems, among other domains. Further, heavy-tail distributed eigenvalues of the covariance matrix of the weight matrices in neural networks have been shown to empirically correlate with test set accuracy in several works (e.g. arXiv:1901.08276), but a formal relationship between heavy-tail distributed parameters and generalization bounds was yet to be demonstrated. In this work, the compression framework of arXiv:1802.05296 is utilized to show that matrices with heavy-tail distributed matrix elements can be compressed, resulting in networks with sparse weight matrices. Since the parameter count has been reduced to a sum of the non-zero elements of sparse matrices, the compression framework allows us to bound the generalization gap of the resulting compressed network with a non-vacuous generalization bound. Further, the action of these matrices on a vector is discussed, and how they may relate to compression and resilient classification is analyzed.
Multilayer Resource-aware Partitioning for Fog Application Placement
Venue: arXiv cs.DC 2021
Year: 2021

Fog computing emerged as a crucial platform for the deployment of IoT applications. The complexity of such applications requires methods that handle the resource diversity and network structure of Fog devices while maximizing the service placement and reducing resource wastage. Prior studies in this domain primarily focused on optimizing specific application requirements and fail to address the network topology combined with the different types of resources encountered in Fog devices. To overcome these problems, we propose a multilayer resource-aware partitioning method to minimize the resource wastage and maximize the service placement and deadline satisfaction rates in a Fog infrastructure with high multi-user application placement requests. Our method represents the heterogeneous Fog resources as a multilayered network graph and partitions them based on network topology and resource features. Afterward, it identifies the appropriate device partitions for placing an application according to its requirements, which need to overlap in the same network topology partition. Simulation results show that our multilayer resource-aware partitioning method is able to place twice as many services, satisfy deadlines for three times as many application requests, and reduce the resource wastage by up to 15-32 times compared to two availability-aware and resource-aware methods.
Deep Visual Anomaly detection with Negative Learning
Venue: arXiv cs.CV 2021
Year: 2021

With the increase in the learning capability of deep convolution-based architectures, various applications of such models have been proposed over time. In the field of anomaly detection, improvements in deep learning opened new prospects of exploration for the researchers whom tried to automate the labor-intensive features of data collection. First, in terms of data collection, it is impossible to anticipate all the anomalies that might exist in a given environment. Second, assuming we limit the possibilities of anomalies, it will still be hard to record all these scenarios for the sake of training a model. Third, even if we manage to record a significant amount of abnormal data, it's laborious to annotate this data on pixel or even frame level. Various approaches address the problem by proposing one-class classification using generative models trained on only normal data. In such methods, only the normal data is used, which is abundantly available and doesn't require significant human input. However, these are trained with only normal data and at the test time, given abnormal data as input, may often generate normal-looking output. This happens due to the hallucination characteristic of generative models. Next, these systems are designed to not use abnormal examples during the training. In this paper, we propose anomaly detection with negative learning (ADNL), which employs the negative learning concept for the enhancement of anomaly detection by utilizing a very small number of labeled anomaly data as compared with the normal data during training. The idea is to limit the reconstruction capability of a generative model using the given a small amount of anomaly examples. This way, the network not only learns to reconstruct normal data but also encloses the normal distribution far from the possible distribution of anomalies.