Reviewerly/exHarmony huggingface dataset:
A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving

As a key technology of intelligent transportation system, the intelligent vehicle is the carrier of comprehensive integration of many technologies. Although vision-based autonomous driving has shown excellent prospects, there is still a problem of how to analyze the complicated traffic situation by the collected data. Recently, autonomous driving has been formulated as many tasks separately by using different models, such as object detection task and intention recognition task. In this study, a vision-based system was developed to detect and identity various objects and predict the intention of pedestrians in the traffic scene. The main contributions of this research are (1) an optimized model was presented to detect 10 kinds of objects based on the structure of YOLOv4; (2) a fine-tuned Part Affinity Fields approach was proposed to estimate the pose of pedestrians; (3) Explainable Artificial Intelligence (XAI) technology is added to explain and assist the estimation results in the risk assessment phase; (4) an elaborate self-driving dataset that includes several different subsets for each corresponding task was introduced; and (5) an end-to-end system containing multiple models with high accuracy was developed. Experimental results proved that the total parameters of optimized YOLOv4 are reduced by 74%, which satisfies the real-time capability. In addition, the detection precision of the optimized YOLOv4 achieved an improvement of 2.6% compared to the state-of-the-art.
Hybrid approaches based on deep whole-sky-image learning to photovoltaic generation forecasting

With the ever-increased penetration of solar energy in the power grid, solar photovoltaic forecasting has become an indispensable aspect in maintaining power system stability and economic operation. At the operating stage, the forecasting accuracy of renewables has a direct influence on energy scheduling and dispatching. In this paper, we propose a series of novel approaches based on deep whole-sky-image learning architectures for very short-term solar photovoltaic generation forecasting, of which the lookahead windows concern the scales from 4 to 20 min. In particular, multiple deep learning models with the integration of both static sky image units and dynamic sky image stream are explicitly investigated. Extensive numerical studies on various models are carried out, through which the experimental results show that the proposed hybrid static image forecaster provides superior performance as compared to the benchmarking methods (i.e. the ones without sky images), with up to 8.3% improvement in general, and up to 32.8% improvement in the cases of ramp events. In addition, case studies at multiple time scales reveal that sky-image-based models can be more robust to the ramp events in solar photovoltaic generation.
SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation

Medical image segmentation is a difficult but important task for many clinical operations such as cardiac bi-ventricular volume estimation. More recently, there has been a shift to utilizing deep learning and fully convolutional neural networks (CNNs) to perform image segmentation that has yielded state-of-the-art results in many public benchmark datasets. Despite the progress of deep learning in medical image segmentation, standard CNNs are still not fully adopted in clinical settings as they lack robustness and interpretability. Shapes are generally more meaningful features than solely textures of images, which are features regular CNNs learn, causing a lack of robustness. Likewise, previous works surrounding model interpretability have been focused on post hoc gradient-based saliency methods. However, gradient-based saliency methods typically require additional computations post hoc and have been shown to be unreliable for interpretability. Thus, we present a new architecture called Shape Attentive U-Net (SAUNet) which focuses on model interpretability and robustness. The proposed architecture attempts to address these limitations by the use of a secondary shape stream that captures rich shape-dependent information in parallel with the regular texture stream. Furthermore, we suggest multi-resolution saliency maps can be learned using our dual-attention decoder module which allows for multi-level interpretability and mitigates the need for additional computations post hoc. Our method also achieves state-of-the-art results on the two large public cardiac MRI image segmentation datasets of SUN09 and AC17.
Good Counterfactuals and Where to Find Them: A Case-Based Technique for Generating Counterfactuals for Explainable AI (XAI)

Recently, a groundswell of research has identified the use of counterfactual explanations as a potentially significant solution to the Explainable AI (XAI) problem. It is argued that (i) technically, these counterfactual cases can be generated by permuting problem-features until a class-change is found, (ii) psychologically, they are much more causally informative than factual explanations, (iii) legally, they are GDPR-compliant. However, there are issues around the finding of “good” counterfactuals using current techniques (e.g. sparsity and plausibility). We show that many commonly-used datasets appear to have few “good” counterfactuals for explanation purposes. So, we propose a new case-based approach for generating counterfactuals, using novel ideas about the counterfactual potential and explanatory coverage of a case-base. The new technique reuses patterns of good counterfactuals, present in a case-base, to generate analogous counterfactuals that can explain new problems and their solutions. Several experiments show how this technique can improve the counterfactual potential and explanatory coverage of case-bases that were previously found wanting.
Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar

Abstract Objective To assess the consistency of machine learning and statistical techniques in predicting individual level and population level risks of cardiovascular disease and the effects of censoring on risk predictions. Design Longitudinal cohort study from 1 January 1998 to 31 December 2018. Setting and participants 3.6 million patients from the Clinical Practice Research Datalink registered at 391 general practices in England with linked hospital admission and mortality records. Main outcome measures Model performance including discrimination, calibration, and consistency of individual risk prediction for the same patients among models with comparable model performance. 19 different prediction techniques were applied, including 12 families of machine learning models (grid searched for best models), three Cox proportional hazards models (local fitted, QRISK3, and Framingham), three parametric survival models, and one logistic model. Results The various models had similar population level performance (C statistics of about 0.87 and similar calibration). However, the predictions for individual risks of cardiovascular disease varied widely between and within different types of machine learning and statistical models, especially in patients with higher risks. A patient with a risk of 9.5-10.5% predicted by QRISK3 had a risk of 2.9-9.2% in a random forest and 2.4-7.2% in a neural network. The differences in predicted risks between QRISK3 and a neural network ranged between –23.2% and 0.1% (95% range). Models that ignored censoring (that is, assumed censored patients to be event free) substantially underestimated risk of cardiovascular disease. Of the 223 815 patients with a cardiovascular disease risk above 7.5% with QRISK3, 57.8% would be reclassified below 7.5% when using another model. Conclusions A variety of models predicted risks for the same patients very differently despite similar model performances. The logistic models and commonly used machine learning models should not be directly applied to the prediction of long term risks without considering censoring. Survival models that consider censoring and that are explainable, such as QRISK3, are preferable. The level of consistency within and between models should be routinely assessed before they are used for clinical decision making.
PointPWC-Net: Cost Volume on Point Clouds for (Self-)Supervised Scene Flow Estimation

We propose a novel end-to-end deep scene flow model, called PointPWC-Net, that directly processes 3D point cloud scenes with large motions in a coarse-to-fine fashion. Flow computed at the coarse level is upsampled and warped to a finer level, enabling the algorithm to accommodate for large motion without a prohibitive search space. We introduce novel cost volume, upsampling, and warping layers to efficiently handle 3D point cloud data. Unlike traditional cost volumes that require exhaustively computing all the cost values on a high-dimensional grid, our point-based formulation discretizes the cost volume onto input 3D points, and a PointConv operation efficiently computes convolutions on the cost volume. Experiment results on FlyingThings3D and KITTI outperform the state-of-the-art by a large margin. We further explore novel self-supervised losses to train our model and achieve comparable results to state-of-the-art trained with supervised loss. Without any fine-tuning, our method also shows great generalization ability on the KITTI Scene Flow 2015 dataset, outperforming all previous methods. The code is released at https://github.com/DylanWusee/PointPWC .
Capsules for biomedical image segmentation

Our work expands the use of capsule networks to the task of object segmentation for the first time in the literature. This is made possible via the introduction of locally-constrained routing and transformation matrix sharing, which reduces the parameter/memory burden and allows for the segmentation of objects at large resolutions. To compensate for the loss of global information in constraining the routing, we propose the concept of “deconvolutional” capsules to create a deep encoder-decoder style network, called SegCaps. We extend the masked reconstruction regularization to the task of segmentation and perform thorough ablation experiments on each component of our method. The proposed convolutional-deconvolutional capsule network, SegCaps, shows state-of-the-art results while using a fraction of the parameters of popular segmentation networks. To validate our proposed method, we perform experiments segmenting pathological lungs from clinical and pre-clinical thoracic computed tomography (CT) scans and segmenting muscle and adipose (fat) tissue from magnetic resonance imaging (MRI) scans of human subjects’ thighs. Notably, our experiments in lung segmentation represent the largest-scale study in pathological lung segmentation in the literature, where we conduct experiments across five extremely challenging datasets, containing both clinical and pre-clinical subjects, and nearly 2000 computed-tomography scans. Our newly developed segmentation platform outperforms other methods across all datasets while utilizing less than 5% of the parameters in the popular U-Net for biomedical image segmentation. Further, we demonstrate capsules’ ability to generalize to unseen handling of rotations/reflections on natural images.
Guided Saliency Feature Learning for Person Re-identification in Crowded Scenes

Person Re-identification (Re-ID) in crowed scenes is a challenging problem, where people are frequently partially occluded by objects and other people. However, few studies have provided flexible solutions to re-identifying people in an image containing a partial occlusion body part. In this paper, we propose a simple occlusion-aware approach to address the problem. The proposed method first leverages a fully convolutional network to generate spatial features. And then we design a combination of a pose-guided and mask-guided layer to generate saliency heatmap to further guide discriminative feature learning. More importantly, we propose a new matching approach, called Guided Adaptive Spatial Matching (GASM), which expects that each spatial feature in the query can find the most similar spatial features of a person in a gallery to match. Especially, We use the saliency heatmap to guide the adaptive spatial matching by assigning the foreground human parts with larger weights adaptively. The effectiveness of the proposed GASM is demonstrated on two occluded person datasets: Crowd REID (51.52%) and Occluded REID (80.25%) and three benchmark person datasets: Market1501 (95.31%), DukeMTMC-reID (88.12%) and MSMT17 (79.52%). Additionally, GASM achieves good performance on cross-domain person Re-ID. The code and models are available at: https://github.com/JDAI-CV/fast-reid/blob/master/projects/CrowdReID .
Focus beyond Quadratic Speedups for Error-Corrected Quantum Advantage

In this perspective, we discuss conditions under which it would be possible for a modest fault-tolerant quantum computer to realize a runtime advantage by executing a quantum algorithm with only a small polynomial speedup over the best classical alternative. The challenge is that the computation must finish within a reasonable amount of time while being difficult enough that the small quantum scaling advantage would compensate for the large constant factor overheads associated with error-correction. We compute several examples of such runtimes using state-of-the-art surface code constructions under a variety of assumptions. We conclude that quadratic speedups will not enable quantum advantage on early generations of such fault-tolerant devices unless there is a significant improvement in how we would realize quantum error-correction. While this conclusion persists even if we were to increase the rate of logical gates in the surface code by more than an order of magnitude, we also repeat this analysis for speedups by other polynomial degrees and find that quartic speedups look significantly more practical.
Deep Network Approximation Characterized by Number of Neurons

This paper quantitatively characterizes the approximation power of deep feed-forward neural networks (FNNs) in terms of the number of neurons. It is shown by construction that ReLU FNNs with width $\mathcal{O}\big(\max\{d\lfloor N^{1/d}\rfloor,\, N+1\}\big)$ and depth $\mathcal{O}(L)$ can approximate an arbitrary H\"older continuous function of order $\alpha\in (0,1]$ on $[0,1]^d$ with a nearly tight approximation rate $\mathcal{O}\big(\sqrt{d} N^{-2\alpha/d}L^{-2\alpha/d}\big)$ measured in $L^p$-norm for any $N,L\in \mathbb{N}^+$ and $p\in[1,\infty]$. More generally for an arbitrary continuous function $f$ on $[0,1]^d$ with a modulus of continuity $\omega_f(\cdot)$, the constructive approximation rate is $\mathcal{O}\big(\sqrt{d}\,\omega_f( N^{-2/d}L^{-2/d})\big)$. We also extend our analysis to $f$ on irregular domains or those localized in an $\varepsilon$-neighborhood of a $d_{\mathcal{M}}$-dimensional smooth manifold $\mathcal{M}\subseteq [0,1]^d$ with $d_{\mathcal{M}}\ll d$. Especially, in the case of an essentially low-dimensional domain, we show an approximation rate $\mathcal{O}\big(\omega_f(\tfrac{\varepsilon}{1-\delta}\sqrt{\tfrac{d}{d_\delta}}+\varepsilon)+\sqrt{d}\,\omega_f(\tfrac{\sqrt{d}}{(1-\delta)\sqrt{d_\delta}}N^{-2/d_\delta}L^{-2/d_\delta})\big)$ for ReLU FNNs to approximate $f$ in the $\varepsilon$-neighborhood, where $d_\delta=\mathcal{O}\big(d_{\mathcal{M}}\tfrac{\ln (d/\delta)}{\delta^2}\big)$ for any $\delta\in(0,1)$ as a relative error for a projection to approximate an isometry when projecting $\mathcal{M}$ to a $d_{\delta}$-dimensional domain.
Efficient Security and Authentication for Edge-Based Internet of Medical Things

Internet of Medical Things (IoMT)-driven smart health and emotional care is revolutionizing the healthcare industry by embracing several technologies related to multimodal physiological data collection, communication, intelligent automation, and efficient manufacturing. The authentication and secure exchange of electronic health records (EHRs), comprising of patient data collected using wearable sensors and laboratory investigations, is of paramount importance. In this article, we present a novel high payload and reversible EHR embedding framework to secure the patient information successfully and authenticate the received content. The proposed approach is based on novel left data mapping (LDM), pixel repetition method (PRM), RC4 encryption, and checksum computation. The input image of size [Formula: see text] is upscaled by using PRM that guarantees reversibility with lesser computational complexity. The binary secret data are encrypted using the RC4 encryption algorithm and then the encrypted data are grouped into 3-bit chunks and converted into decimal equivalents. Before embedding, these decimal digits are encoded by LDM. To embed the shifted data, the cover image is divided into [Formula: see text] blocks and then in each block, two digits are embedded into the counter diagonal pixels. For tamper detection and localization, a checksum digit computed from the block is embedded into one of the main diagonal pixels. A fragile logo is embedded into the cover images in addition to EHR to facilitate early tamper detection. The average peak signal to noise ratio (PSNR) of the stego-images obtained is 41.95 dB for a very high embedding capacity of 2.25 bits per pixel. Furthermore, the embedding time is less than 0.2 s. Experimental results reveal that our approach outperforms many state-of-the-art techniques in terms of payload, imperceptibility, computational complexity, and capability to detect and localize tamper. All the attributes affirm that the proposed scheme is a potential candidate for providing better security and authentication solutions for IoMT-based smart health.
A Deep Learning Approach for Automatic Hate Speech Detection in the Saudi Twittersphere

With the rise of hate speech phenomena in the Twittersphere, significant research efforts have been undertaken in order to provide automatic solutions for detecting hate speech, varying from simple machine learning models to more complex deep neural network models. Despite this, research works investigating hate speech problem in Arabic are still limited. This paper, therefore, aimed to investigate several neural network models based on convolutional neural network (CNN) and recurrent neural network (RNN) to detect hate speech in Arabic tweets. It also evaluated the recent language representation model bidirectional encoder representations from transformers (BERT) on the task of Arabic hate speech detection. To conduct our experiments, we firstly built a new hate speech dataset that contained 9316 annotated tweets. Then, we conducted a set of experiments on two datasets to evaluate four models: CNN, gated recurrent units (GRU), CNN + GRU, and BERT. Our experimental results in our dataset and an out-domain dataset showed that the CNN model gave the best performance, with an F1-score of 0.79 and area under the receiver operating characteristic curve (AUROC) of 0.89.
Kinematic 3D Object Detection in Monocular Video

Perceiving the physical world in 3D is fundamental for self-driving applications. Although temporal motion is an invaluable resource to human vision for detection, tracking, and depth perception, such features have not been thoroughly utilized in modern 3D object detectors. In this work, we propose a novel method for monocular video-based 3D object detection which leverages kinematic motion to extract scene dynamics and improve localization accuracy. We first propose a novel decomposition of object orientation and a self-balancing 3D confidence. We show that both components are critical to enable our kinematic model to work effectively. Collectively, using only a single model, we efficiently leverage 3D kinematics from monocular videos to improve the overall localization precision in 3D object detection while also producing useful by-products of scene dynamics (ego-motion and per-object velocity). We achieve state-of-the-art performance on monocular 3D object detection and the Bird’s Eye View tasks within the KITTI self-driving dataset.
Variation-Aware Federated Learning With Multi-Source Decentralized Medical Image Data

Privacy concerns make it infeasible to construct a large medical image dataset by fusing small ones from different sources/institutions. Therefore, federated learning (FL) becomes a promising technique to learn from multi-source decentralized data with privacy preservation. However, the cross-client variation problem in medical image data would be the bottleneck in practice. In this paper, we propose a variation-aware federated learning (VAFL) framework, where the variations among clients are minimized by transforming the images of all clients onto a common image space. We first select one client with the lowest data complexity to define the target image space and synthesize a collection of images through a privacy-preserving generative adversarial network, called PPWGAN-GP. Then, a subset of those synthesized images, which effectively capture the characteristics of the raw images and are sufficiently distinct from any raw image, is automatically selected for sharing with other clients. For each client, a modified CycleGAN is applied to translate its raw images to the target image space defined by the shared synthesized images. In this way, the cross-client variation problem is addressed with privacy preservation. We apply the framework for automated classification of clinically significant prostate cancer and evaluate it using multi-source decentralized apparent diffusion coefficient (ADC) image data. Experimental results demonstrate that the proposed VAFL framework stably outperforms the current horizontal FL framework. As VAFL is independent of deep learning architectures for classification, we believe that the proposed framework is widely applicable to other medical image classification tasks.
A multi-user collaborative BIM-AR system to support design and construction

Augmented Reality (AR) is fast becoming an established tool for the construction industry. Previous research reports on the conversion of BIM geometric models and the implementation of these with marker-based AR, or the use of more wide area AR taking positional input from GPS. Much of this focused on the use of AR in an individual context, so there is need to align AR with the more collaborative nature of BIM. By implementing marker-based AR, and connecting to a cloud-based database, the presented BIM-AR system provides the ability to view, interact and collaborate with 3D and 2D BIM data via AR with geographically dispersed teams. An Agile Scrum Method was used to develop the prototype system including a mobile AR application and a Large Touch Screen application based on and a Model, View, Controller (MVC) approach. Finally, the system was tested and verified using a focus group of construction practitioners.