Reviewerly/exHarmony huggingface dataset:
Motion Deblurring with Real Events

In this paper, we propose an end-to-end learning framework for event-based motion deblurring in a self-supervised manner, where real-world events are exploited to alleviate the performance degradation caused by data inconsistency. To achieve this end, optical flows are predicted from events, with which the blurry consistency and photometric consistency are exploited to enable self-supervision on the deblurring network with real-world data. Furthermore, a piecewise linear motion model is proposed to take into account motion non-linearities and thus leads to an accurate model for the physical formation of motion blurs in the real-world scenario. Extensive evaluation on both synthetic and real motion blur datasets demonstrates that the proposed algorithm bridges the gap between simulated and real-world motion blurs and shows remarkable performance for eventbased motion deblurring in real-world scenarios.
Rethinking Noise Synthesis and Modeling in Raw Denoising

The lack of large-scale real raw image denoising dataset gives rise to challenges on synthesizing realistic raw image noise for training denoising models. However, the real raw image noise is contributed by many noise sources and varies greatly among different sensors. Existing methods are unable to model all noise sources accurately, and building a noise model for each sensor is also laborious. In this paper, we introduce a new perspective to synthesize noise by directly sampling from the sensor’s real noise. It inherently generates accurate raw image noise for different camera sensors. Two efficient and generic techniques: pattern-aligned patch sampling and high-bit reconstruction help accurate synthesis of spatial-correlated noise and high-bit noise respectively. We conduct systematic experiments on SIDD and ELD datasets. The results show that (1) our method outperforms existing methods and demonstrates wide generalization on different sensors and lighting conditions. (2) Recent conclusions derived from DNN-based noise modeling methods are actually based on inaccurate noise parameters. The DNN-based methods still cannot outperform physics-based statistical methods. The code will be available at https://github.com/zhangyi-3/.
Scene Context-Aware Salient Object Detection

Salient object detection identifies objects in an image that grab visual attention. Although contextual features are considered in recent literature, they often fail in real-world complex scenarios. We observe that this is mainly due to two issues: First, most existing datasets consist of simple foregrounds and backgrounds that hardly represent real-life scenarios. Second, current methods only learn contextual features of salient objects, which are insufficient to model high-level semantics for saliency reasoning in complex scenes. To address these problems, we first construct a new large-scale dataset with complex scenes in this paper. We then propose a context-aware learning approach to explicitly exploit the semantic scene contexts. Specifically, two modules are proposed to achieve the goal: 1) a Semantic Scene Context Refinement module to enhance contextual features learned from salient objects with scene context, and 2) a Contextual Instance Transformer to learn contextual relations between objects and scene context. To our knowledge, such high-level semantic contextual information of image scenes is under-explored for saliency detection in the literature. Extensive experiments demonstrate that the proposed approach outperforms state-of-the-art techniques in complex scenarios for saliency detection, and transfers well to other existing datasets. The code and dataset are available at https://github.com/SirisAvishek/Scene_Context_Aware_Saliency.
Quantum transfer learning for breast cancer detection

One of the areas with the potential to be explored in quantum computing (QC) is machine learning (ML), giving rise to quantum machine learning (QML). In an era when there is so much data, ML may benefit from either speed, complexity or smaller amounts of storage. In this work, we explore a quantum approach to a machine learning problem. Based on the work of Mari et al., we train a set of hybrid classical-quantum neural networks using transfer learning (TL). Our task was to solve the problem of classifying full-image mammograms into malignant and benign, provided by BCDR. Throughout the course of our work, heatmaps were used to highlight the parts of the mammograms that were being targeted by the networks while evaluating different performance metrics. Our work shows that this method may hold benefits regarding the generalization of complex data; however, further tests are needed. We also show that, depending on the task, some architectures perform better than others. Nonetheless, our results were superior to those reported in the state-of-the-art (accuracy of 84% against 76.9%, respectively). In addition, experiments were conducted in a real quantum device, and results were compared with the classical and simulator.
Image Harmonization with Transformer

Image harmonization, aiming to make composite images look more realistic, is an important and challenging task. The composite, synthesized by combining foreground from one image with background from another image, inevitably suffers from the issue of inharmonious appearance caused by distinct imaging conditions, i.e., lights. Current solutions mainly adopt an encoder-decoder architecture with convolutional neural network (CNN) to capture the context of composite images, trying to understand what it looks like in the surrounding background near the foreground. In this work, we seek to solve image harmonization with Transformer, by leveraging its powerful ability of modeling long-range context dependencies, for adjusting foreground light to make it compatible with background light while keeping structure and semantics unchanged. We present the design of our harmonization Transformer frameworks without and with disentanglement, as well as comprehensive experiments and ablation study, demonstrating the power of Transformer and investigating the Transformer for vision. Our method achieves state-of-the-art performance on both image harmonization and image inpainting/enhancement, indicating its superiority. Our code and models are available at https://github.com/zhenglab/HarmonyTransformer.
A New Journey from SDRTV to HDRTV

Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most available resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step solution pipeline including adaptive global color mapping, local enhancement and highlight generation. Moreover, the above analysis inspires us to present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping. In addition, we construct a dataset using HDR videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results achieve state-of-the-art performance in quantitative comparisons and visual quality. The code and dataset are available at https://github.com/chxy95/HDRTVNet.
Offensive Language Detection on Social Media Based on Text Classification

There is a concerning rise of offensive language on the content generated by the crowd over various social platforms. Such language might bully or hurt the feelings of an individual or a community. Recently, the research community has investigated and developed different supervised approaches and training datasets to detect or prevent offensive monologues or dialogues automatically. In this study, we propose a model for text classification consisting of modular cleaning phase and tokenizer, three embedding methods, and eight classifiers. Our experiments shows a promising result for detection of offensive language on our dataset obtained from Twitter. Considering hyperparameter optimization, three methods of AdaBoost, SVM and MLP had highest average of F1-score on popular embedding method of TF-IDF.
A Dual-Encoder-Single-Decoder Based Low-Dose CT Denoising Network

Generative adversarial networks (GAN) have shown great potential for image quality improvement in low-dose CT (LDCT). In general, the shallow features of generator include more shallow visual information such as edges and texture, while the deep features of generator contain more deep semantic information such as organization structure. To improve the network's ability to categorically deal with different kinds of information, this paper proposes a new type of GAN with dual-encoder- single-decoder structure. In the structure of the generator, firstly, a pyramid non-local attention module in the main encoder channel is designed to improve the feature extraction effectiveness by enhancing the features with self-similarity; Secondly, another encoder with shallow feature processing module and deep feature processing module is proposed to improve the encoding capabilities of the generator; Finally, the final denoised CT image is generated by fusing main encoder's features, shallow visual features, and deep semantic features. The quality of the generated images is improved due to the use of feature complementation in the generator. In order to improve the adversarial training ability of discriminator, a hierarchical-split ResNet structure is proposed, which improves the feature's richness and reduces the feature's redundancy in discriminator. The experimental results show that compared with the traditional single-encoder- single-decoder based GAN, the proposed method performs better in both image quality and medical diagnostic acceptability. Code is available in https://github.com/hanzefang/DESDGAN.
A Systematic Literature Review of Virtual, Augmented, and Mixed Reality Game Applications in Healthcare

Virtual reality, augmented reality, and mixed reality (VR/AR/MR) as information and communication technologies have been recognised and implemented in healthcare in recent years. One of the popular application ways is games, due to the potential benefits of providing an engaging and immersive experience in a virtual environment. This study presents a systematic literature review that evaluates the state-of-the-art on VR/AR/MR game applications in healthcare by collecting and analysing related journal and conference papers published from 2014 through to the first half of 2020. After retrieving more than 3,000 papers from six databases, 88 articles, from both computer science and medicine, were selected and analysed in the review. The articles are classified and summarised based on their (1) publication information, (2) design, implementation, and evaluation, and (3) application. The presented review is beneficial for both researchers and developers interested in exploring current research and future trends in VR/AR/MR in healthcare.
LAEDNet: A Lightweight Attention Encoder–Decoder Network for ultrasound medical image segmentation

Automatic ultrasound image segmentation plays an important role in early diagnosis of human diseases. This paper introduces a novel and efficient encoder–decoder network, called Lightweight Attention Encoder–Decoder Network (LAEDNet), for automatic ultrasound image segmentation. In contrast to previous encoder–decoder networks that involve complicated architecture with numerous parameters, our LAEDNet adopts lightweight version of EfficientNet as encoder. On the other hand, a Lightweight Residual Squeeze-and-Excitation (LRSE) block is employed in decoder. To achieve trade-off between segmentation accuracy and implementing efficiency, we also present a family of models, from light to heavy (denoted as LAEDNet-S, LAEDNet-M, and LAEDNet-L, respectively), with varying lightweight version of EfficientNet backbones. To evaluate LAEDNet, we have conducted extensive experiments on Brachial Plexus Dataset (BP), Breast Ultrasound Images Dataset (BUSI), and Head Circumference Ultrasound Images Dataset (HCUS), where ultrasound images are suffered from high noise, blurred borders and low contrast. The experiments show that, compared with U-Net and its variants, e.g., M-Net, U-Net++ and TransUNet, our LAEDNet achieves better results in terms of Dice Coefficient (DSC) and running speed. Particularly, LAEDNet-M only has 10.75M model parameters with 40.7 FPS, yet obtaining 73.0%, 73.8% and 91.3% DSC on BP, BUSI and HCUS datasets, respectively.
Active label cleaning for improved dataset quality under resource constraints

Imperfections in data annotation, known as label noise, are detrimental to the training of machine learning models and have a confounding effect on the assessment of model performance. Nevertheless, employing experts to remove label noise by fully re-annotating large datasets is infeasible in resource-constrained settings, such as healthcare. This work advocates for a data-driven approach to prioritising samples for re-annotation-which we term "active label cleaning". We propose to rank instances according to estimated label correctness and labelling difficulty of each sample, and introduce a simulation framework to evaluate relabelling efficacy. Our experiments on natural images and on a specifically-devised medical imaging benchmark show that cleaning noisy labels mitigates their negative impact on model training, evaluation, and selection. Crucially, the proposed approach enables correcting labels up to 4 × more effectively than typical random selection in realistic conditions, making better use of experts' valuable time for improving dataset quality.
Liver, kidney and spleen segmentation from CT scans and MRI with deep learning: A survey

Deep Learning approaches for automatic segmentation of organs from CT scans and MRI are providing promising results, leading towards a revolution in the radiologists' workflow. Precise delineations of abdominal organs boundaries reveal fundamental for a variety of purposes: surgical planning, volumetric estimation (e.g. Total Kidney Volume – TKV – assessment in Autosomal Dominant Polycystic Kidney Disease – ADPKD), diagnosis and monitoring of pathologies. Fundamental imaging techniques exploited for these tasks are Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), which enable clinicians to perform 3D analyses of all Regions of Interests (ROIs). In the realm of existing methods for segmentation and classification of these zones, Convolutional Neural Networks (CNNs) are emerging as the reference approach. In the last five years an enormous research effort has been done about the possibility of applying CNNs in Medical Imaging, resulting in more than 8000 documents on Scopus and more than 80000 results on Google Scholar. The high accuracy provided by those systems cannot be denied as motivation of all obtained results, though there are still problems to be addressed with. In this survey, major article databases, as Scopus, for instance, were systematically investigated for different kinds of Deep Learning approaches in segmentation of abdominal organs with a particular focus on liver, kidney and spleen. In this work, approaches are accurately classified, both by relevance of each organ (for instance, segmentation of liver has specific properties, if compared to other organs) and by type of computational approach, as well as the architecture of the employed network. For this purpose, a case study of segmentation for each of these organs is presented.
Do clean and dirty cryptocurrency markets herd differently?

In this paper, we investigate the herding behaviour of two types of cryptocurrencies, referred to as "black/dirty" and "green/clean" based on their energy usage levels. Empirical results reveal that herding generally exists only in the dirty cryptocurrency market, and is more significant in down markets. Moreover, we find that clean cryptocurrencies do herd, but with dirty cryptocurrencies, when the two markets are both positive. Our findings are robust across value- and equal-weighted portfolios and provide valuable insights to investors and policy makers.
GHNN: Graph Harmonic Neural Networks for semi-supervised graph-level classification

Graph classification aims to predict the property of the whole graph, which has attracted growing attention in the graph learning community. This problem has been extensively studied in the literature of both graph convolutional networks and graph kernels. Graph convolutional networks can learn effective node representations via message passing to mine graph topology in an implicit way, whereas graph kernels can explicitly utilize graph structural knowledge for classification. Due to the scarcity of labeled data in real-world applications, semi-supervised algorithms are anticipated for this problem. In this paper, we propose Graph Harmonic Neural Network (GHNN) which combines the advantages of both worlds to sufficiently leverage the unlabeled data, and thus overcomes label scarcity in semi-supervised scenarios. Specifically, our GHNN consists of a graph convolutional network (GCN) module and a graph kernel network (GKN) module that explore graph topology information from complementary perspectives. To fully leverage the unlabeled data, we develop a novel harmonic contrastive loss and a harmonic consistency loss to harmonize the training of two modules by giving priority to high-quality unlabeled data, thereby reconciling prediction consistency between both of them. In this manner, the two modules mutually enhance each other to sufficiently explore the graph topology of both labeled and unlabeled data. Extensive experiments on a variety of benchmarks demonstrate the effectiveness of our approach over competitive baselines.
An analysis of different deep learning neural networks for intra-hour solar irradiation forecasting to compute solar photovoltaic generators' energy production

Renewable energies are the alternative that leads to a cleaner generation and a reduction in CO2 emissions. However, their dependency on weather makes them unreliable. Traditional energy operators need a highly accurate estimation of energy to ensure the appropriate control of the network, since energy generation and demand must be balanced. This paper proposes a forecaster to predict solar irradiation, for very short-term, specifically, in the 10 min ahead. This study develops two tools based on artificial neural networks, namely Long-Short Term Memory neural networks and Convolutional Neural Network. The results demonstrate that the Convolutional Neural Network has a higher accuracy. The tool is tested examining the root mean square error, which was of 52.58 W/m2 for the testing step. Compared against the benchmark, it has obtained an improvement of 8.16%. Additionally, for the 82% of the tested days it has given a less than 4% error between the predicted and the actual energy generation. Results indicate that the forecaster is accurate enough to be implemented on a photovoltaic generation plan, improving their integration into the electrical grid, not only for providing power but also ancillary services.
A Survey of IoT-Based Fall Detection for Aiding Elderly Care: Sensors, Methods, Challenges and Future Trends

Remote monitoring of a fall condition or activities and daily life (ADL) of elderly patients has become one of the essential purposes for modern telemedicine. Internet of Things (IoT) and artificial intelligence (AI) techniques, including machine and deep learning models, have been recently applied in the medical field to automate the diagnosis procedures of abnormal and diseased cases. They also have many other applications, including the real-time identification of fall accidents in elderly patients. The goal of this article is to review recent research whose focus is to develop AI algorithms and methods of fall detection systems (FDS) in the IoT environment. In addition, the usability of different sensor types, such as gyroscopes and accelerometers in smartwatches, is described and discussed with the current limitations and challenges for realizing successful FDSs. The availability problem of public fall datasets for evaluating the proposed detection algorithms are also addressed in this study. Finally, this article is concluded by proposing advanced techniques such as lightweight deep models as one of the solutions and prospects of futuristic smart IoT-enabled systems for accurate fall detection in the elderly.
Hybridized Particle Swarm—Gravitational Search Algorithm for Process Optimization

The optimization of industrial processes is a critical task for leveraging profitability and sustainability. To ensure the selection of optimum process parameter levels in any industrial process, numerous metaheuristic algorithms have been proposed so far. However, many algorithms are either computationally too expensive or become trapped in the pit of local optima. To counter these challenges, in this paper, a hybrid metaheuristic called PSO-GSA is employed that works by combining the iterative improvement capability of particle swarm optimization (PSO) and gravitational search algorithm (GSA). A binary PSO is also fused with GSA to develop a BPSO-GSA algorithm. Both the hybrid algorithms i.e., PSO-GSA and BPSO-GSA, are compared against traditional algorithms, such as tabu search (TS), genetic algorithm (GA), differential evolution (DE), GSA and PSO algorithms. Moreover, another popular hybrid algorithm DE-GA is also used for comparison. Since earlier works have already studied the performance of these algorithms on mathematical benchmark functions, in this paper, two real-world-applicable independent case studies on biodiesel production are considered. Based on the extensive comparisons, significantly better solutions are observed in the PSO-GSA algorithm as compared to the traditional algorithms. The outcomes of this work will be beneficial to similar studies that rely on polynomial models.
Personalized Image Aesthetics Assessment via Meta-Learning With Bilevel Gradient Optimization

Typical image aesthetics assessment (IAA) is modeled for the generic aesthetics perceived by an "average" user. However, such generic aesthetics models neglect the fact that users' aesthetic preferences vary significantly depending on their unique preferences. Therefore, it is essential to tackle the issue for personalized IAA (PIAA). Since PIAA is a typical small sample learning (SSL) problem, existing PIAA models are usually built by fine-tuning the well-established generic IAA (GIAA) models, which are regarded as prior knowledge. Nevertheless, this kind of prior knowledge based on "average aesthetics" fails to incarnate the aesthetic diversity of different people. In order to learn the shared prior knowledge when different people judge aesthetics, that is, learn how people judge image aesthetics, we propose a PIAA method based on meta-learning with bilevel gradient optimization (BLG-PIAA), which is trained using individual aesthetic data directly and generalizes to unknown users quickly. The proposed approach consists of two phases: 1) meta-training and 2) meta-testing. In meta-training, the aesthetics assessment of each user is regarded as a task, and the training set of each task is divided into two sets: 1) support set and 2) query set. Unlike traditional methods that train a GIAA model based on average aesthetics, we train an aesthetic meta-learner model by bilevel gradient updating from the support set to the query set using many users' PIAA tasks. In meta-testing, the aesthetic meta-learner model is fine-tuned using a small amount of aesthetic data of a target user to obtain the PIAA model. The experimental results show that the proposed method outperforms the state-of-the-art PIAA metrics, and the learned prior model of BLG-PIAA can be quickly adapted to unseen PIAA tasks.