Reviewerly/exHarmony huggingface dataset:
Benchmarking Low-Light Image Enhancement and Beyond

In this paper, we present a systematic review and evaluation of existing single-image low-light enhancement algorithms. Besides the commonly used low-level vision oriented evaluations, we additionally consider measuring machine vision performance in the low-light condition via face detection task to explore the potential of joint optimization of high-level and low-level vision enhancement. To this end, we first propose a large-scale low-light image dataset serving both low/high-level vision with diversified scenes and contents as well as complex degradation in real scenarios, called Vision Enhancement in the LOw-Light condition (VE-LOL). Beyond paired low/normal-light images without annotations, we additionally include the analysis resource related to human, i.e. face images in the low-light condition with annotated face bounding boxes. Then, efforts are made on benchmarking from the perspective of both human and machine visions. A rich variety of criteria is used for the low-level vision evaluation, including full-reference, no-reference, and semantic similarity metrics. We also measure the effects of the low-light enhancement on face detection in the low-light condition. State-of-the-art face detection methods are used in the evaluation. Furthermore, with the rich material of VE-LOL, we explore the novel problem of joint low-light enhancement and face detection. We develop an enhanced face detector to apply low-light enhancement and face detection jointly. The features extracted by the enhancement module are fed to the successive layer with the same resolution of the detection module. Thus, these features are intertwined together to unitedly learn useful information across two phases, i.e. enhancement and detection. Experiments on VE-LOL provide a comparison of state-of-the-art low-light enhancement algorithms, point out their limitations, and suggest promising future directions. Our dataset has supported the Track “Face Detection in Low Light Conditions” of CVPR UG2+ Challenge (2019–2020) ( http://cvpr2020.ug2challenge.org/ ).
Accurate Magnetic Resonance Image Super-Resolution Using Deep Networks and Gaussian Filtering in the Stationary Wavelet Domain

In this correspondence, we present an accurate Magnetic Resonance (MR) image Super-Resolution (SR) method that uses a Very Deep Residual network (VDR-net) in the training phase. By applying 2D Stationary Wavelet Transform (SWT), we decompose each Low Resolution (LR)-High Resolution (HR) example image pair into its low-frequency and high-frequency subbands. These LR-HR subbands are used to train the VDR-net through the input and output channels. The trained parameters are then used to generate residual subbands of a given LR test image. The obtained residuals are added with their LR subbands to produce the SR subbands. Finally, we attempt to maintain the intrinsic structure of images by implementing the Gaussian edge-preservation step on the SR subbands. Our extensive experimental results show that the proposed MR-SR method outperforms the existing methods in terms of four different objective metrics and subjective quality.
Photovoltaic power forecast based on satellite images considering effects of solar position

The rapid variation of clouds is the main factor that causes the fluctuation of photovoltaic power.1The satellite images contain plenty of information about clouds, applicable for photovoltaic power forecast. However, in practice, two main factors obstruct the application of the satellite images: 1) the relatively low update frequency of the satellite images mismatches the photovoltaic power forecasting frequency, and 2) the cloud region that blocks the sunlight changes significantly with time. In this paper, a novel satellite image-based approach for photovoltaic power forecast is proposed to overcome these obstacles and achieve accurate forecasting results. Firstly, concerning the hourly updated satellite images, a nonlinear cloud movement forecasting model, considering the thickness and shape changes of the cloud, is presented to forecast the hourly variation of the images. Secondly, an active cloud region selection rule is derived based on the changing solar position to dynamically select the cloud region that blocks the concerned photovoltaic power station in a satellite image. Thirdly, a sequential cloud region selection algorithm is provided to estimate the intra-hour variation of the cloud to match the photovoltaic power forecasting frequency. Finally, the photovoltaic power is predicted using the XGBoost algorithm concerning the effects of the cloud and other influencing factors. Testing results show that the proposed method can achieve more accurate photovoltaic power forecasts using the low update frequency satellite images. Meanwhile, the superior performance compared with other benchmarks also verifies the effectiveness of considering cloud information obtained by the proposed method for photovoltaic power forecast.
Blockchain-Empowered Decentralized Horizontal Federated Learning for 5G-Enabled UAVs

Motivated by Industry 4.0, 5G-enabled unmanned aerial vehicles (UAVs; also known as drones) are widely applied in various industries. However, the open nature of 5G networks threatens the safe sharing of data. In particular, privacy leakage can lead to serious losses for users. As a new machine learning paradigm, federated learning (FL) avoids privacy leakage by allowing data models to be shared instead of raw data. Unfortunately, the traditional FL framework is strongly dependent on a centralized aggregation server, which will cause the system to crash if the server is compromised. Unauthorized participants may launch poisoning attacks, thereby reducing the usability of models. In addition, communication barriers hinder collaboration among a large number of cross-domain devices for learning. To address the abovementioned issues, a blockchain-empowered decentralized horizontal FL framework is proposed. The authentication of cross-domain UAVs is accomplished through multisignature smart contracts. Global model updates are computed by using these smart contracts instead of a centralized server. Extensive experimental results show that the proposed scheme achieves high efficiency of cross-domain authentication and good accuracy.
Large-scale real-world radio signal recognition with deep learning

In the past ten years, many high-quality datasets have been released to support the rapid development of deep learning in the fields of computer vision, voice, and natural language processing. Nowadays, deep learning has become a key research component of the Sixth-Generation wireless systems (6G) with numerous regulatory and defense applications. In order to facilitate the application of deep learning in radio signal recognition, in this work, a large-scale real-world radio signal dataset is created based on a special aeronautical monitoring system - Automatic Dependent Surveillance-Broadcast (ADS-B). This paper makes two main contributions. First, an automatic data collection and labeling system is designed to capture over-the-air ADS-B signals in the open and real-world scenario without human participation. Through data cleaning and sorting, a high-quality dataset of ADS-B signals is created for radio signal recognition. Second, we conduct an in-depth study on the performance of deep learning models using the new dataset, as well as comparison with a recognition benchmark using machine learning and deep learning methods. Finally, we conclude this paper with a discussion of open problems in this area.
New challenges in higher education: A study of the digital competence of educators in Covid times

The digital revolution has substantially impacted education. The evolution of the use of technologies in education has been gradual and different, mostly depending on the teaching professionals' predisposition and skills. With the advent of the COVID pandemic, digital tools' impact has been exponential in all educational levels. An study of the new educational hybrid model is offered. The role of the educator is critical in this new model, confirming the necessity to have technical resources and qualified educators to improve students’ competence in order for them to work successfully in the digital economy. We have reflected on the evolution of Educators' Digital Competence (EDC) and its relationship with students' digital skills and impact on the learning process. An analysis of digital competence-learning in the context of the student-centered-learning approach has been studied. A more holistic approach to EDC is suggested. The European Framework for the Digital Competence of Educators (DigCompEdu) has identified 22 competences in six areas of competence. This paper has analyzed 251 responses from a convenience sample of professors from Madrid's universities in business administration based on this validated instrument. The results show the relevance of educators' characteristics to the learning process concerning previous knowledge about technology, training received, and their attitude toward technology. From a theoretical point of view, these competencies have been grouped into four competency areas, and their level of impact on student learning has been determined. In conclusion, digital competence has evolved from being a mere tool to becoming an essential pedagogical element.
UltraGCN

With the recent success of graph convolutional networks (GCNs), they have been widely applied for recommendation, and achieved impressive performance gains. The core of GCNs lies in its message passing mechanism to aggregate neighborhood information. However, we observed that message passing largely slows down the convergence of GCNs during training, especially for large-scale recommender systems, which hinders their wide adoption. LightGCN makes an early attempt to simplify GCNs for collaborative filtering by omitting feature transformations and nonlinear activations. In this paper, we take one step further to propose an ultra-simplified formulation of GCNs (dubbed UltraGCN), which skips infinite layers of message passing for efficient recommendation. Instead of explicit message passing, UltraGCN resorts to directly approximate the limit of infinite-layer graph convolutions via a constraint loss. Meanwhile, UltraGCN allows for more appropriate edge weight assignments and flexible adjustment of the relative importances among different types of relationships. This finally yields a simple yet effective UltraGCN model, which is easy to implement and efficient to train. Experimental results on four benchmark datasets show that UltraGCN not only outperforms the state-of-the-art GCN models but also achieves more than 10x speedup over LightGCN.
Prediction of HER2-positive breast cancer recurrence and metastasis risk from histopathological images and clinical information via multimodal deep learning

HER2-positive breast cancer is a highly heterogeneous tumor, and about 30% of patients still suffer from recurrence and metastasis after trastuzumab targeted therapy. Predicting individual prognosis is of great significance for the further development of precise therapy. With the continuous development of computer technology, more and more attention has been paid to computer-aided diagnosis and prognosis prediction based on Hematoxylin and Eosin (H&E) pathological images, which are available for all breast cancer patients undergone surgical treatment. In this study, we first enrolled 127 HER2-positive breast cancer patients with known recurrence and metastasis status from Cancer Hospital of the Chinese Academy of Medical Sciences. We then proposed a novel multimodal deep learning method integrating whole slide H&E images (WSIs) and clinical information to accurately assess the risk of relapse and metastasis in patients with HER2-positive breast cancer. Specifically, we obtained the whole H&E staining images from the surgical specimens of breast cancer patients, and these images were adjusted to size 512 × 512 pixels. The deep convolutional neural network (CNN) was applied to these images to retrieve image features, which were combined with the clinical data. Based on the combined features. After that, a novel multimodal model was constructed for predicting the prognosis of each patient. The model achieved an area under curve (AUC) of 0.76 in the two-fold cross-validation (CV). To further evaluate the performance of our model, we downloaded the data of all 123 HER2-positive breast cancer patients with available H&E image and known recurrence and metastasis status in The Cancer Genome Atlas (TCGA), which was severed as an independent testing data. Despite the huge differences in race and experimental strategies, our model achieved an AUC of 0.72 in the TCGA samples. As a conclusion, H&E images, in conjunction with clinical information and advanced deep learning models, could be used to evaluate the risk of relapse and metastasis in patients with HER2-positive breast cancer.
The new crypto niche: NFTs, play-to-earn, and metaverse tokens

The combination of blockchain technologies and the gaming industry has given rise to metaverses and play-to-earn games, which incorporate their own economy, commerce, and currencies, namely, metaverse and play-to-earn tokens. In this study, we analysed the performance and dynamics of 174 tokens, showing that this new crypto niche is characterised by (i) a positive performance in the long run, (ii) the absence of high co-movements with the cryptocurrency market, (iii) the emergence of bubbles, (iv) and the absence of high correlations with NFT features, such as number of transactions, sales and Google searches.
Relational Embedding for Few-Shot Classification

We propose to address the problem of few-shot classification by meta-learning "what to observe" and "where to attend" in a relational perspective. Our method lever-ages relational patterns within and between images via self-correlational representation (SCR) and cross-correlational attention (CCA). Within each image, the SCR module transforms a base feature map into a self-correlation tensor and learns to extract structural patterns from the tensor. Between the images, the CCA module computes cross-correlation between two image representations and learns to produce co-attention between them. Our Relational Embedding Network (RENet) combines the two relational modules to learn relational embedding in an end-to-end manner. In experimental evaluation, it achieves consistent improvements over state-of-the-art methods on four widely used few-shot classification benchmarks of miniImageNet, tieredImageNet, CUB-200-2011, and CIFAR-FS.
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

The crux of semi-supervised semantic segmentation is to assign adequate pseudo-labels to the pixels of unlabeled images. A common practice is to select the highly confident predictions as the pseudo ground-truth, but it leads to a problem that most pixels may be left unused due to their unreliability. We argue that every pixel matters to the model training, even its prediction is ambiguous. Intuitively, an unreliable prediction may get confused among the top classes (i.e., those with the highest probabilities), however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative sample to those most unlikely categories. Based on this insight, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative samples, and manage to train the model with all candidate pixels. Considering the training evolution, where the prediction becomes more and more accurate, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the state-of-the-art alternatives. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Project: https://haochen-wang409.github.io/U2PL.
RETRACTED ARTICLE: Eye tracking: empirical foundations for a minimal reporting guideline

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection

Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. While prevalent multi-modal methods [34], [36] simply decorate raw lidar point clouds with camera features and feed them directly to existing 3D detection models, our study shows that fusing camera features with deep lidar features instead of raw points, can lead to better performance. However, as those features are often augmented and aggregated, a key challenge in fusion is how to effectively align the transformed features from two modalities. In this paper, we propose two novel techniques: InverseAug that inverses geometric-related augmentations, e.g., rotation, to enable accurate geometric alignment between lidar points and image pixels, and LearnableAlign that leverages cross-attention to dynamically capture the correlations between image and lidar features during fusion. Based on InverseAug and LearnableAlign, we develop a family of generic multi-modal 3D detection models named DeepFusion, which is more accurate than previous methods. For example, DeepFusion improves Point-Pillars, CenterPoint, and 3D-MAN baselines on Pedestrian detection for 6.7,8.9, and 6.2 LEVEL_2 APH, respectively. Notably, our models achieve state-of-the-art performance on Waymo Open Dataset, and show strong model robustness against input corruptions and out-of-distribution data. Code will be publicly available at https://github.com/tensorflow/lingvo.
Unsupervised feature selection by self-paced learning regularization

Previous feature selection methods equivalently consider the samples to select important features. However, the samples are often diverse. For example, the outliers should have small or even zero weights while the important samples should have large weights. In this paper, we add a self-paced regularization in the sparse feature selection model to reduce the impact of outliers for conducting feature selection. Specifically, the proposed method automatically selects a sample subset which includes the most important samples to build an initial feature selection model, whose generalization ability is then improved by involving other important samples until a robust and generalized feature selection model has been established or all the samples have been used. Experimental results on eight real datasets show that the proposed method outperforms the comparison methods.
Generating Realistic Videos From Keyframes With Concatenated GANs

Given two video frames X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> and X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n+1</sub> , we aim to generate a series of intermediate frames Y <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , Y <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , . .., Y <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> , such that the resulting video consisting of frames X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> , Y <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> - Y <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> , and X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n+1</sub> appears realistic to a human watcher. Such video generation has numerous important applications, including video compression, movie production, slow-motion filming, video surveillance, and forensic analysis. Yet, video generation is highly challenging due to the vast search space of possible frames. Previous methods, mostly based on video prediction and/or video interpolation, tend to generate poor-quality videos with severe motion blur. This paper proposes a novel, end-to-end approach to video generation using generative adversarial networks (GANs). In particular, our design involves two concatenated GANs, one capturing motions and the other generating frame details. The loss function is also carefully engineered to include adversarial loss, gradient difference (for motion learning), and normalized product correlation loss (for frame details). Experiments using three video datasets, namely, Google Robotic Push, KTH human actions, and UCF101, demonstrate that the proposed solution generates high-quality, realistic, and sharp videos, whereas all previous solutions output noisy and blurry results.
In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias toward evaluating performance as accuracy rather than decision quality. We conclude with a concrete set of recommendations for evaluators designed to reduce the mismatch between the conceptualization of uncertainty in visualization versus other fields.