Reviewerly/exHarmony huggingface dataset:
Sampling Signals on Graphs: From Theory to Applications

The study of sampling signals on graphs, with the goal of building an analog of sampling for standard signals in the time and spatial domains, has attracted considerable attention recently.Beyond adding to the growing theory on graph signal processing (GSP), sampling on graphs has various promising applications.In this article, we review current progress on sampling over graphs focusing on theory and potential applications.Although most methodologies used in graph signal sampling are designed to parallel those used in sampling for standard signals, sampling theory for graph signals significantly differs from the theory of Shannon-Nyquist and shift-invariant sampling.This is due in part to the fact that the definitions of several important properties, such as shift invariance and bandlimitedness, are different in GSP systems.Throughout this review, we discuss similarities and differences between standard and graph signal sampling and highlight open problems and challenges.
Deepfakes Detection with Automatic Face Weighting

Altered and manipulated multimedia is increasingly present and widely distributed via social media platforms. Advanced video manipulation tools enable the generation of highly realistic-looking altered multimedia. While many methods have been presented to detect manipulations, most of them fail when evaluated with data outside of the datasets used in research environments. In order to address this problem, the Deepfake Detection Challenge (DFDC) provides a large dataset of videos containing realistic manipulations and an evaluation system that ensures that methods work quickly and accurately, even when faced with challenging data. In this paper, we introduce a method based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) that extracts visual and temporal features from faces present in videos to accurately detect manipulations. The method is evaluated with the DFDC dataset, providing competitive results compared to other techniques.
Towards trustworthy Internet of Things: A survey on Trust Management applications and schemes

Advancement in technology with the proliferation of new wireless communication protocols gave rise to the new era of ubiquitous computing, called Internet of Things (IoT). IoT facilitates connectivity between various heterogeneous physical devices through the internet to advantage users with intelligent and more advanced services. Effective utilization of these services demands a secure system where one can rely on the source of the information together with the received information. Trust Management (TM) is a crucial aspect of security that aims to maintain reliability in a system by ensuring the secure exchange of information. Using the concept of local and global perception about the reputation, TM measures the degree of trust on the system’s entities and endeavors to reduce risk and uncertainty in the system. For IoT, TM paves the way to accomplish various decision-making tasks, like reliable service composition, secure routing, device authentication, access control, etc. However, design and deployment of TM for IoT are hindered by the inherent characteristics of IoT systems that demand to be addressed. In this paper, we identified various applications of TM and examined issues in the design and deployment of TM for IoT. A clear vision towards TM system, explaining the different phases involved in the process of managing the trust, is presented. Furthermore, an exhaustive survey on various TM schemes developed for IoT with their applicability and addressing issues is provided. The survey is conducted considering direct observations and indirect recommendations based distributed, semi-distributed, and centralized schemes along with the review on blockchain technology-based schemes for trust management in IoT. In addition to that, a comparative study of the existing schemes based on the various system measures like computation model, input attributes, evaluation tool, and performance metrics examining their strengths and weaknesses is given. Finally, the paper highlights open research challenges investigated by the survey to present future direction for the researchers.
Image Restoration Using Joint Patch-Group-Based Sparse Representation

Sparse representation has achieved great success in various image processing and computer vision tasks. For image processing, typical patch-based sparse representation (PSR) models usually tend to generate undesirable visual artifacts, while group-based sparse representation (GSR) models lean to produce over-smooth effects. In this paper, we propose a new sparse representation model, termed joint patch-group based sparse representation (JPG-SR). Compared with existing sparse representation models, the proposed JPG-SR provides an effective mechanism to integrate the local sparsity and nonlocal self-similarity of images. We then apply the proposed JPG-SR to image restoration tasks, including image inpainting and image deblocking. An iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is developed to solve the proposed JPG-SR based image restoration problems. Experimental results demonstrate that the proposed JPG-SR is effective and outperforms many state-of-the-art methods in both objective and perceptual quality.
Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation

Nuclei segmentation is a vital step for pathological cancer research. It is still an open problem due to some difficulties, such as color inconsistency introduced by non-uniform manual operations, blurry tumor nucleus boundaries and overlapping tumor cells. In this paper, we aim to leverage the unique optical characteristic of H&E staining images that hematoxylin always stains cell nuclei blue, and eosin always stains the extracellular matrix and cytoplasm pink. Therefore, we extract the Hematoxylin component from RGB images by Beer-Lambert’s Law. According to the optical attribute, the extracted Hematoxylin component is robust to color inconsistency. With the Hematoxylin component, we propose a Hematoxylin-aware CNN model for nuclei segmentation without the necessity of color normalization. Our proposed network is formulated as a Triple U-net structure which includes an RGB branch, a Hematoxylin branch and a Segmentation branch. Then we propose a novel feature aggregation strategy to allow the network to fuse features progressively and to learn better feature representations from different branches. Extensive experiments are performed to qualitatively and quantitatively evaluate the effectiveness of our proposed method. In the meanwhile, it outperforms state-of-the-art methods on three different nuclei segmentation datasets.
Adaptive multi-teacher multi-level knowledge distillation

Knowledge distillation (KD) is an effective learning paradigm for improving the performance of lightweight student networks by utilizing additional supervision knowledge distilled from teacher networks. Most pioneering studies either learn from only a single teacher in their distillation learning methods, neglecting the potential that a student can learn from multiple teachers simultaneously, or simply treat each teacher to be equally important, unable to reveal the different importance of teachers for specific examples. To bridge this gap, we propose a novel adaptive multi-teacher multi-level knowledge distillation learning framework (AMTML-KD), which consists two novel insights: (i) associating each teacher with a latent representation to adaptively learn instance-level teacher importance weights which are leveraged for acquiring integrated soft-targets (high-level knowledge) and (ii) enabling the intermediate-level hints (intermediate-level knowledge) to be gathered from multiple teachers by the proposed multi-group hint strategy. As such, a student model can learn multi-level knowledge from multiple teachers through AMTML-KD. Extensive results on publicly available datasets demonstrate the proposed learning framework ensures student to achieve improved performance than strong competitors.
Augmentation Invariant and Instance Spreading Feature for Softmax Embedding

Deep embedding learning plays a key role in learning discriminative feature representations, where the visually similar samples are pulled closer and dissimilar samples are pushed away in the low-dimensional embedding space. This paper studies the unsupervised embedding learning problem by learning such a representation without using any category labels. This task faces two primary challenges: mining reliable positive supervision from highly similar fine-grained classes, and generalizing to unseen testing categories. To approximate the positive concentration and negative separation properties in category-wise supervised learning, we introduce a data augmentation invariant and instance spreading feature using the instance-wise supervision. We also design two novel domain-agnostic augmentation strategies to further extend the supervision in feature space, which simulates the large batch training using a small batch size and the augmented features. To learn such a representation, we propose a novel instance-wise softmax embedding, which directly perform the optimization over the augmented instance features with the binary discrmination softmax encoding. It significantly accelerates the learning speed with much higher accuracy than existing methods, under both seen and unseen testing categories. The unsupervised embedding performs well even without pre-trained network over samples from fine-grained categories. We also develop a variant using category-wise supervision, namely category-wise softmax embedding, which achieves competitive performance over the state-of-of-the-arts, without using any auxiliary information or restrict sample mining.
SensitiveNets: Learning Agnostic Representations with Application to Face Images

This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task.
Augmented Reality: A Systematic Review of Its Benefits and Challenges in E-learning Contexts

Augmented reality (AR) has received increasing attention in the research literature as a fundamental pedagogical tool that can enhance learning at most educational levels. In academic contexts, this technology permits the superimposing of three-dimensional images onto the real environment. Although AR has been found to enhance learning in the academic environment, no systematic review of it has been conducted to identify, evaluate, and summarize empirical findings on its advantages and challenges in e-learning contexts. Hence, a systematic review of the research literature was conducted on the use of AR in e-learning contexts, with a focus on the key benefits and challenges related to its adoption and implementation. Electronic searches on databases, including Springer, Science Direct, EBSCO, and Google Scholar, were performed to retrieve relevant journal articles; 28 studies were included after they were screened using the inclusion and exclusion criteria. The key benefits of using AR in e-learning included support of kinesthetic (tactile) learning, collaborative learning, distance/remote learning, learner-centered learning, and creative learning. Studies also reported that AR enhanced students’ engagement, motivation, attention/focus, interactivity, verbal participation, concentration, knowledge retention, and spatial abilities, as well as information accessibility. The findings suggest that challenges associated with AR in e-learning include information and cognitive overload, lack of experience in using the technology, resistance from teachers, complex technology, costly technology, and technical issues, such as connectivity problems.
Principle of Neural Network and Its Main Types: Review

In this paper, an overview of the artificial neural networks is presented. Their main and popular types such as the multilayer feedforward neural network (MLFFNN), the recurrent neural network (RNN), and the radial basis function (RBF) are investigated. Furthermore, the main advantages and disadvantages of each type are included as well as the training process.
Vehicle Detection and Tracking in Adverse Weather Using a Deep Learning Framework

Vehicle detection and tracking play an important role in autonomous vehicles and intelligent transportation systems. Adverse weather conditions such as the presence of heavy snow, fog, rain, dust or sandstorm situations are dangerous restrictions on camera’s function by reducing visibility, affecting driving safety. Indeed, these restrictions impact the performance of detection and tracking algorithms utilized in the traffic surveillance systems and autonomous driving applications. In this article, we start by proposing a visibility enhancement scheme consisting of three stages: illumination enhancement, reflection component enhancement, and linear weighted fusion to improve the performance. Then, we introduce a robust vehicle detection and tracking approach using a multi-scale deep convolution neural network. The conventional Gaussian mixture probability hypothesis density filter based tracker is utilized jointly with hierarchical data associations (HDA), which splits into detection-to-track and track-to-track associations. Herein, the cost matrix of each phase is solved using the Hungarian algorithm to compensate for the lost tracks caused by missed detection. Only detection information (i.e., bounding boxes with detection scores) is used in HDA without visual features information for rapid execution. We have also introduced a novel benchmarking dataset designed for research in applications of autonomous vehicles under adverse weather conditions called DAWN. It consists of real-world images collected with different types of adverse weather conditions. The proposed method is tested on DAWN, KITTI, and MS-COCO datasets and compared with 21 vehicle detectors. Experimental results have validated effectiveness of the proposed method which outperforms state-of-the-art vehicle detection and tracking approaches under adverse weather conditions.
Group Sparsity Residual Constraint With Non-Local Priors for Image Restoration

Group sparse representation (GSR) has made great strides in image restoration producing superior performance, realized through employing a powerful mechanism to integrate the local sparsity and nonlocal self-similarity of images. However, due to some form of degradation (e.g., noise, down-sampling or pixels missing), traditional GSR models may fail to faithfully estimate sparsity of each group in an image, thus resulting in a distorted reconstruction of the original image. This motivates us to design a simple yet effective model that aims to address the above mentioned problem. Specifically, we propose group sparsity residual constraint with nonlocal priors (GSRC-NLP) for image restoration. Through introducing the group sparsity residual constraint, the problem of image restoration is further defined and simplified through attempts at reducing the group sparsity residual. Towards this end, we first obtain a good estimation of the group sparse coefficient of each original image group by exploiting the image nonlocal self-similarity (NSS) prior along with self-supervised learning scheme, and then the group sparse coefficient of the corresponding degraded image group is enforced to approximate the estimation. To make the proposed scheme tractable and robust, two algorithms, i.e., iterative shrinkage/thresholding (IST) and alternating direction method of multipliers (ADMM), are employed to solve the proposed optimization problems for different image restoration tasks. Experimental results on image denoising, image inpainting and image compressive sensing (CS) recovery, demonstrate that the proposed GSRC-NLP based image restoration algorithm is comparable to state-of-the-art denoising methods and outperforms several testing image inpainting and image CS recovery methods in terms of both objective and perceptual quality metrics.
An Intelligent Video Analysis Method for Abnormal Event Detection in Intelligent Transportation Systems

Intelligent transportation systems pervasively deploy thousands of video cameras. Analyzing live video streams from these cameras is of significant importance to public safety. As streaming video is increasing, it becomes infeasible to have human operators sitting in front of hundreds of screens to catch suspicious activities or detect objects of interests in real-time. Actually, with millions of traffic surveillance cameras installed, video retrieval is more vital than ever. To that end, this article proposes a long video event retrieval algorithm based on superframe segmentation. By detecting the motion amplitude of the long video, a large number of redundant frames can be effectively removed from the long video, thereby reducing the number of frames that need to be calculated subsequently. Then, by using a superframe segmentation algorithm based on feature fusion, the remaining long video is divided into several Segments of Interest (SOIs) which include the video events. Finally, the trained semantic model is used to match the answer generated by the text question, and the result with the highest matching value is considered as the video segment corresponding to the question. Experimental results demonstrate that our proposed long video event retrieval and description method which significantly improves the efficiency and accuracy of semantic description, and significantly reduces the retrieval time.
Unsupervised Deep Representation Learning for Real-Time Tracking

The advancement of visual tracking has continuously been brought by deep learning models. Typically, supervised learning is employed to train these models with expensive labeled data. In order to reduce the workload of manual annotation and learn to track arbitrary objects, we propose an unsupervised learning method for visual tracking. The motivation of our unsupervised learning is that a robust tracker should be effective in bidirectional tracking. Specifically, the tracker is able to forward localize a target object in successive frames and backtrace to its initial position in the first frame. Based on such a motivation, in the training process, we measure the consistency between forward and backward trajectories to learn a robust tracker from scratch merely using unlabeled videos. We build our framework on a Siamese correlation filter network, and propose a multi-frame validation scheme and a cost-sensitive loss to facilitate unsupervised learning. Without bells and whistles, the proposed unsupervised tracker achieves the baseline accuracy of classic fully supervised trackers while achieving a real-time speed. Furthermore, our unsupervised framework exhibits a potential in leveraging more unlabeled or weakly labeled data to further improve the tracking accuracy.
Are We Evaluating Rigorously? Benchmarking Recommendation for Reproducible Evaluation and Fair Comparison

With tremendous amount of recommendation algorithms proposed every year, one critical issue has attracted a considerable amount of attention: there are no effective benchmarks for evaluation, which leads to two major concerns, i.e., unreproducible evaluation and unfair comparison. This paper aims to conduct rigorous (i.e., reproducible and fair) evaluation for implicit-feedback based top-N recommendation algorithms. We first systematically review 85 recommendation papers published at eight top-tier conferences (e.g., RecSys, SIGIR) to summarize important evaluation factors, e.g., data splitting and parameter tuning strategies, etc. Through a holistic empirical study, the impacts of different factors on recommendation performance are then analyzed in-depth. Following that, we create benchmarks with standardized procedures and provide the performance of seven well-tuned state-of-the-arts across six metrics on six widely-used datasets as a reference for later study. Additionally, we release a user-friendly Python toolkit, which differs from existing ones in addressing the broad scope of rigorous evaluation for recommendation. Overall, our work sheds light on the issues in recommendation evaluation and lays the foundation for further investigation. Our code and datasets are available at GitHub (https://github.com/AmazingDD/daisyRec).
A*3D Dataset: Towards Autonomous Driving in Challenging Environments

With the increasing global popularity of self-driving cars, there is an immediate need for challenging real-world datasets for benchmarking and training various computer vision tasks such as 3D object detection. Existing datasets either represent simple scenarios or provide only day-time data. In this paper, we introduce a new challenging A*3D dataset which consists of RGB images and LiDAR data with a significant diversity of scene, time, and weather. The dataset consists of high-density images (≈ 10 times more than the pioneering KITTI dataset), heavy occlusions, a large number of nighttime frames (≈ 3 times the nuScenes dataset), addressing the gaps in the existing datasets to push the boundaries of tasks in autonomous driving research to more challenging highly diverse environments. The dataset contains 39K frames, 7 classes, and 230K 3D object annotations. An extensive 3D object detection benchmark evaluation on the A*3D dataset for various attributes such as high density, day-time/night-time, gives interesting insights into the advantages and limitations of training and testing 3D object detection in real-world setting.