Reviewerly/exHarmony huggingface dataset:
Cyclical Pruning for Sparse Neural Networks

Current methods for pruning neural network weights iteratively apply magnitude-based pruning on the model weights and re-train the resulting model to recover lost accuracy. In this work, we show that such strategies do not allow for the recovery of erroneously pruned weights. To enable weight recovery, we propose a simple strategy called cyclical pruning which requires the pruning schedule to be periodic and allows for weights pruned erroneously in one cycle to recover in subsequent ones. Experimental results on both linear models and large-scale deep neural networks show that cyclical pruning outperforms existing pruning algorithms, especially at high sparsity ratios. Our approach is easy to tune and can be readily incorporated into existing pruning pipelines to boost performance.
Detecting Real-Time Deep-Fake Videos Using Active Illumination

While many have grown suspicious of viral images and videos found online, there is a general sense that we can and should trust that the person on the other end of our videoconferencing call is who it purports to be. The real-time creation of sophisticated deep fakes, however, is making it more difficult to trust even live video calls. Detecting deep fakes in real time introduces new challenges as compared to off-line forensic analyses. We describe a technique for detecting, in real-time, deep-fake videos transmitted over a live video-conferencing application. This technique leverages the fact that a video call typically places a user in front of a light source (the computer display) which can be manipulated to induce a controlled change in the appearance of the user’s face. Deviations of the expected change in appearance over time can be measured in real time and used to verify the authenticity of a video-call participant.
Variable Few Shot Class Incremental and Open World Learning

Prior work on few-shot class incremental learning has operated with an unnatural assumption: the number of ways and number of shots are assumed to be known and fixed e.g., 10-ways 5-shots, 5-ways 5-shots, etc. Hence, we refer to this setting as Fixed-Few-Shot Class Incremental Learning (FFSCIL). In practice, the pre-specified fixed number of classes and examples per class may not be available, meaning one cannot update the model. Evaluation of FSCIL approaches in such unnatural settings renders their applicability questionable for practical scenarios where such assumptions do not hold. To mitigate the limitation of FFSCIL, we propose Variable-Few-Shot Class Incremental Learning (VFSCIL) and demonstrate it with Up-to N-Ways, Up-to K-Shots class incremental learning; wherein each incremental session, a learner may have up to N classes and up to K samples per class. Consequently, conventional FFSCIL is a special case of herein introduced VFSCIL. Further, we extend VFSCIL to a more practical problem of Variable-Few-Shot Open-World Learning (VFSOWL), where an agent is not only required to perform incremental learning, but must detect unknown samples and enroll only those that it detects correctly. We formulate and study VFSCIL and VFSOWL on two benchmark datasets conventionally employed for FFSCIL i.e., Caltech-UCSD Birds-200-2011 (CUB200) and miniImageNet. First, to serve as a strong baseline, we extend the state-of-the-art FSCIL approach to operate in Up-to N-Ways, Up-to K-Shots class incremental and open-world settings. Then, we propose a novel but simple approach for VFSCIL/VFSOWL where we leverage the current advancements in self-supervised feature learning. Utilizing both benchmark datasets, our proposed approach outperforms the strong baseline on the conventional FFSCIL setting and newly introduced VFSCIL/VFSOWL settings. Our code is available at: https://github.com/TouqeerAhmad/VFSOWL
Exploring Robustness Connection between Artificial and Natural Adversarial Examples

Although recent deep neural network algorithm has shown tremendous success in several computer vision tasks, their vulnerability against minute adversarial perturbations has raised a serious concern. In the early days of crafting these adversarial examples, artificial noises are optimized through the network and added in the images to decrease the confidence of the classifiers against the true class. However, recent efforts are showcasing the presence of natural adversarial examples which can also be effectively used to fool the deep neural networks with high confidence. In this paper, for the first time, we have raised the question that whether there is any robustness connection between artificial and natural adversarial examples. The possible robustness connection between natural and artificial adversarial examples is studied in the form that whether an adversarial example detector trained on artificial examples can detect the natural adversarial examples. We have analyzed several deep neural networks for the possible detection of artificial and natural adversarial examples in seen and unseen settings to set up a robust connection. The extensive experimental results reveal several interesting insights to defend the deep classifiers whether vulnerable against natural or artificially perturbed examples. We believe these findings can pave a way for the development of unified resiliency because defense against one attack is not sufficient for real-world use cases.
Learning Generalized Feature for Temporal Action Detection: Application for Natural Driving Action Recognition Challenge

This paper reports our approach for the 2022 AI City Challenge - Naturalistic Driving Action Recognition (Track 3), where the objective is to detect when and what kinds of actions that a driver performs in a long, untrimmed video. Our solution is built upon the single stage ActionFormer detector, in which temporal location and classification are predicted simultaneously for efficiency. The input feature for the detector is extracted offline using our proposed backbone, which we named "ConvNext-Video". However, due to the small size of the dataset, training the model to avoid over-fitting becomes challenging. To address this problem, we focus on training techniques that can improve the generalization of underlying features. Specifically, we utilize two methods: "learning without forgetting" and semi-weak supervised learning on the unlabeled data A2. Finally, we also add a second-stage classifier (SSC) using our ConvNeXt-Video backbone. The SSC Classifer is designed to combine information from multi-clips and multi-view cameras to improve the prediction precision. Our best result achieves 29.1 F1 score on the public test set. Our source code is released at link.
MAPLE-Edge: A Runtime Latency Predictor for Edge Devices

Neural Architecture Search (NAS) has enabled automatic discovery of more efficient neural network architectures, especially for mobile and embedded vision applications. Although recent research has proposed ways of quickly estimating latency on unseen hardware devices with just a few samples, little focus has been given to the challenges of estimating latency on runtimes using optimized graphs, such as TensorRT and specifically for edge devices. As devices like NVIDIA’s Jetsons get more popular in embedded computing and robotics, we observe a pressing need to more accurately estimate inference latency of neural network architectures on diverse runtimes, including highly optimized ones. In this work, we propose MAPLE-Edge, an edge device-oriented extension of MAPLE, the state-of-the-art latency predictor for general purpose hardware, where we train a regression network on architecture-latency pairs in conjunction with a hardware-runtime descriptor to effectively estimate latency on a diverse pool of edge devices. Compared to MAPLE, MAPLE-Edge can describe the runtime and target device platform using a much smaller set of CPU performance counters that are widely available on all Linux kernels, while still achieving up to +49.6% accuracy gains against previous state-of-the-art baseline methods on optimized edge device runtimes, using just 10 measurements from an unseen target device. We also demonstrate that unlike MAPLE which performs best when trained on a pool of devices sharing a common runtime, MAPLE-Edge can effectively generalize across runtimes by applying a trick of normalizing performance counters by the operator latency, in the measured hardware-runtime descriptor. Lastly, we show that for runtimes exhibiting lower than desired accuracy, performance can be boosted by collecting additional samples from the target device, with an extra 90 samples translating to gains of nearly +40%.
SCVRL: Shuffled Contrastive Video Representation Learning

We propose SCVRL, a novel contrastive-based framework for self-supervised learning for videos. Differently from previous contrast learning based methods that mostly focus on learning visual semantics (e.g., CVRL), SCVRL is capable of learning both semantic and motion patterns. For that, we reformulate the popular shuffling pretext task within a modern contrastive learning paradigm. We show that our transformer-based network has a natural capacity to learn motion in self-supervised settings and achieves strong performance, outperforming CVRL on four benchmarks.
Guided Deep Metric Learning

Deep Metric Learning (DML) methods have been proven relevant for visual similarity learning. However, they sometimes lack generalization properties because they are trained often using an inappropriate sample selection strategy or due to the difficulty of the dataset caused by a distributional shift in the data. These represent a significant drawback when attempting to learn the underlying data manifold. Therefore, there is a pressing need to develop better ways of obtaining generalization and representation of the underlying manifold. In this paper, we propose a novel approach to DML that we call Guided Deep Metric Learning, a novel architecture oriented to learning more compact clusters, improving generalization under distributional shifts in DML. This novel architecture consists of two independent models: A multi-branch master model, inspired from a Few-Shot Learning (FSL) perspective, generates a reduced hypothesis space based on prior knowledge from labeled data, which guides or regularizes the decision boundary of a student model during training under an offline knowledge distillation scheme. Experiments have shown that the proposed method is capable of a better manifold generalization and representation to up to 40% improvement (Recall@1, CIFAR10), using guidelines suggested by Musgrave et al. to perform a more fair and realistic comparison, which is currently absent in the literature.
A TV regularisation sparse light field reconstruction model based on guided-filtering

Obtaining and representing the 4D light field is important for a number of computer vision applications. Due to the high dimensionality, acquiring the light field directly is costly. One way to overcome this deficiency is to reconstruct the light field from a limited number of measurements. Existing approaches involve either a depth estimation process or require a large number of measurements to obtain high-quality reconstructed results. In this paper, we propose a total variation (TV) regularisation sparse model with the alternating direction method of multipliers (ADMM) based on guided filtering, which addresses this depth-dependence problem with only a few measurements. As one of the sparse optimisation methods, TV regularisation based on ADMM is well suited to solve ill-posed problems such as this. Moreover, guided filtering has good edge-preserving smoothing properties, which can be incorporated into the light field reconstruction process. Therefore, high precision light field reconstruction is established with our model. Specifically, the updated image in the iteration step contains the guidance image, and an initialiser for the least squares method using a QR factorisation (LSQR) algorithm is involved in one of the subproblems. The model outperforms other methods in both visual assessments and objective metrics – in simulation experiments from synthetic data and photographic data using produced focal stacks from light field contents – and it works well in experiments using captured focal stacks. We also show a further application for arbitrary refocusing by using the reconstructed light field. • Acquiring the light field from a limited number of measurements is useful. • A general light field imaging model is established. • A total variation regularisation sparse method based on guided-filtering is proposed. • Our method shows higher performance compared with other methods.
Wearable ImageNet: Synthesizing Tileable Textures via Dataset Distillation

Recent methods for Dataset Distillation are able to take in a large set of images of a specific class (e.g., from ImageNet) and synthesize a single image, such that a classifier trained on that image could perform similarly to one trained on the original dataset. It was noticed that the resulting "distilled images" are often quite visually pleasing. In this paper, we describe a simple method for generating tileable distilled textures by sampling random crops from a toroidal canvas of synthetic pixels while enforcing that all such crops serve as effective distilled training data. Such distilled textures not only summarize a given image category in a visually interesting way, but also allow for generation of infinite texture patterns suitable for printing on fabric, clothing, etc. This paper might be just the first step in making the ImageNet dataset into a fashion statement.
Partial Depth Estimation with Single Image Using YOLO and CNN for Robot Arm Control

This paper presents experimental results of partial object detection using YOLO (You Only Look Once) and partial depth estimation using CNN (Convolutional Neural Network) for application to a robot arm control. In recent years, image recognition and its application automation, as an alternative to advanced work, are attracting attention in various fields. In order for robot arms used in factories to perform high-value-added and flexible work, it is necessary to control them by object detection and recognition by deep learning. In this study, the authors propose a new approach for estimating the depth of partial images using YOLO and uses it to control the robot arm. In the experiments, both ends of a pen detected by YOLO are used for the input to a CNN. The detected parts are saved to images with a size of about 60 × 60 pixels, and the depths are estimated by giving the cropped images to the CNN. A desktop-sized robot with 4DOFs can successfully pick the pen by referring the depths. The effectiveness of the proposed method is demonstrated through experiments.
Deep Visual Constraints: Neural Implicit Models for Manipulation Planning From Visual Input

Manipulation planning is the problem of finding a sequence of robot configurations that involves interactions with objects in the scene, e.g., grasping and placing an object, or more general tool-use. To achieve such interactions, traditional approaches require hand-engineering of object representations and interaction constraints, which easily becomes tedious when complex objects/interactions are considered. Inspired by recent advances in 3D modeling, e.g. NeRF, we propose a method to represent objects as continuous functions upon which constraint features are defined and jointly trained. In particular, the proposed pixel-aligned representation is directly inferred from images with known camera geometry and naturally acts as a perception component in the whole manipulation pipeline, thereby enabling long-horizon planning <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only from visual input</i> .
Improved Artificial Potential Field Method for Mobile Robots Path Planning in a Corridor Environment

Road constraints at corridor corners are typical in mine environments, power inspections, and attraction navigation. However, artificial potential field (APF) algorithm research is mainly based on scattered distributed obstacles, and less attention has been paid to robot path planning in corridor environments by using the improved APF method. This paper proposes an improved APF (FV-APF) algorithm to address the local minimal and goal unattainability problems of the classic APF method in the corridor environment by considering the angle constraint and improving the potential field function while combining simple fuzzy ideas. The results show that the FV-APF algorithm can plan paths very well in the complicated cornering environment of the corridor.
Active Disturbance Rejection Control of Quadrotor UAV based on Whale Optimization Algorithm

This paper aims at the problems of nonlinearity, underactuated, strong coupling, and difficult controller parameter tuning in the control process of quadrotor UAV, PID controller and ADRC are designed to control the quadrotor UAV, and the controller parameters are optimized by the Whale Optimization Algorithm. Firstly, the ADRC is designed to control the attitude loop of the quadrotor UAV. The Extended State Observer (ESO) is designed for yaw, pitch, and roll channels to observe and compensate the internal uncertainties and external disturbances in real-time, and to realize decoupling control. Secondly, in the position control system, the PID controller is used to realize the stable tracking of position variables. Finally, the Whale Optimization Algorithm (WOA) is designed to optimize the Active Disturbance Rejection Controller and PID controller for the quadrotor UAV, which has many controller parameters and is difficult to get the optimal control effect. In this method, the parameters of the controller are taken as the objective of WOA optimization to achieve the optimal control effect. The simulation results show that the WOA optimized controller has a smaller overshoot and faster adjustment time compared with human tuning parameters.
Double enhanced residual network for biological image denoising

With the achievements of deep learning, applications of deep convolutional neural networks for the image denoising problem have been widely studied. However, these methods are typically limited by GPU in terms of network layers and other aspects. This paper proposes a multi-level network that can efficiently utilize GPU memory, named Double Enhanced Residual Network (DERNet), for biological-image denoising. The network consists of two sub-networks, and U-Net inspires the basic structure. For each sub-network, the encoder-decoder hierarchical structure is used for down-scaling and up-scaling feature maps so that GPU can yield large receptive fields. In the encoder process, the convolution layers are used for down-sampling to obtain image information, and residual blocks are superimposed for preliminary feature extraction. In the operation of the decoder, transposed convolution layers have the capability to up-sampling and combine with the Residual Dense Instance Normalization (RDIN) block that we propose, extract deep features and restore image details. Finally, both qualitative experiments and visual effects demonstrate the effectiveness of our proposed algorithm.
Self-supervised graph representation learning via positive mining

Inspired by the recent success of self-supervised methods applied on images, self-supervised learning on graph structured data has seen rapid growth especially centered on augmentation-based contrastive methods. However, these methods pay little attention to the inherent distinction between images and graphs: while augmentation is well defined on images, it may behave arbitrarily on graphs. That is, without carefully designed augmentation techniques, augmentations on graphs may behave arbitrarily in that the underlying semantics of graphs can drastically change. As a consequence, the performance of existing augmentation-based methods is highly dependent on the choice of augmentation scheme, i.e., hyperparameters associated with augmentations. In this paper, we propose a novel augmentation-free self-supervised learning method for graphs, named AFGRL, which learns graph representations without using any arbitrary augmentation scheme. Specifically, we generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics within the graph. Extensive experiments toward various node-level tasks (i.e., node classification, clustering, and similarity search) and graph-level tasks (i.e. graph classification) on various real-world datasets demonstrate the superiority of AFGRL.
A Bitter Pill to Swallow? The Consequences of Patient Evaluation in Online Health Question-and-Answer Platforms

Online health question-and-answer platforms (OHQPs), where patients post health-related questions, evaluate advice from multiple doctors and select their most preferred answer, are a prominent channel for patients to receive medical advice in China. They are gaining traction globally and have the potential to circumvent resource-based, geographic, or circumstantial barriers that limit access to care. At the same time, they are underregulated in contrast to brick-and-mortar points of care. Ours is the first study to evaluate the quality of healthcare advice promoted by these platforms and to provide insight into how patients respond to advice. We found that, though advice is generally good, patients cannot discern good advice from bad, choose poor advice (when offered) as often as good advice, and do so to a greater extent in vulnerable categories such as pediatrics, cancers/tumors, and internal medicine. Moreover, we found that OHQPs exacerbate care avoidance. Our findings suggest that platform owners and policymakers should ensure that signals of expert consensus are provided to patients to better assist their choices on OHQPs. We also unveiled bad actors on OHQPs, including drug promoters and spammers, indicating that stronger oversight and accountability mechanisms are needed. Physician peer reviewing and auditing can address both problems.
Secure and Efficient Certificateless Searchable Authenticated Encryption Scheme Without Random Oracle for Industrial Internet of Things

Today, the development of the Industrial Internet of Things (IIoT) has led to a rapid increase in industrial data. More and more industrial enterprises outsource local data to cloud service providers. To ensure the security of user data, encryption technology is necessary. Certificateless searchable authenticated encryption (CLSAE) can solve the problem of ciphertext retrieval for the IIoT. The security of most of the existing CLSAE schemes is proved in the random oracle model, but these schemes may have security risks in real application scenarios. We provide a new CLSAE scheme, which can resist both inside and outside keyword guessing attacks, and its security is proved in the standard model. We compare the scheme proposed in this article with other existing CLSAE schemes. Not using expensive operations, the new scheme is more efficient and more in line with the needs of the IIoT.
Deep Reinforcement Learning based Target Detection for Unmanned Aerial Vehicle

Considering the automation for delivering items using unmanned aerial vehicles has emerged with many research and studies. The research is proposed with one vital aspect to provide foremost necessary medical aids by reaching accurately to patients. This paper deals with developing an integrated camera-based positioning system with GPS for quadcopters to automate the landing process. In addition to the frontal-viewing camera, a downward-viewing camera is used for landing at the identified location based on a QR code. The efficient deep learning-based method for object detection is followed by a pixel-extraction-based method for the code recognition to identify the site accurately.