Reviewerly/exHarmony huggingface dataset:
An Exhaustive Investigation on Resource-aware Client Selection Mechanisms for Cross-device Federated Learning

Federated learning (FL) is a distributed machine learning technique in which each client (FLClient) trains a model without revealing it's local data to the server. Appropriate client selection is a crucial step towards ensuring the quality and robustness of the global model in a cross-device FL set-up. As such, various client selection mechanisms have been proposed, however, most of the mechanism makes the assumption of clients (devices) being mobile phones with uninterrupted power and compute resources supply. On the other hand, due to growing digitization in various industries, clients in a cross-device FL set-up can be resource-constrained IoT edge devices such as single board computers. To this end, there are a few resource-aware client selection mechanisms proposed in the literature. This paper provides a comprehensive, experimental comparative analysis of these mechanisms while resource-constrained IoT edge devices as clients. The effect of varying FL specific hyper-parameters on accuracy, convergence time and client retention is observed for all resource-aware client selection mechanisms so that a cognitive choice of the client selection mechanism can be made for a given application scenario.
Differentiable Neural Architecture Search for Extremely Lightweight Image Super-Resolution

Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. However, the large number of parameters in CNN-based methods for SISR tasks require heavy computations. Although several efficient SISR models have been recently proposed, most are handcrafted and thus lack flexibility. In this work, we propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models. Specifically, the cell-level search space is designed based on an information distillation mechanism, focusing on the combinations of lightweight operations and aiming to build a more lightweight and accurate SR structure. The network-level search space is designed to consider the feature connections among the cells and aims to find which information flow benefits the cell most to boost the performance. Unlike the existing Reinforcement Learning (RL) or Evolutionary Algorithm (EA) based NAS methods for SISR tasks, our search pipeline is fully differentiable, and the lightweight SISR models can be efficiently searched on both the cell-level and network-level jointly on a single GPU. Experiments show that our methods can achieve state-of-the-art performance on the benchmark datasets in terms of PSNR, SSIM, and model complexity with merely 68G Multi-Adds for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 2$ </tex-math></inline-formula> and 18G Multi-Adds for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 4$ </tex-math></inline-formula> SR tasks.
A Comprehensive Survey on Event Analysis Using Deep Learning

We can see multiple breakthroughs in various deep learning algorithms that have been successful in a variety of areas, including audio, video, images, text, and natural language processing (NLP) in the current environment (natural language processing). In this position, deep learning will help you understand the structure and meaning of sentences. This is a contemporary trend, and many academics are switching from standard machine learning to deep learning in their study. Face detection and biometric recognition, for example, have been investigated, and we learned that a significant amount of effort has gone into them. In this report, we examined several research publications from journals and conferences to compare various methodologies in data preparation, face detection, and sentiment analysis.
Local Path Planning for Unmanned Surface Vehicle based on the Improved DWA Algorithm

The dynamic windows approach (DW A) is widely used in local path planning, however the path planned by the traditional dynamic window approach will bypass the periphery of the dense obstacle area, which makes the distance longer and close to the dynamic obstacle. In view of the problem, this paper proposes a local path planning algorithm for usv based on the improved dynamic windows approach algorithm. Firstly, the commonly used path planning algorithm and DWA algorithm are briefly introduced. According to the problem of traditional dynamic windows approach, the concept of obstacle search angle is proposed to better deal with the impact of obstacles on USV navigation. Numerical simulation results show that the improved method is superior to the traditional algorithm in path length, navigation time and average speed, and has stronger obstacle avoidance adjustment ability.
ProGraph: Robust Network Traffic Identification With Graph Propagation

Network traffic identification is critical for effective network management. Existing methods mostly focus on invariant network environments with stable attribute distributions. Unfortunately, however, they can hardly be adaptive to the variation of practical networks and suffer from significant performance degradation. This problem largely stems from the over-dependence of existing methods on the vulnerable side-channel features. To address this issue, in this paper we propose a graph-based approach, namely ProGraph, to ensure robust network traffic classification among various network environments. The core idea of ProGraph is to construct a correlation graph with session clusters aggregated from different networks, based on which graph propagation can be effectively implemented to predict labels of testing nodes in an iterative manner. ProGraph enhances the correlation between clusters of the same class to provide reliable paths for label dissemination from the labeled clusters to the testing ones. It is encouraging to see that the proposed ProGraph achieves an accuracy of 92.25% in networks with constant attributes, while remaining stable with the accuracy of 90.89% when deployed in different networks, which significantly outperforms the state-of-the-art approaches. Meanwhile, ProGraph can accurately identify the novel classes which do not exist in the training dataset, with an AUC of 95.11. Last but not least, a carefully constructed dataset, namely CrossNet2021, containing network traffic of 20 classes of applications from two distinct networking scenarios, is made publicly available to support further research.
Geological Mapping using Mineral Absorption Feature-Guided Band-Ratios Applied to Prisma Satellite Hyperspectral Level 2D Imagery

Spaceborne Earth observation information (EOI) provides cost-effective, spatially comprehensive information for mineral exploration. The spatially contiguous EOI baseline data support geological mapping in addition to information about vegetation, water, and soils and are crucial for a more sustainable development of natural resources. This paper shows examples of PRISMA-derived mineral maps for mapping bedrock geology and regolith. Mineral absorption feature-guided band ratios were applied to PRISMA hyperspectral visible-near infrared (VNIR) and shortwave infrared (SWIR) imagery and compared with previously published airborne hyperspectral imagery for geological mapping. Felsic bedrock is successfully mapped using an Al-sheetsilicate abundance index and discriminated from mafic bedrocks identified by chlorite-epidote abundance and SWIR-active mafic minerals abundance indices. Regolith types are mapped by using the Fe (oxyhydr-) oxides abundance and the kaolin crystallinity index.
Sensorimotor Realities: Formalizing Ability-Mediating Design for Computer-Mediated Reality Environments

We introduce “Sensorimotor Realities,” a new concept in the XR landscape and corresponding technology-agnostic framework for computer-mediated perception and motor action. Sensorimotor Realities capitalize on the heterogeneity of human sensorimotor abilities to support conceptualization, characterization, and design of computer technology that leverages existing abilities in new, computer-mediated worlds. We introduce a conceptual space for Sensorimotor Realities with six dimensions, discuss examples of interactive systems in this space, and show how Sensorimotor Realities are distinct in nature and goal from Augmented, Mixed, Virtual, and Mediated Reality. We capitalize on Sensorimotor Realities to propose “ability-mediating design,” an approach to designing accessible interactive computer systems that complements ability-based design. We discuss how Sensorimotor Realities offer new opportunities for research and development at the intersection of XR, wearable computing, ambient intelligence, and accessible computing, and draw a research roadmap with three major directions of scientific investigation.
Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free

Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular trigger. Several works attempt to detect whether a given DNN has been injected with a specific trigger during the training. In a parallel line of research, the lottery ticket hypothesis reveals the existence of sparse sub-networks which are capable of reaching competitive performance as the dense network after independent training. Connecting these two dots, we investigate the problem of Trojan DNN detection from the brand new lens of sparsity, even when no clean training data is available. Our crucial observation is that the Trojan features are significantly more stable to network pruning than benign features. Leveraging that, we propose a novel Trojan network detection regime: first locating a “winning Trojan lottery ticket” which preserves nearly full Trojan information yet only chance-level performance on clean inputs; then recovering the trigger embedded in this already isolated sub-network. Extensive experiments on various datasets, i.e., CIFAR-10, CIFAR-100, and ImageNet, with different network architectures, i.e., VGG-16, ResNet-18, ResNet-20s, and DenseNet-100 demonstrate the effectiveness of our proposal. Codes are available at https://github.com/VITA-Group/Backdoor-LTH.
List-Decoding and List-Recovery of Reed–Solomon Codes Beyond the Johnson Radius for Every Rate

Understanding the limits of list-decoding and list-recovery of Reed-Solomon (RS) codes is of prime interest in coding theory and has attracted a lot of attention in recent decades. However, the best possible parameters for these problems are still unknown, and in this paper, we take a step in this direction. We show the existence of RS codes that are list-decodable or list-recoverable beyond the Johnson radius for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">every</i> rate, with a polynomial field size in the block length. In particular, we show that for every <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon \in (0,1)$ </tex-math></inline-formula> there exist RS codes that are list-decodable from radius <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1-\epsilon $ </tex-math></inline-formula> and rate less than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\frac {\epsilon }{2-\epsilon }$ </tex-math></inline-formula> , with constant list size. We deduce our results by extending and strengthening a recent result of Ferber, Kwan, and Sauermann on puncturing codes with large minimum distance and by utilizing the underlying code’s linearity.
Hand Gesture Recognition using YOLO Models for Hearing and Speech Impaired People

Sign language recognition can help hearing and speech impaired persons to communicate with the rest of the world. Hand gesture recognition is widely used in the entertainment sector for cars, gaming, and other devices. In the healthcare industry, clinicians can utilise this recognition technique to handle digital images instead of touch screens or computer keyboards during medical procedures where sterilization is necessary, as well as to automatically recognise surgical gestures. For real-time, image-based hand gesture identification, recent improvements in machine learning and object detection approaches provide improved and more efficient results. We used the deep learning-based object detection models YOLOX and YOLOv5 to work on five different hand gestures for recognition in our research project. Both models were released at the same time, but YOLOX was significantly more precise and faster. For YOLOv5, we got the highest mAP score of 98%, the lowest mAP score of 89.6%, the precision score of 98.8%, and the recall score of 82.6%. Our best mAP for YOLOX was 99.55%, while our last mAP was 98.5%. In this research study, we evaluated the performance of the two models and investigated changes in the architecture that enabled YOLOX to perform so much better than YOLOv5.
Optimizing Image Compression via Joint Learning with Denoising

High levels of noise usually exist in today’s captured images due to the relatively small sensors equipped in the smartphone cameras, where the noise brings extra challenges to lossy image compression algorithms. Without the capacity to tell the difference between image details and noise, general image compression methods allocate additional bits to explicitly store the undesired image noise during compression and restore the unpleasant noisy image during decompression. Based on the observations, we optimize the image compression algorithm to be noise-aware as joint denoising and compression to resolve the bits misallocation problem. The key is to transform the original noisy images to noise-free bits by eliminating the undesired noise during compression, where the bits are later decompressed as clean images. Specifically, we propose a novel two-branch, weight-sharing architecture with plug-in feature denoisers to allow a simple and effective realization of the goal with little computational cost. Experimental results show that our method gains a significant improvement over the existing baseline methods on both the synthetic and real-world datasets. Our source code is available at: https://github.com/felixcheng97/DenoiseCompression .
Object Detection: Algorithms and Prospects

As one of the most fundamental and challenging problems in computer vision, object detection has received tremendous attention in recent years. Its development over the past two decades can be seen as a microcosm of the history of computer vision. The goal of face detection is to find the corresponding positions of all the faces in the image. The output of the algorithm is the coordinates of the circumscribing rectangle of the face in the image, and may also include information such as attitude such as tilt angle. Algorithms are constantly updated to address higher accuracy, lower computational complexity, and faster detection. In view of the development of object detection technology, this paper introduces six different algorithms, R-CNN, fast R-CNN, faster R-CNN, YOLO, corner net, and Neural Architecture Search (NAS), to accomplish the purpose of detection. We finally provide the future prospect of object detection in several direction, which can give a further analysis for this area and hope to help the individuals who get started in object detection.
Forward Compatible Training for Large-Scale Embedding Retrieval Systems

In visual retrieval systems, updating the embedding model requires recomputing features for every piece of data. This expensive process is referred to as backfilling. Re-cently, the idea of backward compatible training (BCT) was proposed. To avoid the cost of backfilling, BCT modifies training of the new model to make its representations com-patible with those of the old model. However, BCT can sig-nificantly hinder the performance of the new model. In this work, we propose a new learning paradigm for representation learning: forward compatible training (FCT). In Fct, when the old model is trained, we also prepare for a future unknown version of the model. We propose learning side- information, an auxiliary feature for each sample which fa-cilitates future updates of the model. To develop a powerful and flexible frameworkfor model compatibility, we combine side-information with a forward transformation from old to newembeddings. Training of the new model is not modified, hence, its accuracy is not degraded. We demonstrate sig-nificant retrieval accuracy improvement compared to BCT for various datasets: ImageNet-1k (+18.1%), Places-365 (+5.4%), and VGG-Face2 (+8.3%). Fctobtains model compatibility when the new and old models are trained across different datasets, losses, and architectures. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code available at https://github.com/apple/ml-fct.
A Particle Swarm Optimization With Adaptive Learning Weights Tuned by a Multiple-Input Multiple-Output Fuzzy Logic Controller

In a canonical particle swarm optimization (PSO) algorithm, the fitness is a widely accepted criterion when selecting exemplars for a particle, which exhibits promising performance in simple unimodal functions. To improve a PSO's performance on complicated multimodal functions, various selection strategies based on the fitness value are introduced in PSO community. However, the inherent defects of the fitness-based selections still remain. In this paper, a novelty of a particle is treated as an additional criterion when choosing exemplars for a particle. In each generation, a few of elites and mavericks who have better fitness and novelty values are selected, and saved in two archives, respectively. Hence, in each generation, a particle randomly selects its own learning exemplars from the two archives, respectively. To strengthen a particle's adaptive capability, a multipleinput multiple-output fuzzy logic controller is used to adjust two parameters of the particle, i.e., an acceleration coefficient and a selection proportion of elites. The experimental results and comparisons between our new proposed PSO, named as MFCPSO in this paper, and other 6 PSO variants on CEC2017 test suite with 4 different dimension cases suggest that MFCPSO exhibits very promising characteristics on different types of functions, especially on large scale complicated functions. Furthermore, the effectiveness and efficiency of the fuzzy controlled parameters are discussed based on extensive experiments.
NMI-FGSM-Tri: An Efficient and Targeted Method for Generating Adversarial Examples for Speaker Recognition

Most existing deep neural networks (DNNs) are inexplicable and fragile, which can be easily deceived by carefully designed adversarial example with tiny undetectable noise. This allows attackers to cause serious consequences in many DNN-assisted scenarios without human perception. In the field of speaker recognition, the attack for speaker recognition system has been relatively mature. Most works focus on white-box attacks that assume the information of the DNN is obtainable, and only a few works study gray-box attacks. In this paper, we study blackbox attacks on the speaker recognition system, which can be applied in the real world since we do not need to know the system information. By combining the idea of transferable attack and query attack, our proposed method NMI-FGSM-Tri can achieve the targeted goal by misleading the system to recognize any audio as a registered person. Specifically, our method combines the Nesterov accelerated gradient (NAG), the ensemble attack and the restart trigger to design an attack method that generates the adversarial audios with good performance to attack blackbox DNNs. The experimental results show that the effect of the proposed method is superior to the extant methods, and the attack success rate can reach as high as 94.8% even if only one query is allowed.
Long-term Video Frame Interpolation via Feature Propagation

Video frame interpolation (VFI) works generally predict intermediate frame(s) by first estimating the motion between inputs and then warping the inputs to the target time with the estimated motion. This approach, however, is not optimal when the temporal distance between the input sequence increases as existing motion estimation modules cannot effectively handle large motions. Hence, VFI works perform well for small frame gaps and perform poorly as the frame gap increases. In this work, we propose a novel framework to address this problem. We argue that when there is a large gap between inputs, instead of estimating imprecise motion that will eventually lead to inaccurate interpolation, we can safely propagate from one side of the input up to a reliable time frame using the other input as a reference. Then, the rest of the intermediate frames can be interpolated using standard approaches as the temporal gap is now narrowed. To this end, we propose a propagation network (PNet) by extending the classic feature-level forecasting with a novel motion-to-feature approach. To be thorough, we adopt a simple interpolation model along with PNet as our full model and design a simple procedure to train the full model in an end-to-end manner. Experimental results on several benchmark datasets confirm the effectiveness of our method for long-term VFI compared to state-of-the-art approaches.
NeurMiPs: Neural Mixture of Planar Experts for View Synthesis

We present Neural Mixtures of Planar Experts (Neur-MiPs), a novel planar-based scene representation for modeling geometry and appearance. NeurMiPs leverages a collection of local planar experts in 3D space as the scene representation. Each planar expert consists of the parameters of the local rectangular shape representing geometry and a neural radiance field modeling the color and opacity. We render novel views by calculating ray-plane intersections and composite output colors and densities at intersected points to the image. NeurMiPs blends the efficiency of explicit mesh rendering and flexibility of the neural radiance field. Experiments demonstrate superior performance and speed of our proposed method, compared to other 3D representations in novel view synthesis.