Reviewerly/exHarmony huggingface dataset:
Reinforcement Learning-Based Device Scheduling for Renewable Energy-Powered Federated Learning

Due to its unique privacy protection advantages, emerging federated learning (FL) is regarded as a significant technique to enable Industry 4.0. However, the industrial deployment of FL encounters the primary obstacles of limited device energy and system communication resources. Nowadays, renewable energy-powered devices have been deployed in various industrial fields to tackle the challenges of unsustainable and limited energy of battery-powered devices. Inspired by this, this article proposes a novel FL protocol to groundbreakingly improve the performance of renewable energy-powered FL systems. Specifically, with the underlying theory of FL as the guide, the proposed protocol features a reinforcement learning-based device scheduling solution to adapt to intermittent renewable energy supply. Following this device scheduling solution, an integer linear programming-based bandwidth management scheme is introduced to optimize communication efficiency. Experimental results on two representative data distribution situations demonstrate that compared with the state-of-the-art schemes, our FL protocol can boost up to 36.63% and 50.99% accuracy, respectively.
The Application of Computational Geometry for Automated UUV Mission Planning

UUV survey missions are complicated by environmental factors such as complex bathymetry, seamounts, depth constraints, shorelines, and even entanglement hazards such as wrecks, seaweed, fishing lines, and corals. Missions are also complicated by vehicle performance characteristics and limitations. Often, mission failures are due to planning faults, omissions, and oversights created by human operators while attempting to simultaneously mitigate these factors while contending With less-than-ideal planning conditions such as heat, sand, rain, choppy seas, etc. In this paper, we investigate automating path planning via the application of computational geometry algorithms to reduce the cognitive load on operators during mission planning activities, and accelerate the survey mission planning process. The desired goal is a reduction in mission and vehicle risk, faster overall mission planning which leads to more time actively on mission, and the creation of an automated mission planning pipeline for uncrewed systems. We have focused on the basic problem of automatically generating a set of laivn-moWer patterns that minimize total vehicle travel distance while covering a desired area and avoiding strict exclusion areas (e.g., islands, shoals, seamounts, intakes, or military operations areas). This can be rephrased as the basic computational geometry problem of finding a minimum set of rectangles that span the desired area and do not overlap with exclusion areas. Additionally, we can impose further restrictions such as minimizing coverage area outside the desired area (as in open Water around an island), or requiring no rectangle extend past the boundary of the desired search area (such as in a lake with boundary shorelines). We will describe the process by using well-known computational geometry techniques to automatically generate the set of rectangular regions and hoiv this will lead to the desired outcomes from this work. Further investigation is warranted for arbitrary polygons With more than one arbitrary exclusion region.
3D Random Occlusion and Multi-layer Projection for Deep Multi-camera Pedestrian Localization

Although deep-learning based methods for monocular pedestrian detection have made great progress, they are still vulnerable to heavy occlusions. Using multi-view information fusion is a potential solution but has limited applications, due to the lack of annotated training samples in existing multi-view datasets, which increases the risk of overfitting. To address this problem, a data augmentation method is proposed to randomly generate 3D cylinder occlusions, on the ground plane, which are of the average size of pedestrians and projected to multiple views, to relieve the impact of overfitting in the training. Moreover, the feature map of each view is projected to multiple parallel planes at different heights, by using homographies, which allows the CNNs to fully utilize the features across the height of each pedestrian to infer the locations of pedestrians on the ground plane. The proposed 3DROM method has a greatly improved performance in comparison with the state-of-the-art deep-learning based methods for multi-view pedestrian detection. Code is available at https://github.com/xjtlu-cvlab/3DROM .
Deep Convolution Neural Network for RBC Images

The suggested study’s objectives are to develop an unique criterion-based method for classifying RBC pictures and to increase classification accuracy by utilizing Deep Convolutional Neural Networks instead of Conventional CNN Algorithm. Materials and Procedures A dataset-master image dataset of 790 pictures is used to apply Deep Convolutional Neural Network. Convolutional Neural Network and Deep Convolutional Neural Network comparison using deep learning has been suggested and developed to improve classification accuracy of RBC pictures. Using Gpower, the sample size was calculated to be 27 for each group. Results: When compared to Convolutional Neural Network, Deep Convolutional Neural Network had the highest accuracy in classifying blood cell pictures (95.2%) and the lowest mean error (85.8 percent). Between the classifiers, there is a statistically significant difference of p=0.005. The study demonstrates that Deep Convolutional Neural Networks perform more accurately than Conventional Neural Networks while classifying photos of blood cells[1].
Generalizing Interactive Backpropagating Refinement for Dense Prediction Networks

As deep neural networks become the state-of-the-art approach in the field of computer vision for dense prediction tasks, many methods have been developed for automatic estimation of the target outputs given the visual inputs. Although the estimation accuracy of the proposed automatic methods continues to improve, interactive refinement is oftentimes necessary for further correction. Recently, feature backpropagating refinement scheme [25] (f-BRS) has been proposed for the task of interactive segmentation, which enables efficient optimization of a small set of auxiliary variables inserted into the pretrained network to produce object segmentation that better aligns with user inputs. However, the proposed auxiliary variables only contain channel-wise scale and bias, limiting the optimization to global refinement only. In this work, in order to generalize backpropagating refinement for a wide range of dense prediction tasks, we introduce a set of G-BRS (Generalized Backpropagating Refinement Scheme) layers that enable both global and localized refinement for the following tasks: interactive segmentation, semantic segmentation, image matting and monocular depth estimation. Experiments on SBD, Cityscapes, Mapillary Vista, Composition-1k and NYU-Depth-V2 show that our method can successfully generalize and significantly improve performance of existing pretrained state-of-the-art models with only a few clicks.
Dynamics Modeling and Simulation of Multi-rotor UAV based on the Composite Wind Field Model

To study the influence of compound wind field disturbance on the stability of multi-rotor UAV(unmanned aerial vehicle), a dynamics modeling and simulation method for multi-rotor UAV under the compound wind field model is proposed. Firstly, based on the idea of equivalent modeling in practical engineering, the models of uniform wind, gust, turbulence and wind shear are constructed respectively. The numerical analysis and simulation research are then carried out. Second, the dynamic model of multi rotor UAV under the action of composite wind field is deduced, and the attitude control model is derived. Finally, the numerical analysis software is utilized to simulate and analyze the attitude response of the multi-rotor UAV under the condition of wind and no wind, and analyze its control performance under the interference of compound wind field. The simulation results show that the dynamic model of the multi-rotor UAV under the influence of the wind field can accurately reflect its dynamic performance under the action of the wind field, and the output attitude angle fluctuation is less than 2°. The obtained results show that the established wind field model can be effectively applied to the research on the flight performance of multi-rotor UAV under the interference of compound wind field.
Structure and Motion from Casual Videos

Casual videos, such as those captured in daily life using a hand-held camera, pose problems for conventional structure-from-motion (SfM) techniques: the camera is often roughly stationary (not much parallax), and a large portion of the video may contain moving objects. Under such conditions, state-of-the-art SfM methods tend to produce erroneous results, often failing entirely. To address these issues, we propose CasualSAM, a method to estimate camera poses and dense depth maps from a monocular, casually-captured video. Like conventional SfM, our method performs a joint optimization over 3D structure and camera poses, but uses a pretrained depth prediction network to represent 3D structure rather than sparse keypoints. In contrast to previous approaches, our method does not assume motion is rigid or determined by semantic segmentation, instead optimizing for a per-pixel motion map based on reprojection error. Our method sets a new state-of-the-art for pose and depth estimation on the Sintel dataset, and produces high-quality results for the DAVIS dataset where most prior methods fail to produce usable camera poses.
A knowledge-driven activity recognition framework for learning unknown activities

Human activity recognition has increasingly received attention in recent years to track regular activities of people. Existing activity recognition approaches considerably contributed to the analysis of human behavior. However, they still confront numerous issues related to the variability of activities performed by people within dynamic environments. Generally, this variability renders the used training or ontology models with predefined activities unsuitable. Therefore, creating an activity recognition approach that is able to leverage dynamically new and unknown activities at runtime becomes important. In this paper, we propose a novel knowledge-driven activity recognition framework using smartphone. This framework envisions taking a knowledge-driven approach to reinforce the recognition accuracy and people's quality of life in the context of dynamic environments at runtime. More specifically, we propose an ontology-based context evolution along with a dynamic decision-making, so that new and unknown performed activities can be accurately recognized. Furthermore, we use a public activity recognition dataset to demonstrate the effectiveness of the proposed framework and show its advantage over a data-driven baselines in terms of accuracy. Experimental results reveal that our framework not only reinforces the accuracy, but also enables an effective activity learning when facing unknown activities at runtime.
Network Calibration by Temperature Scaling based on the Predicted Confidence

Calibrating neural networks is crucial in applications where the decision making depends on the predicted probabilities. Modern neural networks can be poorly calibrated. They tend to overestimate probabilities when compared to the expected accuracy. This results in a misleading reliability that corrupts our decision policy. We show that the magnitude of calibration error depends on the predicted confidence for each sample. This prediction confidence calibration paradigm is then applied to the concept of temperature scaling. We describe an optimization method that finds the suitable temperature scaling for each bin of a discretized value of prediction confidence. We report extensive experiments on a variety of image datasets and network architectures. Our approach achieves state-of-the-art calibration with a guarantee that the classification accuracy is not altered.
KAFL: Achieving High Training Efficiency for Fast-K Asynchronous Federated Learning

Federated Averaging (FedAvg) and its variants are prevalent optimization algorithms adopted in Federated Learning (FL) as they show good model convergence. However, such optimization methods are mostly running in a synchronous flavor which is plagued by the straggler problem, especially in the real-world FL scenario. Federated learning involves a massive number of resource-weak edge devices connected to the intermittent networks, exhibiting a vastly heterogeneous training environment. The asynchronous setting is a plausible solution to fulfill the resources utilization. Yet, due to data and device heterogeneity, the training bias and model staleness dramatically downgrade the model performance. This paper presents KAFL, a fast-K Asynchronous Federated Learning framework, to improve the system and statistical efficiency. KAFL allows the global server to iteratively collect and aggregate (1) the parameters uploaded by the fastest K edge clients (K-FedAsync); or (2) the first M updated parameters sent from any clients (Mstep-FedAsync). Compared to the fully asynchronous setting, KAFL helps the server obtain a better direction toward the global optima as it collects the information from at least K clients or M parameters. To further improve the convergence speed of KAFL, we propose a new weighted aggregation method which dynamically adjusts the aggregation weights according to the weight deviation matrix and client contribution frequency. Experimental results show that KAFL achieves a significant time-to-target-accuracy speedup on both IID and Non-IID datasets. To achieve the same model accuracy, KAFL reduces more than 50% training time for five CNN and RNN models, demonstrating the high training efficiency of our proposed framework.
Whitening Transformation inspired Self-Attention for Powerline Element Detection

Powerline inspection operations involve capturing and inspecting visual footage of powerline elements from elevated positions above and around the powerline and are currently performed with the help of helicopters and/or Unmanned Aerial Vehicles (UAVs). Current technological advances in the areas of robotics and machine learning are towards enabling fully autonomous operations. To this end, one of the tasks to be addressed is the robust, precise and fast powerline object detection problem. Recently introduced Transformer-based object detection methods demonstrate time and accuracy advances with respect to previous works. In this work, we present an enhanced Transformer-based architecture that further improves the state-of-the-art by incorporating a content-specific object query generator and by substituting the original attention operation with a whitening-inspired transformation at certain stages of the architecture. We evaluate our method in a recently captured powerline detection dataset and we show that our novel contributions offer a significant boost regarding detection accuracy.
Using Supplementary Properties to Reduce the Need for Documentation

The purpose of programming languages is to instruct computers and to communicate descriptions of computational processes among people. However, a program has to be supplemented by additional documentation using natural language, diagrams, specifications, models, etc. that also serve the purpose of supporting design and validation. A number of languages have been developed for this purpose, with the cost of having to maintain several descriptions. In this paper, we describe how to include supplementary properties in a programming language to reduce the need for additional languages and to reduce the need for additional documentation. Supplementary properties may restrict access to global variables, enforce restrictions on subclasses, specify required interfaces of modules, etc.
Energy-Aware Planning-Scheduling for Autonomous Aerial Robots

In this paper, we present an online planning-scheduling approach for battery-powered autonomous aerial robots. The approach consists of simultaneously planning a coverage path and scheduling onboard computational tasks. We further derive a novel variable coverage motion robust to airborne constraints and an empirically motivated energy model. The model includes the energy contribution of the schedule based on an automatic computational energy modeling tool. Our experiments show how an initial flight plan is adjusted online as a function of the available battery, accounting for uncertainty. Our approach remedies possible in-flight failure in case of unexpected battery drops, e.g., due to adverse atmospheric conditions, and increases the overall fault tolerance.
AziNorm: Exploiting the Radial Symmetry of Point Cloud for Azimuth-Normalized 3D Perception

Studying the inherent symmetry of data is of great importance in machine learning. Point cloud, the most important data format for 3D environmental perception, is naturally endowed with strong radial symmetry. In this work, we exploit this radial symmetry via a divide-and-conquer strategy to boost 3D perception performance and ease optimization. We propose Azimuth Normalization (AziNorm), which normalizes the point clouds along the radial direction and eliminates the variability brought by the difference of azimuth. AziNorm can be flexibly incorporated into most LiDAR-based perception methods. To validate its effectiveness and generalization ability, we apply AziNorm in both object detection and semantic segmentation. For detection, we integrate AziNorm into two representative detection methods, the one-stage SECOND detector and the state-of-the-art two-stage PV-RCNN detector. Experiments on Waymo Open Dataset demonstrate that AziNorm improves SECOND and PV-RCNN by 7.03 mAPH and 3.01 mAPH respectively. For segmentation, we integrate AziNorm into KPConv. On SemanticKitti dataset, AziNorm improves KPConv by 1.6/1.1 mIoU on val/test set. Besides, AziNorm remarkably improves data efficiency and accelerates convergence, reducing the requirement of data amounts or training epochs by an order of magnitude. SECOND w/ AziNorm can significantly outperform fully trained vanilla SECOND, even trained with only 10% data or 10% epochs. Code and models are available at https://github.com/hustvl/AziNorm.
Blockchain-Assisted Verifiable and Secure Remote Sensing Image Retrieval in Cloud Environment

Secure retrieval of remote sensing images in an outsourced cloud environment garners considerable attention. Since the cloud service provider (CSP) is considered as a semitrusted third party that may return incorrect retrieval results to save computational resources or defraud retrieval fees for profit, it becomes a critical challenge to achieve secure and verifiable remote sensing image retrieval. This article presents a secure retrieval and blockchain-assisted verifiable scheme for encrypted remote sensing images in the cloud environment. In response to the characteristic that geographical objects in remote sensing images with clear category attributes, we design a remote sensing image retrieval method to facilitate secure and efficient retrieval. In addition, we propose a verifiable method combined with blockchain and Merkle trees for checking the integrity and correctness of the storage and retrieval services provided by CSP, which can replace the traditional third-party auditor. The security analysis and experimental evaluation demonstrate the security, verifiability, and feasibility of the proposed scheme, achieving secure remote sensing image retrieval while preventing malicious behavior of CSP.
VizWiz-FewShot: Locating Objects in Images Taken by People with Visual Impairments

We introduce a few-shot localization dataset originating from photographers who authentically were trying to learn about the visual content in the images they took. It includes nearly 10,000 segmentations of 100 categories in over 4,500 images that were taken by people with visual impairments. Compared to existing few-shot object detection and instance segmentation datasets, our dataset is the first to locate holes in objects (e.g., found in 12.3% of our segmentations), it shows objects that occupy a much larger range of sizes relative to the images, and text is over five times more common in our objects (e.g., found in 22.4% of our segmentations). Analysis of three modern few-shot localization algorithms demonstrates that they generalize poorly to our new dataset. The algorithms commonly struggle to locate objects with holes, very small and very large objects, and objects lacking text. To encourage a larger community to work on these unsolved challenges, we publicly share our annotated few-shot dataset at https://vizwiz.org .
Quaternion-Based Graph Contrastive Learning for Recommendation

Graph Convolution Network (GCN) has been applied in recommendation with various architectures for its representation learning capability in graph-structured data. Despite existing GCN-based recommendation models successfully capturing the user-item interactions, they still suffer from two limitations. On the one hand, they model users and items in the Euclidean space with real-value embeddings, which have high distortion when modeling complex graphs. On the other hand, they have not fully explored contrastive learning for GCN-based recommendations. Simply applying augmentation pairs of the same type may make features less generalizable and lead to sub-optimal performance. To this end, in this paper, we propose a Quaternion-based Graph Contrastive Learning (QGCL) recommendation model. It embeds all users and items into the Quaternion space and performs message propagation with quaternion graph convolution layers. Moreover, we attempt to compose different types of data augmentations for augmented views in graph contrastive learning as an auxiliary task. We evaluate the proposed model using three public datasets, and experimental results demonstrate significant improvements over the state-of-the-art methods by a large margin.
YOLOv2-tiny Target Detection System Based on FPGA Platform

Convolutional neural networks (CNN) are increasingly used for target detection due to the development of deep learning. Currently, most CNNs are predominantly deployed on CPU and GPU platforms. Due to the restrictions of speed and power, it is difficult for the CPU and GPU-based CNN target identification system to match the needs of mobile scenarios. Field Programmable Gate Array (FPGA) may overcome the issues of complex target detection network architecture, extensive computation, and high storage overhead due to its high parallelism and low power consumption. So, an FPGA-based target detection system is designed in this paper. This study examines the network structure and data processing flow of the YOLOv2-Tiny model to lower the system's computing and resource needs by combining layers and employing 16-bit fixed-point quantization. In addition, a dual cache mechanism and a multi-channel transmission design are implemented to improve data storage and transfer. This YOLOv2-Tiny target detection system implemented using MZ7100 FPGA achieves a throughput of 47.33 GOPs at 150 MHz with batch processing while consuming 2.74 W of on-chip power and consumes 26.93 ms to recognize a single image frame. Compared with the Intel Core i5 CPU, the efficiency is 53 times. Although the recognition speed is not as fast as the GPU, the power consumption is significantly reduced, which meets the requirements for target recognition in embedded environments.