Reviewerly/exHarmony huggingface dataset:
Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks

This paper studies a federated learning (FL) system, where <i>multiple</i> FL services co-exist in a wireless network and share common wireless resources. It fills the void of wireless resource allocation for multiple simultaneous FL services in the existing literature. Our method designs a two-level resource allocation framework comprising <i>intra-service</i> resource allocation and <i>inter-service</i> resource allocation. The intra-service resource allocation problem aims to minimize the length of FL rounds by optimizing the bandwidth allocation among the clients of each FL service. Based on this, an inter-service resource allocation problem is further considered, which distributes bandwidth resources among multiple simultaneous FL services. We consider both cooperative and selfish providers of the FL services. For cooperative FL service providers, we design a distributed bandwidth allocation algorithm to optimize the overall performance of multiple FL services, meanwhile catering it to the fairness among FL services and the privacy of clients. For selfish FL service providers, a new auction scheme is designed with the FL service providers as the bidders and the network operator as the auctioneer. The designed auction scheme strikes a balance between the overall FL performance and fairness. Our simulation results show that the proposed algorithms outperform other benchmarks under various network conditions.
Gradient Distribution Alignment Certificates Better Adversarial Domain Adaptation

The latest heuristic for handling the domain shift in un-supervised domain adaptation tasks is to reduce the data distribution discrepancy using adversarial learning. Recent studies improve the conventional adversarial domain adaptation methods with discriminative information by integrating the classifier’s outputs into distribution divergence measurement. However, they still suffer from the equilibrium problem of adversarial learning in which even if the discriminator is fully confused, sufficient similarity between two distributions cannot be guaranteed. To overcome this problem, we propose a novel approach named feature gradient distribution alignment (FGDA) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> . We demonstrate the rationale of our method both theoretically and empirically. In particular, we show that the distribution discrepancy can be reduced by constraining feature gradients of two domains to have similar distributions. Meanwhile, our method enjoys a theoretical guarantee that a tighter error upper bound for target samples can be obtained than that of conventional adversarial domain adaptation methods. By integrating the proposed method with existing adversarial domain adaptation models, we achieve state-of-the-art performance on two real-world benchmark datasets.
Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild

This paper addresses the problem of 3D human body shape and pose estimation from an RGB image. This is often an ill-posed problem, since multiple plausible 3D bodies may match the visual evidence present in the input - particularly when the subject is occluded. Thus, it is desirable to estimate a distribution over 3D body shape and pose conditioned on the input image instead of a single 3D re-construction. We train a deep neural network to estimate a hierarchical matrix-Fisher distribution over relative 3D joint rotation matrices (i.e. body pose), which exploits the human body’s kinematic tree structure, as well as a Gaussian distribution over SMPL body shape parameters. To further ensure that the predicted shape and pose distributions match the visual evidence in the input image, we implement a differentiable rejection sampler to impose a reprojection loss between ground-truth 2D joint coordinates and samples from the predicted distributions, projected onto the image plane. We show that our method is competitive with the state-of-the-art in terms of 3D shape and pose metrics on the SSP-3D and 3DPW datasets, while also yielding a structured probability distribution over 3D body shape and pose, with which we can meaningfully quantify prediction uncertainty and sample multiple plausible 3D reconstructions to explain a given input image.
Geometry Attention Transformer with position-aware LSTMs for image captioning

In recent years, Transformer structures have been widely applied in image captioning with impressive performance. However, previous works often neglect the geometry and position relations of different visual objects. These relations are often thought of as crucial information for good captioning results. Aiming to further promote the image captioning by Transformers, this paper proposes an improved Geometry Attention Transformer (GAT) framework. In order to obtain geometric representation ability, two novel geometry-aware architectures are designed respectively for the encoder and decoder in our GAT by i) a geometry gate-controlled self-attention refiner, and ii) a group of position-LSTMs. The first one explicitly incorporates relative spatial information into the image representations in encoding steps, and the second one precisely informs the decoder of relative word positions for generating caption texts. The image representations and spatial information are extracted by a pretrained Faster-RCNN network. Our ablation study has proved that these two designed optimization modules could efficiently improve the performance of image captioning. The experiment comparisons on the datasets MS COCO and Flickr30K, also show that our GAT could often outperform current state-of-the-art image captioning models.
Copyright and Artificial Creation: Does EU Copyright Law Protect AI-Assisted Output?

Abstract This article queries whether and to what extent works produced with the aid of AI systems – AI-assisted output – are protected under EU copyright standards. We carry out a doctrinal legal analysis to scrutinise the concepts of “work”, “originality” and “creative freedom”, as well as the notion of authorship, as set forth in the EU copyright acquis and developed in the case-law of the Court of Justice. On this basis, we develop a four-step test to assess whether AI-assisted output qualifies as an original work of authorship under EU law, and how the existing rules on authorship may apply. Our conclusion is that current EU copyright rules are generally suitable and sufficiently flexible to deal with the challenges posed by AI-assisted output.
Visual-inertial hand motion tracking with robustness against occlusion, interference, and contact

State-of-the-art technologies for hand (and finger) motion tracking do not always provide accurate and robust tracking. For example, severe occlusions can affect tracking with vision sensors, electromagnetic interference affects tracking with inertial measurement units (IMUs) and compasses, and ambiguous mechanical contact can affect tracking with soft sensors (i.e., the inability to distinguish motion-induced deformation). Here, we report a visual-inertial skeleton tracking (VIST) framework that provides robust and accurate hand tracking in a variety of real-world scenarios. Our proposed VIST framework comprises a sensor glove with multiple IMUs and passive visual markers as well as a head-mounted stereo camera. VIST also uses a tightly coupled filtering-based visual-inertial fusion algorithm to estimate the hand/finger motion and autocalibrates hand/glove-related kinematic parameters simultaneously while taking into account the hand anatomical constraints. Our VIST framework exhibits good tracking accuracy and robustness, affordable material cost, lightweight hardware and software, and durability to permit washing. We validate our VIST framework through quantitative and qualitative experiments in real-world conditions. Our approach to hand tracking has the potential to enrich not only human-robot interaction applications (e.g., direct humanoid hand teleoperation, hand-based collaborative robot programming, and drone swarm control) but also the user experience in many virtual reality and augmented reality applications.
Rehearsal revealed: The limits and merits of revisiting samples in continual learning

Learning from non-stationary data streams and overcoming catastrophic forgetting still poses a serious challenge for machine learning research. Rather than aiming to improve state-of-the-art, in this work we provide insight into the limits and merits of rehearsal, one of continual learning’s most established methods. We hypothesize that models trained sequentially with rehearsal tend to stay in the same low-loss region after a task has finished, but are at risk of overfitting on its sample memory, hence harming generalization. We provide both conceptual and strong empirical evidence on three benchmarks for both behaviors, bringing novel insights into the dynamics of rehearsal and continual learning in general. Finally, we interpret important continual learning works in the light of our findings, allowing for a deeper understanding of their successes. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Gillis: Serving Large Neural Networks in Serverless Functions with Automatic Model Partitioning

The increased use of deep neural networks has stimulated the growing demand for cloud-based model serving platforms. Serverless computing offers a simplified solution: users deploy models as serverless functions and let the platform handle provisioning and scaling. However, serverless functions have constrained resources in CPU and memory, making them inefficient or infeasible to serve large neural networks-which have become increasingly popular. In this paper, we present Gillis, a serverless-based model serving system that automatically partitions a large model across multiple serverless functions for faster inference and reduced memory footprint per function. Gillis employs two novel model partitioning algorithms that respectively achieve latency-optimal serving and cost-optimal serving with SLO compliance. We have implemented Gillis on three serverless platforms-AWS Lambda, Google Cloud Functions, and KNIX-with MXNet as the serving backend. Experimental evaluations against popular models show that Gillis supports serving very large neural networks, reduces the inference latency substantially, and meets various SLOs with a low serving cost.
Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting

Although CNNs have achieved remarkable progress on the shadow detection task, they tend to make mistakes in dark non-shadow regions and relatively bright shadow regions. They are also susceptible to brightness change. These two phenomenons reveal that deep shadow detectors heavily depend on the intensity cue, which we refer to as intensity bias. In this paper, we propose a novel feature decomposition and reweighting scheme to mitigate this intensity bias, in which multi-level integrated features are decomposed into intensity-variant and intensity-invariant components through self-supervision. By reweighting these two types of features, our method can reallocate the attention to the corresponding latent semantics and achieves balanced exploitation of them. Extensive experiments on three popular datasets show that the proposed method outperforms state-of-the-art shadow detectors.
Dynamic modeling for SO2-NOx emission concentration of circulating fluidized bed units based on quantum genetic algorithm - Extreme learning machine

To meet the stringent emission standards and achieve cleaner production of circulating fluidized bed units, it is necessary to build a dynamic model of pollutants emission for creating an economical and environmentally friendly pollutant removal operation mode. This article fources on the modeling and accurate prediction of SO2-NOx emission concentration of the circulating fluidized bed unit. According to the generation and reduction mechanism of pollutants, the model inputs are selected and determined by Pearson coefficient. Then, a dynamic model of SO2-NOx emission concentration based on extreme learning machine is developed, and quantum genetic algorithm is used to optimize the connection weight between the input layer and the hidden layer and threshold of the extreme learning machine, which contributes to increase prediction performance. The test result shows that the optimized model can effectively imitate the dynamic trends of actual measured data with appropriate accuracy, the mean absolute percentage error of SO2 concentration and NOx concentration are 4.63% and 3.09% respectively. And the model's satisfactory generalization ability is demonstrated based on the generalization experiment. In addition, compared with other methods, the proposed modeling method for SO2-NOx concentration has more suitability and accuracy under dynamic conditions, which contributes to online optimization of pollutant control and intelligent development of circulating fluidized bed units.
A Lightweight Adaptive RoI Extraction Network for Precise Aerial Image Instance Segmentation

Bounding boxes have been widely implemented into aerial object detection for its simplicity. They perform instance-level location with the coordinates and orientation for each target. But the defects such as coarse edge information impede semantic interpretation in earth observation. Besides, in terms of the aerial imaging instruments, it’s essential to recognize the exterior appearance and contour of the objects. In this paper, we propose a novel aerial instance segmentation method termed adaptive RoI extraction network (ARE-Net) which bridges the gap of accurately delineating instances under the complex back-ground of aerial images. To exert instance segmentation under the proprietary property, e.g., complex background, densely distributed instances, of aerial images, RoIs are pooled from multi-level feature maps and integral region proposals. On this basis, global attention RoI extractor (GA-RoIE) and perceptual RoI extractor (PRoIE) are respectively introduced for detection branch and mask branch to perform adaptive RoI extraction for aerial images. Meanwhile, to reconcile the probability distribution regional distribution of pixel-wise prediction in aerial images, we present the Adaptive Compound loss function to improve the integrating degree of the predicted binary mask to ground truth mask. Additionally, we adopt RegNetx with Deformable Convolution to optimize ARE-Net, and name it as R-ARE-Net. Despite implementing pixel-wise prediction, comprehensive experiments on iSAID and NWPU VHR-10 instance segmentation dataset still have verified the effectiveness and efficiency of ARE-Net and R-ARE-Net. Experimental results indicate that our proposed methods receive the highest AP value (38.0% AP on iSAID and 64.2% AP on NWPU VHR-10 instance segmentation dataset) and lowest FLOPs and Parameters consumption (~46% reduced FLOPs and 61.5% reduced Parameters than SCNet) among the mainstream methods. Besides, the false alarms, missing segmentations, poorly predicted masks, and under-segmentations that appeared in the mainstream methods can be avoided to some extend for R-ARE-Net.
Exploring Interactions Between Trust, Anthropomorphism, and Relationship Development in Voice Assistants

Modern conversational agents such as Alexa and Google Assistant represent significant progress in speech recognition, natural language processing, and speech synthesis. But as these agents have grown more realistic, concerns have been raised over how their social nature might unconsciously shape our interactions with them. Through a survey of 500 voice assistant users, we explore whether users' relationships with their voice assistants can be quantified using the same metrics as social, interpersonal relationships; as well as if this correlates with how much they trust their devices and the extent to which they anthropomorphise them. Using Knapp's staircase model of human relationships, we find that not only can human-device interactions be modelled in this way, but also that relationship development with voice assistants correlates with increased trust and anthropomorphism.
Toward AI-enabled augmented reality to enhance the safety of highway work zones: Feasibility, requirements, and challenges

Highway work zones are considered among the most hazardous working environments. In 2018 alone, 124 workers lost their lives to fatal accidents. The lack of predictive safety systems that notify workers of upcoming dangers in advance is a major reason to blame in the highway maintenance and operation community. This article presents an integrative design framework for bringing recent advances in Augmented Reality (AR) and Artificial Intelligence (AI) to enhance the safety of highway workers through real-time multimodal notifications on-spot. To this end, this article conceptualizes and co-designs three major pillars: (1) AR user interface design for multimodal notification, (2) real-time AI at the edge for vehicle detection/classification from distance, and (3) real-time wireless communication in work zone setting to enable latency-aware operation between AI and AR components. Our early results demonstrate that we can achieve 24.83 FPS end-to-end execution latency on the Xavier AGX Jetson board with 48.7% mAP on BDD100K dataset, and a real-time communication covering 120 meters with an average latency of 5.1 ms at the farthest distance. Our mixed-method user research also reveals an acceptable level of excitement and engagement from the body of highway workers toward both the proposed technology and the designed user interface. Overall, this article provides a proof-of-concept toward AI-enabled AR safety systems in highway work zones.
Reachable Polyhedral Marching (RPM): A Safety Verification Algorithm for Robotic Systems with Deep Neural Network Components

We present a method for computing exact reachable sets for deep neural networks with rectified linear unit (ReLU) activation. Our method is well-suited for use in rigorous safety analysis of robotic perception and control systems with deep neural network components. Our algorithm can compute both forward and backward reachable sets for a ReLU network iterated over multiple time steps, as would be found in a perception-action loop in a robotic system. Our algorithm is unique in that it builds the reachable sets by incrementally enumerating polyhedral cells in the input space, rather than iterating layer-by-layer through the network as in other methods. If an unsafe cell is found, our algorithm can return this result without completing the full reachability computation, thus giving an anytime property that accelerates safety verification. In addition, our method requires less memory during execution compared to existing methods where memory can be a limiting factor. We demonstrate our algorithm on safety verification of the ACAS Xu aircraft advisory system. We find unsafe actions many times faster than the fastest existing method and certify no unsafe actions exist in about twice the time of the existing method. We also compute forward and backward reachable sets for a learned model of pendulum dynamics over a 50 time step horizon in 87s on a laptop computer. Algorithm source code: https://github.com/StanfordMSL/Neural-Network-Reach.
Neighbor-view Enhanced Model for Vision and Language Navigation

Vision and Language Navigation (VLN) requires an agent to navigate to a target location by following natural language instructions. Most of existing works represent a navigation candidate by the feature of the corresponding single view where the candidate lies in. However, an instruction may mention landmarks out of the single view as references, which might lead to failures of textual-visual matching of existing methods. In this work, we propose a multi-module Neighbor-View Enhanced Model (NvEM) to adaptively incorporate visual contexts from neighbor views for better textual-visual matching. Specifically, our NvEM utilizes a subject module and a reference module to collect contexts from neighbor views. The subject module fuses neighbor views at a global level, and the reference module fuses neighbor objects at a local level. Subjects and references are adaptively determined via attention mechanisms. Our model also includes an action module to utilize the strong orientation guidance (e.g., "turn left'') in instructions. Each module predicts navigation action separately and their weighted sum is used for predicting the final action. Extensive experimental results demonstrate the effectiveness of the proposed method on the R2R and R4R benchmarks against several state-of-the-art navigators, and NvEM even beats some pre-training ones. Our code is available at https://github.com/MarSaKi/NvEM.
Collaborative Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images

In the past few years, multitask learning (MTL) has been widely used in a single model to solve the problems of multiple businesses. MTL enables each task to achieve high performance and greatly reduces computational resource overhead. In this work, we designed a collaborative network that simultaneously solves the super-resolution semantic segmentation and super-resolution image reconstruction. This algorithm can obtain high-resolution semantic segmentation and super-resolution reconstruction results by taking relatively low-resolution images as input when high-resolution data are inconvenient or computing resources are limited. The framework consists of three parts: the semantic segmentation branch (SSB), the super-resolution branch (SRB), and the structural affinity block (SAB). Specifically, the SSB, SRB, and SAB are responsible for completing super-resolution semantic segmentation, image super-resolution reconstruction, and associated features, respectively. Our proposed method is simple and efficient, and it can replace the different branches with most of the state-of-the-art models. The International Society for Photogrammetry and Remote Sensing (ISPRS) segmentation benchmarks were used to evaluate our models. In particular, super-resolution semantic segmentation on the Potsdam dataset reduced Intersection over Union (IoU) by only 1.8% when the resolution of the input image was reduced by a factor of two. The experimental results showed that our framework can obtain more accurate semantic segmentation and super-resolution reconstruction results than the single model.
Mutation Sensitive Correlation Filter for Real-Time UAV Tracking with Adaptive Hybrid Label

Unmanned aerial vehicle (UAV) based visual tracking has been confronted with numerous challenges, e.g., object motion and occlusion. These challenges generally introduce unexpected mutations of target appearance and result in tracking failure. However, prevalent discriminative correlation filter (DCF) based trackers are insensitive to target mutations due to a predefined label, which concentrates on merely the centre of the training region. Meanwhile, appearance mutations caused by occlusion or similar objects usually lead to the inevitable learning of wrong information. To cope with appearance mutations, this paper proposes a novel DCF-based method to enhance the sensitivity and resistance to mutations with an adaptive hybrid label, i.e., MSCF. The ideal label is optimized jointly with the correlation filter and remains temporal consistency. Besides, a novel measurement of mutations called mutation threat factor (MTF) is applied to correct the label dynamically. Considerable experiments are conducted on widely used UAV benchmarks. The results indicate that the performance of MSCF tracker surpasses other 26 state-of-the- art DCF-based and deep-based trackers. With a real-time speed of ~38 frames/s, the proposed approach is sufficient for UAV tracking commissions.