OpenRaiser/Intern-Atlas huggingface dataset:
A Survey on Participant Selection for Federated Learning in Mobile Networks
Venue: arXiv cs.DC 2022
Year: 2022

Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy-preserving manner. The main challenges of FL is that end devices usually possess various computation and communication capabilities and their training data are not independent and identically distributed (non-IID). Due to limited communication bandwidth and unstable availability of such devices in a mobile network, only a fraction of end devices (also referred to as the participants or clients in a FL process) can be selected in each round. Hence, it is of paramount importance to utilize an efficient participant selection scheme to maximize the performance of FL including final model accuracy and training time. In this paper, we provide a review of participant selection techniques for FL. First, we introduce FL and highlight the main challenges during participant selection. Then, we review the existing studies and categorize them based on their solutions. Finally, we provide some future directions on participant selection for FL based on our analysis of the state-of-the-art in this topic area.
SST-Calib: Simultaneous Spatial-Temporal Parameter Calibration between LIDAR and Camera
Venue: arXiv cs.CV 2022
Year: 2022

With information from multiple input modalities, sensor fusion-based algorithms usually out-perform their single-modality counterparts in robotics. Camera and LIDAR, with complementary semantic and depth information, are the typical choices for detection tasks in complicated driving environments. For most camera-LIDAR fusion algorithms, however, the calibration of the sensor suite will greatly impact the performance. More specifically, the detection algorithm usually requires an accurate geometric relationship among multiple sensors as the input, and it is often assumed that the contents from these sensors are captured at the same time. Preparing such sensor suites involves carefully designed calibration rigs and accurate synchronization mechanisms, and the preparation process is usually done offline. In this work, a segmentation-based framework is proposed to jointly estimate the geometrical and temporal parameters in the calibration of a camera-LIDAR suite. A semantic segmentation mask is first applied to both sensor modalities, and the calibration parameters are optimized through pixel-wise bidirectional loss. We specifically incorporated the velocity information from optical flow for temporal parameters. Since supervision is only performed at the segmentation level, no calibration label is needed within the framework. The proposed algorithm is tested on the KITTI dataset, and the result shows an accurate real-time calibration of both geometric and temporal parameters.
Video-based Smoky Vehicle Detection with A Coarse-to-Fine Framework
Venue: arXiv cs.CV 2022
Year: 2022

Automatic smoky vehicle detection in videos is a superior solution to the traditional expensive remote sensing one with ultraviolet-infrared light devices for environmental protection agencies. However, it is challenging to distinguish vehicle smoke from shadow and wet regions coming from rear vehicle or clutter roads, and could be worse due to limited annotated data. In this paper, we first introduce a real-world large-scale smoky vehicle dataset with 75,000 annotated smoky vehicle images, facilitating the effective training of advanced deep learning models. To enable fair algorithm comparison, we also build a smoky vehicle video dataset including 163 long videos with segment-level annotations. Moreover, we present a new Coarse-to-fine Deep Smoky vehicle detection (CoDeS) framework for efficient smoky vehicle detection. The CoDeS first leverages a light-weight YOLO detector for fast smoke detection with high recall rate, and then applies a smoke-vehicle matching strategy to eliminate non-vehicle smoke, and finally uses a elaborately-designed 3D model to further refine the results in spatial temporal space. Extensive experiments in four metrics demonstrate that our framework is significantly superior to those hand-crafted feature based methods and recent advanced methods. The code and dataset will be released at https://github.com/pengxj/smokyvehicle.
Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation
Venue: arXiv cs.CV 2022
Year: 2022

Sign language is the window for people differently-abled to express their feelings as well as emotions. However, it remains challenging for people to learn sign language in a short time. To address this real-world challenge, in this work, we study the motion transfer system, which can transfer the user photo to the sign language video of specific words. In particular, the appearance content of the output video comes from the provided user image, while the motion of the video is extracted from the specified tutorial video. We observe two primary limitations in adopting the state-of-the-art motion transfer methods to sign language generation:(1) Existing motion transfer works ignore the prior geometrical knowledge of the human body. (2) The previous image animation methods only take image pairs as input in the training stage, which could not fully exploit the temporal information within videos. In an attempt to address the above-mentioned limitations, we propose Structure-aware Temporal Consistency Network (STCNet) to jointly optimize the prior structure of human with the temporal consistency for sign language video generation. There are two main contributions in this paper. (1) We harness a fine-grained skeleton detector to provide prior knowledge of the body keypoints. In this way, we ensure the keypoint movement in a valid range and make the model become more explainable and robust. (2) We introduce two cycle-consistency losses, i.e., short-term cycle loss and long-term cycle loss, which are conducted to assure the continuity of the generated video. We optimize the two losses and keypoint detector network in an end-to-end manner.
Revisiting Chernoff Information with Likelihood Ratio Exponential Families
Venue: arXiv cs.IT 2022
Year: 2022

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback--Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.
Enabling and Assessing Trust when Cooperating with Robots in Disaster Response (EASIER)
Venue: arXiv cs.RO 2022
Year: 2022

This paper presents a conceptual overview of the EASIER project and its scope. EASIER focuses on supporting emergency forces in disaster response scenarios with a semi-autonomous mobile manipulator. Specifically, we examine the operator's trust in the system and his/her cognitive load generated by its use. We plan to address different research topics, exploring how shared autonomy, interaction design, and transparency relate to trust and cognitive load. Another goal is to develop non-invasive methods to continuously measure trust and cognitive load in the context of disaster response using a multilevel approach. This project is conducted by multiple academic partners specializing in artificial intelligence, interaction design, and psychology, as well as an industrial partner for disaster response equipment and end-users for framing the project and the experiments in real use-cases.
XR Hackathon Going Online: Lessons Learned from a Case Study with Goethe-Institut
Venue: arXiv cs.HC 2022
Year: 2022

In this article we report a case study of a Language and Culture-oriented transdisciplinary XR hackathon organized with Goethe-Institut. The hackathon was hosted as an online event in November 2020 by our University Lab in collaboration with Goethe-Institut as a follow-up to our previous co-organized event within our research group Living Lab. We have improved the formula of the event based on lessons learned from its previous edition. First, in one of the two hackathon tracks we provided the participants with a custom VR framework, to serve as a starting point for their designs to skip the repetitive early development stage. In cooperation with our partner, Goethe-Institut, we have also outlined best modern research-backed language-learning practices and methods and gathered them into actionable evaluation criteria.
Participatory Action for Citizens' Engagement to Develop a Pro-Environmental Research Application
Venue: arXiv cs.CY 2022
Year: 2022

To understand and begin to address the challenge of air pollution in Europe we conducted participatory research, art and design activities with the residents of one of the areas most affected by smog in Poland. The participatory research events, described in detail in this article, centered around the theme of ecology and served to design an application that would allow us to conduct field research on pro-environmental behaviours at a larger scale. As a result we developed a research application, rooted in local culture and history and place attachment, which makes use of gamification techniques. The application gathers air quality data from the densest network of air pollution sensors in Europe, thereby aligning the visible signs of pollution in the app with the local sensor data. At the same time it reinforces the users' pro-environmental habits and exposes them to educational messages about air quality and the environment. The data gathered with this application will validate the efficacy of this kind of an intervention in addressing residents' smog-causing behaviours.
Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain
Venue: arXiv cs.CV 2022
Year: 2022

Currently, under supervised learning, a model pretrained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated the knowledge transfer learning. It has reached the status of consensus solution for task-aware model training in remote sensing domain (RSD). Unfortunately, due to different categories of imaging data and stiff challenges of data annotation, there is not a large enough and uniform remote sensing dataset to support large-scale pretraining in RSD. Moreover, pretraining models on large-scale nature scene datasets by supervised learning and then directly fine-tuning on diverse downstream tasks seems to be a crude method, which is easily affected by inevitable labeling noise, severe domain gaps and task-aware discrepancies. Thus, in this paper, considering the self-supervised pretraining and powerful vision transformer (ViT) architecture, a concise and effective knowledge transfer learning strategy called ConSecutive PreTraining (CSPT) is proposed based on the idea of not stopping pretraining in natural language processing (NLP), which can gradually bridge the domain gap and transfer knowledge from the nature scene domain to the RSD. The proposed CSPT also can release the huge potential of unlabeled data for task-aware model training. Finally, extensive experiments are carried out on twelve datasets in RSD involving three types of downstream tasks (e.g., scene classification, object detection and land cover classification) and two types of imaging data (e.g., optical and SAR). The results show that by utilizing the proposed CSPT for task-aware model training, almost all downstream tasks in RSD can outperform the previous method of supervised pretraining-then-fine-tuning and even surpass the state-of-the-art (SOTA) performance without any expensive labeling consumption and careful model design.
Pixel-level Correspondence for Self-Supervised Learning from Video
Venue: arXiv cs.CV 2022
Year: 2022

While self-supervised learning has enabled effective representation learning in the absence of labels, for vision, video remains a relatively untapped source of supervision. To address this, we propose Pixel-level Correspondence (PiCo), a method for dense contrastive learning from video. By tracking points with optical flow, we obtain a correspondence map which can be used to match local features at different points in time. We validate PiCo on standard benchmarks, outperforming self-supervised baselines on multiple dense prediction tasks, without compromising performance on image classification.
The Power of Transfer Learning in Agricultural Applications: AgriNet
Venue: arXiv cs.CV 2022
Year: 2022

Advances in deep learning and transfer learning have paved the way for various automation classification tasks in agriculture, including plant diseases, pests, weeds, and plant species detection. However, agriculture automation still faces various challenges, such as the limited size of datasets and the absence of plant-domain-specific pretrained models. Domain specific pretrained models have shown state of art performance in various computer vision tasks including face recognition and medical imaging diagnosis. In this paper, we propose AgriNet dataset, a collection of 160k agricultural images from more than 19 geographical locations, several images captioning devices, and more than 423 classes of plant species and diseases. We also introduce AgriNet models, a set of pretrained models on five ImageNet architectures: VGG16, VGG19, Inception-v3, InceptionResNet-v2, and Xception. AgriNet-VGG19 achieved the highest classification accuracy of 94 % and the highest F1-score of 92%. Additionally, all proposed models were found to accurately classify the 423 classes of plant species, diseases, pests, and weeds with a minimum accuracy of 87% for the Inception-v3 model.Finally, experiments to evaluate of superiority of AgriNet models compared to ImageNet models were conducted on two external datasets: pest and plant diseases dataset from Bangladesh and a plant diseases dataset from Kashmir.
Generative Adversarial Networks and Other Generative Models
Venue: arXiv cs.CV 2022
Year: 2022

Generative networks are fundamentally different in their aim and methods compared to CNNs for classification, segmentation, or object detection. They have initially not been meant to be an image analysis tool, but to produce naturally looking images. The adversarial training paradigm has been proposed to stabilize generative methods, and has proven to be highly successful -- though by no means from the first attempt. This chapter gives a basic introduction into the motivation for Generative Adversarial Networks (GANs) and traces the path of their success by abstracting the basic task and working mechanism, and deriving the difficulty of early practical approaches. Methods for a more stable training will be shown, and also typical signs for poor convergence and their reasons. Though this chapter focuses on GANs that are meant for image generation and image analysis, the adversarial training paradigm itself is not specific to images, and also generalizes to tasks in image analysis. Examples of architectures for image semantic segmentation and abnormality detection will be acclaimed, before contrasting GANs with further generative modeling approaches lately entering the scene. This will allow a contextualized view on the limits but also benefits of GANs.
Periodic and Event-Triggering for Joint Capacity Maximization and Safe Intersection Crossing
Venue: arXiv cs.RO 2022
Year: 2022

Intersection crossing represents a bottleneck for transportation systems and Connected Autonomous Vehicles (CAVs) may be the groundbreaking solution to the problem. This work proposes a novel framework, i.e, AVOID-PERIOD, where an Intersection Manager (IM) controls CAVs approaching an intersection in order to maximize intersection capacity while minimizing the CAVs' gas consumption. Contrary to most of the works in the literature, the CAVs' location uncertainty is accounted for and periodic communication and re-optimization allows for the creation of safe trajectories for the CAVs. To improve scalability for high-traffic intersections, an event-triggering approach is also developed (AVOID-EVENT) that minimizes computational and communication complexity. AVOID-EVENT reduces the number of re-optimizations required by 92.2%, while retaining most of the benefits introduced by AVOID-PERIOD.
Copy Propagation subsumes Constant Propagation
Venue: arXiv cs.PL 2022
Year: 2022

Constant propagation and copy propagation are code transformations that may avoid some load operations and can enable other optimizations. In literature, constant and copy propagations are considered two independent transformations requiring two different data flow analyses. Here we give a generic definition for copy propagation which enables us to view constant propagation as a particular case of copy propagation and formulate a novel data flow analysis that unifies these two transformations.
An Approach to Ensure Fairness in News Articles
Venue: arXiv cs.IR 2022
Year: 2022

Recommender systems, information retrieval, and other information access systems present unique challenges for examining and applying concepts of fairness and bias mitigation in unstructured text. This paper introduces Dbias, which is a Python package to ensure fairness in news articles. Dbias is a trained Machine Learning (ML) pipeline that can take a text (e.g., a paragraph or news story) and detects if the text is biased or not. Then, it detects the biased words in the text, masks them, and recommends a set of sentences with new words that are bias-free or at least less biased. We incorporate the elements of data science best practices to ensure that this pipeline is reproducible and usable. We show in experiments that this pipeline can be effective for mitigating biases and outperforms the common neural network architectures in ensuring fairness in the news articles.
Rate-Optimal Streaming Codes Over the Three-Node Decode-And-Forward Relay Network
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study the three-node Decode-and-Forward (D&F) relay network subject to random and burst packet erasures. The source wishes to transmit an infinite stream of packets to the destination via the relay. The three-node D&F relay network is constrained by a decoding delay of T packets, i.e., the packet transmitted by the source at time i must be decoded by the destination by time i+T. For the individual channels from source to relay and relay to destination, we assume a delay-constrained sliding-window (DCSW) based packet-erasure model that can be viewed as a tractable approximation to the commonly-accepted Gilbert-Elliot channel model. Under the model, any time-window of width w contains either up to a random erasure or else erasure burst of length at most b (>= a). Thus the source-relay and relay-destination channels are modeled as (a_1, b_1, w_1, T_1) and (a_2, b_2, w_2, T_2) DCSW channels. We first derive an upper bound on the capacity of the three-node D&F relay network. We then show that the upper bound is tight for the parameter regime: max{b_1, b_2}|(T-b_1-b_2-max{a_1, a_2}+1), a1=a2 OR b1=b2 by constructing streaming codes achieving the bound. The code construction requires field size linear in T, and has decoding complexity equivalent to that of decoding an MDS code.
A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection
Venue: arXiv cs.SD 2022
Year: 2022

Audio commands are a preferred communication medium to keep inspectors in the loop of civil infrastructure inspection performed by a semi-autonomous drone. To understand job-specific commands from a group of heterogeneous and dynamic inspectors, a model must be developed cost-effectively for the group and easily adapted when the group changes. This paper is motivated to build a multi-tasking deep learning model that possesses a Share-Split-Collaborate architecture. This architecture allows the two classification tasks to share the feature extractor and then split subject-specific and keyword-specific features intertwined in the extracted features through feature projection and collaborative training. A base model for a group of five authorized subjects is trained and tested on the inspection keyword dataset collected by this study. The model achieved a 95.3% or higher mean accuracy in classifying the keywords of any authorized inspectors. Its mean accuracy in speaker classification is 99.2%. Due to the richer keyword representations that the model learns from the pooled training data, adapting the base model to a new inspector requires only a little training data from that inspector, like five utterances per keyword. Using the speaker classification scores for inspector verification can achieve a success rate of at least 93.9% in verifying authorized inspectors and 76.1% in detecting unauthorized ones. Further, the paper demonstrates the applicability of the proposed model to larger-size groups on a public dataset. This paper provides a solution to addressing challenges facing AI-assisted human-robot interaction, including worker heterogeneity, worker dynamics, and job heterogeneity.
Few-Example Clustering via Contrastive Learning
Venue: arXiv cs.LG 2022
Year: 2022

We propose Few-Example Clustering (FEC), a novel algorithm that performs contrastive learning to cluster few examples. Our method is composed of the following three steps: (1) generation of candidate cluster assignments, (2) contrastive learning for each cluster assignment, and (3) selection of the best candidate. Based on the hypothesis that the contrastive learner with the ground-truth cluster assignment is trained faster than the others, we choose the candidate with the smallest training loss in the early stage of learning in step (3). Extensive experiments on the \textit{mini}-ImageNet and CUB-200-2011 datasets show that FEC outperforms other baselines by about 3.2% on average under various scenarios. FEC also exhibits an interesting learning curve where clustering performance gradually increases and then sharply drops.
Causal Discovery using Model Invariance through Knockoff Interventions
Venue: arXiv cs.LG 2022
Year: 2022

Cause-effect analysis is crucial to understand the underlying mechanism of a system. We propose to exploit model invariance through interventions on the predictors to infer causality in nonlinear multivariate systems of time series. We model nonlinear interactions in time series using DeepAR and then expose the model to different environments using Knockoffs-based interventions to test model invariance. Knockoff samples are pairwise exchangeable, in-distribution and statistically null variables generated without knowing the response. We test model invariance where we show that the distribution of the response residual does not change significantly upon interventions on non-causal predictors. We evaluate our method on real and synthetically generated time series. Overall our method outperforms other widely used causality methods, i.e, VAR Granger causality, VARLiNGAM and PCMCI+.