Reviewerly/exHarmony huggingface dataset:
Privacy-Preserving Aggregation for Federated Learning-Based Navigation in Vehicular Fog

Federated learning-based automotive navigation has recently received considerable attention, as it can potentially address the issue of weak global positioning system (GPS) signals under severe blockages, such as in downtowns and tunnels. Specifically, the data-driven navigation framework combines the position estimation offered by the high-sampling inertial measurement units and the position calibration provided by the low-sampling GPS signals. Despite its promise, the privacy preservation and flexibility of the participating users in the federated learning process are still problematic. To address these challenges, in this article, we propose an efficient, flexible, and privacy-preserving model aggregation scheme under a federated learning-based navigation framework named FedLoc. Specifically, our proposed scheme efficiently protects the locally trained model updates, flexibly supports the fluctuation of participants, and is robust against unregistered malicious users by exploiting a homomorphic threshold cryptosystem, together with the bounded Laplace mechanism and the skip list. We perform a detailed security analysis to demonstrate the security properties in terms of privacy preservation and dishonest user detection. In addition, we evaluate and compare the computational efficiency with two traditional schemes, and the simulation results show that our scheme greatly improves the computational efficiency during participant fluctuation. To validate the effectiveness of our scheme, we also show that only part of the model update is excluded from aggregation in the case of a dishonest user.
Dust potency in the context of solar photovoltaic (PV) soiling loss

We report on the impact of natural soiling on PV module performance loss in terms of dust potency (i.e., soiling loss per unit area dust mass) affected by dust composition and environmental conditions. Samples of accumulated dust mass (ADM) on PV panels from Doha, Qatar, were collected at different periods over two years. Our analysis demonstrated that dust accumulation rate and dust potency exhibited seasonal patterns influenced by environmental conditions. The measured dust potency varied significantly from month to month, ranging from 2 to 3.7% g−1 m2 that situates in the range of 2 – 8% g−1 m2 found in the literature. The lower dust potency was observed during the summer months, characterized by strong winds and low humidity. Soiling loss may be seen as a function of ADM and the dust potency, with the latter being a function of dust physico-chemical properties. ADM also has a significant seasonal fluctuation both in terms of particle size distribution and chemical composition. The dust potency is found to be impacted by the particle size, elemental carbon (EC), organic carbon (OC), and ions contents of the dust. ADM with lower dust potency has shown larger size particles and low EC/OC contents, while the effect of soluble ions on dust potency is found to be complex, due most probably to the interactions with humidity. Finally, based on the experimental and literature data, we developed a practical and straightforward model to estimate PV soiling loss as a function of dust mass and dust potency.
Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement

Enhancing the quality of low-light (LOL) images plays a very important role in many image processing and multimedia applications. In recent years, a variety of deep learning techniques have been developed to address this challenging task. A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces, causing many unfavorable outcomes, e.g., details loss, color unsaturation, and artifacts. To address these issues, we develop a new context-sensitive decomposition network (CSDNet) architecture to exploit the scene-level contextual dependencies on spatial scales. More concretely, we build a two-stream estimation mechanism including reflectance and illumination estimation network. We design a novel context-sensitive decomposition connection to bridge the two-stream mechanism by incorporating the physical principle. The spatially varying illumination guidance is further constructed for achieving the edge-aware smoothness property of the illumination component. According to different training patterns, we construct CSDNet (paired supervision) and context-sensitive decomposition generative adversarial network (CSDGAN) (unpaired supervision) to fully evaluate our designed architecture. We test our method on seven testing benchmarks [including massachusetts institute of technology (MIT)-Adobe FiveK, LOL, ExDark, and naturalness preserved enhancement (NPE)] to conduct plenty of analytical and evaluated experiments. Thanks to our designed context-sensitive decomposition connection, we successfully realized excellent enhanced results (with sufficient details, vivid colors, and few noises), which fully indicates our superiority against existing state-of-the-art approaches. Finally, considering the practical needs for high efficiency, we develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels. Furthermore, by sharing an encoder for these two components, we obtain a more lightweight version (SLiteCSDNet for short). SLiteCSDNet just contains 0.0301M parameters but achieves the almost same performance as CSDNet. Code is available at https://github.com/KarelZhang/CSDNet-CSDGAN.
Event generation and statistical sampling for physics with deep generative models and a density information buffer

Abstract Simulating nature and in particular processes in particle physics require expensive computations and sometimes would take much longer than scientists can afford. Here, we explore ways to a solution for this problem by investigating recent advances in generative modeling and present a study for the generation of events from a physical process with deep generative models. The simulation of physical processes requires not only the production of physical events, but to also ensure that these events occur with the correct frequencies. We investigate the feasibility of learning the event generation and the frequency of occurrence with several generative machine learning models to produce events like Monte Carlo generators. We study three processes: a simple two-body decay, the processes e + e − → Z → l + l − and $$pp\to t\bar{t}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>p</mml:mi> <mml:mi>p</mml:mi> <mml:mo>→</mml:mo> <mml:mi>t</mml:mi> <mml:mover> <mml:mrow> <mml:mi>t</mml:mi> </mml:mrow> <mml:mo>¯</mml:mo> </mml:mover> </mml:math> including the decay of the top quarks and a simulation of the detector response. By buffering density information of encoded Monte Carlo events given the encoder of a Variational Autoencoder we are able to construct a prior for the sampling of new events from the decoder that yields distributions that are in very good agreement with real Monte Carlo events and are generated several orders of magnitude faster. Applications of this work include generic density estimation and sampling, targeted event generation via a principal component analysis of encoded ground truth data, anomaly detection and more efficient importance sampling, e.g., for the phase space integration of matrix elements in quantum field theories.
Towards Automating Code Review Activities

Code reviews are popular in both industrial and open source projects. The benefits of code reviews are widely recognized and include better code quality and lower likelihood of introducing bugs. However, since code review is a manual activity it comes at the cost of spending developers' time on reviewing their teammates' code. Our goal is to make the first step towards partially automating the code review process, thus, possibly reducing the manual costs associated with it. We focus on both the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">contributor</i> and the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">reviewer</i> sides of the process, by training two different Deep Learning architectures. The first one learns code changes performed by developers during real code review activities, thus providing the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">contributor</i> with a revised version of her code implementing code transformations usually recommended during code review before the code is even submitted for review. The second one automatically provides the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">reviewer</i> commenting on a submitted code with the revised code implementing her comments expressed in natural language. The empirical evaluation of the two models shows that, on the contributor side, the trained model succeeds in replicating the code transformations applied during code reviews in up to 16% of cases. On the reviewer side, the model can correctly implement a comment provided in natural language in up to 31% of cases. While these results are encouraging, more research is needed to make these models usable by developers.
Towards Practical Lipreading with Distilled and Efficient Models

Lipreading has witnessed a lot of progress due to the resurgence of neural networks. Recent works have placed emphasis on aspects such as improving performance by finding the optimal architecture or improving generalization. However, there is still a significant gap between the current methodologies and the requirements for an effective deployment of lipreading in practical scenarios. In this work, we propose a series of innovations that significantly bridge that gap: first, we raise the state-of-the-art performance by a wide margin on LRW and LRW-1000 to 88.5 % and 46.6 %, respectively using self-distillation. Secondly, we propose a series of architectural changes, including a novel Depthwise Separable Temporal Convolutional Network (DS-TCN) head, that slashes the computational cost to a fraction of the (already quite efficient) original model. Thirdly, we show that knowledge distillation is a very effective tool for recovering performance of the lightweight models. This results in a range of models with different accuracy-efficiency trade-offs. However, our most promising lightweight models are on par with the current state-of-the-art while showing a reduction of 8.2× and 3.9× in terms of computational cost and number of parameters, respectively, which we hope will enable the deployment of lipreading models in practical applications.
Advancing RNN Transducer Technology for Speech Recognition

We investigate a set of techniques for RNN Transducers (RNN-Ts) that were instrumental in lowering the word error rate on three different tasks (Switchboard 300 hours, conversational Spanish 780 hours and conversational Italian 900 hours). The techniques pertain to architectural changes, speaker adaptation, language model fusion, model combination and general training recipe. First, we introduce a novel multiplicative integration of the encoder and prediction network vectors in the joint network (as opposed to additive). Second, we discuss the applicability of i-vector speaker adaptation to RNN-Ts in conjunction with data perturbation. Third, we explore the effectiveness of the recently proposed density ratio language model fusion for these tasks. Last but not least, we describe the other components of our training recipe and their effect on recognition performance. We report a 5.9% and 12.5% word error rate on the Switchboard and CallHome test sets of the NIST Hub5 2000 evaluation and a 12.7% WER on the Mozilla CommonVoice Italian test set.
Deep Spectral Representation Learning From Multi-View Data

Multi-view representation learning (MvRL) aims to learn a consensus representation from diverse sources or domains to facilitate downstream tasks such as clustering, retrieval, and classification. Due to the limited representative capacity of the adopted shallow models, most existing MvRL methods may yield unsatisfactory results, especially when the labels of data are unavailable. To enjoy the representative capacity of deep learning, this paper proposes a novel multi-view unsupervised representation learning method, termed as Multi-view Laplacian Network (MvLNet), which could be the first deep version of the multi-view spectral representation learning method. Note that, such an attempt is nontrivial because simply combining Laplacian embedding (i.e., spectral representation) with neural networks will lead to trivial solutions. To solve this problem, MvLNet enforces an orthogonal constraint and reformulates it as a layer with the help of Cholesky decomposition. The orthogonal layer is stacked on the embedding network so that a common space could be learned for consensus representation. Compared with numerous recent-proposed approaches, extensive experiments on seven challenging datasets demonstrate the effectiveness of our method in three multi-view tasks including clustering, recognition, and retrieval. The source code could be found at www.pengxi.me.
Intelligent Radio Signal Processing: A Survey

Intelligent signal processing for wireless communications is a vital task in modern wireless systems, but it faces new challenges because of network heterogeneity, diverse service requirements, a massive number of connections, and various radio characteristics. Owing to recent advancements in big data and computing technologies, artificial intelligence (AI) has become a useful tool for radio signal processing and has enabled the realization of intelligent radio signal processing. This survey covers four intelligent signal processing topics for the wireless physical layer, including modulation classification, signal detection, beamforming, and channel estimation. In particular, each theme is presented in a dedicated section, starting with the most fundamental principles, followed by a review of up-to-date studies and a summary. To provide the necessary background, we first present a brief overview of AI techniques such as machine learning, deep learning, and federated learning. Finally, we highlight a number of research challenges and future directions in the area of intelligent radio signal processing. We expect this survey to be a good source of information for anyone interested in intelligent radio signal processing, and the perspectives we provide therein will stimulate many more novel ideas and contributions in the future.
Progressive Semantic Segmentation

The objective of this work is to segment high-resolution images without overloading GPU memory usage or losing the fine details in the output segmentation map. The memory constraint means that we must either downsample the big image or divide the image into local patches for separate processing. However, the former approach would lose the fine details, while the latter can be ambiguous due to the lack of a global picture. In this work, we present MagNet, a multi-scale framework that resolves local ambiguity by looking at the image at multiple magnification levels. MagNet has multiple processing stages, where each stage corresponds to a magnification level, and the output of one stage is fed into the next stage for coarse-to-fine information propagation. Each stage analyzes the image at a higher resolution than the previous stage, recovering the previously lost details due to the lossy downsampling step, and the segmentation output is progressively refined through the processing stages. Experiments on three high-resolution datasets of urban views, aerial scenes, and medical images show that MagNet consistently outperforms the state-of-the-art methods by a significant margin. Code is available at https://github.com/VinAIResearch/MagNet.
Artificial Intelligence Internet of Things for the Elderly: From Assisted Living to Health-Care Monitoring

An aging population is increasingly prevalent in both developed and developing countries, raising a series of social challenges and economic burdens. In particular, more elderly people are staying alone at home than are living with people who can take care of them. Therefore, assisted living (AL) and health-care monitoring (HM) can be critical issues in this era of human-centered artificial intelligence (AI). In this context, we aim to provide an encompassing review summarizing the state-of-the-art works combining AI and the Internet of Things (IoT) to help the elderly live easier and better. We systematically and comprehensively compare paradigms in terms of methodologies and application scenarios. The pros and cons among these technologies are discussed in detail. Then, we summarize current achievements and indicate their limitations. Finally, perspectives on highly promising future work are presented.
Hallucination Improves Few-Shot Object Detection

Learning to detect novel objects from few annotated examples is of great practical importance. A particularly challenging yet common regime occurs when there are extremely limited examples (less than three). One critical factor in improving few-shot detection is to address the lack of variation in training data. We propose to build a better model of variation for novel classes by transferring the shared within-class variation from base classes. To this end, we introduce a hallucinator network that learns to generate additional, useful training examples in the region of interest (RoI) feature space, and incorporate it into a modern object detection model. Our approach yields significant performance improvements on two state-of-the-art few-shot detectors with different proposal generation procedures. In particular, we achieve new state of the art in the extremely-few-shot regime on the challenging COCO benchmark.
Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

Tremendous efforts have been made on instance segmentation but the mask quality is still not satisfactory. The boundaries of predicted instance masks are usually imprecise due to the low spatial resolution of feature maps and the imbalance problem caused by the extremely low proportion of boundary pixels. To address these issues, we propose a conceptually simple yet effective post-processing refinement framework to improve the boundary quality based on the results of any instance segmentation model, termed BPR. Following the idea of looking closer to segment boundaries better, we extract and refine a series of small boundary patches along the predicted instance boundaries. The refinement is accomplished by a boundary patch refinement network at higher resolution. The proposed BPR framework yields significant improvements over the Mask R-CNN baseline on Cityscapes benchmark, especially on the boundary-aware metrics. Moreover, by applying the BPR framework to the "PolyTransform + SegFix" baseline, we reached 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> place on the Cityscapes leaderboard. Code is available at https://github.com/tinyalpha/BPR.
Invertible Image Signal Processing

Unprocessed RAW data is a highly valuable image format for image editing and computer vision. However, since the file size of RAW data is huge, most users can only get access to processed and compressed sRGB images. To bridge this gap, we design an Invertible Image Signal Processing (InvISP) pipeline, which not only enables rendering visually appealing sRGB images but also allows recovering nearly perfect RAW data. Due to our framework’s inherent reversibility, we can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead. We also integrate a differentiable JPEG compression simulator that empowers our framework to re-construct RAW data from JPEG images. Extensive quantitative and qualitative experiments on two DSLR demonstrate that our method obtains much higher quality in both rendered sRGB images and reconstructed RAW data than alternative methods.
VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval

Cross-view image geo-localization aims to determine the locations of street-view query images by matching with GPS-tagged reference images from aerial view. Recent works have achieved surprisingly high retrieval accuracy on city-scale datasets. However, these results rely on the assumption that there exists a reference image exactly centered at the location of any query image, which is not applicable for practical scenarios. In this paper, we redefine this problem with a more realistic assumption that the query image can be arbitrary in the area of interest and the reference images are captured before the queries emerge. This assumption breaks the one-to-one retrieval setting of existing datasets as the queries and reference images are not perfectly aligned pairs, and there may be multiple reference images covering one query location. To bridge the gap between this realistic setting and existing datasets, we propose a new large-scale benchmark –VIGOR– for cross-View Image Geo-localization beyond One-to-one Retrieval. We benchmark existing state-of-the-art methods and propose a novel end-to-end framework to localize the query in a coarse-to-fine manner. Apart from the image-level retrieval accuracy, we also evaluate the localization accuracy in terms of the actual distance (meters) using the raw GPS data. Extensive experiments are conducted under different application scenarios to validate the effectiveness of the proposed method. The results indicate that cross-view geo-localization in this realistic setting is still challenging, fostering new research in this direction. Our dataset and code will be released at https://github.com/JeffZilence/VIGOR.
Comprehensive Review on Facemask Detection Techniques in the Context of Covid-19

The outbreak of Coronavirus Disease 2019 (Covid-19) had an enormous impact on humanity. Till May 2021, almost 172 million people have been affected globally due to the contagious spread of Covid-19. Although the distribution of vaccines has been started, the worldwide mass distribution is yet to happen. According to the World Health Organization (WHO), wearing a facemask can reduce the contagious spread of Covid-19 significantly. The governments of different countries have recommended implementing the “no mask, no service” method to impede the spread of Covid-19. However, even the improper wearing of a facemask can obstruct the goal and lead to the spread of the virus. Therefore, to ensure public safety, a system for monitoring facemasks on faces, commonly known as a facemask detection algorithm, is essential for overcoming this crisis. The facemask detection algorithms are part of the object detection algorithms which are used to detect objects in an image. Among the various object detection algorithms, deep learning showed tremendous performance in facemask detection for its excellent feature extraction capability than the traditional machine learning algorithms. However, there remains a lot of scope for future research to build an efficient facemask detection system. Therefore, this study aims to draw attention to the researchers by providing a narrative and meta-analytic review on all the published works related to facemask detection in the context of Covid-19. Because facemask detection algorithms are run mainly by adopting object detection algorithms, this paper also explores the progress of object detection algorithms over the last few decades. A comprehensive analysis of different datasets used in facemask detection techniques by many studies has been explored. The performance comparison among these algorithms is discussed in narrative and meta-analytic approaches. Finally, this study concludes with a discussion of some of the major challenges and future scope in the related field.