Reviewerly/exHarmony huggingface dataset:
GCM-Net: Towards Effective Global Context Modeling for Image Inpainting

Deep learning based inpainting methods have obtained promising performance for image restoration, however current image inpainting methods still tend to produce unreasonable structures and blurry textures when processing the damaged images with heavy corruptions. In this paper, we propose a new image inpainting method termed Global Context Modeling Network (GCM-Net). By capturing the global contextual information, GCM-Net can potentially improve the performance of recovering the missing region in the damaged images with irregular masks. To be specific, we first use four convolution layers to extract the shadow features. Then, we design a progressive multi-scale fusion block termed PMSFB to extract and fuse the multi-scale features for obtaining local features. Besides, a dense context extraction (DCE) module is also designed to aggregate the local features extracted by PMSFBs. To improve the information flow, a channel attention guided residual learning module is deployed in both the DCE and PMSFB, which can reweight the learned residual features and refine the extracted information. To capture more global contextual information and enhance the representation ability, a coordinate context attention (CCA) based module is also presented. Finally, the extracted features with rich information are decoded as the image inpainting result. Extensive results on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our method can better recover the structures and textures, and deliver significant improvements, compared with some related inpainting methods.
Transfer learning for radioactive particle tracking

Radioactive particle tracking (RPT) is a non-invasive technique used to monitor opaque multiphase flow systems. Achieving highly accurate particle tracing is challenging and time-consuming because of the need to build a new RPT model from calibration data each time the experimental conditions change. This paper aims to examine if RPT calibration data under previous conditions can be leveraged with the help of transfer learning (TL) when creating an RPT model for a new condition. Several TL strategies for exploiting historical calibration data are evaluated in conjunction with Geant4 simulations to understand their applicability to RPT. The results show that when it is impractical to collect a lot of calibration data, TL is often superior to training an RPT model only on new data. Moreover, when new calibration data collection is not feasible, an RPT model trained on the historical data can be very accurate if the new condition is sufficiently similar to the historical conditions.
Seeing science: using graphics to communicate research

Abstract Proper science communication is quickly becoming a professional imperative in ecology, but many researchers are not practiced in diverse and effective communication strategies. Academic jargon and text‐heavy content are often cited as barriers for laypeople trying to access and understand research results. Here, we have presented scientific visualizations (infographics, figures, and illustrations) as a useful tool to reduce the information transfer gaps between researchers and the public. The graphics we have proposed are images with minimal text that convey ecological research concepts, methods, processes, and results. They are more captivating than text alone and more efficient at disseminating information to a broad audience because they reduce cognitive load. We applied tools and best practices from the fields of marketing and design to explain graphic construction and demonstrated how to build a visualization that is both aesthetic and effective. The basic principles of design are paramount to image composition, and we reviewed experimental literature to support the notions that the proper use of color, proximity, and balance helps to illuminate the main message or story that we wanted to communicate. We presented examples from wildlife ecology research in Alaska to highlight how researchers can use graphics for their own communication efforts, and emphasized the power of visual narratives to explain complex techniques and ecosystem processes. The best practices we outlined here are meant to help researchers understand the composition of science visualizations, build productive collaborations with artists, and ultimately create appealing and informative images that communicate research.
Idiomatic Expression Identification using Semantic Compatibility

Abstract Idiomatic expressions are an integral part of natural language and constantly being added to a language. Owing to their non-compositionality and their ability to take on a figurative or literal meaning depending on the sentential context, they have been a classical challenge for NLP systems. To address this challenge, we study the task of detecting whether a sentence has an idiomatic expression and localizing it when it occurs in a figurative sense. Prior research for this task has studied specific classes of idiomatic expressions offering limited views of their generalizability to new idioms. We propose a multi-stage neural architecture with attention flow as a solution. The network effectively fuses contextual and lexical information at different levels using word and sub-word representations. Empirical evaluations on three of the largest benchmark datasets with idiomatic expressions of varied syntactic patterns and degrees of non-compositionality show that our proposed model achieves new state-of-the-art results. A salient feature of the model is its ability to identify idioms unseen during training with gains from 1.4% to 30.8% over competitive baselines on the largest dataset.
Smoothness-Driven Consensus Based on Compact Representation for Robust Feature Matching

For robust feature matching, a popular and particularly effective method is to recover smooth functions from the data to differentiate the true correspondences (inliers) from false correspondences (outliers). In the existing works, the well-established regularization theory has been extensively studied and exploited to estimate the functions while controlling its complexity to enforce the smoothness constraint, which has shown prominent advantages in this task. However, despite the theoretical optimality properties, the high complexities in both time and space are induced and become the main obstacle of their application. In this article, we propose a novel method for multivariate regression and point matching, which exploits the sparsity structure of smooth functions. Specifically, we use compact Fourier bases for constructing the function, which inherently allows a coarse-to-fine representation. The smoothness constraint can be explicitly imposed by adopting a few low-frequency bases for representation, resulting in reduced computational complexities of the induced multivariate regression algorithm. To cope with potential gross outliers, we formulate the learning problem into a Bayesian framework with latent variables indicating the inliers and outliers and a mixture model accounting for the distribution of data, where a fast expectation-maximization solution can be derived. Extensive experiments are conducted on synthetic data and real-world image matching, and point set registration datasets, which demonstrates the advantages of our method against the current state-of-the-art methods in terms of both scalability and robustness.
Exploiting collisions for sampling-based multicopter motion planning

Multicopters with collision-resilient designs can operate with trajectories involving collisions. This paper presents a sampling-based method that can exploit collisions for better motion planning. The method is built upon the basis of the RRT* algorithm and takes advantages of fast motion primitive generation and collision checking for multicopters. It generates collision states by detecting potential intersections between motion primitives and obstacles, and connects these states with other sampled states to form collision-inclusive trajectories. We show that allowing collision helps improve the performance of the sampling-based planner in narrow spaces like tunnels. Finally, an experiment of tracking the trajectory generated by the collision-inclusive planner is presented.
NeuralDiff: Segmenting 3D objects that move in egocentric videos

Given a raw video sequence taken from a freely-moving camera, we study the problem of decomposing the observed 3D scene into a static background and a dynamic foreground containing the objects that move within the scene. This task is reminiscent of the classic background subtraction problem, but is significantly harder because all parts of the scene, static and dynamic, generate a large apparent motion due to the camera large viewpoint change and parallax. In particular, we consider egocentric videos and further separate the dynamic component into objects and the actor that observes and moves them. We achieve this factorization by reconstructing the video via a triple-stream neural rendering network that explains the different motions based on corresponding inductive biases. We demonstrate that our method can successfully separate the different types of motion, outperforming recent neural rendering baselines at this task, and can accurately segment the moving objects. We do so by assessing the method empirically on challenging videos from the EPIC-KITCHENS dataset which we augment with appropriate annotations to create a new benchmark for the task of dynamic object segmentation on unconstrained video sequences, for complex 3D environments.
Fedsld: Federated Learning with Shared Label Distribution for Medical Image Classification

Federated learning (FL) enables collaboratively training a joint model for multiple medical centers, while keeping the data decentralized due to privacy concerns. However, federated optimizations often suffer from the heterogeneity of the data distribution across medical centers. In this work, we propose Federated Learning with Shared Label Distribution (FedSLD) for classification tasks, a method that adjusts the contribution of each data sample to the local objective during optimization via knowledge of clients' label distribution, mitigating the instability brought by data heterogeneity. We conduct extensive experiments on four publicly available image datasets with different types of non-IID data distributions. Our results show that FedSLD achieves better convergence performance than the compared leading FL optimization algorithms, increasing the test accuracy by up to 5.50 percentage points.
Cancer Cell Profiling Using Image Moments and Neural Networks with Model Agnostic Explainability: A Case Study of Breast Cancer Histopathological (BreakHis) Database

With the evolution of modern digital pathology, examining cancer cell tissues has paved the way to quantify subtle symptoms, for example, by means of image staining procedures using Eosin and Hematoxylin. Cancer tissues in the case of breast and lung cancer are quite challenging to examine by manual expert analysis of patients suffering from cancer. Merely relying on the observable characteristics by histopathologists for cell profiling may under-constrain the scale and diagnostic quality due to tedious repetition with constant concentration. Thus, automatic analysis of cancer cells has been proposed with algorithmic and soft-computing techniques to leverage speed and reliability. The paper’s novelty lies in the utility of Zernike image moments to extract complex features from cancer cell images and using simple neural networks for classification, followed by explainability on the test results using the Local Interpretable Model-Agnostic Explanations (LIME) technique and Explainable Artificial Intelligence (XAI). The general workflow of the proposed high throughput strategy involves acquiring the BreakHis public dataset, which consists of microscopic images, followed by the application of image processing and machine learning techniques. The recommended technique has been mathematically substantiated and compared with the state-of-the-art to justify the empirical basis in the pursuit of our algorithmic discovery. The proposed system is able to classify malignant and benign cancer cell images of 40× resolution with 100% recognition rate. XAI interprets and reasons the test results obtained from the machine learning model, making it reliable and transparent for analysis and parameter tuning.
Domain-Aware SE Network for Sketch-based Image Retrieval with Multiplicative Euclidean Margin Softmax

This paper proposes a novel approach for Sketch-Based Image Retrieval (SBIR), for which the key is to bridge the gap between sketches and photos in terms of the data representation. Inspired by channel-wise attention explored in recent years, we present a Domain-Aware Squeeze-and-Excitation (DASE) network, which seamlessly incorporates the prior knowledge of sample sketch or photo into SE module and make the SE module capable of emphasizing appropriate channels according to domain signal. Accordingly, the proposed network can switch its mode to achieve a better domain feature with lower intra-class discrepancy. Moreover, while previous works simply focus on minimizing intra-class distance and maximizing inter-class distance, we introduce a loss function, named Multiplicative Euclidean Margin Softmax (MEMS), which introduces multiplicative Euclidean margin into feature space and ensure that the maximum intra-class distance is smaller than the minimum inter-class distance. This facilitates learning a highly discriminative feature space and ensures a more accurate image retrieval result. Extensive experiments are conducted on two widely used SBIR benchmark datasets. Our approach achieves better results on both datasets, surpassing the state-of-the-art methods by a large margin.
High Perceptual Quality Image Denoising with a Posterior Sampling CGAN

The vast work in Deep Learning (DL) has led to a leap in image denoising research. Most DL solutions for this task have chosen to put their efforts on the denoiser’s architecture while maximizing distortion performance. However, distortion driven solutions lead to blurry results with sub-optimal perceptual quality, especially in immoderate noise levels. In this paper we propose a different perspective, aiming to produce sharp and visually pleasing denoised images that are still faithful to their clean sources. Formally, our goal is to achieve high perceptual quality with acceptable distortion. This is attained by a stochastic denoiser that samples from the posterior distribution, trained as a generator in the framework of conditional generative adversarial networks (CGAN). Contrary to distortion-based regularization terms that conflict with perceptual quality, we introduce to the CGAN objective a theoretically founded penalty term that does not force a distortion requirement on individual samples, but rather on their mean. We showcase our proposed method with a novel denoiser architecture that achieves the reformed denoising goal and produces vivid and diverse outcomes in immoderate noise levels.
Estimation of Spatially-Correlated Ocean Currents from Ensemble Forecasts and Online Measurements

We present a method to estimate two-dimensional, time-invariant oceanic flow fields based on data from both ensemble forecasts and online measurements. Our method produces a realistic estimate in a computationally efficient manner suitable for use in marine robotics for path planning and related applications. We use kernel methods and singular value decomposition to find a compact model of the ensemble data that is represented as a linear combination of basis flow fields and that preserves the spatial correlations present in the data. Online measurements of ocean current, taken for example by marine robots, can then be incorporated using recursive Bayesian estimation. We provide computational analysis, performance comparisons with related methods, and demonstration with real-world ensemble data to show the computational efficiency and validity of our method. Possible applications in addition to path planning include active perception for model improvement through deliberate choice of measurement locations.
Entanglement Rate Optimization in Heterogeneous Quantum Communication Networks

Quantum communication networks are emerging as a promising technology that could constitute a key building block in future communication networks in the 6G era and beyond. These networks have an inherent feature of parallelism that allows them to boost the capacity and enhance the security of communication systems. Recent advances led to the deployment of small- and large-scale quantum communication networks with real quantum hardware. In quantum networks, entanglement is a key resource that allows for data transmission between different nodes. However, to reap the benefits of entanglement and enable efficient quantum communication, the number of generated entangled pairs must be optimized. Indeed, if the entanglement generation rates are not optimized, then some of these valuable resources will be discarded and lost. In this paper, the problem of optimizing the entanglement generation rates and their distribution over a quantum memory is studied. In particular, a quantum network in which users have heterogeneous distances and applications is considered. This problem is posed as a mixed integer nonlinear programming optimization problem whose goal is to efficiently utilize the available quantum memory by distributing the quantum entangled pairs in a way that maximizes the user satisfaction. An interior point optimization method is used to solve the optimization problem and extensive simulations are conducted to evaluate the effectiveness of the proposed system. Simulation results show the key design considerations for efficient quantum networks, and the effect of different network parameters on the network performance.
Multi-task Learning for Jersey Number Recognition in Ice Hockey

Identifying players in sports videos by recognizing their jersey numbers is a challenging task in computer vision. We have designed and implemented a multi-task learning network for jersey number recognition. In order to train a network to recognize jersey numbers, two output label representations are used (1) Holistic - considers the entire jersey number as one class, and (2) Digit-wise - considers the two digits in a jersey number as two separate classes. The proposed network learns both holistic and digit-wise representations through a multi-task loss function. We determine the optimal weights to be assigned to holistic and digit-wise losses through an ablation study. Experimental results demonstrate that the proposed multi-task learning network performs better than the constituent holistic and digit-wise single-task learning networks.
Toward the Capacity of Private Information Retrieval From Coded and Colluding Servers

In this work, two practical concepts related to private information retrieval (PIR) are introduced and coined <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">full support-rank</i> PIR and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">strongly linear</i> PIR. Being of full support-rank is a technical, yet natural condition required to prove a converse result for a capacity expression and satisfied by almost all currently known capacity-achieving schemes, while strong linearity is a practical requirement enabling implementation over small finite fields with low subpacketization degree. Then, the capacity of MDS-coded, linear, full support-rank PIR in the presence of colluding servers is derived, as well as the capacity of symmetric, linear PIR with colluding, adversarial, and nonresponsive servers for the recently introduced concept of matched randomness. This positively settles the capacity conjectures stated by Freij-Hollanti <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> and Tajeddine <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> in the presented cases. It is also shown that, further restricting to strongly-linear PIR schemes with deterministic linear interference cancellation, the so-called star product scheme proposed by Freij-Hollanti <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> is essentially optimal and induces no capacity loss.
Mapless-Planner: A Robust and Fast Planning Framework for Aggressive Autonomous Flight without Map Fusion

Maintaining a map online is resource-consuming while a robust navigation system usually needs environment abstraction via a well-fused map. In this paper, we propose a mapless local planner which directly conducts such abstraction on the unfused sensor data. A limited-memory data structure with a reliable proximity query algorithm is proposed for maintaining raw historical information. A sampling-based scheme is designed to extract the free-space skeleton. A smart waypoint selection strategy enables to generate high-quality trajectories within the resultant flight corridors. Our planner differs from other mapless ones in that it can abstract and exploit the environment information more efficiently. The online replan consistency and success rate are both significantly improved against conventional mapless methods.
Sparse to Dense Motion Transfer for Face Image Animation

Face image animation from a single image has achieved remarkable progress. However, it remains challenging when only sparse landmarks are available as the driving signal. Given a source face image and a sequence of sparse face landmarks, our goal is to generate a video of the face imitating the motion of landmarks. We develop an efficient and effective method for motion transfer from sparse landmarks to the face image. We then combine global and local motion estimation in a unified model to faithfully transfer the motion. The model can learn to segment the moving foreground from the background and generate not only global motion, such as rotation and translation of the face, but also subtle local motion such as the gaze change. We further improve face landmark detection on videos. With temporally better aligned landmark sequences for training, our method can generate temporally coherent videos with higher visual quality. Experiments suggest we achieve results comparable to the state-of-the-art image driven method on the same identity testing and better results on cross identity testing.