Reviewerly/exHarmony huggingface dataset:
Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection

In this paper, we introduce a new framework for unsupervised deep homography estimation. Our contributions are 3 folds. First, unlike previous methods that regress 4 offsets for a homography, we propose a homography flow representation, which can be estimated by a weighted sum of 8 pre-defined homography flow bases. Second, considering a homography contains 8 Degree-of-Freedoms (DOFs) that is much less than the rank of the network features, we propose a Low Rank Representation (LRR) block that reduces the feature rank, so that features corresponding to the dominant motions are retained while others are rejected. Last, we propose a Feature Identity Loss (FIL) to enforce the learned image feature warp-equivariant, meaning that the result should be identical if the order of warp operation and feature extraction is swapped. With this constraint, the unsupervised optimization is achieved more effectively and more stable features are learned. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the state-of-the-art on the homography benchmark datasets both qualitatively and quantitatively. Code is available at https://github.com/megvii-research/BasesHomo
Acoustic Anomaly Detection of Mechanical Failures in Noisy Real-Life Factory Environments

Anomaly detection without employing dedicated sensors for each industrial machine is recognized as one of the essential techniques for preventive maintenance and is especially important for factories with low automatization levels, a number of which remain much larger than autonomous manufacturing lines. We have based our research on the hypothesis that real-life sound data from working industrial machines can be used for machine diagnostics. However, the sound data can be contaminated and drowned out by typical factory environmental sound, making the application of sound data-based anomaly detection an overly complicated process and, thus, the main problem we are solving with our approach. In this paper, we present a noise-tolerant deep learning-based methodology for real-life sound-data-based anomaly detection within real-world industrial machinery sound data. The main element of the proposed methodology is a generative adversarial network (GAN) used for the reconstruction of sound signal reconstruction and the detection of anomalies. The experimental results obtained in the Malfunctioning Industrial Machine Investigation and Inspection (MIMII) show the superiority of the proposed methodology over baseline approaches based on the One-Class Support Vector Machine (OC-SVM) and the Autoencoder–Decoder neural network. The proposed schematics using the unscented Kalman Filter (UKF) and the mean square error (MSE) loss function with the L2 regularization term showed an improvement of the Area Under Curve (AUC) for the noisy pump data of the pump.
Topology-Aware Neural Model for Highly Accurate QoS Prediction

With the widespread deployment of various cloud computing and service-oriented systems, there is a rapidly increasing demand for collaborative quality-of-service (QoS) prediction. Existing QoS prediction methods have made great progress in modeling users and services as well as exploiting contexts of service invocations. However, they ignore the completion of service requests/responses relies on the underlying network topology and the complex interactions between Autonomous Systems. To tackle this challenge, we propose a topology-aware neural (TAN) model for collaborative QoS prediction. In the TAN model, the features of users, services, and intermediate nodes on the communication path are projected to a shared latent space as input features. To jointly characterize the invocation process, the path features and end-cross features are captured respectively through an explicit path modeling layer and an implicit cross-modeling layer. After that, a gating layer fuses and transmits these features to the prediction layer for estimating unknown QoS values. In this way, TAN provides a flexible framework that can comprehensively capture the invocation context for making accurate QoS prediction. Experimental results on two real-world datasets demonstrate that TAN significantly outperforms state-of-the-art methods on the tasks of response time, throughput, and reliability prediction. Also, TAN shows better extensibility of using auxiliary information.
Beyond OCR + VQA

Text-based visual question answering (TextVQA) requires analyzing both the visual contents and texts in an image to answer a question, which is more practical than general visual question answering (VQA). Existing efforts tend to regard optical character recognition (OCR) as a pre-processing and then combine it with a VQA framework. It makes the performance of multimodal reasoning and question answering highly depend on the accuracy of OCR. In this work, we address this issue with two perspectives. First, we take advantages of multimodal cues to complete the semantic information of texts. A visually enhanced text embedding is proposed to enable understanding of texts without accurately recognizing them. Second, we further leverage rich contextual information to modify the answer texts even if the OCR module does not correctly recognize them. In addition, the visual objects are endued with semantic representations to enable objects in the same semantic space as OCR tokens. Equipped with these techniques, the cumulative error propagation caused by poor OCR performance is effectively suppressed. Extensive experiments on TextVQA and ST-VQA datasets demonstrate that our approach achieves the state-of-the-art performance in terms of accuracy and robustness.
Information systems security research agenda: Exploring the gap between research and practice

This paper undertakes a systematic review of the Information Systems Security literature. The literature review consists of three parts: First, we perform topic modeling of major Information Systems journals to understand the field's debates. Second, we conduct a Delphi Study composed of the Chief Information Security Officers of major corporations in the US to identify security issues that they view as important. Third, we compare Topic Modeling and the Delphi Study results and discuss key debates, gaps, and contradictions within the academic literature. Further, extant Information Systems Security literature is reviewed to discuss where the academic community has placed the research emphasis and what is now required in the discipline. Based on our analysis, we propose a future agenda for Information Systems security research.
Scale-Aware Detailed Matching for Few-Shot Aerial Image Semantic Segmentation

Few-shot semantic segmentation, aiming to segment query images with a few annotated support samples, has drawn increasing attention. Most existing few-shot methods leverage the single prototype obtained from global average pooling to represent all support information and further use the extracted prototype to segment the query images in a matching manner. Although promising results for natural images have been reported, these methods cannot be directly applied on aerial images. The main reason comes from that the extracted single support prototype can only provide a coarse guidance for matching between query and support images and could not handle the large variance of objects’ appearances and scales. To deal with these challenges on aerial images, we propose a scale-aware few-shot semantic segmentation network to perform detailed matching with multiple prototypes. More specifically, the detailed matching module is first constructed to compute the pixel-level similarity between the query features and the extracted multiple support prototypes for providing more accurate parsing guidance. Subsequently, to address the problem of scale imbalance, the scale-aware focal loss is designed to dynamically down-weight the loss assigned to large well-parsed objects and focus training on tiny hard-parsed objects. To facilitate the reproducible research on the task of few-shot semantic segmentation in aerial images, we further provide a few-shot segmentation benchmark iSAID- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5^{\mathrm {i}}$ </tex-math></inline-formula> constructed from the large-scale iSAID dataset <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> . Comprehensive experiments and comparisons with the state-of-the-art few-shot segmentation methods on the iSAID- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5^{\mathrm {i}}$ </tex-math></inline-formula> dataset clearly demonstrate the superiority of our proposed method. The code and dataset are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/caoql98/SDM</uri> .
Distributed Heuristic Multi-Agent Path Finding with Communication

Multi-Agent Path Finding (MAPF) is essential to large-scale robotic systems. Recent methods have applied reinforcement learning (RL) to learn decentralized polices in partially observable environments. A fundamental challenge of obtaining collision-free policy is that agents need to learn co-operation to handle congested situations. This paper combines communication with deep Q-learning to provide a novel learning based method for MAPF, where agents achieve cooperation via graph convolution. To guide RL algorithm on long-horizon goal-oriented tasks, we embed the potential choices of shortest paths from single source as heuristic guidance instead of using a specific path as in most existing works. Our method treats each agent independently and trains the model from a single agent&#x2019;s perspective. The final trained policy is applied to each agent for decentralized execution. The whole system is distributed during training and is trained under a curriculum learning strategy. Empirical evaluation in obstacle-rich environment indicates the high success rate with low average step of our method.
Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks

Emerging edge computing platforms often contain machine learning (ML) accelerators that can accelerate inference for a wide range of neural network (NN) models. These models are designed to fit within the limited area and energy constraints of the edge computing platforms, each targeting various applications (e.g., face detection, speech recognition, translation, image captioning, video analytics). To understand how edge ML accelerators perform, we characterize the performance of a commercial Google Edge TPU, using 24 Google edge NN models (which span a wide range of NN model types) and analyzing each NN layer within each model. We find that the Edge TPU suffers from three major shortcomings: (1) it operates significantly below peak computational throughput, (2) it operates significantly below its theoretical energy efficiency, and (3) its memory system is a large energy and performance bottleneck. Our characterization reveals that the one-size-fits-all, monolithic design of the Edge TPU ignores the high degree of heterogeneity both across different NN models and across different NN layers within the same NN model, leading to the shortcomings we observe. We propose a new acceleration framework called Mensa. Mensa incorporates multiple heterogeneous edge ML accelerators (including both on-chip and near-data accelerators), each of which caters to the characteristics of a particular subset of NN models and layers. During NN inference, for each NN layer, Mensa decides which accelerator to schedule the layer on, taking into account both the optimality of each accelerator for the layer and layer-to-layer communication costs. Our comprehensive analysis of the Google edge NN models shows that all of the layers naturally group into a small number of clusters, which allows us to design an efficient implementation of Mensa for these models with only three specialized accelerators. Averaged across all 24 Google edge NN models, Mensa improves energy efficiency and throughput by 3.0x and 3.1x over the Edge TPU, and by 2.4x and 4.3x over Eyeriss v2, a state-of-the-art accelerator.
DWA-RL: Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation among Mobile Obstacles

We present a novel Deep Reinforcement Learning (DRL) based policy to compute dynamically feasible and spatially aware velocities for a robot navigating among mobile obstacles. Our approach combines the benefits of the Dynamic Window Approach (DWA) in terms of satisfying the robot’s dynamics constraints with state-of-the-art DRL-based navigation methods that can handle moving obstacles and pedestrians well. Our formulation achieves these goals by embedding the environmental obstacles’ motions in a novel low-dimensional observation space. It also uses a novel reward function to positively reinforce velocities that move the robot away from the obstacle’s heading direction leading to significantly lower number of collisions. We evaluate our method in realistic 3-D simulated environments and on a real differential drive robot in challenging dense indoor scenarios with several walking pedestrians. We compare our method with state-of-the-art collision avoidance methods and observe significant improvements in terms of success rate (up to 33% increase), number of dynamics constraint violations (up to 61% decrease), and smoothness. We also conduct ablation studies to highlight the advantages of our observation space formulation, and reward structure.
Automated detection of leukemia by pretrained deep neural networks and transfer learning: A comparison

Leukemia is usually diagnosed by viewing the smears of blood and bone marrow using microscopes and complex Cytochemical tests can be used to authorize and classify leukemia. But these methods are costly, slow and affected by the proficiency and expertise of the specialists concerned. Leukemia can be detected with the help of image processing-based methods by analyzing microscopic smear images to detect the presence of leukemic cells and such techniques are simple, fast, cheap and not biased by the specialists. The proposed study presents a computer aided diagnosis system that uses pretrained deep Convolutional Neural Networks (CNNs) for detection of leukemia images against normal images. The use of pretrained networks is comparatively an easy method of applying deep learning for image analysis and the comparison results of the present study can be used to choose appropriate networks for diagnostic tasks. The microscopic images used in the proposed work were downloaded from a public dataset ALL-IDB. In the proposed work, image classification is done without using any image segregation and feature extraction practices and the study used pretrained series network AlexNet, VGG-16, VGG-19, Directed Acyclic Graph (DAG) networks GoogLeNet, Inceptionv3, MobileNet-v2, Xception, DenseNet-201, Inception-ResNet-v2 and residual networks ResNet-18, ResNet-50 and ResNet-101 for performing the classification and comparison. A classification accuracy of 100% is obtained with all the pretrained networks used in the study for ALL_IDB1 dataset and for ALL_IDB2 dataset, 100% accuracy is obtained with all networks except the AlexNet and VGG-16. The efficacy of three optimization algorithms Stochastic Gradient Descent with Momentum (SGDM), Root Mean Square propagation (RMSprop) and Adaptive Moment estimation (ADAM) is also compared in all the classifications performed. The study considered the detection of leukemia in general only, and classification of leukemia into different types can be attempted as a future work.
Why and how is Scrum being adapted in practice: A systematic review

Scrum, recognized today as the most popular agile development methodology, has been used in a wide range of settings and for varying purposes, in- and outside of the traditional software development context. The use of Scrum in non-traditional settings and for different needs led to a considerable corpus of academic literature that investigates, presents, and discusses modifications to the original method, aimed to make it fit such novel forms of application. Based on a large-scale review of extant literature, this study systematically analyses why and how Scrum was reportedly modified in different instances and contributes with a synthesis that can serve as a basis for a more systematic approach to future research and practice. We explicate nine common modification objectives for change (e.g., attaining high performance, non-standard contexts, distributed development) mapped against seven generic modification strategies (e.g., method guidance, new procedures, or artifacts). Building on our extensive literature analysis we highlight research gaps and identify promising areas for future research.
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning

Federated learning enables a cluster of decentralized mobile devices at the edge to collaboratively train a shared machine learning model, while keeping all the raw training samples on device. This decentralized training approach is demonstrated as a practical solution to mitigate the risk of privacy leakage. However, enabling efficient FL deployment at the edge is challenging because of non-IID training data distribution, wide system heterogeneity and stochastic-varying runtime effects in the field. This paper jointly optimizes time-to-convergence and energy efficiency of state-of-the-art FL use cases by taking into account the stochastic nature of edge execution. We propose AutoFL by tailor-designing a reinforcement learning algorithm that learns and determines which K participant devices and per-device execution targets for each FL model aggregation round in the presence of stochastic runtime variance, system and data heterogeneity. By considering the unique characteristics of FL edge deployment judiciously, AutoFL achieves 3.6 times faster model convergence time and 4.7 and 5.2 times higher energy efficiency for local clients and globally over the cluster of K participants, respectively.
Relationship-Preserving Knowledge Distillation for Zero-Shot Sketch Based Image Retrieval

Zero-shot sketch-based image retrieval is challenging for the modal gap between distributions of sketches and images and the inconsistency of label spaces during training and testing. Previous methods mitigate the modal gap by projecting sketches and images into a joint embedding space. Most of them also bridge seen and unseen classes by leveraging semantic embeddings, i.e., word vectors and hierarchical similarities. In this paper, we propose Relationship-Preserving Knowledge Distillation (RPKD) to study generalizable embeddings from the perspective of knowledge distillation bypassing the usage of semantic embeddings. In particular, we firstly distill the instance-level knowledge to preserve inter-class relationships without semantic similarities that require extra effort to collect. We also reconcile the contrastive relationships among instances between different embedding spaces, which is complementary to instance-level relationships. Furthermore, embedding-induced supervision, which measures the similarities of an instance to partial class embedding centers from the teacher, is developed to align the student's classification confidences. Extensive experiments conducted on three benchmark ZS-SBIR datasets, i.e., Sketchy, TU-Berlin, and QuickDraw, demonstrate the superiority of our proposed RPKD approach comparing to the state-of-the-art methods.
Photon Condensation and Enhanced Magnetism in Cavity QED

A system of magnetic molecules coupled to microwave cavities (LC resonators) undergoes the equilibrium superradiant phase transition. The transition is experimentally observable. The effect of the coupling is first illustrated by the vacuum-induced ferromagnetic order in a quantum Ising model and then by the modification of the magnetic phase diagram of Fe_{8} dipolar crystals, exemplifying the cooperation between intrinsic and photon-induced spin-spin interactions. Finally, a transmission experiment is shown to resolve the transition, measuring the quantum electrodynamical control of magnetism.
Cycle-Consistent Inverse GAN for Text-to-Image Synthesis

This paper investigates an open research task of text-to-image synthesis for automatically generating or manipulating images from text descriptions. Prevailing methods mainly take the textual descriptions as the conditional input for the GAN generation, and need to train different models for the text-guided image generation and manipulation tasks. In this paper, we propose a novel unified framework of Cycle-consistent Inverse GAN (CI-GAN) for both text-to-image generation and text-guided image manipulation tasks. Specifically, we first train a GAN model without text input, aiming to generate images with high diversity and quality. Then we learn a GAN inversion model to convert the images back to the GAN latent space and obtain the inverted latent codes for each image, where we introduce the cycle-consistency training to learn more robust and consistent inverted latent codes. We further uncover the semantics of the latent space of the trained GAN model, by learning a similarity model between text representations and the latent codes. In the text-guided optimization module, we can generate images with the desired semantic attributes through optimization on the inverted latent codes. Extensive experiments on the Recipe1M and CUB datasets validate the efficacy of our proposed framework.
Learning Tracking Representations via Dual-Branch Fully Transformer Networks

We present a Siamese-like Dual-branch network based on solely Transformers for tracking. Given a template and a search image, we divide them into non-overlapping patches and extract a feature vector for each patch based on its matching results with others within an attention window. For each token, we estimate whether it contains the target object and the corresponding size. The advantage of the approach is that the features are learned from matching, and ultimately, for matching. So the features are aligned with the object tracking task. The method achieves better or comparable results as the best-performing methods which first use CNN to extract features and then use Transformer to fuse them. It outperforms the state-of-the-art methods on the GOT-10k and VOT2020 benchmarks. In addition, the method achieves real-time inference speed (about 40 fps) on one GPU. The code and models are released at https://github.com/phiphiphi31/DualTFR.