OpenRaiser/Intern-Atlas huggingface dataset:
Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data
Venue: arXiv cs.CV 2021
Year: 2021

We present a network architecture which compares RGB images and untextured 3D models by the similarity of the represented shape. Our system is optimised for zero-shot retrieval, meaning it can recognise shapes never shown in training. We use a view-based shape descriptor and a siamese network to learn object geometry from pairs of 3D models and 2D images. Due to scarcity of datasets with exact photograph-mesh correspondences, we train our network with only synthetic data. Our experiments investigate the effect of different qualities and quantities of training data on retrieval accuracy and present insights from bridging the domain gap. We show that increasing the variety of synthetic data improves retrieval accuracy and that our system's performance in zero-shot mode can match that of the instance-aware mode, as far as narrowing down the search to the top 10% of objects.
Deterministic Massively Parallel Connectivity
Venue: arXiv cs.DS 2021
Year: 2021

We consider the problem of designing fundamental graph algorithms on the model of Massive Parallel Computation (MPC). The input to the problem is an undirected graph $G$ with $n$ vertices and $m$ edges, and with $D$ being the maximum diameter of any connected component in $G$. We consider the MPC with low local space, allowing each machine to store only $Θ(n^δ)$ words for an arbitrarily constant $δ> 0$, and with linear global space (which is equal to the number of machines times the local space available), that is, with optimal utilization. In a recent breakthrough, Andoni et al. (FOCS 18) and Behnezhad et al. (FOCS 19) designed parallel randomized algorithms that in $O(\log D + \log \log n)$ rounds on an MPC with low local space determine all connected components of an input graph, improving upon the classic bound of $O(\log n)$ derived from earlier works on PRAM algorithms. In this paper, we show that asymptotically identical bounds can be also achieved for deterministic algorithms: we present a deterministic MPC low local space algorithm that in $O(\log D + \log \log n)$ rounds determines all connected components of the input graph.
Mobile Blockchain Decentralized Applications (DApps): A Case Study Of IPTM Blockchain Certificate Verification System
Venue: arXiv cs.CR 2021
Year: 2021

A variety of mobile devices and applications have spread the usability of blockchain solutions to over 5.27 billion unique mobile phone users. The rising of Bitcoin price up to USD 50,000 in March 2021 has made many blockchain mobile wallets and smart contracts DApps popular for current and future investment of cryptocurrency and digital-asset managements. To understand the trend, this chapter will present the design and implementation of mobile blockchain DApps using Android Studio together with Ethereum smart contract as the digital-asset management tool. Java Android and Ethereum Web3-Java APIs will be demonstrated as a practical deployment of the mobile DApps. The logic and decision-making of the mobile DApps will be demonstrated and coded as a smart contract. The source codes of the mobile DApps and smart-contract were published in Github as open-source codes for those who are interested to build and run the project.
Effect of stepwise adjustment of Damping factor upon PageRank
Venue: arXiv cs.DM 2021
Year: 2021

The effect of adjusting damping factor α, from a small initial value α0 to the final desired αf value, upon then iterations needed for PageRank computation is observed. Adjustment of the damping factor is done in one or more steps. Results show no improvement in performance over a fixed damping factor based PageRank.
End-to-end Malaria Diagnosis and 3D Cell Rendering with Deep Learning
Venue: arXiv cs.CV 2021
Year: 2021

Malaria is a parasitic infection that poses a significant burden on global health. It kills one child every 30 seconds and over one million people annually. If diagnosed in a timely manner, however, most people can be effectively treated with antimalarial therapy. Several deaths due to malaria are byproducts of disparities in the social determinants of health; the current gold standard for diagnosing malaria requires microscopes, reagents, and other equipment that most patients of low socioeconomic brackets do not have access to. In this paper, we propose a convolutional neural network (CNN) architecture that allows for rapid automated diagnosis of malaria (achieving a high classification accuracy of 98%), as well as a deep neural network (DNN) based three-dimensional (3D) modeling algorithm that renders 3D models of parasitic cells in augmented reality (AR). This creates an opportunity to optimize the current workflow for malaria diagnosis and demonstrates potential for deep learning models to improve telemedicine practices and patient health literacy on a global scale.
ABD-Net: Attention Based Decomposition Network for 3D Point Cloud Decomposition
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose Attention Based Decomposition Network (ABD-Net), for point cloud decomposition into basic geometric shapes namely, plane, sphere, cone and cylinder. We show improved performance of 3D object classification using attention features based on primitive shapes in point clouds. Point clouds, being the simple and compact representation of 3D objects have gained increasing popularity. They demand robust methods for feature extraction due to unorderness in point sets. In ABD-Net the proposed Local Proximity Encapsulator captures the local geometric variations along with spatial encoding around each point from the input point sets. The encapsulated local features are further passed to proposed Attention Feature Encoder to learn basic shapes in point cloud. Attention Feature Encoder models geometric relationship between the neighborhoods of all the points resulting in capturing global point cloud information. We demonstrate the results of our proposed ABD-Net on ANSI mechanical component and ModelNet40 datasets. We also demonstrate the effectiveness of ABD-Net over the acquired attention features by improving the performance of 3D object classification on ModelNet40 benchmark dataset and compare them with state-of-the-art techniques.
Adversarial Motorial Prototype Framework for Open Set Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Open set recognition is designed to identify known classes and to reject unknown classes simultaneously. Specifically, identifying known classes and rejecting unknown classes correspond to reducing the empirical risk and the open space risk, respectively. First, the motorial prototype framework (MPF) is proposed, which classifies known classes according to the prototype classification idea. Moreover, a motorial margin constraint term is added into the loss function of the MPF, which can further improve the clustering compactness of known classes in the feature space to reduce both risks. Second, this paper proposes the adversarial motorial prototype framework (AMPF) based on the MPF. On the one hand, this model can generate adversarial samples and add these samples into the training phase; on the other hand, it can further improve the differential mapping ability of the model to known and unknown classes with the adversarial motion of the margin constraint radius. Finally, this paper proposes an upgraded version of the AMPF, AMPF++, which adds much more generated unknown samples into the training phase. In this paper, a large number of experiments prove that the performance of the proposed models is superior to that of other current works.
Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge
Venue: arXiv cs.CV 2021
Year: 2021

In this report, we introduce our real-time 2D object detection system for the realistic autonomous driving scenario. Our detector is built on a newly designed YOLO model, called YOLOX. On the Argoverse-HD dataset, our system achieves 41.0 streaming AP, which surpassed second place by 7.8/6.1 on detection-only track/fully track, respectively. Moreover, equipped with TensorRT, our model achieves the 30FPS inference speed with a high-resolution input size (e.g., 1440-2304). Code and models will be available at https://github.com/Megvii-BaseDetection/YOLOX
Adding Visibility to Visibility Graphs: Weighting Visibility Analysis with Attenuation Coefficients
Venue: arXiv cs.CV 2021
Year: 2021

Evaluating the built environment based on visibility has been long used as a tool for human-centric design. The origins of isovists and visibility graphs are within interior spaces, while more recently, these evaluation techniques have been applied in the urban context. One of the key differentiators of an outside environment is the weather, which has largely been ignored in the design computation and space-syntax research areas. While a visibility graph is a straightforward metric for determining connectivity between regions of space through a line of sight calculation, this approach largely ignores the actual visibility of one point to another. This paper introduces a new method for weighting a visibility graph based on weather conditions (i.e. rain, fog, snow). These new factors are integrated into visibility graphs and applied to sample environments to demonstrate the variance between assuming a straight line of sight and reduced visibility.
An optical biomimetic eyes with interested object imaging
Venue: arXiv cs.CV 2021
Year: 2021

We presented an optical system to perform imaging interested objects in complex scenes, like the creature easy see the interested prey in the hunt for complex environments. It utilized Deep-learning network to learn the interested objects's vision features and designed the corresponding "imaging matrices", furthermore the learned matrixes act as the measurement matrix to complete compressive imaging with a single-pixel camera, finally we can using the compressed image data to only image the interested objects without the rest objects and backgrounds of the scenes with the previous Deep-learning network. Our results demonstrate that no matter interested object is single feature or rich details, the interference can be successfully filtered out and this idea can be applied in some common applications that effectively improve the performance. This bio-inspired optical system can act as the creature eye to achieve success on interested-based object imaging, object detection, object recognition and object tracking, etc.
Generative Adversarial Neural Cellular Automata
Venue: arXiv cs.NE 2021
Year: 2021

Motivated by the interaction between cells, the recently introduced concept of Neural Cellular Automata shows promising results in a variety of tasks. So far, this concept was mostly used to generate images for a single scenario. As each scenario requires a new model, this type of generation seems contradictory to the adaptability of cells in nature. To address this contradiction, we introduce a concept using different initial environments as input while using a single Neural Cellular Automata to produce several outputs. Additionally, we introduce GANCA, a novel algorithm that combines Neural Cellular Automata with Generative Adversarial Networks, allowing for more generalization through adversarial training. The experiments show that a single model is capable of learning several images when presented with different inputs, and that the adversarially trained model improves drastically on out-of-distribution data compared to a supervised trained model.
VirtualConductor: Music-driven Conducting Video Generation System
Venue: arXiv cs.CV 2021
Year: 2021

In this demo, we present VirtualConductor, a system that can generate conducting video from any given music and a single user's image. First, a large-scale conductor motion dataset is collected and constructed. Then, we propose Audio Motion Correspondence Network (AMCNet) and adversarial-perceptual learning to learn the cross-modal relationship and generate diverse, plausible, music-synchronized motion. Finally, we combine 3D animation rendering and a pose transfer model to synthesize conducting video from a single given user's image. Therefore, any user can become a virtual conductor through the system.
Hyperparameter Analysis for Derivative Compressive Sampling
Venue: arXiv cs.CV 2021
Year: 2021

Derivative compressive sampling (DCS) is a signal reconstruction method from measurements of the spatial gradient with sub-Nyquist sampling rate. Applications of DCS include optical image reconstruction, photometric stereo, and shape-from-shading. In this work, we study the sensitivity of DCS with respect to algorithmic hyperparameters using a brute-force search algorithm. We perform experiments on a dataset of surface images and deduce guidelines for the user to setup values for the hyperparameters for improved signal recovery performance.
Gemini2: Generating Keyframe-Oriented Animated Transitions Between Statistical Graphics
Venue: arXiv cs.HC 2021
Year: 2021

Complex animated transitions may be easier to understand when divided into separate, consecutive stages. However, effective staging requires careful attention to both animation semantics and timing parameters. We present Gemini2, a system for creating staged animations from a sequence of chart keyframes. Given only a start state and an end state, Gemini2 can automatically recommend intermediate keyframes for designers to consider. The Gemini2 recommendation engine leverages Gemini, our prior work, and GraphScape to itemize the given complex change into semantic edit operations and to recombine operations into stages with a guided order for clearly conveying the semantics. To evaluate Gemini2's recommendations, we conducted a human-subject study in which participants ranked recommended animations from both Gemini2 and Gemini. We find that Gemini2's animation recommendation ranking is well aligned with subjects' preferences, and Gemini2 can recommend favorable animations that Gemini cannot support.
Issue Link Label Recovery and Prediction for Open Source Software
Venue: arXiv cs.SE 2021
Year: 2021

Modern open source software development heavily relies on the issue tracking systems to manage their feature requests, bug reports, tasks, and other similar artifacts. Together, those "issues" form a complex network with links to each other. The heterogeneous character of issues inherently results in varied link types and therefore poses a great challenge for users to create and maintain the label of the link manually. The goal of most existing automated issue link construction techniques ceases with only examining the existence of links between issues. In this work, we focus on the next important question of whether we can assess the type of issue link automatically through a data-driven method. We analyze the links between issues and their labels used the issue tracking system for 66 open source projects. Using three projects, we demonstrate promising results when using supervised machine learning classification for the task of link label recovery with careful model selection and tuning, achieving F1 scores of between 0.56-0.70 for the three studied projects. Further, the performance of our method for future link label prediction is convincing when there is sufficient historical data. Our work signifies the first step in systematically manage and maintain issue links faced in practice.
Privacy-Preserving Machine Learning: Methods, Challenges and Directions
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning (ML) is increasingly being adopted in a wide variety of application domains. Usually, a well-performing ML model relies on a large volume of training data and high-powered computational resources. Such a need for and the use of huge volumes of data raise serious privacy concerns because of the potential risks of leakage of highly privacy-sensitive information; further, the evolving regulatory environments that increasingly restrict access to and use of privacy-sensitive data add significant challenges to fully benefiting from the power of ML for data-driven applications. A trained ML model may also be vulnerable to adversarial attacks such as membership, attribute, or property inference attacks and model inversion attacks. Hence, well-designed privacy-preserving ML (PPML) solutions are critically needed for many emerging applications. Increasingly, significant research efforts from both academia and industry can be seen in PPML areas that aim toward integrating privacy-preserving techniques into ML pipeline or specific algorithms, or designing various PPML architectures. In particular, existing PPML research cross-cut ML, systems and applications design, as well as security and privacy areas; hence, there is a critical need to understand state-of-the-art research, related challenges and a research roadmap for future research in PPML area. In this paper, we systematically review and summarize existing privacy-preserving approaches and propose a Phase, Guarantee, and Utility (PGU) triad based model to understand and guide the evaluation of various PPML solutions by decomposing their privacy-preserving functionalities. We discuss the unique characteristics and challenges of PPML and outline possible research directions that leverage as well as benefit multiple research communities such as ML, distributed systems, security and privacy.
A $D$-competitive algorithm for the Multilevel Aggregation Problem with Deadlines
Venue: arXiv cs.DS 2021
Year: 2021

In this paper, we consider the multi-level aggregation problem with deadlines (MLAPD) previously studied by Bienkowski et al. (2015), Buchbinder et al. (2017), and Azar and Touitou (2019). This is an online problem where the algorithm services requests arriving over time and can save costs by aggregating similar requests. Costs are structured in the form of a rooted tree. This problem has applications to many important areas such as multicasting, sensor networks, and supply-chain management. In particular, the TCP-acknowledgment problem, joint-replenishment problem, and assembly problem are all special cases of the delay version of the problem. We present a $D$-competitive algorithm for MLAPD. This beats the $6(D+1)$-competitive algorithm given in Buchbinder et al. (2017). Our approach illuminates key structural aspects of the problem and provides an algorithm that is simpler to implement than previous approaches. We also give improved competitive ratios for special cases of the problem.
Method Towards CVPR 2021 Image Matching Challenge
Venue: arXiv cs.CV 2021
Year: 2021

This report describes Megvii-3D team's approach towards CVPR 2021 Image Matching Workshop.
CPNet: Cross-Parallel Network for Efficient Anomaly Detection
Venue: arXiv cs.CV 2021
Year: 2021

Anomaly detection in video streams is a challenging problem because of the scarcity of abnormal events and the difficulty of accurately annotating them. To alleviate these issues, unsupervised learning-based prediction methods have been previously applied. These approaches train the model with only normal events and predict a future frame from a sequence of preceding frames by use of encoder-decoder architectures so that they result in small prediction errors on normal events but large errors on abnormal events. The architecture, however, comes with the computational burden as some anomaly detection tasks require low computational cost without sacrificing performance. In this paper, Cross-Parallel Network (CPNet) for efficient anomaly detection is proposed here to minimize computations without performance drops. It consists of N smaller parallel U-Net, each of which is designed to handle a single input frame, to make the calculations significantly more efficient. Additionally, an inter-network shift module is incorporated to capture temporal relationships among sequential frames to enable more accurate future predictions.The quantitative results show that our model requires less computational cost than the baseline U-Net while delivering equivalent performance in anomaly detection.
A heuristic for estimating Nash equilibria in first-price auctions with correlated values
Venue: arXiv cs.GT 2021
Year: 2021

Our paper concerns the computation of Nash equilibria of first-price auctions with correlated values. While there exist several equilibrium computation methods for auctions with independent values, the correlation of the bidders' values introduces significant complications that render existing methods unsatisfactory in practice. Our contribution is a step towards filling this gap: inspired by the seminal fictitious play process of Brown and Robinson, we present a learning heuristic-that we call fictitious bidding (FB)-for estimating Bayes-Nash equilibria of first-price auctions with correlated values, and we assess the performance of this heuristic on several relevant examples.
TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization
Venue: arXiv cs.CV 2021
Year: 2021

Finding tampered regions in images is a hot research topic in machine learning and computer vision. Although many image manipulation location algorithms have been proposed, most of them only focus on the RGB images with different color spaces, and the frequency information that contains the potential tampering clues is often ignored. In this work, a novel end-to-end two-stream boundary-aware network (abbreviated as TBNet) is proposed for generic image manipulation localization in which the RGB stream, the frequency stream, and the boundary artifact location are explored in a unified framework. Specifically, we first design an adaptive frequency selection module (AFS) to adaptively select the appropriate frequency to mine inconsistent statistics and eliminate the interference of redundant statistics. Then, an adaptive cross-attention fusion module (ACF) is proposed to adaptively fuse the RGB feature and the frequency feature. Finally, the boundary artifact location network (BAL) is designed to locate the boundary artifacts for which the parameters are jointly updated by the outputs of the ACF, and its results are further fed into the decoder. Thus, the parameters of the RGB stream, the frequency stream, and the boundary artifact location network are jointly optimized, and their latent complementary relationships are fully mined. The results of extensive experiments performed on four public benchmarks of the image manipulation localization task, namely, CASIA1.0, COVER, Carvalho, and In-The-Wild, demonstrate that the proposed TBNet can significantly outperform state-of-the-art generic image manipulation localization methods in terms of both MCC and F1.