OpenRaiser/Intern-Atlas huggingface dataset:
Product Re-identification System in Fully Automated Defect Detection
Venue: arXiv cs.CV 2021
Year: 2021

In this work, we introduce a method and present an improved neural work to perform product re-identification, which is an essential core function of a fully automated product defect detection system. Our method is based on feature distance. It is the combination of feature extraction neural networks, such as VGG16, AlexNet, with an image search engine - Vearch. The dataset that we used to develop product re-identification systems is a water-bottle dataset that consists of 400 images of 18 types of water bottles. This is a small dataset, which was the biggest challenge of our work. However, the combination of neural networks with Vearch shows potential to tackle the product re-identification problems. Especially, our new neural network - AlphaAlexNet that a neural network was improved based on AlexNet could improve the production identification accuracy by four percent. This indicates that an ideal production identification accuracy could be achieved when efficient feature extraction methods could be introduced and redesigned for image feature extractions of nearly identical products. In order to solve the biggest challenges caused by the small size of the dataset and the difficult nature of identifying productions that have little differences from each other. In our future work, we propose a new roadmap to tackle nearly-identical production identifications: to introduce or develop new algorithms that need very few images to train themselves.
A Comprehensive Analytical Survey on Unsupervised and Semi-Supervised Graph Representation Learning Methods
Venue: arXiv cs.LG 2021
Year: 2021

Graph representation learning is a fast-growing field where one of the main objectives is to generate meaningful representations of graphs in lower-dimensional spaces. The learned embeddings have been successfully applied to perform various prediction tasks, such as link prediction, node classification, clustering, and visualization. The collective effort of the graph learning community has delivered hundreds of methods, but no single method excels under all evaluation metrics such as prediction accuracy, running time, scalability, etc. This survey aims to evaluate all major classes of graph embedding methods by considering algorithmic variations, parameter selections, scalability, hardware and software platforms, downstream ML tasks, and diverse datasets. We organized graph embedding techniques using a taxonomy that includes methods from manual feature engineering, matrix factorization, shallow neural networks, and deep graph convolutional networks. We evaluated these classes of algorithms for node classification, link prediction, clustering, and visualization tasks using widely used benchmark graphs. We designed our experiments on top of PyTorch Geometric and DGL libraries and run experiments on different multicore CPU and GPU platforms. We rigorously scrutinize the performance of embedding methods under various performance metrics and summarize the results. Thus, this paper may serve as a comparative guide to help users select methods that are most suitable for their tasks.
Image Animation with Keypoint Mask
Venue: arXiv cs.CV 2021
Year: 2021

Motion transfer is the task of synthesizing future video frames of a single source image according to the motion from a given driving video. In order to solve it, we face the challenging complexity of motion representation and the unknown relations between the driving video and the source image. Despite its difficulty, this problem attracted great interests from researches at the recent years, with gradual improvements. The goal is often thought as the decoupling of motion and appearance, which is may be solved by extracting the motion from keypoint movement. We chose to tackle the generic, unsupervised setting, where we need to apply animation to any arbitrary object, without any domain specific model for the structure of the input. In this work, we extract the structure from a keypoint heatmap, without an explicit motion representation. Then, the structures from the image and the video are extracted to warp the image according to the video, by a deep generator. We suggest two variants of the structure from different steps in the keypoint module, and show superior qualitative pose and quantitative scores.
Improving Energy Efficiency of Massive MIMO Relay Systems using Power Bisection Allocation for Cell-Edge Users
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies downlink energy efficiency (EE) of a cellular massive Multiple Input Multiple Output (MIMO) system with massive number of antennas at both base station (BS) and relay considering cell-edge users which is less investigated in the literature. Some improving methods for base station (BS) and relay power allocation under quality of service constraints are proposed. The EE optimization problem is provided over BS and relay power variables (power dimensions). Considering the quasi-concavity property of the EE function relative to each power variable, we propose to use a power bisection algorithm (PBA) in one dimension, followed by the exhaustive search in the other dimension (ODS algorithm) which is called PBODS algorithm and also a method to limit the range of power variables. The results show that the performance of the PBODS algorithm approaches the performance of optimal solution which is exhaustive search in both dimensions while it has a lower complexity. In addition to that we suggest to use the PBA in two dimensions alternatively as the sub-optimal alternative optimization (AOP) algorithm. The complexity can highly be reduced using the AOP algorithm with a slight but acceptable degradation in performance which makes the algorithm much suitable for practical use.
Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization
Venue: arXiv cs.LG 2021
Year: 2021

Deep reinforcement learning algorithms can perform poorly in real-world tasks due to the discrepancy between source and target environments. This discrepancy is commonly viewed as the disturbance in transition dynamics. Many existing algorithms learn robust policies by modeling the disturbance and applying it to source environments during training, which usually requires prior knowledge about the disturbance and control of simulators. However, these algorithms can fail in scenarios where the disturbance from target environments is unknown or is intractable to model in simulators. To tackle this problem, we propose a novel model-free actor-critic algorithm -- namely, state-conservative policy optimization (SCPO) -- to learn robust policies without modeling the disturbance in advance. Specifically, SCPO reduces the disturbance in transition dynamics to that in state space and then approximates it by a simple gradient-based regularizer. The appealing features of SCPO include that it is simple to implement and does not require additional knowledge about the disturbance or specially designed simulators. Experiments in several robot control tasks demonstrate that SCPO learns robust policies against the disturbance in transition dynamics.
A Memristor-Based Bayesian Machine
Venue: arXiv cs.ET 2021
Year: 2021

In recent years, a considerable research effort has shown the energy benefits of implementing neural networks with memristors or other emerging memory technologies. However, for extreme-edge applications with high uncertainty, access to reduced amounts of data, and where explainable decisions are required, neural networks may not provide an acceptable form of intelligence. Bayesian reasoning can solve these concerns, but it is computationally expensive and, unlike neural networks, does not translate naturally to memristor-based architectures. In this work, we introduce, demonstrate experimentally on a fully fabricated hybrid CMOS-memristor system, and analyze a Bayesian machine designed for highly-energy efficient Bayesian reasoning. The architecture of the machine is obtained by writing Bayes' law in a way making its implementation natural by the principles of distributed memory and stochastic computing, allowing the circuit to function using solely local memory and minimal data movement. Measurements on a fabricated small-scale Bayesian machine featuring 2,048 memristors and 30,080 transistors show the viability of this approach and the possibility of overcoming the challenges associated with its design: the inherent imperfections of memristors, as well as the need to distribute very locally higher-than-nominal supply voltages. The design of a scaled-up version of the machine shows its outstanding energy efficiency on a real-life gesture recognition task: a gesture can be recognized using 5,000 times less energy than using a microcontroller unit. The Bayesian machine also features several desirable features, e.g., instant on/off operation, compatibility with low supply voltages, and resilience to single-event upsets. These results open the road for Bayesian reasoning as an attractive way for energy-efficient, robust, and explainable intelligence at the edge.
Exact Shapley Values for Local and Model-True Explanations of Decision Tree Ensembles
Venue: arXiv cs.LG 2021
Year: 2021

Additive feature explanations using Shapley values have become popular for providing transparency into the relative importance of each feature to an individual prediction of a machine learning model. While Shapley values provide a unique additive feature attribution in cooperative game theory, the Shapley values that can be generated for even a single machine learning model are far from unique, with theoretical and implementational decisions affecting the resulting attributions. Here, we consider the application of Shapley values for explaining decision tree ensembles and present a novel approach to Shapley value-based feature attribution that can be applied to random forests and boosted decision trees. This new method provides attributions that accurately reflect details of the model prediction algorithm for individual instances, while being computationally competitive with one of the most widely used current methods. We explain the theoretical differences between the standard and novel approaches and compare their performance using synthetic and real data.
Differentially Private Regret Minimization in Episodic Markov Decision Processes
Venue: arXiv cs.LG 2021
Year: 2021

We study regret minimization in finite horizon tabular Markov decision processes (MDPs) under the constraints of differential privacy (DP). This is motivated by the widespread applications of reinforcement learning (RL) in real-world sequential decision making problems, where protecting users' sensitive and private information is becoming paramount. We consider two variants of DP -- joint DP (JDP), where a centralized agent is responsible for protecting users' sensitive data and local DP (LDP), where information needs to be protected directly on the user side. We first propose two general frameworks -- one for policy optimization and another for value iteration -- for designing private, optimistic RL algorithms. We then instantiate these frameworks with suitable privacy mechanisms to satisfy JDP and LDP requirements, and simultaneously obtain sublinear regret guarantees. The regret bounds show that under JDP, the cost of privacy is only a lower order additive term, while for a stronger privacy protection under LDP, the cost suffered is multiplicative. Finally, the regret bounds are obtained by a unified analysis, which, we believe, can be extended beyond tabular MDPs.
FuSeBMC v.4: Smart Seed Generation for Hybrid Fuzzing
Venue: arXiv cs.CR 2021
Year: 2021

FuSeBMC is a test generator for finding security vulnerabilities in C programs. In earlier work [4], we described a previous version that incrementally injected labels to guide Bounded Model Checking (BMC) and Evolutionary Fuzzing engines to produce test cases for code coverage and bug finding. This paper introduces a new version of FuSeBMC that utilizes both engines to produce smart seeds. First, the engines are run with a short time limit on a lightly instrumented version of the program to produce the seeds. The BMC engine is particularly useful in producing seeds that can pass through complex mathematical guards. Then, FuSeBMC runs its engines with more extended time limits using the smart seeds created in the previous round. FuSeBMC manages this process in two main ways using its Tracer subsystem. Firstly, it uses shared memory to record the labels covered by each test case. Secondly, it evaluates test cases, and those of high impact are turned into seeds for subsequent test fuzzing. As a result, we significantly increased our code coverage score from last year, outperforming all tools that participated in this year's competition in every single category.
Proving Theorems using Incremental Learning and Hindsight Experience Replay
Venue: arXiv cs.AI 2021
Year: 2021

Traditional automated theorem provers for first-order logic depend on speed-optimized search and many handcrafted heuristics that are designed to work best over a wide range of domains. Machine learning approaches in literature either depend on these traditional provers to bootstrap themselves or fall short on reaching comparable performance. In this paper, we propose a general incremental learning algorithm for training domain specific provers for first-order logic without equality, based only on a basic given-clause algorithm, but using a learned clause-scoring function. Clauses are represented as graphs and presented to transformer networks with spectral features. To address the sparsity and the initial lack of training data as well as the lack of a natural curriculum, we adapt hindsight experience replay to theorem proving, so as to be able to learn even when no proof can be found. We show that provers trained this way can match and sometimes surpass state-of-the-art traditional provers on the TPTP dataset in terms of both quantity and quality of the proofs.
Learning Semi-Structured Representations of Radiology Reports
Venue: arXiv cs.CL 2021
Year: 2021

Beyond their primary diagnostic purpose, radiology reports have been an invaluable source of information in medical research. Given a corpus of radiology reports, researchers are often interested in identifying a subset of reports describing a particular medical finding. Because the space of medical findings in radiology reports is vast and potentially unlimited, recent studies proposed mapping free-text statements in radiology reports to semi-structured strings of terms taken from a limited vocabulary. This paper aims to present an approach for the automatic generation of semi-structured representations of radiology reports. The approach consists of matching sentences from radiology reports to manually created semi-structured representations, followed by learning a sequence-to-sequence neural model that maps matched sentences to their semi-structured representations. We evaluated the proposed approach on the OpenI corpus of manually annotated chest x-ray radiology reports. The results indicate that the proposed approach is superior to several baselines, both in terms of (1) quantitative measures such as BLEU, ROUGE, and METEOR and (2) qualitative judgment of a radiologist. The results also demonstrate that the trained model produces reasonable semi-structured representations on an out-of-sample corpus of chest x-ray radiology reports from a different medical provider.
Mask2Former for Video Instance Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline. In this report, we show universal image segmentation architectures trivially generalize to video segmentation by directly predicting 3D segmentation volumes. Specifically, Mask2Former sets a new state-of-the-art of 60.4 AP on YouTubeVIS-2019 and 52.6 AP on YouTubeVIS-2021. We believe Mask2Former is also capable of handling video semantic and panoptic segmentation, given its versatility in image segmentation. We hope this will make state-of-the-art video segmentation research more accessible and bring more attention to designing universal image and video segmentation architectures.
Natural language processing to identify lupus nephritis phenotype in electronic health records
Venue: arXiv cs.LG 2021
Year: 2021

Systemic lupus erythematosus (SLE) is a rare autoimmune disorder characterized by an unpredictable course of flares and remission with diverse manifestations. Lupus nephritis, one of the major disease manifestations of SLE for organ damage and mortality, is a key component of lupus classification criteria. Accurately identifying lupus nephritis in electronic health records (EHRs) would therefore benefit large cohort observational studies and clinical trials where characterization of the patient population is critical for recruitment, study design, and analysis. Lupus nephritis can be recognized through procedure codes and structured data, such as laboratory tests. However, other critical information documenting lupus nephritis, such as histologic reports from kidney biopsies and prior medical history narratives, require sophisticated text processing to mine information from pathology reports and clinical notes. In this study, we developed algorithms to identify lupus nephritis with and without natural language processing (NLP) using EHR data. We developed four algorithms: a rule-based algorithm using only structured data (baseline algorithm) and three algorithms using different NLP models. The three NLP models are based on regularized logistic regression and use different sets of features including positive mention of concept unique identifiers (CUIs), number of appearances of CUIs, and a mixture of three components respectively. The baseline algorithm and the best performed NLP algorithm were external validated on a dataset from Vanderbilt University Medical Center (VUMC). Our best performing NLP model incorporating features from both structured data, regular expression concepts, and mapped CUIs improved F measure in both the NMEDW (0.41 vs 0.79) and VUMC (0.62 vs 0.96) datasets compared to the baseline lupus nephritis algorithm.
Understanding User Perspectives on Prompts for Brief Reflection on Troubling Emotions
Venue: arXiv cs.HC 2021
Year: 2021

We investigate users' perspectives on an online reflective question activity (RQA) that prompts people to externalize their underlying emotions on a troubling situation. Inspired by principles of cognitive behavioral therapy, our 15-minute activity encourages self-reflection without a human or automated conversational partner. A deployment of our RQA on Amazon Mechanical Turk suggests that people perceive several benefits from our RQA, including structured awareness of their thoughts and problem-solving around managing their emotions. Quantitative evidence from a randomized experiment suggests people find that our RQA makes them feel less worried by their selected situation and worth the minimal time investment. A further two-week technology probe deployment with 11 participants indicates that people see benefits to doing this activity repeatedly, although the activity may get monotonous over time. In summary, this work demonstrates the promise of online reflection activities that carefully leverage principles of psychology in their design.
Spatiotemporal Motion Synchronization for Snowboard Big Air
Venue: arXiv cs.CV 2021
Year: 2021

During the training for snowboard big air, one of the most popular winter sports, athletes and coaches extensively shoot and check their jump attempts using a single camera or smartphone. However, by watching videos sequentially, it is difficult to compare the precise difference in performance between two trials. Therefore, side-by-side display or overlay of two videos may be helpful for training. To accomplish this, the spatial and temporal alignment of multiple performances must be ensured. In this study, we propose a conventional but plausible solution using the existing image processing techniques for snowboard big air training. We conducted interviews with expert snowboarders who stated that the spatiotemporally aligned videos enabled them to precisely identify slight differences in body movements. The results suggest that the proposed method can be used during the training of snowboard big air.
Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODE
Venue: arXiv cs.CV 2021
Year: 2021

In order to perform unconditional video generation, we must learn the distribution of the real-world videos. In an effort to synthesize high-quality videos, various studies attempted to learn a mapping function between noise and videos, including recent efforts to separate motion distribution and appearance distribution. Previous methods, however, learn motion dynamics in discretized, fixed-interval timesteps, which is contrary to the continuous nature of motion of a physical body. In this paper, we propose a novel video generation approach that learns separate distributions for motion and appearance, the former modeled by neural ODE to learn natural motion dynamics. Specifically, we employ a two-stage approach where the first stage converts a noise vector to a sequence of keypoints in arbitrary frame rates, and the second stage synthesizes videos based on the given keypoints sequence and the appearance noise vector. Our model not only quantitatively outperforms recent baselines for video generation, but also demonstrates versatile functionality such as dynamic frame rate manipulation and motion transfer between two datasets, thus opening new doors to diverse video generation applications.
Generalizing Interactive Backpropagating Refinement for Dense Prediction
Venue: arXiv cs.CV 2021
Year: 2021

As deep neural networks become the state-of-the-art approach in the field of computer vision for dense prediction tasks, many methods have been developed for automatic estimation of the target outputs given the visual inputs. Although the estimation accuracy of the proposed automatic methods continues to improve, interactive refinement is oftentimes necessary for further correction. Recently, feature backpropagating refinement scheme (f-BRS) has been proposed for the task of interactive segmentation, which enables efficient optimization of a small set of auxiliary variables inserted into the pretrained network to produce object segmentation that better aligns with user inputs. However, the proposed auxiliary variables only contain channel-wise scale and bias, limiting the optimization to global refinement only. In this work, in order to generalize backpropagating refinement for a wide range of dense prediction tasks, we introduce a set of G-BRS (Generalized Backpropagating Refinement Scheme) layers that enable both global and localized refinement for the following tasks: interactive segmentation, semantic segmentation, image matting and monocular depth estimation. Experiments on SBD, Cityscapes, Mapillary Vista, Composition-1k and NYU-Depth-V2 show that our method can successfully generalize and significantly improve performance of existing pretrained state-of-the-art models with only a few clicks.
On the role of the Fibonacci matrix as key in modified ECC
Venue: arXiv cs.CR 2021
Year: 2021

In this paper, we have proposed a modified cryptographic scheme based on the application of recursive matrices as key in ECC and ElGamal. For encryption, we consider mapping analogous to affine Hill cipher in which a plaintext matrix has been constructed by points corresponding to letters on elliptic curves. In the formation of key-space, the generalized Fibonacci matrices have been taken into account, which is the sequence of matrices. The beauty of considering Fibonacci matrices is their construction where we need only two parameters(integers) in place of $n^2$ elements. The use of a recursive matrix makes a large keyspace for our proposed scheme and increases its efficiency. Thus, it reduces time as well space complexity, and its security \& strength is based on EC-DLP which is a hard problem in number theory.