OpenRaiser/Intern-Atlas huggingface dataset:
Novel Approach to Use HU Moments with Image Processing Techniques for Real Time Sign Language Communication
Venue: arXiv cs.CV 2020
Year: 2020

Sign language is the fundamental communication method among people who suffer from speech and hearing defects. The rest of the world doesn't have a clear idea of sign language. "Sign Language Communicator" (SLC) is designed to solve the language barrier between the sign language users and the rest of the world. The main objective of this research is to provide a low cost affordable method of sign language interpretation. This system will also be very useful to the sign language learners as they can practice the sign language. During the research available human computer interaction techniques in posture recognition was tested and evaluated. A series of image processing techniques with Hu-moment classification was identified as the best approach. To improve the accuracy of the system, a new approach height to width ratio filtration was implemented along with Hu-moments. System is able to recognize selected Sign Language signs with the accuracy of 84% without a controlled background with small light adjustments
Learning Gaussian Instance Segmentation in Point Clouds
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a novel method for instance segmentation of 3D point clouds. The proposed method is called Gaussian Instance Center Network (GICN), which can approximate the distributions of instance centers scattered in the whole scene as Gaussian center heatmaps. Based on the predicted heatmaps, a small number of center candidates can be easily selected for the subsequent predictions with efficiency, including i) predicting the instance size of each center to decide a range for extracting features, ii) generating bounding boxes for centers, and iii) producing the final instance masks. GICN is a single-stage, anchor-free, and end-to-end architecture that is easy to train and efficient to perform inference. Benefited from the center-dictated mechanism with adaptive instance size selection, our method achieves state-of-the-art performance in the task of 3D instance segmentation on ScanNet and S3DIS datasets.
Attention Sequence to Sequence Model for Machine Remaining Useful Life Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Accurate estimation of remaining useful life (RUL) of industrial equipment can enable advanced maintenance schedules, increase equipment availability and reduce operational costs. However, existing deep learning methods for RUL prediction are not completely successful due to the following two reasons. First, relying on a single objective function to estimate the RUL will limit the learned representations and thus affect the prediction accuracy. Second, while longer sequences are more informative for modelling the sensor dynamics of equipment, existing methods are less effective to deal with very long sequences, as they mainly focus on the latest information. To address these two problems, we develop a novel attention-based sequence to sequence with auxiliary task (ATS2S) model. In particular, our model jointly optimizes both reconstruction loss to empower our model with predictive capabilities (by predicting next input sequence given current input sequence) and RUL prediction loss to minimize the difference between the predicted RUL and actual RUL. Furthermore, to better handle longer sequence, we employ the attention mechanism to focus on all the important input information during training process. Finally, we propose a new dual-latent feature representation to integrate the encoder features and decoder hidden states, to capture rich semantic information in data. We conduct extensive experiments on four real datasets to evaluate the efficacy of the proposed method. Experimental results show that our proposed method can achieve superior performance over 13 state-of-the-art methods consistently.
Parallel Oculomotor Plant Mathematical Model for Large Scale Eye Movement Simulation
Venue: arXiv cs.HC 2020
Year: 2020

The usage of eye tracking sensors is expected to grow in virtual (VR) and augmented reality (AR) platforms. Provided that users of these platforms consent to employing captured eye movement signals for authentication and health assessment, it becomes important to estimate oculomotor plant and brain function characteristics in real time. This paper shows a path toward that goal by presenting a parallel processing architecture capable of estimating oculomotor plant characteristics and comparing its performance to a single-threaded implementation. Results show that the parallel implementation improves the speed, accuracy, and throughput of oculomotor plant characteristic estimation versus the original serial version for both large-scale and real-time simulation.
Distractor-Aware Neuron Intrinsic Learning for Generic 2D Medical Image Classifications
Venue: arXiv cs.CV 2020
Year: 2020

Medical image analysis benefits Computer Aided Diagnosis (CADx). A fundamental analyzing approach is the classification of medical images, which serves for skin lesion diagnosis, diabetic retinopathy grading, and cancer classification on histological images. When learning these discriminative classifiers, we observe that the convolutional neural networks (CNNs) are vulnerable to distractor interference. This is due to the similar sample appearances from different categories (i.e., small inter-class distance). Existing attempts select distractors from input images by empirically estimating their potential effects to the classifier. The essences of how these distractors affect CNN classification are not known. In this paper, we explore distractors from the CNN feature space via proposing a neuron intrinsic learning method. We formulate a novel distractor-aware loss that encourages large distance between the original image and its distractor in the feature space. The novel loss is combined with the original classification loss to update network parameters by back-propagation. Neuron intrinsic learning first explores distractors crucial to the deep classifier and then uses them to robustify CNN inherently. Extensive experiments on medical image benchmark datasets indicate that the proposed method performs favorably against the state-of-the-art approaches.
Incentive-Based Selection and Composition of IoT Energy Services
Venue: arXiv cs.DC 2020
Year: 2020

We propose a novel incentive-based framework for composing energy service requests. An incentive model is designed that considers the context of the providers and consumers to determine rewards for sharing wireless energy. We propose a novel priority scheduling approach to compose energy service requests that maximizes the reward of the provider. A set of exhaustive experiments with a dataset and collected IoT users' behavior is conducted to evaluate the proposed approach. Experimental results prove the efficiency of the proposed approach.
Power Minimization for Multi-cell Uplink NOMA with Imperfect SIC
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we investigate a multi-cell uplink non-orthogonal multiple access (NOMA) system with imperfect successive interference cancellation (SIC). The objective of the formulated optimization problem is to minimize the total power consumption under users' quality-of-service constraints. The considered problem is first transformed into a linear programming problem, upon which centralized and distributed optimal solutions are proposed. Numerical results are presented to verify the performance of the proposed solutions and evaluate the impact of imperfect SIC on the system performance.
Energy-Efficient Resource Allocation for IRS-Assisted Multi-Antenna Uplink Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study the resource allocation for an intelligent reflecting surface (IRS)-assisted uplink system, where the base station is equipped with multiple antennas. We propose to jointly optimize the transmit power of the users, active beamforming at the base station, and passive beamforming at the IRS to maximize the overall system energy efficiency while maintaining users' minimum rate constraints. This problem belongs to a highly intractable non-convex bi-quadratic programming problem, for which an iterative solution based on block coordinate descent is proposed. Extensive simulations are conducted to demonstrate the effectiveness of the proposed scheme and the benefits of having more elements at the IRS.
On Randomized Searching for Multi-robot Coordination
Venue: arXiv cs.RO 2020
Year: 2020

In this chapter, we propose a novel approach for solving the coordination of a fleet of mobile robots, which consists of finding a set of collision-free trajectories for individual robots in the fleet. This problem is studied for several decades, and many approaches have been introduced. However, only a small minority is applicable in practice because of their properties - small computational requirement, producing solutions near-optimum, and completeness. The approach we present is based on a multi-robot variant of Rapidly Exploring Random Tree algorithm (RRT) for discrete environments and significantly improves its performance. Although the solutions generated by the approach are slightly worse than one of the best state-of-the-art algorithms presented in [23], it solves problems where ter Morses algorithm fails.
Modelling, Simulation, and Planning for the MoleMOD System
Venue: arXiv cs.RO 2020
Year: 2020

MoleMOD is a heterogeneous self-reconfigurable modular robotic system to be employed in architecture and civil engineering. In this paper we present two components of the MoleMOD infrastructure - a test environment and a planning algorithm. The test environment for simulation and visualization of active parts as well as passive blocks of MoleMOD is based on Gazebo - a powerful general-purpose robotic simulator. The key effort has been put into preparation of realistic models of passive and active components taking into account their physical characteristics. Moreover, given a starting configuration of the MoleMOD system and a final configuration an approach to plan collision-free trajectories for a fleet of active parts is introduced.
Knowledge Graph Extraction from Videos
Venue: arXiv cs.CL 2020
Year: 2020

Nearly all existing techniques for automated video annotation (or captioning) describe videos using natural language sentences. However, this has several shortcomings: (i) it is very hard to then further use the generated natural language annotations in automated data processing, (ii) generating natural language annotations requires to solve the hard subtask of generating semantically precise and syntactically correct natural language sentences, which is actually unrelated to the task of video annotation, (iii) it is difficult to quantitatively measure performance, as standard metrics (e.g., accuracy and F1-score) are inapplicable, and (iv) annotations are language-specific. In this paper, we propose the new task of knowledge graph extraction from videos, i.e., producing a description in the form of a knowledge graph of the contents of a given video. Since no datasets exist for this task, we also include a method to automatically generate them, starting from datasets where videos are annotated with natural language. We then describe an initial deep-learning model for knowledge graph extraction from videos, and report results on MSVD* and MSR-VTT*, two datasets obtained from MSVD and MSR-VTT using our method.
Nonobtuse triangulations of PSLGs
Venue: arXiv cs.CG 2020
Year: 2020

We show that any planar straight line graph (PSLG) with $n$ vertices has a conforming triangulation by $O(n^{2.5})$ nonobtuse triangles (all angles $\leq 90^\circ$), answering the question of whether any polynomial bound exists. A nonobtuse triangulation is Delaunay, so this result also improves a previous $O(n^3)$ bound of Eldesbrunner and Tan for conforming Delaunay triangulations of PSLGs. In the special case that the PSLG is the triangulation of a simple polygon, we will show that only $O(n^2)$ triangles are needed, improving an $O(n^4)$ bound of Bern and Eppstein. We also show that for any $ε>0$, every PSLG has a conforming triangulation with $O(n^2/ε^2)$ elements and with all angles bounded above by $90^\circ + ε$. This improves a result of S. Mitchell when $ε= 3 π/8 = 67.5^\circ $ and Tan when $ε= 7π/30 =42^\circ$.
Cephalometric Landmark Regression with Convolutional Neural Networks on 3D Computed Tomography Data
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we address the problem of automatic three-dimensional cephalometric analysis. Cephalometric analysis performed on lateral radiographs doesn't fully exploit the structure of 3D objects due to projection onto the lateral plane. With the development of three-dimensional imaging techniques such as CT, several analysis methods have been proposed that extend to the 3D case. The analysis based on these methods is invariant to rotations and translations and can describe difficult skull deformation, where 2D cephalometry has no use. In this paper, we provide a wide overview of existing approaches for cephalometric landmark regression. Moreover, we perform a series of experiments with state of the art 3D convolutional neural network (CNN) based methods for keypoint regression: direct regression with CNN, heatmap regression and Softargmax regression. For the first time, we extensively evaluate the described methods and demonstrate their effectiveness in the estimation of Frankfort Horizontal and cephalometric points locations for patients with severe skull deformations. We demonstrate that Heatmap and Softargmax regression models provide sufficient regression error for medical applications (less than 4 mm). Moreover, the Softargmax model achieves 1.15o inclination error for the Frankfort horizontal. For the fair comparison with the prior art, we also report results projected on the lateral plane.
Parallel Performance of ARM ThunderX2 for Atomistic Simulation Algorithms
Venue: arXiv cs.DC 2020
Year: 2020

Atomistic simulation drives scientific advances in modern material science and accounts for a significant proportion of wall time on High Performance Computing facilities. It is important that algorithms are efficient and implementations are performant in a continuously diversifying hardware landscape. Furthermore, they have to be portable to make best use of the available computing resource. In this paper we assess the parallel performance of some key algorithms implemented in a performance portable framework developed by us. We consider Molecular Dynamics with short range interactions, the Fast Multipole Method and Kinetic Monte Carlo. To assess the performance of emerging architectures, we compare the Marvell ThunderX2 (ARM) architecture to traditional x86_64 hardware made available through the Azure cloud computing service.
The Geometry of Information Cocoon: Analyzing the Cultural Space with Word Embedding Models
Venue: arXiv cs.CY 2020
Year: 2020

Accompanied by the development of digital media, the threat of information cocoon has become a significant issue. However, little is known about the measure of information cocoon as a cultural space and its relationship with social class. This study addresses this problem by constructing the cultural space with word embedding models and random shuffling methods among three large-scale digital media use datasets. In the light of field theory of cultural production, we investigate the information cocoon effect on different social classes among 979 computer users, 100,000 smartphone users, and 159,373 mobile reading application users. Our analysis reveals that information cocoons widely exist in the daily use of digital media. Moreover, people of lower social class have a higher probability of getting stuck in the information cocoon filled with the entertainment content. In contrast, the people of higher social class have more capability to stride over the constraints of the information cocoon. The results suggest that the disadvantages for vulnerable groups in acquiring knowledge may further widen social inequality.
Shopping in the Multiverse: A Counterfactual Approach to In-Session Attribution
Venue: arXiv cs.AI 2020
Year: 2020

We tackle the challenge of in-session attribution for on-site search engines in eCommerce. We phrase the problem as a causal counterfactual inference, and contrast the approach with rule-based systems from industry settings and prediction models from the multi-touch attribution literature. We approach counterfactuals in analogy with treatments in formal semantics, explicitly modeling possible outcomes through alternative shopper timelines; in particular, we propose to learn a generative browsing model over a target shop, leveraging the latent space induced by prod2vec embeddings; we show how natural language queries can be effectively represented in the same space and how "search intervention" can be performed to assess causal contribution. Finally, we validate the methodology on a synthetic dataset, mimicking important patterns emerged in customer interviews and qualitative analysis, and we present preliminary findings on an industry dataset from a partnering shop.
Time Series Source Separation with Slow Flows
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we show that slow feature analysis (SFA), a common time series decomposition method, naturally fits into the flow-based models (FBM) framework, a type of invertible neural latent variable models. Building upon recent advances on blind source separation, we show that such a fit makes the time series decomposition identifiable.
Can we cover navigational perception needs of the visually impaired by panoptic segmentation?
Venue: arXiv cs.CV 2020
Year: 2020

Navigational perception for visually impaired people has been substantially promoted by both classic and deep learning based segmentation methods. In classic visual recognition methods, the segmentation models are mostly object-dependent, which means a specific algorithm has to be devised for the object of interest. In contrast, deep learning based models such as instance segmentation and semantic segmentation allow to individually recognize part of the entire scene, namely things or stuff, for blind individuals. However, both of them can not provide a holistic understanding of the surroundings for the visually impaired. Panoptic segmentation is a newly proposed visual model with the aim of unifying semantic segmentation and instance segmentation. Motivated by that, we propose to utilize panoptic segmentation as an approach to navigating visually impaired people by offering both things and stuff awareness in the proximity of the visually impaired. We demonstrate that panoptic segmentation is able to equip the visually impaired with a holistic real-world scene perception through a wearable assistive system.
Graph Convolutional Network-based Suspicious Communication Pair Estimation for Industrial Control Systems
Venue: arXiv cs.CR 2020
Year: 2020

Whitelisting is considered an effective security monitoring method for networks used in industrial control systems, where the whitelists consist of observed tuples of the IP address of the server, the TCP/UDP port number, and IP address of the client (communication triplets). However, this method causes frequent false detections. To reduce false positives due to a simple whitelist-based judgment, we propose a new framework for scoring communications to judge whether the communications not present in whitelists are normal or anomalous. To solve this problem, we developed a graph convolutional network-based suspicious communication pair estimation using relational graph convolution networks, and evaluated its performance. For this, we collected the network traffic of three factories owned by Panasonic Corporation, Japan. The proposed method achieved a receiver operating characteristic area under the curve of 0.957, which outperforms baseline approaches such as DistMult, a method that directly optimizes the node embeddings, and heuristics, which score the triplets using first- and second-order proximities of multigraphs. This method enables security operators to concentrate on significant alerts.
Integrating Network Embedding and Community Outlier Detection via Multiclass Graph Description
Venue: arXiv cs.SI 2020
Year: 2020

Network (or graph) embedding is the task to map the nodes of a graph to a lower dimensional vector space, such that it preserves the graph properties and facilitates the downstream network mining tasks. Real world networks often come with (community) outlier nodes, which behave differently from the regular nodes of the community. These outlier nodes can affect the embedding of the regular nodes, if not handled carefully. In this paper, we propose a novel unsupervised graph embedding approach (called DMGD) which integrates outlier and community detection with node embedding. We extend the idea of deep support vector data description to the framework of graph embedding when there are multiple communities present in the given network, and an outlier is characterized relative to its community. We also show the theoretical bounds on the number of outliers detected by DMGD. Our formulation boils down to an interesting minimax game between the outliers, community assignments and the node embedding function. We also propose an efficient algorithm to solve this optimization framework. Experimental results on both synthetic and real world networks show the merit of our approach compared to state-of-the-arts.
Unsupervised Domain Adaptation in the Absence of Source Data
Venue: arXiv cs.CV 2020
Year: 2020

Current unsupervised domain adaptation methods can address many types of distribution shift, but they assume data from the source domain is freely available. As the use of pre-trained models becomes more prevalent, it is reasonable to assume that source data is unavailable. We propose an unsupervised method for adapting a source classifier to a target domain that varies from the source domain along natural axes, such as brightness and contrast. Our method only requires access to unlabeled target instances and the source classifier. We validate our method in scenarios where the distribution shift involves brightness, contrast, and rotation and show that it outperforms fine-tuning baselines in scenarios with limited labeled data.
Privacy Implications of Eye Tracking in Mixed Reality
Venue: arXiv cs.HC 2020
Year: 2020

Mixed Reality (MR) devices require a world with always-on sensors and real-time processing applied to their outputs. We have grappled with some of the ethical concerns presented by this scenario, such as bystander privacy issues with smartphones and cameras. However, MR technologies demand that we define and defend privacy in this new paradigm. This paper focuses on the challenges presented by eye tracking and gaze tracking, techniques that have commonly been deployed in the HCI community for years but are now being integrated into MR devices by default.
Addressing the Privacy Implications of Mixed Reality: A Regulatory Approach
Venue: arXiv cs.CR 2020
Year: 2020

Mixed reality (MR) technologies are emerging into the mainstream with affordable devices like the Oculus Quest. These devices blend the physical and virtual in novel ways that blur the lines that exist in legal precedent, like those between speech and conduct. In this paper, we discuss the challenges of regulating immersive technologies, focusing on the potential for extensive data collection, and examine the trade-offs of three potential approaches to protecting data privacy in the context of mixed reality environments.