OpenRaiser/Intern-Atlas huggingface dataset:
ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments
Venue: arXiv cs.CL 2020
Year: 2020

For embodied agents, navigation is an important ability but not an isolated goal. Agents are also expected to perform specific tasks after reaching the target location, such as picking up objects and assembling them into a particular arrangement. We combine Vision-and-Language Navigation, assembling of collected objects, and object referring expression comprehension, to create a novel joint navigation-and-assembly task, named ArraMon. During this task, the agent (similar to a PokeMON GO player) is asked to find and collect different target objects one-by-one by navigating based on natural language instructions in a complex, realistic outdoor environment, but then also ARRAnge the collected objects part-by-part in an egocentric grid-layout environment. To support this task, we implement a 3D dynamic environment simulator and collect a dataset (in English; and also extended to Hindi) with human-written navigation and assembling instructions, and the corresponding ground truth trajectories. We also filter the collected instructions via a verification stage, leading to a total of 7.7K task instances (30.8K instructions and paths). We present results for several baseline models (integrated and biased) and metrics (nDTW, CTC, rPOD, and PTC), and the large model-human performance gap demonstrates that our task is challenging and presents a wide scope for future work. Our dataset, simulator, and code are publicly available at: https://arramonunc.github.io
Reinforced Medical Report Generation with X-Linear Attention and Repetition Penalty
Venue: arXiv cs.CL 2020
Year: 2020

To reduce doctors' workload, deep-learning-based automatic medical report generation has recently attracted more and more research efforts, where attention mechanisms and reinforcement learning are integrated with the classic encoder-decoder architecture to enhance the performance of deep models. However, these state-of-the-art solutions mainly suffer from two shortcomings: (i) their attention mechanisms cannot utilize high-order feature interactions, and (ii) due to the use of TF-IDF-based reward functions, these methods are fragile with generating repeated terms. Therefore, in this work, we propose a reinforced medical report generation solution with x-linear attention and repetition penalty mechanisms (ReMRG-XR) to overcome these problems. Specifically, x-linear attention modules are used to explore high-order feature interactions and achieve multi-modal reasoning, while repetition penalty is used to apply penalties to repeated terms during the model's training process. Extensive experimental studies have been conducted on two public datasets, and the results show that ReMRG-XR greatly outperforms the state-of-the-art baselines in terms of all metrics.
Measuring agreement on linguistic expressions in medical treatment scenarios
Venue: arXiv cs.AI 2020
Year: 2020

Quality of life assessment represents a key process of deciding treatment success and viability. As such, patients' perceptions of their functional status and well-being are important inputs for impairment assessment. Given that patient completed questionnaires are often used to assess patient status and determine future treatment options, it is important to know the level of agreement of the words used by patients and different groups of medical professionals. In this paper, we propose a measure called the Agreement Ratio which provides a ratio of overall agreement when modelling words through Fuzzy Sets (FSs). The measure has been specifically designed for assessing this agreement in fuzzy sets which are generated from data such as patient responses. The measure relies on using the Jaccard Similarity Measure for comparing the different levels of agreement in the FSs generated.
Influence of User Emotion on Information Propagation with Public Sentiment in the Chinese Sina-microblog
Venue: arXiv cs.SI 2020
Year: 2020

Social networks are flooded with different pieces of emotional information, the propagation of which helps to shape the development of public sentiment. To help designing effective communication strategies during the entire development of an event,we propose an emotion-based susceptible-forwarding-immune (E-SFI) propagation dynamic model, that takes into account of the categories of emotions into positive, neutral and negative and the emotional choices of user communities, to investigate the information propagation process that leads to public sentiment. Our Model-based analytic and numerical analyses show that three types of forwarding probabilities involved in our E-SFI model are in accordance with the actual accident situation, and our sensitivity analyses describe important factors that affect the emotional choices of user communities in support for decision strategies to guide the public sentiment.
Spatial Social Network (SSN) Hot Spot Detection: Scan Methods for Non-Planar Networks
Venue: arXiv cs.SI 2020
Year: 2020

Moving window and hot spot detection analyses are statistical methods used to analyze point patterns within a given area. Such methods have been used to successfully detect clusters of point events such as car thefts or incidences of cancer. Yet, these methods do not account for the connections between individual events, such as social ties within a neighborhood. This paper presents two GIS methods, EdgeScan and NDScan, for capturing areas with high and low levels of local social connections. Both methods are moving window processes that count the number of edges and network density, respectively, in a given focal area (window area). The focal window attaches resultant EdgeScan and NDScan statistics to nodes at the center of the focal window area. We implement these methods on a case study of 1960s connections between members of the Mafia in New York City. We use various definitions of a focal neighborhood including Euclidean, Manhattan and K Nearest Neighbor (KNN) definitions. We find that KNN tends to overstate the values of local networks, and that there is more variation in outcome values for nodes on the periphery of the study area. We find that, location-wise, EdgeScan and NDScan hot spots differ from traditional spatial hot spots in the study area. These methods can be extended to future studies that detect local triads and motifs, which can capture the local network structure in more detail.
DARE: AI-based Diver Action Recognition System using Multi-Channel CNNs for AUV Supervision
Venue: arXiv cs.CV 2020
Year: 2020

With the growth of sensing, control and robotic technologies, autonomous underwater vehicles (AUVs) have become useful assistants to human divers for performing various underwater operations. In the current practice, the divers are required to carry expensive, bulky, and waterproof keyboards or joystick-based controllers for supervision and control of AUVs. Therefore, diver action-based supervision is becoming increasingly popular because it is convenient, easier to use, faster, and cost effective. However, the various environmental, diver and sensing uncertainties present underwater makes it challenging to train a robust and reliable diver action recognition system. In this regard, this paper presents DARE, a diver action recognition system, that is trained based on Cognitive Autonomous Driving Buddy (CADDY) dataset, which is a rich set of data containing images of different diver gestures and poses in several different and realistic underwater environments. DARE is based on fusion of stereo-pairs of camera images using a multi-channel convolutional neural network supported with a systematically trained tree-topological deep neural network classifier to enhance the classification performance. DARE is fast and requires only a few milliseconds to classify one stereo-pair, thus making it suitable for real-time underwater implementation. DARE is comparatively evaluated against several existing classifier architectures and the results show that DARE supersedes the performance of all classifiers for diver action recognition in terms of overall as well as individual class accuracies and F1-scores.
Blind Decision Making: Reinforcement Learning with Delayed Observations
Venue: arXiv cs.LG 2020
Year: 2020

Reinforcement learning typically assumes that the state update from the previous actions happens instantaneously, and thus can be used for making future decisions. However, this may not always be true. When the state update is not available, the decision taken is partly in the blind since it cannot rely on the current state information. This paper proposes an approach, where the delay in the knowledge of the state can be used, and the decisions are made based on the available information which may not include the current state information. One approach could be to include the actions after the last-known state as a part of the state information, however, that leads to an increased state-space making the problem complex and slower in convergence. The proposed algorithm gives an alternate approach where the state space is not enlarged, as compared to the case when there is no delay in the state update. Evaluations on the basic RL environments further illustrate the improved performance of the proposed algorithm.
Gram Regularization for Multi-view 3D Shape Retrieval
Venue: arXiv cs.CV 2020
Year: 2020

How to obtain the desirable representation of a 3D shape is a key challenge in 3D shape retrieval task. Most existing 3D shape retrieval methods focus on capturing shape representation with different neural network architectures, while the learning ability of each layer in the network is neglected. A common and tough issue that limits the capacity of the network is overfitting. To tackle this, L2 regularization is applied widely in existing deep learning frameworks. However,the effect on the generalization ability with L2 regularization is limited as it only controls large value in parameters. To make up the gap, in this paper, we propose a novel regularization term called Gram regularization which reinforces the learning ability of the network by encouraging the weight kernels to extract different information on the corresponding feature map. By forcing the variance between weight kernels to be large, the regularizer can help to extract discriminative features. The proposed Gram regularization is data independent and can converge stably and quickly without bells and whistles. Moreover, it can be easily plugged into existing off-the-shelf architectures. Extensive experimental results on the popular 3D object retrieval benchmark ModelNet demonstrate the effectiveness of our method.
Unified Composite Distribution and Its Applications to Double Shadowed $α-κ-μ$ Fading Channels
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we propose a mixture Gamma shadowed (MGS) distribution as a unified composite model via representing the shadowing by an inverse Nakagami-$m$. Accordingly, the exact expression and the asymptotic behaviour at high average signal-to-noise ratio (SNR) regime of the fundamental statistics of a MGS distribution are derived first. These statistics are then applied to analyze the performance of the wireless communications systems over double shadowed $α-κ-μ$ fading via providing the outage probability (OP), the average bit error probability (ABEP), the average channel capacity (ACC), and the effective capacity (EC). The average area under the receiver operating characteristics curve (AUC) of energy detection (ED) is also analyzed. The numerical and simulation results are presented to verify the validation of our analysis.
An End-to-end Method for Producing Scanning-robust Stylized QR Codes
Venue: arXiv cs.CV 2020
Year: 2020

Quick Response (QR) code is one of the most worldwide used two-dimensional codes.~Traditional QR codes appear as random collections of black-and-white modules that lack visual semantics and aesthetic elements, which inspires the recent works to beautify the appearances of QR codes. However, these works adopt fixed generation algorithms and therefore can only generate QR codes with a pre-defined style. In this paper, combining the Neural Style Transfer technique, we propose a novel end-to-end method, named ArtCoder, to generate the stylized QR codes that are personalized, diverse, attractive, and scanning-robust.~To guarantee that the generated stylized QR codes are still scanning-robust, we propose a Sampling-Simulation layer, a module-based code loss, and a competition mechanism. The experimental results show that our stylized QR codes have high-quality in both the visual effect and the scanning-robustness, and they are able to support the real-world application.
A Survey on the Explainability of Supervised Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.
Conversational agents for learning foreign languages -- a survey
Venue: arXiv cs.HC 2020
Year: 2020

Conversational practice, while crucial for all language learners, can be challenging to get enough of and very expensive. Chatbots are computer programs developed to engage in conversations with humans. They are designed as software avatars with limited, but growing conversational capability. The most natural and potentially powerful application of chatbots is in line with their fundamental nature - language practice. However, their role and outcomes within (in)formal language learning are currently tangential at best. Existing research in the area has generally focused on chatbots' comprehensibility and the motivation they inspire in their users. In this paper, we provide an overview of the chatbots for learning languages, critically analyze existing approaches, and discuss the major challenges for future work.
Text Information Aggregation with Centrality Attention
Venue: arXiv cs.CL 2020
Year: 2020

A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves aggregation process of combining the representations of all the words, such as pooling or self-attention. However, these widely used aggregation approaches did not take higher-order relationship among the words into consideration. Hence we propose a new way of obtaining aggregation weights, called eigen-centrality self-attention. More specifically, we build a fully-connected graph for all the words in a sentence, then compute the eigen-centrality as the attention score of each word. The explicit modeling of relationships as a graph is able to capture some higher-order dependency among words, which helps us achieve better results in 5 text classification tasks and one SNLI task than baseline models such as pooling, self-attention and dynamic routing. Besides, in order to compute the dominant eigenvector of the graph, we adopt power method algorithm to get the eigen-centrality measure. Moreover, we also derive an iterative approach to get the gradient for the power method process to reduce both memory consumption and computation requirement.}
A grammar compressor for collections of reads with applications to the construction of the BWT
Venue: arXiv cs.DS 2020
Year: 2020

We describe a grammar for DNA sequencing reads from which we can compute the BWT directly. Our motivation is to perform in succinct space genomic analyses that require complex string queries not yet supported by repetition-based self-indexes. Our approach is to store the set of reads as a grammar, but when required, compute its BWT to carry out the analysis by using self-indexes. Our experiments in real data showed that the space reduction we achieve with our compressor is competitive with LZ-based methods and better than entropy-based approaches. Compared to other popular grammars, in this kind of data, we achieve, on average, 12\% of extra compression and require less working space and time.
High-level Prior-based Loss Functions for Medical Image Segmentation: A Survey
Venue: arXiv cs.CV 2020
Year: 2020

Today, deep convolutional neural networks (CNNs) have demonstrated state of the art performance for supervised medical image segmentation, across various imaging modalities and tasks. Despite early success, segmentation networks may still generate anatomically aberrant segmentations, with holes or inaccuracies near the object boundaries. To mitigate this effect, recent research works have focused on incorporating spatial information or prior knowledge to enforce anatomically plausible segmentation. If the integration of prior knowledge in image segmentation is not a new topic in classical optimization approaches, it is today an increasing trend in CNN based image segmentation, as shown by the growing literature on the topic. In this survey, we focus on high level prior, embedded at the loss function level. We categorize the articles according to the nature of the prior: the object shape, size, topology, and the inter-regions constraints. We highlight strengths and limitations of current approaches, discuss the challenge related to the design and the integration of prior-based losses, and the optimization strategies, and draw future research directions.
Research Needed in Computational Social Science for Power System Reliability, Resilience, and Restoration
Venue: arXiv cs.MA 2020
Year: 2020

In the literature, smart grids are modeled as cyber-physical power systems without considering the computational social aspects. However, end-users are playing a key role in their operation and response to disturbances via demand response and distributed energy resources. Therefore, due to the critical role of active and passive end-users and the intermittency of renewable energy, smart grids must be planned and operated by considering the computational social aspects in addition to the technical aspects. The level of cooperation, flexibility, and other social features of the various stakeholders, including consumers, prosumers, and microgrids, affect the system efficiency, reliability, and resilience. In this paper, we design an artificial society simulating the interaction between power systems and the social communities that they serve via agent-based modeling inspired by Barsade's theory on the emotional spread. The simulation results show a decline in the consumers' and prosumers' satisfaction levels induced by a shortage of electricity. It also shows the effects of social diffusion via the Internet and mass media on the satisfaction level. In view of the importance of computational social science for power system applications and the limited number of publications devoted to it, we provide a list of research topics that need to be achieved to enhance the reliability and resilience of power systems' operation and planning.
Stylized Neural Painting
Venue: arXiv cs.CV 2020
Year: 2020

This paper proposes an image-to-painting translation method that generates vivid and realistic painting artworks with controllable styles. Different from previous image-to-image translation methods that formulate the translation as pixel-wise prediction, we deal with such an artistic creation process in a vectorized environment and produce a sequence of physically meaningful stroke parameters that can be further used for rendering. Since a typical vector render is not differentiable, we design a novel neural renderer which imitates the behavior of the vector renderer and then frame the stroke prediction as a parameter searching process that maximizes the similarity between the input and the rendering output. We explored the zero-gradient problem on parameter searching and propose to solve this problem from an optimal transportation perspective. We also show that previous neural renderers have a parameter coupling problem and we re-design the rendering network with a rasterization network and a shading network that better handles the disentanglement of shape and color. Experiments show that the paintings generated by our method have a high degree of fidelity in both global appearance and local textures. Our method can be also jointly optimized with neural style transfer that further transfers visual style from other images. Our code and animated results are available at \url{https://jiupinjia.github.io/neuralpainter/}.
Grant-Free Opportunistic Uplink Transmission in Wireless-powered IoT: A Spatio-temporal Model
Venue: arXiv cs.IT 2020
Year: 2020

Ambient radio frequency (RF) energy harvesting is widely promoted as an enabler for wireless-power Internet of Things (IoT) networks. This paper jointly characterizes energy harvesting and packet transmissions in grant-free opportunistic uplink IoT networks energized via harvesting downlink energy. To do that, a joint queuing theory and stochastic geometry model is utilized to develop a spatio-temporal analytical model. Particularly, the harvested energy and packet transmission success probability are characterized using tools from stochastic geometry. {Moreover, each device is modeled using a two-dimensional discrete-time Markov chain (DTMC). Such two dimensions are utilized to jointly track the scavenged/depleted energy to/from the batteries along with the arrival/departure of packets to/from devices buffers over time. Consequently, the adopted queuing model represents the devices as spatially interacting queues. To that end, the network performance is assessed in light of the packet throughput, the average delay, and the average buffer size. The effect of base stations (BSs) densification is discussed and several design insights are provided. The results show that the parameters for uplink power control and opportunistic channel access should be jointly optimized to maximize average network packet throughput, and hence, minimize delay.
Hierarchical clustering in particle physics through reinforcement learning
Venue: arXiv cs.AI 2020
Year: 2020

Particle physics experiments often require the reconstruction of decay patterns through a hierarchical clustering of the observed final-state particles. We show that this task can be phrased as a Markov Decision Process and adapt reinforcement learning algorithms to solve it. In particular, we show that Monte-Carlo Tree Search guided by a neural policy can construct high-quality hierarchical clusterings and outperform established greedy and beam search baselines.
Machine Learning and Soil Humidity Sensing: Signal Strength Approach
Venue: arXiv cs.LG 2020
Year: 2020

The IoT vision of ubiquitous and pervasive computing gives rise to future smart irrigation systems comprising physical and digital world. Smart irrigation ecosystem combined with Machine Learning can provide solutions that successfully solve the soil humidity sensing task in order to ensure optimal water usage. Existing solutions are based on data received from the power hungry/expensive sensors that are transmitting the sensed data over the wireless channel. Over time, the systems become difficult to maintain, especially in remote areas due to the battery replacement issues with large number of devices. Therefore, a novel solution must provide an alternative, cost and energy effective device that has unique advantage over the existing solutions. This work explores a concept of a novel, low-power, LoRa-based, cost-effective system which achieves humidity sensing using Deep learning techniques that can be employed to sense soil humidity with the high accuracy simply by measuring signal strength of the given underground beacon device.