OpenRaiser/Intern-Atlas huggingface dataset:
On The Plurality of Graphs
Venue: arXiv cs.AI 2020
Year: 2020

We conduct a series of experiments designed to empirically demonstrate the effects of varying the structural features of a multi-agent emergent communication game framework. Specifically, we model the interactions (edges) between individual agents (nodes)as the structure of a graph generated according to a series of known random graph generating algorithms. Confirming the hypothesis proposed in [10], we show that the two factors of variation induced in this work, namely 1) the graph-generating process and 2) the centrality measure according to which edges are sampled, in fact play a significant role in determining the dynamics of language emergence within the population at hand.
Interactive Text Graph Mining with a Prolog-based Dialog Engine
Venue: arXiv cs.CL 2020
Year: 2020

On top of a neural network-based dependency parser and a graph-based natural language processing module we design a Prolog-based dialog engine that explores interactively a ranked fact database extracted from a text document. We reorganize dependency graphs to focus on the most relevant content elements of a sentence and integrate sentence identifiers as graph nodes. Additionally, after ranking the graph we take advantage of the implicit semantic information that dependency links and WordNet bring in the form of subject-verb-object, is-a and part-of relations. Working on the Prolog facts and their inferred consequences, the dialog engine specializes the text graph with respect to a query and reveals interactively the document's most relevant content elements. The open-source code of the integrated system is available at https://github.com/ptarau/DeepRank . Under consideration in Theory and Practice of Logic Programming (TPLP).
SSGMT: A Secure Smart Grid Monitoring Technique
Venue: arXiv cs.NI 2020
Year: 2020

Critical infrastructure systems like power grid require an improved critical in-formation infrastructure (CII) that can not only help in monitoring of the crit-ical entities but also take part in failure analysis and self-healing. Efficient designing of a CII is challenging as each kind of communication technology has its own advantages and disadvantages. Wired networks are highly scala-ble and secure, but they are neither cost effective nor dynamic in nature. Wireless communication technologies on the other hand are easy to deploy, low cost etc. but they are vulnerable to cyber-attacks. In order to optimize cost, power consumption, dynamic nature, accuracy and scalability a hybrid communication network is designed in this paper where a portion of the communication network is built using wireless sensor networks (WSN) and the rest is a wired network of fiber optic channels. To offer seamless opera-tion of the hybrid communication network and provide security a Secure Smart Grid Monitoring Technique (SSGMT) is also proposed. The perfor-mance of the proposed hybrid CII for the generation and transmission sys-tem of power grid coupled with the SSGMT during different cyber-attacks is tested using NS2 simulator. The simulation results show that the SSGMT for a joint power communication network of IEEE 118-Bus system performs better than the prevailing wireless CIIs like Lo-ADI and Modified AODV.
Anakatabatic Inertia: Particle-wise Adaptive Inertia for PSO
Venue: arXiv cs.NE 2020
Year: 2020

Throughout the course of the development of Particle Swarm Optimization, particle inertia has been established as an important aspect of the method for researching possible method improvements. As a continuation of our previous research, we propose a novel generalized technique of inertia weight adaptation based on individual particle's fitness improvement, called anakatabatic inertia. This technique allows for adapting inertia weight value for each particle corresponding to the particle's increasing or decreasing fitness, i.e. conditioned by particle's ascending (anabatic) or descending (katabatic) movement. The proposed inertia weight control framework was metaoptimized and tested on the 30 test functions of the CEC 2014 test suite. The conducted procedure produced four anakatabatic models, two for each of the PSO methods used (Standard PSO and TVAC-PSO). The benchmark testing results show that using the proposed anakatabatic inertia models reliably yield moderate improvements in accuracy of Standard PSO (final fitness minimum reduced up to 0.09 orders of magnitude) and rather strong improvements for TVAC-PSO (final fitness minimum reduced up to 0.59 orders of magnitude), mostly without any adverse effects on the method's performance.
The Splay-List: A Distribution-Adaptive Concurrent Skip-List
Venue: arXiv cs.DC 2020
Year: 2020

The design and implementation of efficient concurrent data structures have seen significant attention. However, most of this work has focused on concurrent data structures providing good \emph{worst-case} guarantees. In real workloads, objects are often accessed at different rates, since access distributions may be non-uniform. Efficient distribution-adaptive data structures are known in the sequential case, e.g. the splay-trees; however, they often are hard to translate efficiently in the concurrent case. In this paper, we investigate distribution-adaptive concurrent data structures and propose a new design called the splay-list. At a high level, the splay-list is similar to a standard skip-list, with the key distinction that the height of each element adapts dynamically to its access rate: popular elements ``move up,'' whereas rarely-accessed elements decrease in height. We show that the splay-list provides order-optimal amortized complexity bounds for a subset of operations while being amenable to efficient concurrent implementation. Experimental results show that the splay-list can leverage distribution-adaptivity to improve on the performance of classic concurrent designs, and can outperform the only previously-known distribution-adaptive design in certain settings.
RareAct: A video dataset of unusual interactions
Venue: arXiv cs.CV 2020
Year: 2020

This paper introduces a manually annotated video dataset of unusual actions, namely RareAct, including actions such as "blend phone", "cut keyboard" and "microwave shoes". RareAct aims at evaluating the zero-shot and few-shot compositionality of action recognition models for unlikely compositions of common action verbs and object nouns. It contains 122 different actions which were obtained by combining verbs and nouns rarely co-occurring together in the large-scale textual corpus from HowTo100M, but that frequently appear separately. We provide benchmarks using a state-of-the-art HowTo100M pretrained video and text model and show that zero-shot and few-shot compositionality of actions remains a challenging and unsolved task.
From Design Draft to Real Attire: Unaligned Fashion Image Translation
Venue: arXiv cs.CV 2020
Year: 2020

Fashion manipulation has attracted growing interest due to its great application value, which inspires many researches towards fashion images. However, little attention has been paid to fashion design draft. In this paper, we study a new unaligned translation problem between design drafts and real fashion items, whose main challenge lies in the huge misalignment between the two modalities. We first collect paired design drafts and real fashion item images without pixel-wise alignment. To solve the misalignment problem, our main idea is to train a sampling network to adaptively adjust the input to an intermediate state with structure alignment to the output. Moreover, built upon the sampling network, we present design draft to real fashion item translation network (D2RNet), where two separate translation streams that focus on texture and shape, respectively, are combined tactfully to get both benefits. D2RNet is able to generate realistic garments with both texture and shape consistency to their design drafts. We show that this idea can be effectively applied to the reverse translation problem and present R2DNet accordingly. Extensive experiments on unaligned fashion design translation demonstrate the superiority of our method over state-of-the-art methods. Our project website is available at: https://victoriahy.github.io/MM2020/ .
Enhancing autonomy transparency: an option-centric rationale approach
Venue: arXiv cs.HC 2020
Year: 2020

While the advances in artificial intelligence and machine learning empower a new generation of autonomous systems for assisting human performance, one major concern arises from the human factors perspective: Humans have difficulty deciphering autonomy-generated solutions and increasingly perceive autonomy as a mysterious black box. The lack of transparency contributes to the lack of trust in autonomy and sub-optimal team performance. To enhance autonomy transparency, this study proposed an option-centric rationale display and evaluated its effectiveness. We developed a game Treasure Hunter wherein a human uncovers a map for treasures with the help from an intelligent assistant, and conducted a human-in-the-loop experiment with 34 participants. Results indicated that by conveying the intelligent assistant's decision-making rationale via the option-centric rationale display, participants had higher trust in the system and calibrated their trust faster. Additionally, higher trust led to higher acceptance of recommendations from the intelligent assistant, and in turn higher task performance.
AiRound and CV-BrCT: Novel Multi-View Datasets for Scene Classification
Venue: arXiv cs.CV 2020
Year: 2020

It is undeniable that aerial/satellite images can provide useful information for a large variety of tasks. But, since these images are always looking from above, some applications can benefit from complementary information provided by other perspective views of the scene, such as ground-level images. Despite a large number of public repositories for both georeferenced photographs and aerial images, there is a lack of benchmark datasets that allow the development of approaches that exploit the benefits and complementarity of aerial/ground imagery. In this paper, we present two new publicly available datasets named \thedataset~and CV-BrCT. The first one contains triplets of images from the same geographic coordinate with different perspectives of view extracted from various places around the world. Each triplet is composed of an aerial RGB image, a ground-level perspective image, and a Sentinel-2 sample. The second dataset contains pairs of aerial and street-level images extracted from southeast Brazil. We design an extensive set of experiments concerning multi-view scene classification, using early and late fusion. Such experiments were conducted to show that image classification can be enhanced using multi-view data.
Characterizing Communities of Hashtag Usage on Twitter During the 2020 COVID-19 Pandemic by Multi-view Clustering
Venue: arXiv cs.SI 2020
Year: 2020

The COVID-19 pandemic has produced a flurry of online activity on social media sites. As such, analysis of social media data during the COVID-19 pandemic can produce unique insights into discussion topics and how those topics evolve over the course of the pandemic. In this study, we propose analyzing discussion topics on Twitter by clustering hashtags. In order to obtain high-quality clusters of the Twitter hashtags, we also propose a novel multi-view clustering technique that incorporates multiple different data types that can be used to describe how users interact with hashtags. The results of our multi-view clustering show that there are distinct temporal and topical trends present within COVID-19 twitter discussion. In particular, we find that some topical clusters of hashtags shift over the course of the pandemic, while others are persistent throughout, and that there are distinct temporal trends in hashtag usage. This study is the first to use multi-view clustering to analyze hashtags and the first analysis of the greater trends of discussion occurring online during the COVID-19 pandemic.
Recognition and 3D Localization of Pedestrian Actions from Monocular Video
Venue: arXiv cs.CV 2020
Year: 2020

Understanding and predicting pedestrian behavior is an important and challenging area of research for realizing safe and effective navigation strategies in automated and advanced driver assistance technologies in urban scenes. This paper focuses on monocular pedestrian action recognition and 3D localization from an egocentric view for the purpose of predicting intention and forecasting future trajectory. A challenge in addressing this problem in urban traffic scenes is attributed to the unpredictable behavior of pedestrians, whereby actions and intentions are constantly in flux and depend on the pedestrians pose, their 3D spatial relations, and their interaction with other agents as well as with the environment. To partially address these challenges, we consider the importance of pose toward recognition and 3D localization of pedestrian actions. In particular, we propose an action recognition framework using a two-stream temporal relation network with inputs corresponding to the raw RGB image sequence of the tracked pedestrian as well as the pedestrian pose. The proposed method outperforms methods using a single-stream temporal relation network based on evaluations using the JAAD public dataset. The estimated pose and associated body key-points are also used as input to a network that estimates the 3D location of the pedestrian using a unique loss function. The evaluation of our 3D localization method on the KITTI dataset indicates the improvement of the average localization error as compared to existing state-of-the-art methods. Finally, we conduct qualitative tests of action recognition and 3D localization on HRI's H3D driving dataset.
Retrieve-Then-Adapt: Example-based Automatic Generation for Proportion-related Infographics
Venue: arXiv cs.HC 2020
Year: 2020

Infographic is a data visualization technique which combines graphic and textual descriptions in an aesthetic and effective manner. Creating infographics is a difficult and time-consuming process which often requires significant attempts and adjustments even for experienced designers, not to mention novice users with limited design expertise. Recently, a few approaches have been proposed to automate the creation process by applying predefined blueprints to user information. However, predefined blueprints are often hard to create, hence limited in volume and diversity. In contrast, good infogrpahics have been created by professionals and accumulated on the Internet rapidly. These online examples often represent a wide variety of design styles, and serve as exemplars or inspiration to people who like to create their own infographics. Based on these observations, we propose to generate infographics by automatically imitating examples. We present a two-stage approach, namely retrieve-then-adapt. In the retrieval stage, we index online examples by their visual elements. For a given user information, we transform it to a concrete query by sampling from a learned distribution about visual elements, and then find appropriate examples in our example library based on the similarity between example indexes and the query. For a retrieved example, we generate an initial drafts by replacing its content with user information. However, in many cases, user information cannot be perfectly fitted to retrieved examples. Therefore, we further introduce an adaption stage. Specifically, we propose a MCMC-like approach and leverage recursive neural networks to help adjust the initial draft and improve its visual appearance iteratively, until a satisfactory result is obtained. We implement our approach on proportion-related infographics, and demonstrate its effectiveness by sample results and expert reviews.
Making Coherence Out of Nothing At All: Measuring the Evolution of Gradient Alignment
Venue: arXiv cs.LG 2020
Year: 2020

We propose a new metric ($m$-coherence) to experimentally study the alignment of per-example gradients during training. Intuitively, given a sample of size $m$, $m$-coherence is the number of examples in the sample that benefit from a small step along the gradient of any one example on average. We show that compared to other commonly used metrics, $m$-coherence is more interpretable, cheaper to compute ($O(m)$ instead of $O(m^2)$) and mathematically cleaner. (We note that $m$-coherence is closely connected to gradient diversity, a quantity previously used in some theoretical bounds.) Using $m$-coherence, we study the evolution of alignment of per-example gradients in ResNet and Inception models on ImageNet and several variants with label noise, particularly from the perspective of the recently proposed Coherent Gradients (CG) theory that provides a simple, unified explanation for memorization and generalization [Chatterjee, ICLR 20]. Although we have several interesting takeaways, our most surprising result concerns memorization. Naively, one might expect that when training with completely random labels, each example is fitted independently, and so $m$-coherence should be close to 1. However, this is not the case: $m$-coherence reaches much higher values during training (100s), indicating that over-parameterized neural networks find common patterns even in scenarios where generalization is not possible. A detailed analysis of this phenomenon provides both a deeper confirmation of CG, but at the same point puts into sharp relief what is missing from the theory in order to provide a complete explanation of generalization in neural networks.
Multi-Class 3D Object Detection Within Volumetric 3D Computed Tomography Baggage Security Screening Imagery
Venue: arXiv cs.CV 2020
Year: 2020

Automatic detection of prohibited objects within passenger baggage is important for aviation security. X-ray Computed Tomography (CT) based 3D imaging is widely used in airports for aviation security screening whilst prior work on automatic prohibited item detection focus primarily on 2D X-ray imagery. These works have proven the possibility of extending deep convolutional neural networks (CNN) based automatic prohibited item detection from 2D X-ray imagery to volumetric 3D CT baggage security screening imagery. However, previous work on 3D object detection in baggage security screening imagery focused on the detection of one specific type of objects (e.g., either {\it bottles} or {\it handguns}). As a result, multiple models are needed if more than one type of prohibited item is required to be detected in practice. In this paper, we consider the detection of multiple object categories of interest using one unified framework. To this end, we formulate a more challenging multi-class 3D object detection problem within 3D CT imagery and propose a viable solution (3D RetinaNet) to tackle this problem. To enhance the performance of detection we investigate a variety of strategies including data augmentation and varying backbone networks. Experimentation carried out to provide both quantitative and qualitative evaluations of the proposed approach to multi-class 3D object detection within 3D CT baggage security screening imagery. Experimental results demonstrate the combination of the 3D RetinaNet and a series of favorable strategies can achieve a mean Average Precision (mAP) of 65.3\% over five object classes (i.e. {\it bottles, handguns, binoculars, glock frames, iPods}). The overall performance is affected by the poor performance on {\it glock frames} and {\it iPods} due to the lack of data and their resemblance with the baggage clutter.
A Combination of Theta*, ORCA and Push and Rotate for Multi-agent Navigation
Venue: arXiv cs.MA 2020
Year: 2020

We study the problem of multi-agent navigation in static environments when no centralized controller is present. Each agent is controlled individually and relies on three algorithmic components to achieve its goal while avoiding collisions with the other agents and the obstacles: i) individual path planning which is done by Theta* algorithm; ii) collision avoidance while path following which is performed by ORCA* algorithm; iii) locally-confined multi-agent path planning done by Push and Rotate algorithm. The latter component is crucial to avoid deadlocks in confined areas, such as narrow passages or doors. We describe how the suggested components interact and form a coherent navigation pipeline. We carry out an extensive empirical evaluation of this pipeline in simulation. The obtained results clearly demonstrate that the number of occurring deadlocks significantly decreases enabling more agents to reach their goals compared to techniques that rely on collision-avoidance only and do not include multi-agent path planning component
Central object segmentation by deep learning for fruits and other roundish objects
Venue: arXiv cs.CV 2020
Year: 2020

We present CROP (Central Roundish Object Painter), which identifies and paints the object at the center of an RGB image. Primarily CROP works for roundish fruits in various illumination conditions, but surprisingly, it could also deal with images of other organic or inorganic materials, or ones by optical and electron microscopes, although CROP was trained solely by 172 images of fruits. The method involves image segmentation by deep learning, and the architecture of the neural network is a deeper version of the original U-Net. This technique could provide us with a means of automatically collecting statistical data of fruit growth in farms. As an example, we describe our experiment of processing 510 time series photos automatically to collect the data on the size and the position of the target fruit. Our trained neural network CROP and the above automatic programs are available on GitHub with user-friendly interface programs.
A Novel Indoor Positioning System for unprepared firefighting scenarios
Venue: arXiv cs.CV 2020
Year: 2020

Situational awareness and Indoor location tracking for firefighters is one of the tasks with paramount importance in search and rescue operations. For Indoor Positioning systems (IPS), GPS is not the best possible solution. There are few other techniques like dead reckoning, Wifi and bluetooth based triangulation, Structure from Motion (SFM) based scene reconstruction for Indoor positioning system. However due to high temperatures, the rapidly changing environment of fires, and low parallax in the thermal images, these techniques are not suitable for relaying the necessary information in a fire fighting environment needed to increase situational awareness in real time. In fire fighting environments, thermal imaging cameras are used due to smoke and low visibility hence obtaining relative orientation from the vanishing point estimation is very difficult. The following technique that is the content of this research implements a novel optical flow based video compass for orientation estimation and fused IMU data based activity recognition for IPS. This technique helps first responders to go into unprepared, unknown environments and still maintain situational awareness like the orientation and, position of the victim fire fighters.