OpenRaiser/Intern-Atlas huggingface dataset:
1st Place Solution of LVIS Challenge 2020: A Good Box is not a Guarantee of a Good Mask
Venue: arXiv cs.CV 2020
Year: 2020

This article introduces the solutions of the team lvisTraveler for LVIS Challenge 2020. In this work, two characteristics of LVIS dataset are mainly considered: the long-tailed distribution and high quality instance segmentation mask. We adopt a two-stage training pipeline. In the first stage, we incorporate EQL and self-training to learn generalized representation. In the second stage, we utilize Balanced GroupSoftmax to promote the classifier, and propose a novel proposal assignment strategy and a new balanced mask loss for mask head to get more precise mask predictions. Finally, we achieve 41.5 and 41.2 AP on LVIS v1.0 val and test-dev splits respectively, outperforming the baseline based on X101-FPN-MaskRCNN by a large margin.
Deep Learning in Science
Venue: arXiv cs.CY 2020
Year: 2020

Much of the recent success of Artificial Intelligence (AI) has been spurred on by impressive achievements within a broader family of machine learning methods, commonly referred to as Deep Learning (DL). This paper provides insights on the diffusion and impact of DL in science. Through a Natural Language Processing (NLP) approach on the arXiv.org publication corpus, we delineate the emerging DL technology and identify a list of relevant search terms. These search terms allow us to retrieve DL-related publications from Web of Science across all sciences. Based on that sample, we document the DL diffusion process in the scientific system. We find i) an exponential growth in the adoption of DL as a research tool across all sciences and all over the world, ii) regional differentiation in DL application domains, and iii) a transition from interdisciplinary DL applications to disciplinary research within application domains. In a second step, we investigate how the adoption of DL methods affects scientific development. Therefore, we empirically assess how DL adoption relates to re-combinatorial novelty and scientific impact in the health sciences. We find that DL adoption is negatively correlated with re-combinatorial novelty, but positively correlated with expectation as well as variance of citation performance. Our findings suggest that DL does not (yet?) work as an autopilot to navigate complex knowledge landscapes and overthrow their structure. However, the 'DL principle' qualifies for its versatility as the nucleus of a general scientific method that advances science in a measurable way.
Evaluation of a Skill-based Control Architecture for a Visual Inspection-oriented Aerial Platform
Venue: arXiv cs.RO 2020
Year: 2020

The periodic inspection of vessels is a fundamental task to ensure their integrity and avoid maritime accidents. Currently, these inspections represent a high cost for the ship owner, in addition to the danger that this kind of hostile environment entails for the surveyors. In these situations, robotic platforms turn out to be useful not only for safety reasons, but also to reduce vessel downtimes and simplify the inspection procedures. Under this context, in this paper we report on the evaluation of a new control architecture devised to drive an aerial platform during these inspection procedures. The control architecture, based on an extensive use of behaviour-based high-level control, implements visual inspection-oriented functionalities, while releases the operator from the complexities of inspection flights and ensures the integrity of the platform. Apart from the control software, the full system comprises a multi-rotor platform equipped with a suitable set of sensors to permit teleporting the surveyor to the areas that need inspection. The paper provides an extensive set of testing results in different scenarios, under different operational conditions and over real vessels, in order to demonstrate the suitability of the platform for this kind of tasks.
Message Dissemination in Connected Vehicles
Venue: arXiv cs.NI 2020
Year: 2020

Advances in connected vehicles based on Vehicular Ad-hoc Networks (VANETs) in recent years have gained significant attention in Intelligent Transport Systems (ITS) in terms of disseminating messages in an efficient manner. VANET uses Dedicated Short Range Communication (DSRC) for disseminating messages between vehicles and between infrastructures. Though DSRC based communications are viable, it is still challenging to disseminate messages in a timely manner when vehicles are not in the transmission range of each other. Furthermore, DSRC communication channels are heavily congested when the vehicle density increases on the road. To address these limitations, two emerging paradigms: 1) vehicular cloud computing and 2) vehicular fog computing are been adopted to disseminate message between the vehicles in a connected vehicular environment. Vehicular fog computing uses fog nodes for the dissemination of messages among vehicles. Any real-world object can be formed as a fog node by acquiring the properties such as 1) network connectivity, 2) computation, and 3) storage. In this book chapter, we highlight the significance of message dissemination in connected vehicles based on techniques like DSRC, vehicular cloud computing, and vehicular fog computing. Our objective is to help the readers better understand the fundamentals of connected vehicles and communication techniques while disseminating messages between vehicles and between infrastructures.
Future Frame Prediction of a Video Sequence
Venue: arXiv cs.CV 2020
Year: 2020

Predicting future frames of a video sequence has been a problem of high interest in the field of Computer Vision as it caters to a multitude of applications. The ability to predict, anticipate and reason about future events is the essence of intelligence and one of the main goals of decision-making systems such as human-machine interaction, robot navigation and autonomous driving. However, the challenge lies in the ambiguous nature of the problem as there may be multiple future sequences possible for the same input video shot. A naively designed model averages multiple possible futures into a single blurry prediction. Recently, two distinct approaches have attempted to address this problem as: (a) use of latent variable models that represent underlying stochasticity and (b) adversarially trained models that aim to produce sharper images. A latent variable model often struggles to produce realistic results, while an adversarially trained model underutilizes latent variables and thus fails to produce diverse predictions. These methods have revealed complementary strengths and weaknesses. Combining the two approaches produces predictions that appear more realistic and better cover the range of plausible futures. This forms the basis and objective of study in this project work. In this paper, we proposed a novel multi-scale architecture combining both approaches. We validate our proposed model through a series of experiments and empirical evaluations on Moving MNIST, UCF101, and Penn Action datasets. Our method outperforms the results obtained using the baseline methods.
The ADAPT Enhanced Dependency Parser at the IWPT 2020 Shared Task
Venue: arXiv cs.CL 2020
Year: 2020

We describe the ADAPT system for the 2020 IWPT Shared Task on parsing enhanced Universal Dependencies in 17 languages. We implement a pipeline approach using UDPipe and UDPipe-future to provide initial levels of annotation. The enhanced dependency graph is either produced by a graph-based semantic dependency parser or is built from the basic tree using a small set of heuristics. Our results show that, for the majority of languages, a semantic dependency parser can be successfully applied to the task of parsing enhanced dependencies. Unfortunately, we did not ensure a connected graph as part of our pipeline approach and our competition submission relied on a last-minute fix to pass the validation script which harmed our official evaluation scores significantly. Our submission ranked eighth in the official evaluation with a macro-averaged coarse ELAS F1 of 67.23 and a treebank average of 67.49. We later implemented our own graph-connecting fix which resulted in a score of 79.53 (language average) or 79.76 (treebank average), which would have placed fourth in the competition evaluation.
Data-First Visualization Design Studies
Venue: arXiv cs.HC 2020
Year: 2020

We introduce the notion of a data-first design study which is triggered by the acquisition of real-world data instead of specific stakeholder analysis questions. We propose an adaptation of the design study methodology framework to provide practical guidance and to aid transferability to other data-first design processes. We discuss opportunities and risks by reflecting on two of our own data-first design studies. We review 64 previous design studies and identify 16 of them as edge cases with characteristics that may indicate a data-first design process in action.
Robust Homomorphic Video Hashing
Venue: arXiv cs.CR 2020
Year: 2020

The Internet has been weaponized to carry out cybercriminal activities at an unprecedented pace. The rising concerns for preserving the privacy of personal data while availing modern tools and technologies is alarming. End-to-end encrypted solutions are in demand for almost all commercial platforms. On one side, it seems imperative to provide such solutions and give people trust to reliably use these platforms. On the other side, this creates a huge opportunity to carry out unchecked cybercrimes. This paper proposes a robust video hashing technique, scalable and efficient in chalking out matches from an enormous bulk of videos floating on these commercial platforms. The video hash is validated to be robust to common manipulations like scaling, corruptions by noise, compression, and contrast changes that are most probable to happen during transmission. It can also be transformed into the encrypted domain and work on top of encrypted videos without deciphering. Thus, it can serve as a potential forensic tool that can trace the illegal sharing of videos without knowing the underlying content. Hence, it can help preserve privacy and combat cybercrimes such as revenge porn, hateful content, child abuse, or illegal material propagated in a video.
A Visual Analytics Approach to Debugging Cooperative, Autonomous Multi-Robot Systems' Worldviews
Venue: arXiv cs.HC 2020
Year: 2020

Autonomous multi-robot systems, where a team of robots shares information to perform tasks that are beyond an individual robot's abilities, hold great promise for a number of applications, such as planetary exploration missions. Each robot in a multi-robot system that uses the shared-world coordination paradigm autonomously schedules which robot should perform a given task, and when, using its worldview--the robot's internal representation of its belief about both its own state, and other robots' states. A key problem for operators is that robots' worldviews can fall out of sync (often due to weak communication links), leading to desynchronization of the robots' scheduling decisions and inconsistent emergent behavior (e.g., tasks not performed, or performed by multiple robots). Operators face the time-consuming and difficult task of making sense of the robots' scheduling decisions, detecting de-synchronizations, and pinpointing the cause by comparing every robot's worldview. To address these challenges, we introduce MOSAIC Viewer, a visual analytics system that helps operators (i) make sense of the robots' schedules and (ii) detect and conduct a root cause analysis of the robots' desynchronized worldviews. Over a year-long partnership with roboticists at the NASA Jet Propulsion Laboratory, we conduct a formative study to identify the necessary system design requirements and a qualitative evaluation with 12 roboticists. We find that MOSAIC Viewer is faster- and easier-to-use than the users' current approaches, and it allows them to stitch low-level details to formulate a high-level understanding of the robots' schedules and detect and pinpoint the cause of the desynchronized worldviews.
Clustering in VANET: Algorithms and Challenges
Venue: arXiv cs.NI 2020
Year: 2020

Clustering is an important concept in vehicular ad hoc network (VANET) where several vehicles join to form a group based on common features. Mobility-based clustering strategies are the most common in VANET clustering; however, machine learning and fuzzy logic algorithms are also the basis of many VANET clustering algorithms. Some VANET clustering algorithms integrate machine learning and fuzzy logic algorithms to make the cluster more stable and efficient. Network mobility (NEMO) and multi-hop-based strategies are also used for VANET clustering. Mobility and some other clustering strategies are presented in the existing literature reviews; however, extensive study of intelligence-based, mobility-based, and multi-hop-based strategies still missing in the VANET clustering reviews. In this paper, we presented a classification of intelligence-based clustering algorithms, mobility-based algorithms, and multi-hop-based algorithms with an analysis on the mobility metrics, evaluation criteria, challenges, and future directions of machine learning, fuzzy logic, mobility, NEMO, and multi-hop clustering algorithms.
Attribute Adaptive Margin Softmax Loss using Privileged Information
Venue: arXiv cs.CV 2020
Year: 2020

We present a novel framework to exploit privileged information for recognition which is provided only during the training phase. Here, we focus on recognition task where images are provided as the main view and soft biometric traits (attributes) are provided as the privileged data (only available during training phase). We demonstrate that more discriminative feature space can be learned by enforcing a deep network to adjust adaptive margins between classes utilizing attributes. This tight constraint also effectively reduces the class imbalance inherent in the local data neighborhood, thus carving more balanced class boundaries locally and using feature space more efficiently. Extensive experiments are performed on five different datasets and the results show the superiority of our method compared to the state-of-the-art models in both tasks of face recognition and person re-identification.
Secure Millimeter-Wave Ad Hoc Communications Using Physical Layer Security
Venue: arXiv cs.IT 2020
Year: 2020

Millimeter-wave (mmWave) communications are highly promising to improve the capacity of modern wireless networks, while the physical layer security (PLS) techniques hold great potential to enhance the critical secrecy performance therein. By carefully exploiting the significant signal difference between the Non-Light-of-Sight (NLoS) and Line-of-Sight (LoS) mmWave links, this paper proposes a Sight-based Cooperative Jamming (SCJ) scheme to improve the PLS performance of mmWave ad hoc communications. In this scheme, each potential jammer that has no LoS link to its nearest receiver but may have LoS links to eavesdroppers is selected with a certain probability to generate artificial noise such that channel advantages at legitimate receivers can be achieved. For performance modeling of the new jamming scheme, novel and efficient theoretical approximation approaches are firstly developed to enable the challenging issue of interference distribution modeling to be tackled, and then a theoretical framework based on stochastic geometry is proposed to capture the secrecy transmission capacity behavior under the SCJ scheme. Finally, extensive numerical results are provided to illustrate the SCJ scheme under various network scenarios.
About Graph Degeneracy, Representation Learning and Scalability
Venue: arXiv cs.LG 2020
Year: 2020

Graphs or networks are a very convenient way to represent data with lots of interaction. Recently, Machine Learning on Graph data has gained a lot of traction. In particular, vertex classification and missing edge detection have very interesting applications, ranging from drug discovery to recommender systems. To achieve such tasks, tremendous work has been accomplished to learn embedding of nodes and edges into finite-dimension vector spaces. This task is called Graph Representation Learning. However, Graph Representation Learning techniques often display prohibitive time and memory complexities, preventing their use in real-time with business size graphs. In this paper, we address this issue by leveraging a degeneracy property of Graphs - the K-Core Decomposition. We present two techniques taking advantage of this decomposition to reduce the time and memory consumption of walk-based Graph Representation Learning algorithms. We evaluate the performances, expressed in terms of quality of embedding and computational resources, of the proposed techniques on several academic datasets. Our code is available at https://github.com/SBrandeis/kcore-embedding
Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity
Venue: arXiv cs.GR 2020
Year: 2020

For human-like agents, including virtual avatars and social robots, making proper gestures while speaking is crucial in human--agent interaction. Co-speech gestures enhance interaction experiences and make the agents look alive. However, it is difficult to generate human-like gestures due to the lack of understanding of how people gesture. Data-driven approaches attempt to learn gesticulation skills from human demonstrations, but the ambiguous and individual nature of gestures hinders learning. In this paper, we present an automatic gesture generation model that uses the multimodal context of speech text, audio, and speaker identity to reliably generate gestures. By incorporating a multimodal context and an adversarial training scheme, the proposed model outputs gestures that are human-like and that match with speech content and rhythm. We also introduce a new quantitative evaluation metric for gesture generation models. Experiments with the introduced metric and subjective human evaluation showed that the proposed gesture generation model is better than existing end-to-end generation models. We further confirm that our model is able to work with synthesized audio in a scenario where contexts are constrained, and show that different gesture styles can be generated for the same speech by specifying different speaker identities in the style embedding space that is learned from videos of various speakers. All the code and data is available at https://github.com/ai4r/Gesture-Generation-from-Trimodal-Context.
Evaluating the Impact of COVID-19 on Cyberbullying through Bayesian Trend Analysis
Venue: arXiv cs.SI 2020
Year: 2020

COVID-19's impact has surpassed from personal and global health to our social life. In terms of digital presence, it is speculated that during pandemic, there has been a significant rise in cyberbullying. In this paper, we have examined the hypothesis of whether cyberbullying and reporting of such incidents have increased in recent times. To evaluate the speculations, we collected cyberbullying related public tweets (N=454,046) posted between January 1st, 2020 -- June 7th, 2020. A simple visual frequentist analysis ignores serial correlation and does not depict changepoints as such. To address correlation and a relatively small number of time points, Bayesian estimation of the trends is proposed for the collected data via an autoregressive Poisson model. We show that this new Bayesian method detailed in this paper can clearly show the upward trend on cyberbullying-related tweets since mid-March 2020. However, this evidence itself does not signify a rise in cyberbullying but shows a correlation of the crisis with the discussion of such incidents by individuals. Our work emphasizes a critical issue of cyberbullying and how a global crisis impacts social media abuse and provides a trend analysis model that can be utilized for social media data analysis in general.
Technical Report: The Policy Graph Improvement Algorithm
Venue: arXiv cs.AI 2020
Year: 2020

Optimizing a partially observable Markov decision process (POMDP) policy is challenging. The policy graph improvement (PGI) algorithm for POMDPs represents the policy as a fixed size policy graph and improves the policy monotonically. Due to the fixed policy size, computation time for each improvement iteration is known in advance. Moreover, the method allows for compact understandable policies. This report describes the technical details of the PGI [1] and particle based PGI [2] algorithms for POMDPs in a more accessible way than [1] or [2] allowing practitioners and students to understand and implement the algorithms.
Over-the-Air Computing for Wireless Data Aggregation in Massive IoT
Venue: arXiv cs.NI 2020
Year: 2020

Wireless data aggregation (WDA), referring to aggregating data distributed at devices (e.g., sensors and smartphone), is a common operation in 5G-and-beyond machine-type communications to support Internet-of-Things (IoT), which lays the foundation for diversified applications such as distributed sensing, learning, and control. Conventional WDA techniques that are designed based on a separated-communication-and-computation principle encounter difficulty in accommodating the massive access under the limited radio resource and stringent latency constraints imposed by emerging applications (e.g, auto-driving). To address this issue, over-the-air computation (AirComp) is being developed as a new WDA solution by seamlessly integrating computation and communication. By exploiting the waveform superposition property of a multiple-access channel, AirComp turns the air into a computer for computing and communicating functions of distributed data at many devices, thereby allowing low-latency WDA over massive devices. In view of growing interests on AirComp, this article provides a timely overview of the technology by introducing basic principles, discussing advanced techniques and applications, and identifying promising research opportunities.
InterLock: An Intercorrelated Logic and Routing Locking
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we propose a canonical prune-and-SAT (CP&SAT) attack for breaking state-of-the-art routing-based obfuscation techniques. In the CP&SAT attack, we first encode the key-programmable routing blocks (keyRBs) based on an efficient SAT encoding mechanism suited for detailed routing constraints, and then efficiently re-encode and reduce the CNF corresponded to the keyRB using a bounded variable addition (BVA) algorithm. In the CP&SAT attack, this is done before subjecting the circuit to the SAT attack. We illustrate that this encoding and BVA-based pre-processing significantly reduces the size of the CNF corresponded to the routing-based obfuscated circuit, in the result of which we observe 100% success rate for breaking prior art routing-based obfuscation techniques. Further, we propose a new intercorrelated logic and routing locking technique, or in short InterLock, as a countermeasure to mitigate the CP&SAT attack. In Interlock, in addition to hiding the connectivity, a part of the logic (gates) in the selected timing paths are also implemented in the keyRB(s). We illustrate that when the logic gates are twisted with keyRBs, the BVA could not provide any advantage as a pre-processing step. Our experimental results show that, by using InterLock, with only three 8$\times$8 or only two 16x16 keyRBs (twisted with actual logic gates), the resilience against existing attacks as well as our new proposed CP&SAT attack would be guaranteed while, on average, the delay/area overhead is less than 10% for even medium-size benchmark circuits.