OpenRaiser/Intern-Atlas huggingface dataset:
Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks
Venue: arXiv cs.CV 2021
Year: 2021

Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time- and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available.
A New Approach to CNF-SAT From a Probabilistic Point of View
Venue: arXiv cs.LO 2021
Year: 2021

The following paper proposes a new approach to determine whether a logical (CNF) formula is satisfiable or not using probability theory methods. Furthermore, we will introduce an algorithm that speeds up the standard solution for (CNF-SAT) in some cases. It is known that any (CNF) formula is solved with a time complexity of $2^n$ where n is the number of different literals in the (CNF) formula. In our approach, we will follow an enhanced method from a probabilistic point of view that does not always increase exponentially with the number of different literals. This will enhance the chance of determining whether a large formula is satisfiable or not in many cases. Additionally, we will point out at some promising properties that follow from applying probability theory concepts and axioms to logic, which might originate more insights about the satisfiability of logical formulas.
Landmark-Aware and Part-based Ensemble Transfer Learning Network for Facial Expression Recognition from Static images
Venue: arXiv cs.CV 2021
Year: 2021

Facial Expression Recognition from static images is a challenging problem in computer vision applications. Convolutional Neural Network (CNN), the state-of-the-art method for various computer vision tasks, has had limited success in predicting expressions from faces having extreme poses, illumination, and occlusion conditions. To mitigate this issue, CNNs are often accompanied by techniques like transfer, multi-task, or ensemble learning that often provide high accuracy at the cost of increased computational complexity. In this work, we propose a Part-based Ensemble Transfer Learning network that models how humans recognize facial expressions by correlating the spatial orientation pattern of the facial features with a specific expression. It consists of 5 sub-networks, and each sub-network performs transfer learning from one of the five subsets of facial landmarks: eyebrows, eyes, nose, mouth, or jaw to expression classification. We show that our proposed ensemble network uses visual patterns emanating from facial muscles' motor movements to predict expressions and demonstrate the usefulness of transfer learning from Facial Landmark Localization to Facial Expression Recognition. We test the proposed network on the CK+, JAFFE, and SFEW datasets, and it outperforms the benchmark for CK+ and JAFFE datasets by 0.51% and 5.34%, respectively. Additionally, the proposed ensemble network consists of only 1.65M model parameters, ensuring computational efficiency during training and real-time deployment. Grad-CAM visualizations of our proposed ensemble highlight the complementary nature of its sub-networks, a key design parameter of an effective ensemble network. Lastly, cross-dataset evaluation results reveal that our proposed ensemble has a high generalization capacity, making it suitable for real-world usage.
Improving Hierarchy Storage for Video Streaming in Cloud
Venue: arXiv cs.MM 2021
Year: 2021

Frequently accessed video streams are pre-transcoded into several formats to satisfy the characteristics of all display devices. Storing several video stream formats imposes a high cost on video stream providers using the old classical way. Alternatively, cloud providers offer a high flexibility of using their services and at a low cost relatively. Therefore, video stream companies adopted cloud technology to store their video streams. Generally, having all video streams stored in one type of cloud storage, the cost rises gradually. More importantly, the variation of the access pattern to frequently accessed video streams impacts negatively the storage cost and increases it significantly. To optimize storage usage and lower its cost, we propose a method that manages the cloud hierarchy storage. Particularly, we develop an algorithm that operates on parts of different videos that are frequently accessed and stores them in their suitable storage type cloud. Experiments came up with promising results on reducing the cost of using cloud storage by 18.75 %.
Design not Lost in Translation: A Case Study of an Intimate-Space Socially Assistive Robot for Emotion Regulation
Venue: arXiv cs.HC 2021
Year: 2021

We present a Research-through-Design case study of the design and development of an intimate-space tangible device perhaps best understood as a socially assistive robot, aimed at scaffolding children's efforts at emotional regulation. This case study covers the initial research device development, as well as knowledge transfer to a product development company towards translating the research into a workable commercial product that could also serve as a robust research product for field trials. Key contributions to the literature include: 1. sharing of lessons learned from the knowledge transfer process that can be useful to others interested in developing robust products, whether commercial or research, that preserve design values, while allowing for large scale deployment and research; 2. articulation of a design space in HCI/HRI--Human Robot Interaction--of intimate space socially assistive robots, with the current artifact as a central exemplar, contextualized alongside other related HRI artifacts.
Towards Automated Acceptance testing for industrial robots
Venue: arXiv cs.RO 2021
Year: 2021

Industrial robots are important machines applied in numerous modern industries that execute repetitive tasks with high accuracy, replacing or supporting dangerous jobs. In this kind of system, with increased complexity in which cost is related to the time the system keeps working, the system must operate with a minimum number of failures. In other words, a quality aspect important in industry is reliability. We hypothesize that Automated Acceptance Testing improves reliability for industrial robot program. We present the research question, the motivation for this study, our hypothesis and future research efforts.
BERT-CoQAC: BERT-based Conversational Question Answering in Context
Venue: arXiv cs.CL 2021
Year: 2021

As one promising way to inquire about any particular information through a dialog with the bot, question answering dialog systems have gained increasing research interests recently. Designing interactive QA systems has always been a challenging task in natural language processing and used as a benchmark to evaluate a machine's ability of natural language understanding. However, such systems often struggle when the question answering is carried out in multiple turns by the users to seek more information based on what they have already learned, thus, giving rise to another complicated form called Conversational Question Answering (CQA). CQA systems are often criticized for not understanding or utilizing the previous context of the conversation when answering the questions. To address the research gap, in this paper, we explore how to integrate conversational history into the neural machine comprehension system. On one hand, we introduce a framework based on a publically available pre-trained language model called BERT for incorporating history turns into the system. On the other hand, we propose a history selection mechanism that selects the turns that are relevant and contributes the most to answer the current question. Experimentation results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC leader board. We also conduct a number of experiments to show the side effects of using entire context information which brings unnecessary information and noise signals resulting in a decline in the model's performance.
A modularity comparison of Long Short-Term Memory and Morphognosis neural networks
Venue: arXiv cs.LG 2021
Year: 2021

This study compares the modularity performance of two artificial neural network architectures: a Long Short-Term Memory (LSTM) recurrent network, and Morphognosis, a neural network based on a hierarchy of spatial and temporal contexts. Mazes are used to measure performance, defined as the ability to utilize independently learned mazes to solve mazes composed of them. A maze is a sequence of rooms connected by doors. The modular task is implemented as follows: at the beginning of the maze, an initial door choice forms a context that must be retained until the end of an intervening maze, where the same door must be chosen again to reach the goal. For testing, the door-association mazes and separately trained intervening mazes are presented together for the first time. While both neural networks perform well during training, the testing performance of Morphognosis is significantly better than LSTM on this modular task.
TricubeNet: 2D Kernel-Based Object Representation for Weakly-Occluded Oriented Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

We present a novel approach for oriented object detection, named TricubeNet, which localizes oriented objects using visual cues ($i.e.,$ heatmap) instead of oriented box offsets regression. We represent each object as a 2D Tricube kernel and extract bounding boxes using simple image-processing algorithms. Our approach is able to (1) obtain well-arranged boxes from visual cues, (2) solve the angle discontinuity problem, and (3) can save computational complexity due to our anchor-free modeling. To further boost the performance, we propose some effective techniques for size-invariant loss, reducing false detections, extracting rotation-invariant features, and heatmap refinement. To demonstrate the effectiveness of our TricubeNet, we experiment on various tasks for weakly-occluded oriented object detection: detection in an aerial image, densely packed object image, and text image. The extensive experimental results show that our TricubeNet is quite effective for oriented object detection. Code is available at https://github.com/qjadud1994/TricubeNet.
Tractability of Quantified Temporal Constraints To The Max
Venue: arXiv cs.LO 2021
Year: 2021

A temporal constraint language is a set of relations that are first-order definable over (Q;<). We show that several temporal constraint languages whose constraint satisfaction problem is maximally tractable are also maximally tractable for the more expressive quantified constraint satisfaction problem. These constraint languages are defined in terms of preservation under certain binary polymorphisms. We also present syntactic characterizations of the relations in these languages.
OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we propose a cloud-based benchmark for robotic grasping and manipulation, called the OCRTOC benchmark. The benchmark focuses on the object rearrangement problem, specifically table organization tasks. We provide a set of identical real robot setups and facilitate remote experiments of standardized table organization scenarios in varying difficulties. In this workflow, users upload their solutions to our remote server and their code is executed on the real robot setups and scored automatically. After each execution, the OCRTOC team resets the experimental setup manually. We also provide a simulation environment that researchers can use to develop and test their solutions. With the OCRTOC benchmark, we aim to lower the barrier of conducting reproducible research on robotic grasping and manipulation and accelerate progress in this field. Executing standardized scenarios on identical real robot setups allows us to quantify algorithm performances and achieve fair comparisons. Using this benchmark we held a competition in the 2020 International Conference on Intelligence Robots and Systems (IROS 2020). In total, 59 teams took part in this competition worldwide. We present the results and our observations of the 2020 competition, and discuss our adjustments and improvements for the upcoming OCRTOC 2021 competition. The homepage of the OCRTOC competition is www.ocrtoc.org, and the OCRTOC software package is available at https://github.com/OCRTOC/OCRTOC_software_package.
Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL
Venue: arXiv cs.MA 2021
Year: 2021

How cooperation emerges is a long-standing and interdisciplinary problem. Game-theoretical studies on social dilemmas reveal that altruistic incentives are critical to the emergence of cooperation but their analyses are limited to stateless games. For more realistic scenarios, multi-agent reinforcement learning has been used to study sequential social dilemmas (SSDs). Recent works show that learning to incentivize other agents can promote cooperation in SSDs. However, we find that, with these incentivizing mechanisms, the team cooperation level does not converge and regularly oscillates between cooperation and defection during learning. We show that a second-order social dilemma resulting from the incentive mechanisms is the main reason for such fragile cooperation. We formally analyze the dynamics of second-order social dilemmas and find that a typical tendency of humans, called homophily, provides a promising solution. We propose a novel learning framework to encourage homophilic incentives and show that it achieves stable cooperation in both SSDs of public goods and tragedy of the commons.
Weakly-supervised Multi-task Learning for Multimodal Affect Recognition
Venue: arXiv cs.CL 2021
Year: 2021

Multimodal affect recognition constitutes an important aspect for enhancing interpersonal relationships in human-computer interaction. However, relevant data is hard to come by and notably costly to annotate, which poses a challenging barrier to build robust multimodal affect recognition systems. Models trained on these relatively small datasets tend to overfit and the improvement gained by using complex state-of-the-art models is marginal compared to simple baselines. Meanwhile, there are many different multimodal affect recognition datasets, though each may be small. In this paper, we propose to leverage these datasets using weakly-supervised multi-task learning to improve the generalization performance on each of them. Specifically, we explore three multimodal affect recognition tasks: 1) emotion recognition; 2) sentiment analysis; and 3) sarcasm recognition. Our experimental results show that multi-tasking can benefit all these tasks, achieving an improvement up to 2.9% accuracy and 3.3% F1-score. Furthermore, our method also helps to improve the stability of model performance. In addition, our analysis suggests that weak supervision can provide a comparable contribution to strong supervision if the tasks are highly correlated.
Automating Cyber Threat Hunting Using NLP, Automated Query Generation, and Genetic Perturbation
Venue: arXiv cs.CR 2021
Year: 2021

Scaling the cyber hunt problem poses several key technical challenges. Detecting and characterizing cyber threats at scale in large enterprise networks is hard because of the vast quantity and complexity of the data that must be analyzed as adversaries deploy varied and evolving tactics to accomplish their goals. There is a great need to automate all aspects, and, indeed, the workflow of cyber hunting. AI offers many ways to support this. We have developed the WILEE system that automates cyber threat hunting by translating high-level threat descriptions into many possible concrete implementations. Both the (high-level) abstract and (low-level) concrete implementations are represented using a custom domain specific language (DSL). WILEE uses the implementations along with other logic, also written in the DSL, to automatically generate queries to confirm (or refute) any hypotheses tied to the potential adversarial workflows represented at various layers of abstraction.
Identifying and Modeling Security Threats for IoMT Edge Network using Markov Chain and Common Vulnerability Scoring System (CVSS)
Venue: arXiv cs.CR 2021
Year: 2021

In this work, we defined an attack vector for networks utilizing the Internet of Medical Things (IoMT) devices and compute the probability distribution of IoMT security threats based on Markov chain and Common Vulnerability Scoring System (CVSS). IoMT is an emerging technology that improves patients' quality of life by permitting personalized e-health services without restrictions on time and site. The IoMT consists of embedded objects, sensors, and actuators that transmit and receive medical data. These Medical devices are vulnerable to different types of security threats, and thus, they pose a significant risk to patient's privacy and safety. Because security is a critical factor for successfully merging IoMT into pervasive healthcare systems, there is an urgent need for new security mechanisms to prevent threats on the IoMT edge network. Toward this direction, the first step is defining an attack vector that an attacker or unauthorized user can take advantage of to penetrate and tamper with medical data. In this article, we specify a threat model for the IoMT edge network. We identify any vulnerabilities or weaknesses within the IoMT network that allow unauthorized privileges and threats that can utilize these weaknesses to compromise the IoMT edge network. Finally, we compute the probability distribution of IoMT threats based on the Markov transition probability matrix.
A Picture is Worth a Collaboration: Accumulating Design Knowledge for Computer-Vision-based Hybrid Intelligence Systems
Venue: arXiv cs.CV 2021
Year: 2021

Computer vision (CV) techniques try to mimic human capabilities of visual perception to support labor-intensive and time-consuming tasks like the recognition and localization of critical objects. Nowadays, CV increasingly relies on artificial intelligence (AI) to automatically extract useful information from images that can be utilized for decision support and business process automation. However, the focus of extant research is often exclusively on technical aspects when designing AI-based CV systems while neglecting socio-technical facets, such as trust, control, and autonomy. For this purpose, we consider the design of such systems from a hybrid intelligence (HI) perspective and aim to derive prescriptive design knowledge for CV-based HI systems. We apply a reflective, practice-inspired design science approach and accumulate design knowledge from six comprehensive CV projects. As a result, we identify four design-related mechanisms (i.e., automation, signaling, modification, and collaboration) that inform our derived meta-requirements and design principles. This can serve as a basis for further socio-technical research on CV-based HI systems.
Genetic Constrained Graph Variational Autoencoder for COVID-19 Drug Discovery
Venue: arXiv cs.LG 2021
Year: 2021

In the past several months, COVID-19 has spread over the globe and caused severe damage to the people and the society. In the context of this severe situation, an effective drug discovery method to generate potential drugs is extremely meaningful. In this paper, we provide a methodology of discovering potential drugs for the treatment of Severe Acute Respiratory Syndrome Corona-Virus 2 (commonly known as SARS-CoV-2). We proposed a new model called Genetic Constrained Graph Variational Autoencoder (GCGVAE) to solve this problem. We trained our model based on the data of various viruses' protein structure, including that of the SARS, HIV, Hep3, and MERS, and used it to generate possible drugs for SARS-CoV-2. Several optimization algorithms, including valency masking and genetic algorithm, are deployed to fine tune our model. According to the simulation, our generated molecules have great effectiveness in inhibiting SARS-CoV-2. We quantitatively calculated the scores of our generated molecules and compared it with the scores of existing drugs, and the result shows our generated molecules scores much better than those existing drugs. Moreover, our model can be also applied to generate effective drugs for treating other viruses given their protein structure, which could be used to generate drugs for future viruses.
Interventional Aspect-Based Sentiment Analysis
Venue: arXiv cs.CL 2021
Year: 2021

Recent neural-based aspect-based sentiment analysis approaches, though achieving promising improvement on benchmark datasets, have reported suffering from poor robustness when encountering confounder such as non-target aspects. In this paper, we take a causal view to addressing this issue. We propose a simple yet effective method, namely, Sentiment Adjustment (SENTA), by applying a backdoor adjustment to disentangle those confounding factors. Experimental results on the Aspect Robustness Test Set (ARTS) dataset demonstrate that our approach improves the performance while maintaining accuracy in the original test set.
Secure Artificial Intelligence of Things for Implicit Group Recommendations
Venue: arXiv cs.AI 2021
Year: 2021

The emergence of Artificial Intelligence of Things (AIoT) has provided novel insights for many social computing applications such as group recommender systems. As distance among people has been greatly shortened, it has been a more general demand to provide personalized services to groups instead of individuals. In order to capture group-level preference features from individuals, existing methods were mostly established via aggregation and face two aspects of challenges: secure data management workflow is absent, and implicit preference feedbacks is ignored. To tackle current difficulties, this paper proposes secure Artificial Intelligence of Things for implicit Group Recommendations (SAIoT-GR). As for hardware module, a secure IoT structure is developed as the bottom support platform. As for software module, collaborative Bayesian network model and non-cooperative game are can be introduced as algorithms. Such a secure AIoT architecture is able to maximize the advantages of the two modules. In addition, a large number of experiments are carried out to evaluate the performance of the SAIoT-GR in terms of efficiency and robustness.