OpenRaiser/Intern-Atlas huggingface dataset:
UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation
Venue: arXiv cs.CL 2022
Year: 2022

Sarcasm is a term that refers to the use of words to mock, irritate, or amuse someone. It is commonly used on social media. The metaphorical and creative nature of sarcasm presents a significant difficulty for sentiment analysis systems based on affective computing. The methodology and results of our team, UTNLP, in the SemEval-2022 shared task 6 on sarcasm detection are presented in this paper. We put different models, and data augmentation approaches to the test and report on which one works best. The tests begin with traditional machine learning models and progress to transformer-based and attention-based models. We employed data augmentation based on data mutation and data generation. Using RoBERTa and mutation-based data augmentation, our best approach achieved an F1-sarcastic of 0.38 in the competition's evaluation phase. After the competition, we fixed our model's flaws and achieved an F1-sarcastic of 0.414.
TigerLily: Finding drug interactions in silico with the Graph
Venue: arXiv cs.LG 2022
Year: 2022

Tigerlily is a TigerGraph based system designed to solve the drug interaction prediction task. In this machine learning task, we want to predict whether two drugs have an adverse interaction. Our framework allows us to solve this highly relevant real-world problem using graph mining techniques in these steps: (a) Using PyTigergraph we create a heterogeneous biological graph of drugs and proteins. (b) We calculate the personalized PageRank scores of drug nodes in the TigerGraph Cloud. (c) We embed the nodes using sparse non-negative matrix factorization of the personalized PageRank matrix. (d) Using the node embeddings we train a gradient boosting based drug interaction predictor.
OMG: Observe Multiple Granularities for Natural Language-Based Vehicle Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Retrieving tracked-vehicles by natural language descriptions plays a critical role in smart city construction. It aims to find the best match for the given texts from a set of tracked vehicles in surveillance videos. Existing works generally solve it by a dual-stream framework, which consists of a text encoder, a visual encoder and a cross-modal loss function. Although some progress has been made, they failed to fully exploit the information at various levels of granularity. To tackle this issue, we propose a novel framework for the natural language-based vehicle retrieval task, OMG, which Observes Multiple Granularities with respect to visual representation, textual representation and objective functions. For the visual representation, target features, context features and motion features are encoded separately. For the textual representation, one global embedding, three local embeddings and a color-type prompt embedding are extracted to represent various granularities of semantic features. Finally, the overall framework is optimized by a cross-modal multi-granularity contrastive loss function. Experiments demonstrate the effectiveness of our method. Our OMG significantly outperforms all previous methods and ranks the 9th on the 6th AI City Challenge Track2. The codes are available at https://github.com/dyhBUPT/OMG.
GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval
Venue: arXiv cs.IR 2022
Year: 2022

Recently, retrieval models based on dense representations are dominant in passage retrieval tasks, due to their outstanding ability in terms of capturing semantics of input text compared to the traditional sparse vector space models. A common practice of dense retrieval models is to exploit a dual-encoder architecture to represent a query and a passage independently. Though efficient, such a structure loses interaction between the query-passage pair, resulting in inferior accuracy. To enhance the performance of dense retrieval models without loss of efficiency, we propose a GNN-encoder model in which query (passage) information is fused into passage (query) representations via graph neural networks that are constructed by queries and their top retrieved passages. By this means, we maintain a dual-encoder structure, and retain some interaction information between query-passage pairs in their representations, which enables us to achieve both efficiency and efficacy in passage retrieval. Evaluation results indicate that our method significantly outperforms the existing models on MSMARCO, Natural Questions and TriviaQA datasets, and achieves the new state-of-the-art on these datasets.
Fast optimization of common basis for matrix set through Common Singular Value Decomposition
Venue: arXiv cs.LG 2022
Year: 2022

SVD (singular value decomposition) is one of the basic tools of machine learning, allowing to optimize basis for a given matrix. However, sometimes we have a set of matrices $\{A_k\}_k$ instead, and would like to optimize a single common basis for them: find orthogonal matrices $U$, $V$, such that $\{U^T A_k V\}$ set of matrices is somehow simpler. For example DCT-II is orthonormal basis of functions commonly used in image/video compression - as discussed here, this kind of basis can be quickly automatically optimized for a given dataset. While also discussed gradient descent optimization might be computationally costly, there is proposed CSVD (common SVD): fast general approach based on SVD. Specifically, we choose $U$ as built of eigenvectors of $\sum_i (w_k)^q (A_k A_k^T)^p$ and $V$ of $\sum_k (w_k)^q (A_k^T A_k)^p$, where $w_k$ are their weights, $p,q>0$ are some chosen powers e.g. 1/2, optionally with normalization e.g. $A \to A - rc^T$ where $r_i=\sum_j A_{ij}, c_j =\sum_i A_{ij}$.
Configuration-Aware Safe Control for Mobile Robotic Arm with Control Barrier Functions
Venue: arXiv cs.RO 2022
Year: 2022

Collision avoidance is a widely investigated topic in robotic applications. When applying collision avoidance techniques to a mobile robot, how to deal with the spatial structure of the robot still remains a challenge. In this paper, we design a configuration-aware safe control law by solving a Quadratic Programming (QP) with designed Control Barrier Functions (CBFs) constraints, which can safely navigate a mobile robotic arm to a desired region while avoiding collision with environmental obstacles. The advantage of our approach is that it correctly and in an elegant way incorporates the spatial structure of the mobile robotic arm. This is achieved by merging geometric restrictions among mobile robotic arm links into CBFs constraints. Simulations on a rigid rod and the modeled mobile robotic arm are performed to verify the feasibility and time-efficiency of proposed method. Numerical results about the time consuming for different degrees of freedom illustrate that our method scales well with dimension.
Unsupervised domain adaptation and super resolution on drone images for autonomous dry herbage biomass estimation
Venue: arXiv cs.CV 2022
Year: 2022

Herbage mass yield and composition estimation is an important tool for dairy farmers to ensure an adequate supply of high quality herbage for grazing and subsequently milk production. By accurately estimating herbage mass and composition, targeted nitrogen fertiliser application strategies can be deployed to improve localised regions in a herbage field, effectively reducing the negative impacts of over-fertilization on biodiversity and the environment. In this context, deep learning algorithms offer a tempting alternative to the usual means of sward composition estimation, which involves the destructive process of cutting a sample from the herbage field and sorting by hand all plant species in the herbage. The process is labour intensive and time consuming and so not utilised by farmers. Deep learning has been successfully applied in this context on images collected by high-resolution cameras on the ground. Moving the deep learning solution to drone imaging, however, has the potential to further improve the herbage mass yield and composition estimation task by extending the ground-level estimation to the large surfaces occupied by fields/paddocks. Drone images come at the cost of lower resolution views of the fields taken from a high altitude and requires further herbage ground-truth collection from the large surfaces covered by drone images. This paper proposes to transfer knowledge learned on ground-level images to raw drone images in an unsupervised manner. To do so, we use unpaired image style translation to enhance the resolution of drone images by a factor of eight and modify them to appear closer to their ground-level counterparts. We then ... ~\url{www.github.com/PaulAlbert31/Clover_SSL}.
Communication Bounds for Convolutional Neural Networks
Venue: arXiv cs.DC 2022
Year: 2022

Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential. Moving words of data between levels of a memory hierarchy or between processors on a network is much more expensive than the cost of arithmetic, so minimizing communication is critical to optimizing performance. In this paper, we present new lower bounds on data movement for mixed precision convolutions in both single-processor and parallel distributed memory models, as well as algorithms that outperform current implementations such as Im2Col. We obtain performance figures using GEMMINI, a machine learning accelerator, where our tiling provides improvements between 13% and 150% over a vendor supplied algorithm.
UMass PCL at SemEval-2022 Task 4: Pre-trained Language Model Ensembles for Detecting Patronizing and Condescending Language
Venue: arXiv cs.CL 2022
Year: 2022

Patronizing and condescending language (PCL) is everywhere, but rarely is the focus on its use by media towards vulnerable communities. Accurately detecting PCL of this form is a difficult task due to limited labeled data and how subtle it can be. In this paper, we describe our system for detecting such language which was submitted to SemEval 2022 Task 4: Patronizing and Condescending Language Detection. Our approach uses an ensemble of pre-trained language models, data augmentation, and optimizing the threshold for detection. Experimental results on the evaluation dataset released by the competition hosts show that our work is reliably able to detect PCL, achieving an F1 score of 55.47% on the binary classification task and a macro F1 score of 36.25% on the fine-grained, multi-label detection task.
Saliency in Augmented Reality
Venue: arXiv cs.CV 2022
Year: 2022

With the rapid development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary theory underlying AR is human visual confusion, which allows users to perceive the real-world scenes and augmented contents (virtual-world scenes) simultaneously by superimposing them together. To achieve good Quality of Experience (QoE), it is important to understand the interaction between two scenarios, and harmoniously display AR contents. However, studies on how this superimposition will influence the human visual attention are lacking. Therefore, in this paper, we mainly analyze the interaction effect between background (BG) scenes and AR contents, and study the saliency prediction problem in AR. Specifically, we first construct a Saliency in AR Dataset (SARD), which contains 450 BG images, 450 AR images, as well as 1350 superimposed images generated by superimposing BG and AR images in pair with three mixing levels. A large-scale eye-tracking experiment among 60 subjects is conducted to collect eye movement data. To better predict the saliency in AR, we propose a vector quantized saliency prediction method and generalize it for AR saliency prediction. For comparison, three benchmark methods are proposed and evaluated together with our proposed method on our SARD. Experimental results demonstrate the superiority of our proposed method on both of the common saliency prediction problem and the AR saliency prediction problem over benchmark methods. Our dataset and code are available at: https://github.com/DuanHuiyu/ARSaliency.
BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment
Venue: arXiv cs.CV 2022
Year: 2022

This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To overcome the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Transformer Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.
Detecting, Tracking and Counting Motorcycle Rider Traffic Violations on Unconstrained Roads
Venue: arXiv cs.CV 2022
Year: 2022

In many Asian countries with unconstrained road traffic conditions, driving violations such as not wearing helmets and triple-riding are a significant source of fatalities involving motorcycles. Identifying and penalizing such riders is vital in curbing road accidents and improving citizens' safety. With this motivation, we propose an approach for detecting, tracking, and counting motorcycle riding violations in videos taken from a vehicle-mounted dashboard camera. We employ a curriculum learning-based object detector to better tackle challenging scenarios such as occlusions. We introduce a novel trapezium-shaped object boundary representation to increase robustness and tackle the rider-motorcycle association. We also introduce an amodal regressor that generates bounding boxes for the occluded riders. Experimental results on a large-scale unconstrained driving dataset demonstrate the superiority of our approach compared to existing approaches and other ablative variants.
Strolling in Room-Scale VR: Hex-Core-MK1 Omnidirectional Treadmill
Venue: arXiv cs.HC 2022
Year: 2022

The natural locomotion interface is critical to the development of many VR applications. For household VR applications, there are two basic requirements: natural immersive experience and minimized space occupation. The existing locomotion strategies generally do not simultaneously satisfy these two requirements well. This paper presents a novel omnidirectional treadmill (ODT) system, named Hex-Core-MK1 (HCMK1). By implementing two kinds of mirror symmetrical spiral rollers to generate the omnidirectional velocity field, this proposed system is capable of providing real walking experiences with a full-degree of freedom in an area as small as 1.76 m^2, while delivering great advantages over several existing ODT systems in terms of weight, volume, latency and dynamic performance. Compared with the sizes of Infinadeck and HCP, the two best motor-driven ODTs so far, the 8 cm height of HCMK1 is only 20% of Infinadeck and 50% of HCP. In addition, HCMK1 is a lightweight device weighing only 110 kg, which provides possibilities of further expanding VR scenarios, such as terrain simulation. The latency of HCMK1 is only 23ms. The experiments show that HCMK1 can deliver on a starting acceleration of 16.00 m/s^2 and a braking acceleration of 30.00 m/s^2.
Face recognition with small and large size databases
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents experimental results using the ORL (40 people) and FERET (994 people) databases. The ORL database can be useful for securing applications where few users attempting to access are expected. This is the case, for instance, of a PDA or PC where the password is the face of the user. On the other hand, the FERET database is useful for studying those situations where the number of authorized users is around a thousand people.
Research on Domain Information Mining and Theme Evolution of Scientific Papers
Venue: arXiv cs.DL 2022
Year: 2022

In recent years, with the increase of social investment in scientific research, the number of research results in various fields has increased significantly. Cross-disciplinary research results have gradually become an emerging frontier research direction. There is a certain dependence between a large number of research results. It is difficult to effectively analyze today's scientific research results when looking at a single research field in isolation. How to effectively use the huge number of scientific papers to help researchers becomes a challenge. This paper introduces the research status at home and abroad in terms of domain information mining and topic evolution law of scientific and technological papers from three aspects: the semantic feature representation learning of scientific and technological papers, the field information mining of scientific and technological papers, and the mining and prediction of research topic evolution rules of scientific and technological papers.
Inductive Biases for Object-Centric Representations in the Presence of Complex Textures
Venue: arXiv cs.CV 2022
Year: 2022

Understanding which inductive biases could be helpful for the unsupervised learning of object-centric representations of natural scenes is challenging. In this paper, we systematically investigate the performance of two models on datasets where neural style transfer was used to obtain objects with complex textures while still retaining ground-truth annotations. We find that by using a single module to reconstruct both the shape and visual appearance of each object, the model learns more useful representations and achieves better object separation. In addition, we observe that adjusting the latent space size is insufficient to improve segmentation performance. Finally, the downstream usefulness of the representations is significantly more strongly correlated with segmentation quality than with reconstruction accuracy.
KANT: A tool for Grounding and Knowledge Management
Venue: arXiv cs.RO 2022
Year: 2022

The intelligent robotics community usually organizes knowledge into symbolic and sub-symbolic levels. These two levels establish the set of symbols and rules for manipulating knowledge based on their (symbol system - dictionary). Thus, the correspondences -- Grounding or knowledge representation -- require specific software techniques for anchoring continuous and discrete state variables between these two levels. This paper presents the design and evaluation of an Open Source tool called KANT(Knowledge mAnagemeNT) to let different components of the system architecture controlling the robot query, save, edit, and delete the data from the Knowledge Base without having to worry about the type and the implementation of the source data. Using KANT, components managing subsymbolic information can smoothly interact with symbolic components. Besides, implementation mechanisms used in KANT, such as the use of in-memory and non-SQL databases, improve the performance of the knowledge management systems in ROS middleware, as shown by the evaluations presented in this work.
Spot the Difference: A Novel Task for Embodied Agents in Changing Environments
Venue: arXiv cs.CV 2022
Year: 2022

Embodied AI is a recent research area that aims at creating intelligent agents that can move and operate inside an environment. Existing approaches in this field demand the agents to act in completely new and unexplored scenes. However, this setting is far from realistic use cases that instead require executing multiple tasks in the same environment. Even if the environment changes over time, the agent could still count on its global knowledge about the scene while trying to adapt its internal representation to the current state of the environment. To make a step towards this setting, we propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment and needs to recover the correct layout in a fixed time budget. To this end, we collect a new dataset of occupancy maps starting from existing datasets of 3D spaces and generating a number of possible layouts for a single environment. This dataset can be employed in the popular Habitat simulator and is fully compliant with existing methods that employ reconstructed occupancy maps during navigation. Furthermore, we propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents. Experimental results show that the proposed architecture outperforms existing state-of-the-art models for exploration on this new setting.
Inclusion unlocks the creative potential of gender diversity in teams
Venue: arXiv cs.SI 2022
Year: 2022

Diversity in teams can boost creativity, and gender diversity was shown to be a contributor to collective creativity. We show that gender diversity requires inclusion to lead to benefits in creativity by analyzing teams in 4011 video game projects. Recording data on the weighted network from past collaborations, we developed four measures of inclusion, depending on a lack of segregation, strong ties across genders, and the incorporation of women into the core of the team s network. We found that gender diversity without inclusion does not contribute to creativity, while with maximal inclusion one standard deviation change in diversity results in .04 to .09 standard deviation change in creativity, depending on the measure of inclusion. To reap creative benefits of diversity, developer firms need to include 23 percent or more female developers (as opposed to the 15 percent mean female proportion) and include them in the team along all dimensions. Inclusion at low diversity has a negative effect. By analyzing the sequences of diversity and inclusion across games within firms, we found that adding diversity first, and developing inclusion later can lead to higher diversity and inclusion, compared to adding female developers with already existing cross-gender ties to the team.
Designing an AI-Based Adaptive Controller Augmented with a System Identifier for a Micro-Class Robot Equipped with a Vibrating Actuator
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, an adaptive control scheme based on using neural networks is designed to guarantee the desired behavior of a micro-robot which is equipped with vibrating actuators and follows the principle of slip-stick movement. There are two tiny shaking motors which have been utilized to run the micro-class robotic system. Dynamic modeling equations are expressed by considering the spring coefficient of the bases. After that, the effect of the spring on the foundations was investigated. In addition to designing neural-based controller, an AI-based system identifier has been developed to help the controller update its parameters and achieve its desired targets. Using this method, several specific paths for the movement of this micro robot are simulated. Based on the simulation results, the proposed controlling strategy guarantees acceptable performance for tracking different paths due to plotted near-zero errors and handles the nonlinear behavior of the micro-robot system.