OpenRaiser/Intern-Atlas huggingface dataset:
A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset
Venue: arXiv cs.CV 2022
Year: 2022

Many human pose estimation methods estimate Skinned Multi-Person Linear (SMPL) models and regress the human joints from these SMPL estimates. In this work, we show that the most widely used SMPL-to-joint linear layer (joint regressor) is inaccurate, which may mislead pose evaluation results. To achieve a more accurate joint regressor, we propose a method to create pseudo-ground-truth SMPL poses, which can then be used to train an improved regressor. Specifically, we optimize SMPL estimates coming from a state-of-the-art method so that its projection matches the silhouettes of humans in the scene, as well as the ground-truth 2D joint locations. While the quality of this pseudo-ground-truth is challenging to assess due to the lack of actual ground-truth SMPL, with the Human 3.6m dataset, we qualitatively show that our joint locations are more accurate and that our regressor leads to improved pose estimations results on the test set without any need for retraining. We release our code and joint regressor at https://github.com/ubc-vision/joint-regressor-refinement
Who's the Expert? On Multi-source Belief Change
Venue: arXiv cs.AI 2022
Year: 2022

Consider the following belief change/merging scenario. A group of information sources gives a sequence of reports about the state of the world at various instances (e.g. different points in time). The true states at these instances are unknown to us. The sources have varying levels of expertise, also unknown to us, and may be knowledgeable on some topics but not others. This may cause sources to report false statements in areas they lack expertise. What should we believe on the basis of these reports? We provide a framework in which to explore this problem, based on an extension of propositional logic with expertise formulas. This extended language allows us to express beliefs about the state of the world at each instance, as well as beliefs about the expertise of each source. We propose several postulates, provide a couple of families of concrete operators, and analyse these operators with respect to the postulates.
Baba is You is Undecidable
Venue: arXiv cs.CC 2022
Year: 2022

We establish the undecidability of 2019 puzzle game Baba is You through a reduction from the Post correspondence problem. In particular, we consider a restricted form of the Post correspondence problem introduced by Neary (arXiv:1312.6700) that is limited to five pairs of words. Baba is You is an award winning tile-based game in which the player can reprogram the game's mechanisms by pushing blocks that spell out the rules. We achieve undecidability through a generalization of the size of the playfield in the horizontal direction, adding a "hallway" to one side of the level. The undecidability of Baba is You has been claimed several times online using different source problems, including the simulation of Turing machines and Conway's Game of Life, however, this contribution appears to be the first formal proof of the result.
Gaze-enhanced Crossmodal Embeddings for Emotion Recognition
Venue: arXiv cs.LG 2022
Year: 2022

Emotional expressions are inherently multimodal -- integrating facial behavior, speech, and gaze -- but their automatic recognition is often limited to a single modality, e.g. speech during a phone call. While previous work proposed crossmodal emotion embeddings to improve monomodal recognition performance, despite its importance, an explicit representation of gaze was not included. We propose a new approach to emotion recognition that incorporates an explicit representation of gaze in a crossmodal emotion embedding framework. We show that our method outperforms the previous state of the art for both audio-only and video-only emotion classification on the popular One-Minute Gradual Emotion Recognition dataset. Furthermore, we report extensive ablation experiments and provide detailed insights into the performance of different state-of-the-art gaze representations and integration strategies. Our results not only underline the importance of gaze for emotion recognition but also demonstrate a practical and highly effective approach to leveraging gaze information for this task.
Learn to Understand Negation in Video Retrieval
Venue: arXiv cs.MM 2022
Year: 2022

Negation is a common linguistic skill that allows human to express what we do NOT want. Naturally, one might expect video retrieval to support natural-language queries with negation, e.g., finding shots of kids sitting on the floor and not playing with a dog. However, the state-of-the-art deep learning based video retrieval models lack such ability, as they are typically trained on video description datasets such as MSR-VTT and VATEX that lack negated descriptions. Their retrieved results basically ignore the negator in the sample query, incorrectly returning videos showing kids playing with dog. This paper presents the first study on learning to understand negation in video retrieval and make contributions as follows. By re-purposing two existing datasets (MSR-VTT and VATEX), we propose a new evaluation protocol for video retrieval with negation. We propose a learning based method for training a negation-aware video retrieval model. The key idea is to first construct a soft negative caption for a specific training video by partially negating its original caption, and then compute a bidirectionally constrained loss on the triplet. This auxiliary loss is weightedly added to a standard retrieval loss. Experiments on the re-purposed benchmarks show that re-training the CLIP (Contrastive Language-Image Pre-Training) model by the proposed method clearly improves its ability to handle queries with negation. In addition, the model performance on the original benchmarks is also improved.
Multimodal Representation Learning With Text and Images
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, multimodal AI has seen an upward trend as researchers are integrating data of different types such as text, images, speech into modelling to get the best results. This project leverages multimodal AI and matrix factorization techniques for representation learning, on text and image data simultaneously, thereby employing the widely used techniques of Natural Language Processing (NLP) and Computer Vision. The learnt representations are evaluated using downstream classification and regression tasks. The methodology adopted can be extended beyond the scope of this project as it uses Auto-Encoders for unsupervised representation learning.
Local permutation polynomials and the action of e-Klenian groups
Venue: arXiv cs.DM 2022
Year: 2022

Permutation polynomials of finite fields have many applications in Coding Theory, Cryptography and Combinatorics. In the first part of this paper we present a new family of local permutation polynomials based on a class of symmetric subgroups without fixed points, the so called e-Klenian groups. In the second part we use the fact that bivariate local permutation polynomials define Latin Squares, to discuss several constructions of Mutually Orthogonal Latin Squares (MOLS) and, in particular, we provide a new family of MOLS on size a prime power.
Finding Strategies Against Misinformation in Social Media: A Qualitative Study
Venue: arXiv cs.HC 2022
Year: 2022

Misinformation spread through social media has become a fundamental challenge in modern society. Recent studies have evaluated various strategies for addressing this problem, such as by modifying social media platforms or educating people about misinformation, to varying degrees of success. Our goal is to develop a new strategy for countering misinformation: intelligent tools that encourage social media users to foster metacognitive skills "in the wild." As a first step, we conducted focus groups with social media users to discover how they can be best supported in combating misinformation. Qualitative analyses of the discussions revealed that people find it difficult to detect misinformation. Findings also indicated a need for but lack of resources to support cross-validation of information. Moreover, misinformation had a nuanced emotional impact on people. Suggestions for the design of intelligent tools that support social media users in information selection, information engagement, and emotional response management are presented.
Trust in Human-AI Interaction: Scoping Out Models, Measures, and Methods
Venue: arXiv cs.HC 2022
Year: 2022

Trust has emerged as a key factor in people's interactions with AI-infused systems. Yet, little is known about what models of trust have been used and for what systems: robots, virtual characters, smart vehicles, decision aids, or others. Moreover, there is yet no known standard approach to measuring trust in AI. This scoping review maps out the state of affairs on trust in human-AI interaction (HAII) from the perspectives of models, measures, and methods. Findings suggest that trust is an important and multi-faceted topic of study within HAII contexts. However, most work is under-theorized and under-reported, generally not using established trust models and missing details about methods, especially Wizard of Oz. We offer several targets for systematic review work as well as a research agenda for combining the strengths and addressing the weaknesses of the current literature.
DefakeHop++: An Enhanced Lightweight Deepfake Detector
Venue: arXiv cs.CV 2022
Year: 2022

On the basis of DefakeHop, an enhanced lightweight Deepfake detector called DefakeHop++ is proposed in this work. The improvements lie in two areas. First, DefakeHop examines three facial regions (i.e., two eyes and mouth) while DefakeHop++ includes eight more landmarks for broader coverage. Second, for discriminant features selection, DefakeHop uses an unsupervised approach while DefakeHop++ adopts a more effective approach with supervision, called the Discriminant Feature Test (DFT). In DefakeHop++, rich spatial and spectral features are first derived from facial regions and landmarks automatically. Then, DFT is used to select a subset of discriminant features for classifier training. As compared with MobileNet v3 (a lightweight CNN model of 1.5M parameters targeting at mobile applications), DefakeHop++ has a model of 238K parameters, which is 16% of MobileNet v3. Furthermore, DefakeHop++ outperforms MobileNet v3 in Deepfake image detection performance in a weakly-supervised setting.
Recognising Known Configurations of Garments For Dual-Arm Robotic Flattening
Venue: arXiv cs.RO 2022
Year: 2022

Robotic deformable-object manipulation is a challenge in the robotic industry because deformable objects have complicated and various object states. Predicting those object states and updating manipulation planning is time-consuming and computationally expensive. In this paper, we propose learning known configurations of garments to allow a robot to recognise garment states and choose a pre-designed manipulation plan for garment flattening.
PGD: A Large-scale Professional Go Dataset for Data-driven Analytics
Venue: arXiv cs.AI 2022
Year: 2022

Lee Sedol is on a winning streak--does this legend rise again after the competition with AlphaGo? Ke Jie is invincible in the world championship--can he still win the title this time? Go is one of the most popular board games in East Asia, with a stable professional sports system that has lasted for decades in China, Japan, and Korea. There are mature data-driven analysis technologies for many sports, such as soccer, basketball, and esports. However, developing such technology for Go remains nontrivial and challenging due to the lack of datasets, meta-information, and in-game statistics. This paper creates the Professional Go Dataset (PGD), containing 98,043 games played by 2,148 professional players from 1950 to 2021. After manual cleaning and labeling, we provide detailed meta-information for each player, game, and tournament. Moreover, the dataset includes analysis results for each move in the match evaluated by advanced AlphaZero-based AI. To establish a benchmark for PGD, we further analyze the data and extract meaningful in-game features based on prior knowledge related to Go that can indicate the game status. With the help of complete meta-information and constructed in-game features, our results prediction system achieves an accuracy of 75.30%, much higher than several state-of-the-art approaches (64%-65%). As far as we know, PGD is the first dataset for data-driven analytics in Go and even in board games. Beyond this promising result, we provide more examples of tasks that benefit from our dataset. The ultimate goal of this paper is to bridge this ancient game and the modern data science community. It will advance research on Go-related analytics to enhance the fan experience, help players improve their ability, and facilitate other promising aspects. The dataset will be made publicly available.
Deep Learning-Enabled Semantic Communication Systems with Task-Unaware Transmitter and Dynamic Data
Venue: arXiv cs.IT 2022
Year: 2022

Existing deep learning-enabled semantic communication systems often rely on shared background knowledge between the transmitter and receiver that includes empirical data and their associated semantic information. In practice, the semantic information is defined by the pragmatic task of the receiver and cannot be known to the transmitter. The actual observable data at the transmitter can also have non-identical distribution with the empirical data in the shared background knowledge library. To address these practical issues, this paper proposes a new neural network-based semantic communication system for image transmission, where the task is unaware at the transmitter and the data environment is dynamic. The system consists of two main parts, namely the semantic coding (SC) network and the data adaptation (DA) network. The SC network learns how to extract and transmit the semantic information using a receiver-leading training process. By using the domain adaptation technique from transfer learning, the DA network learns how to convert the data observed into a similar form of the empirical data that the SC network can process without retraining. Numerical experiments show that the proposed method can be adaptive to observable datasets while keeping high performance in terms of both data recovery and task execution.
Directly wireless communication of human minds via non-invasive brain-computer-metasurface platform
Venue: arXiv cs.IT 2022
Year: 2022

Brain-computer interfaces (BCIs), invasive or non-invasive, have projected unparalleled vision and promise for assisting patients in need to better their interaction with the surroundings. Inspired by the BCI-based rehabilitation technologies for nerve-system impairments and amputation, we propose an electromagnetic brain-computer-metasurface (EBCM) paradigm, regulated by human's cognition by brain signals directly and non-invasively. We experimentally show that our EBCM platform can translate human's mind from evoked potentials of P300-based electroencephalography to digital coding information in the electromagnetic domain non-invasively, which can be further processed and transported by an information metasurface in automated and wireless fashions. Directly wireless communications of the human minds are performed between two EBCM operators with accurate text transmissions. Moreover, several other proof-of-concept mind-control schemes are presented using the same EBCM platform, exhibiting flexibly-customized capabilities of information processing and synthesis like visual-beam scanning, wave modulations, and pattern encoding.
Leveraging Emotion-specific Features to Improve Transformer Performance for Emotion Classification
Venue: arXiv cs.CL 2022
Year: 2022

This paper describes the approach to the Emotion Classification shared task held at WASSA 2022 by team PVGs AI Club. This Track 2 sub-task focuses on building models which can predict a multi-class emotion label based on essays from news articles where a person, group or another entity is affected. Baseline transformer models have been demonstrating good results on sequence classification tasks, and we aim to improve this performance with the help of ensembling techniques, and by leveraging two variations of emotion-specific representations. We observe better results than our baseline models and achieve an accuracy of 0.619 and a macro F1 score of 0.520 on the emotion classification task.
Learning to Get Up
Venue: arXiv cs.GR 2022
Year: 2022

Getting up from an arbitrary fallen state is a basic human skill. Existing methods for learning this skill often generate highly dynamic and erratic get-up motions, which do not resemble human get-up strategies, or are based on tracking recorded human get-up motions. In this paper, we present a staged approach using reinforcement learning, without recourse to motion capture data. The method first takes advantage of a strong character model, which facilitates the discovery of solution modes. A second stage then learns to adapt the control policy to work with progressively weaker versions of the character. Finally, a third stage learns control policies that can reproduce the weaker get-up motions at much slower speeds. We show that across multiple runs, the method can discover a diverse variety of get-up strategies, and execute them at a variety of speeds. The results usually produce policies that use a final stand-up strategy that is common to the recovery motions seen from all initial states. However, we also find policies for which different strategies are seen for prone and supine initial fallen states. The learned get-up control strategies often have significant static stability, i.e., they can be paused at a variety of points during the get-up motion. We further test our method on novel constrained scenarios, such as having a leg and an arm in a cast.
Dual-Beam Intelligent Reflecting Surface for Millimeter and THz Communications
Venue: arXiv cs.IT 2022
Year: 2022

Intelligent reflecting surface (IRS) is a cost-efficient technique to improve power efficiency and spectral efficiency. However, IRS-aided multi-antenna transmission needs to jointly optimize the passive and active beamforming, imposing a high computational burden and high latency due to its iterative optimization process. Making use of hybrid analog-digital beamforming in high-frequency transmission systems, a novel technique, coined dual-beam IRS, is proposed in this paper. The key idea is to form a pair of beams towards the IRS and user, respectively. Then, the optimization of passive and active beamforming can be decoupled, resulting in a simplified system design. Simulation results corroborate that it achieves a good balance between the cell-edge and cell-center performance. Compared with the performance bound, the gap is moderate, but it remarkably outperforms other sub-optimal schemes.
Graph Anisotropic Diffusion
Venue: arXiv cs.LG 2022
Year: 2022

Traditional Graph Neural Networks (GNNs) rely on message passing, which amounts to permutation-invariant local aggregation of neighbour features. Such a process is isotropic and there is no notion of `direction' on the graph. We present a new GNN architecture called Graph Anisotropic Diffusion. Our model alternates between linear diffusion, for which a closed-form solution is available, and local anisotropic filters to obtain efficient multi-hop anisotropic kernels. We test our model on two common molecular property prediction benchmarks (ZINC and QM9) and show its competitive performance.
Combined Learning of Neural Network Weights for Privacy in Collaborative Tasks
Venue: arXiv cs.LG 2022
Year: 2022

We introduce CoLN, Combined Learning of Neural network weights, a novel method to securely combine Machine Learning models over sensitive data with no sharing of data. With CoLN, local hosts use the same Neural Network architecture and base parameters to train a model using only locally available data. Locally trained models are then submitted to a combining agent, which produces a combined model. The new model's parameters can be sent back to hosts, and can then be used as initial parameters for a new training iteration. CoLN is capable of combining several distributed neural networks of the same kind but is not restricted to any single neural architecture. In this paper we detail the combination algorithm and present experiments with feed-forward, convolutional, and recurrent Neural Network architectures, showing that the CoLN combined model approximates the performance of a hypothetical ideal centralized model, trained using the combination of the local datasets. CoLN can contribute to secure collaborative research, as required in the medical area, where privacy issues preclude data sharing, but where the limitations of local data demand information derived from larger datasets.
Abnormal-aware Multi-person Evaluation System with Improved Fuzzy Weighting
Venue: arXiv cs.CY 2022
Year: 2022

There exists a phenomenon that subjectivity highly lies in the daily evaluation process. Our research primarily concentrates on a multi-person evaluation system with anomaly detection to minimize the possible inaccuracy that subjective assessment brings. We choose the two-stage screening method, which consists of rough screening and score-weighted Kendall-$τ$ Distance to winnow out abnormal data, coupled with hypothesis testing to narrow global discrepancy. Then we use Fuzzy Synthetic Evaluation Method(FSE) to determine the significance of scores given by reviewers as well as their reliability, culminating in a more impartial weight for each reviewer in the final conclusion. The results demonstrate a clear and comprehensive ranking instead of unilateral scores, and we get to have an efficiency in filtering out abnormal data as well as a reasonably objective weight determination mechanism. We can sense that through our study, people will have a chance of modifying a multi-person evaluation system to attain both equity and a relatively superior competitive atmosphere.
Learning user-defined sub-goals using memory editing in reinforcement learning
Venue: arXiv cs.AI 2022
Year: 2022

The aim of reinforcement learning (RL) is to allow the agent to achieve the final goal. Most RL studies have focused on improving the efficiency of learning to achieve the final goal faster. However, the RL model is very difficult to modify an intermediate route in the process of reaching the final goal. That is, the agent cannot be under control to achieve other sub-goals in the existing studies. If the agent can go through the sub-goals on the way to the destination, the RL can be applied and studied in various fields. In this study, I propose a methodology to achieve the user-defined sub-goals as well as the final goal using memory editing. The memory editing is performed to generate various sub-goals and give an additional reward to the agent. In addition, the sub-goals are separately learned from the final goal. I set two simple environments and various scenarios in the test environments. As a result, the agent almost successfully passed the sub-goals as well as the final goal under control. Moreover, the agent was able to be induced to visit the novel state indirectly in the environments. I expect that this methodology can be used in the fields that need to control the agent in a variety of scenarios.
Quality-Aware Real-Time Augmented Reality Visualization under Delay Constraints
Venue: arXiv cs.DC 2022
Year: 2022

Augmented reality (AR) is one of emerging applications in modern multimedia systems research. Due to intensive time-consuming computations for AR visualization in mobile devices, quality-aware real-time computing under delay constraints is essentially required. Inspired by Lyapunov optimization framework, this paper proposes a time-average quality maximization method for the AR visualization under delay considerations.