OpenRaiser/Intern-Atlas huggingface dataset:
Detecting Harmful Memes and Their Targets
Venue: arXiv cs.CL 2021
Year: 2021

Among the various modes of communication in social media, the use of Internet memes has emerged as a powerful means to convey political, psychological, and socio-cultural opinions. Although memes are typically humorous in nature, recent days have witnessed a proliferation of harmful memes targeted to abuse various social entities. As most harmful memes are highly satirical and abstruse without appropriate contexts, off-the-shelf multimodal models may not be adequate to understand their underlying semantics. In this work, we propose two novel problem formulations: detecting harmful memes and the social entities that these harmful memes target. To this end, we present HarMeme, the first benchmark dataset, containing 3,544 memes related to COVID-19. Each meme went through a rigorous two-stage annotation process. In the first stage, we labeled a meme as very harmful, partially harmful, or harmless; in the second stage, we further annotated the type of target(s) that each harmful meme points to: individual, organization, community, or society/general public/other. The evaluation results using ten unimodal and multimodal models highlight the importance of using multimodal signals for both tasks. We further discuss the limitations of these models and we argue that more research is needed to address these problems.
SAM: A Self-adaptive Attention Module for Context-Aware Recommendation System
Venue: arXiv cs.IR 2021
Year: 2021

Recently, textual information has been proved to play a positive role in recommendation systems. However, most of the existing methods only focus on representation learning of textual information in ratings, while potential selection bias induced by the textual information is ignored. In this work, we propose a novel and general self-adaptive module, the Self-adaptive Attention Module (SAM), which adjusts the selection bias by capturing contextual information based on its representation. This module can be embedded into recommendation systems that contain learning components of contextual information. Experimental results on three real-world datasets demonstrate the effectiveness of our proposal, and the state-of-the-art models with SAM significantly outperform the original ones.
SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference
Venue: arXiv cs.AR 2021
Year: 2021

Edge computing devices inherently face tight resource constraints, which is especially apparent when deploying Deep Neural Networks (DNN) with high memory and compute demands. FPGAs are commonly available in edge devices. Since these reconfigurable circuits can achieve higher throughput and lower power consumption than general purpose processors, they are especially well-suited for DNN acceleration. However, existing solutions for designing FPGA-based DNN accelerators for edge devices come with high development overheads, given the cost of repeated FPGA synthesis passes, reimplementation in a Hardware Description Language (HDL) of the simulated design, and accelerator system integration. In this paper we propose SECDA, a new hardware/software co-design methodology to reduce design time of optimized DNN inference accelerators on edge devices with FPGAs. SECDA combines cost-effective SystemC simulation with hardware execution, streamlining design space exploration and the development process via reduced design evaluation time. As a case study, we use SECDA to efficiently develop two different DNN accelerator designs on a PYNQ-Z1 board, a platform that includes an edge FPGA. We quickly and iteratively explore the system's hardware/software stack, while identifying and mitigating performance bottlenecks. We evaluate the two accelerator designs with four common DNN models, achieving an average performance speedup across models of up to 3.5$\times$ with a 2.9$\times$ reduction in energy consumption over CPU-only inference. Our code is available at https://github.com/gicLAB/SECDA
Learning to Ask for Data-Efficient Event Argument Extraction
Venue: arXiv cs.CL 2021
Year: 2021

Event argument extraction (EAE) is an important task for information extraction to discover specific argument roles. In this study, we cast EAE as a question-based cloze task and empirically analyze fixed discrete token template performance. As generating human-annotated question templates is often time-consuming and labor-intensive, we further propose a novel approach called "Learning to Ask," which can learn optimized question templates for EAE without human annotations. Experiments using the ACE-2005 dataset demonstrate that our method based on optimized questions achieves state-of-the-art performance in both the few-shot and supervised settings.
Robustly Removing Deep Sea Lighting Effects for Visual Mapping of Abyssal Plains
Venue: arXiv cs.CV 2021
Year: 2021

The majority of Earth's surface lies deep in the oceans, where no surface light reaches. Robots diving down to great depths must bring light sources that create moving illumination patterns in the darkness, such that the same 3D point appears with different color in each image. On top, scattering and attenuation of light in the water makes images appear foggy and typically blueish, the degradation depending on each pixel's distance to its observed seafloor patch, on the local composition of the water and the relative poses and cones of the light sources. Consequently, visual mapping, including image matching and surface albedo estimation, severely suffers from the effects that co-moving light sources produce, and larger mosaic maps from photos are often dominated by lighting effects that obscure the actual seafloor structure. In this contribution a practical approach to estimating and compensating these lighting effects on predominantly homogeneous, flat seafloor regions, as can be found in the Abyssal plains of our oceans, is presented. The method is essentially parameter-free and intended as a preprocessing step to facilitate visual mapping, but already produces convincing lighting artefact compensation up to a global white balance factor. It does not require to be trained beforehand on huge sets of annotated images, which are not available for the deep sea. Rather, we motivate our work by physical models of light propagation, perform robust statistics-based estimates of additive and multiplicative nuisances that avoid explicit parameters for light, camera, water or scene, discuss the breakdown point of the algorithms and show results on imagery captured by robots in several kilometer water depth.
TEACh: Task-driven Embodied Agents that Chat
Venue: arXiv cs.CV 2021
Year: 2021

Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes. To study this, we introduce TEACh, a dataset of over 3,000 human--human, interactive dialogues to complete household tasks in simulation. A Commander with access to oracle information about a task communicates in natural language with a Follower. The Follower navigates through and interacts with the environment to complete tasks varying in complexity from "Make Coffee" to "Prepare Breakfast", asking questions and getting additional information from the Commander. We propose three benchmarks using TEACh to study embodied intelligence challenges, and we evaluate initial models' abilities in dialogue understanding, language grounding, and task execution.
Self-Supervised Decomposition, Disentanglement and Prediction of Video Sequences while Interpreting Dynamics: A Koopman Perspective
Venue: arXiv cs.CV 2021
Year: 2021

Human interpretation of the world encompasses the use of symbols to categorize sensory inputs and compose them in a hierarchical manner. One of the long-term objectives of Computer Vision and Artificial Intelligence is to endow machines with the capacity of structuring and interpreting the world as we do. Towards this goal, recent methods have successfully been able to decompose and disentangle video sequences into their composing objects and dynamics, in a self-supervised fashion. However, there has been a scarce effort in giving interpretation to the dynamics of the scene. We propose a method to decompose a video into moving objects and their attributes, and model each object's dynamics with linear system identification tools, by means of a Koopman embedding. This allows interpretation, manipulation and extrapolation of the dynamics of the different objects by employing the Koopman operator K. We test our method in various synthetic datasets and successfully forecast challenging trajectories while interpreting them.
Video Temporal Relationship Mining for Data-Efficient Person Re-identification
Venue: arXiv cs.CV 2021
Year: 2021

This paper is a technical report to our submission to the ICCV 2021 VIPriors Re-identification Challenge. In order to make full use of the visual inductive priors of the data, we treat the query and gallery images of the same identity as continuous frames in a video sequence. And we propose one novel post-processing strategy for video temporal relationship mining, which not only calculates the distance matrix between query and gallery images, but also the matrix between gallery images. The initial query image is used to retrieve the most similar image from the gallery, then the retrieved image is treated as a new query to retrieve its most similar image from the gallery. By iteratively searching for the closest image, we can achieve accurate image retrieval and finally obtain a robust retrieval sequence.
Natural language understanding for logical games
Venue: arXiv cs.CL 2021
Year: 2021

We developed a system able to automatically solve logical puzzles in natural language. Our solution is composed by a parser and an inference module. The parser translates the text into first order logic (FOL), while the MACE4 model finder is used to compute the models of the given FOL theory. We also empower our software agent with the capability to provide Yes/No answers to natural language questions related to each puzzle. Moreover, in line with Explainalbe Artificial Intelligence (XAI), the agent can back its answer, providing a graphical representation of the proof. The advantage of using reasoning for Natural Language Understanding (NLU) instead of Machine learning is that the user can obtain an explanation of the reasoning chain. We illustrate how the system performs on various types of natural language puzzles, including 382 knights and knaves puzzles. These features together with the overall performance rate of 80.89\% makes the proposed solution an improvement upon similar solvers for natural language understanding in the puzzles domain.
Conditional Deep Gaussian Processes: empirical Bayes hyperdata learning
Venue: arXiv cs.LG 2021
Year: 2021

It is desirable to combine the expressive power of deep learning with Gaussian Process (GP) in one expressive Bayesian learning model. Deep kernel learning showed success in adopting a deep network for feature extraction followed by a GP used as function model. Recently,it was suggested that, albeit training with marginal likelihood, the deterministic nature of feature extractor might lead to overfitting while the replacement with a Bayesian network seemed to cure it. Here, we propose the conditional Deep Gaussian Process (DGP) in which the intermediate GPs in hierarchical composition are supported by the hyperdata and the exposed GP remains zero mean. Motivated by the inducing points in sparse GP, the hyperdata also play the role of function supports, but are hyperparameters rather than random variables. We follow our previous moment matching approach to approximate the marginal prior for conditional DGP with a GP carrying an effective kernel. Thus, as in empirical Bayes, the hyperdata are learned by optimizing the approximate marginal likelihood which implicitly depends on the hyperdata via the kernel. We shall show the equivalence with the deep kernel learning in the limit of dense hyperdata in latent space. However, the conditional DGP and the corresponding approximate inference enjoy the benefit of being more Bayesian than deep kernel learning. Preliminary extrapolation results demonstrate expressive power from the depth of hierarchy by exploiting the exact covariance and hyperdata learning, in comparison with GP kernel composition, DGP variational inference and deep kernel learning. We also address the non-Gaussian aspect of our model as well as way of upgrading to a full Bayes inference.
Cuttlefish: Library for Achieving Energy Efficiency in Multicore Parallel Programs
Venue: arXiv cs.DC 2021
Year: 2021

A low-cap power budget is challenging for exascale computing. Dynamic Voltage and Frequency Scaling (DVFS) and Uncore Frequency Scaling (UFS) are the two widely used techniques for limiting the HPC application's energy footprint. However, existing approaches fail to provide a unified solution that can work with different types of parallel programming models and applications. This paper proposes Cuttlefish, a programming model oblivious C/C++ library for achieving energy efficiency in multicore parallel programs running over Intel processors. An online profiler periodically profiles model-specific registers to discover a running application's memory access pattern. Using a combination of DVFS and UFS, Cuttlefish then dynamically adapts the processor's core and uncore frequencies, thereby improving its energy efficiency. The evaluation on a 20-core Intel Xeon processor using a set of widely used OpenMP benchmarks, consisting of several irregular-tasking and work-sharing pragmas, achieves geometric mean energy savings of 19.4% with a 3.6% slowdown.
On the complexity of the optimal transport problem with graph-structured cost
Venue: arXiv cs.LG 2021
Year: 2021

Multi-marginal optimal transport (MOT) is a generalization of optimal transport to multiple marginals. Optimal transport has evolved into an important tool in many machine learning applications, and its multi-marginal extension opens up for addressing new challenges in the field of machine learning. However, the usage of MOT has been largely impeded by its computational complexity which scales exponentially in the number of marginals. Fortunately, in many applications, such as barycenter or interpolation problems, the cost function adheres to structures, which has recently been exploited for developing efficient computational methods. In this work we derive computational bounds for these methods. With $m$ marginal distributions supported on $n$ points, we provide a $ \mathcal{\tilde O}(d(G)m n^2ε^{-2})$ bound for a $ε$-accuracy when the problem is associated with a tree with diameter $d(G)$. For the special case of the Wasserstein barycenter problem, which corresponds to a star-shaped tree, our bound is in alignment with the existing complexity bound for it.
Automatic Home-based Screening of Obstructive Sleep Apnea Using Single Channel Electrocardiogram and SPO2 Signals
Venue: arXiv cs.LG 2021
Year: 2021

Obstructive sleep apnea (OSA) is one of the most widespread respiratory diseases today. Complete or relative breathing cessations due to upper airway subsidence during sleep is OSA. It has confirmed potential influence on Covid-19 hospitalization and mortality, and is strongly associated with major comorbidities of severe Covid-19 infection. Un-diagnosed OSA may also lead to a variety of severe physical and mental side-effects. To score OSA severity, nocturnal sleep monitoring is performed under defined protocols and standards called polysomnography (PSG). This method is time-consuming, expensive, and requiring professional sleep technicians. Automatic home-based detection of OSA is welcome and in great demand. It is a fast and effective way for referring OSA suspects to sleep clinics for further monitoring. On-line OSA detection also can be a part of a closed-loop automatic control of the OSA therapeutic/assistive devices. In this paper, several solutions for online OSA detection are introduced and tested on 155 subjects of three different databases. The best combinational solution uses mutual information (MI) analysis for selecting out of ECG and SpO2-based features. Several methods of supervised and unsupervised machine learning are employed to detect apnoeic episodes. To achieve the best performance, the most successful classifiers in four different ternary combination methods are used. The proposed configurations exploit limited use of biological signals, have online working scheme, and exhibit uniform and acceptable performance (over 85%) in all the employed databases. The benefits have not been gathered all together in the previous published methods.
ReDUCE: Reformulation of Mixed Integer Programs using Data from Unsupervised Clusters for Learning Efficient Strategies
Venue: arXiv cs.RO 2021
Year: 2021

Mixed integer convex and nonlinear programs, MICP and MINLP, are expressive but require long solving times. Recent work that combines learning methods on solver heuristics has shown potential to overcome this issue allowing for applications on larger scale practical problems. Gathering sufficient training data to employ these methods still present a challenge since getting data from traditional solvers are slow and newer learning approaches still require large amounts of data. In order to scale up and make these hybrid learning approaches more manageable we propose ReDUCE, a method that exploits structure within small to medium size datasets. We also introduce the bookshelf organization problem as an MINLP as a way to measure performance of solvers with ReDUCE. Results show that existing algorithms with ReDUCE can solve this problem within a few seconds, a significant improvement over the original formulation. ReDUCE is demonstrated as a high level planner for a robotic arm for the bookshelf problem.
Graph Compression with Application to Model Selection
Venue: arXiv cs.IT 2021
Year: 2021

Many multivariate data such as social and biological data exhibit complex dependencies that are best characterized by graphs. Unlike sequential data, graphs are, in general, unordered structures. This means we can no longer use classic, sequential-based compression methods on these graph-based data. Therefore, it is necessary to develop new methods for graph compression. In this paper, we present universal source coding methods for the lossless compression of unweighted, undirected, unlabelled graphs. We encode in two steps: 1) transforming graph into a rooted binary tree, 2) the encoding rooted binary tree using graph statistics. Our coders showed better compression performance than other source coding methods on both synthetic and real-world graphs. We then applied our graph coding methods for model selection of Gaussian graphical models using minimum description length (MDL) principle finding the description length of the conditional independence graph. Experiments on synthetic data show that our approach gives better performance compared to common model selection methods. We also applied our approach to electrocardiogram (ECG) data in order to explore the differences between graph models of two groups of subjects.
Simplify Your Law: Using Information Theory to Deduplicate Legal Documents
Venue: arXiv cs.CL 2021
Year: 2021

Textual redundancy is one of the main challenges to ensuring that legal texts remain comprehensible and maintainable. Drawing inspiration from the refactoring literature in software engineering, which has developed methods to expose and eliminate duplicated code, we introduce the duplicated phrase detection problem for legal texts and propose the Dupex algorithm to solve it. Leveraging the Minimum Description Length principle from information theory, Dupex identifies a set of duplicated phrases, called patterns, that together best compress a given input text. Through an extensive set of experiments on the Titles of the United States Code, we confirm that our algorithm works well in practice: Dupex will help you simplify your law.
A Survey of COVID-19 Misinformation: Datasets, Detection Techniques and Open Issues
Venue: arXiv cs.SI 2021
Year: 2021

Misinformation during pandemic situations like COVID-19 is growing rapidly on social media and other platforms. This expeditious growth of misinformation creates adverse effects on the people living in the society. Researchers are trying their best to mitigate this problem using different approaches based on Machine Learning (ML), Deep Learning (DL), and Natural Language Processing (NLP). This survey aims to study different approaches of misinformation detection on COVID-19 in recent literature to help the researchers in this domain. More specifically, we review the different methods used for COVID-19 misinformation detection in their research with an overview of data pre-processing and feature extraction methods to get a better understanding of their work. We also summarize the existing datasets which can be used for further research. Finally, we discuss the limitations of the existing methods and highlight some potential future research directions along this dimension to combat the spreading of misinformation during a pandemic.
Do Facial Trait Correlates with Roll Call Voting in Parliament? Using fWHR to Study Performance in Politics
Venue: arXiv cs.SI 2021
Year: 2021

Research has shown that people recognize and select leaders based on their facial appearance. However, considering the correlation between the performance of leaders and their facial traits, empirical findings are mixed. This paper adds to the debate by focusing on two previously understudied aspects of facial traits among political leaders: (i) previous studies have focused on electoral success and achievement drive of politicians omitting their actual daily performance after elections; (ii) previous research has analyzed individual politicians omitting the context of social circumstances which potentially influence their performance. We address these issues by analyzing Ukrainian members of parliament (MPs) who voted for bills in six consecutive Verkhovna Rada starting from Rada 4 (2002-06) to Rada 9 (2019-Present) to study politicians' performance, which is defined as co-voting or cooperation between MPs in voting on the same bill. In simple words, we analyze whether politicians tend to follow leaders when voting. This ability to summon the votes of others is interpreted as better performance. To measure performance, we proposed a generic methodology named Feature Importance For Measuring Performance (FIMP) that can be used in various scenarios. Using FIMP, our data suggest that MPs vote has no impact from their colleagues with higher or lower facial width-to-height ratio (fWHR), a popular measure of the facial trait.
Terahertz Wireless Communications in Space
Venue: arXiv cs.ET 2021
Year: 2021

The New Space Era has increased communication traffic in space by new space missions led by public space agencies and private companies. Mars colonization is also targeted by crewed missions in the near future. Due to increasing space traffic near Earth and Mars, the bandwidth is getting congested. Moreover, the downlink performance of the current missions is not satisfactory in terms of delay and data rate. Therefore, to meet the increasing demand in space links, Terahertz band (0.1-10 THz) wireless communications are proposed in this study. In line with this, we discuss the major challenges that the realization of THz band space links pose and possible solutions. Moreover, we simulate Mars-space THz links for the case of a clear Mars atmosphere, and a heavy dust storm to show that even in the worst conditions, a large bandwidth is available for Mars communication traffic.