Miracle-dz/UniFAR-FTUs huggingface dataset:
How to See Smells: Extracting Olfactory References from Artworks

1 PROBLEM Although being an essential part of how we experience the world, smell is severely undervalued in the context of cultural heritage. The Odeuropa project aims at preserving and recreating the olfactory heritage of Europe. State-of-the-art methods of artificial intelligence are applied to large corpora of visual and textual data ranging from the 16th to 20th century of European history to extract olfactory references. Creating an ontology of smells, this information is stored in the “European Olfactory Knowledge Graph (EOKG)” following standards of the semantic web. My Ph.D. addresses the visual extraction part of the project. We will create a taxonomy of visual smell references and acquire a large corpus of artworks from various early modern European digital collections. Using computer vision techniques, we will implement a pipeline for the combined recognition of olfactory objects, poses, and iconographies and annotate the images from our image corpus accordingly. Following these steps, we will address the following research questions: (i)What visual representations of smell exist in European 16th to 20th century works of art and how can these be represented in the EOKG as an ontology shared with the other work packages of the Odeuropa project? (ii)Whichmachine-learning techniques exist for the automated extraction of olfactory references in the visual arts? Particularly, which techniques are suited to cope with the domain shift problem when applying computer vision techniques to our field of research? (iii) How do the identified techniques perform in terms of established evaluation metrics? Which ones work best for the extraction of olfactory references? Both the preservation of olfactory heritage [3] and the application of machine learning (ML) to cultural heritage [1] have been addressed before. However, in most cases machine learning algorithms are treated as “black boxes” and their application does not contribute back to ML [4]. Computer vision techniques like object detection and pose estimation have successfully been applied to the domain of visual arts ([8], [2]) but have not achieved performance comparable to their application in the photographic domain. One reason for the success of computer vision on photographs is the availability of huge labeled datasets like ImageNet [10]. Datasets containing artworks
Few-Shot Knowledge Validation using Rules

Knowledge graphs (KGs) form the basis of modern intelligent search systems – their network structure helps with the semantic reasoning and interpretation of complex tasks. A KG is a highly dynamic structure in which facts are continuously updated, added, and removed. A typical approach to ensure data quality in the presence of continuous changes is to apply logic rules. These rules are automatically mined from the data using frequency-based approaches. As a result, these approaches depend on the data quality of the KG and are susceptible to errors and incompleteness. To address these issues, we propose Colt, a few-shot rule-based knowledge validation framework that enables the interactive quality assessment of logic rules. It evaluates the quality of any rule by asking a user to validate only a few facts entailed by such rule on the KG. We formalize the problem as learning a validation function over the rule’s outcomes and study the theoretical connections to the generalized maximum coverage problem. Our model obtains (i) an accurate estimate of the quality of a rule with fewer than 20 user interactions and (ii) 75% quality (F1) with 5% annotations in the task of validating facts entailed by any rule.
Escape from An Echo Chamber

An echo chamber effect refers to the phenomena that online users revealed selective exposure and ideological segregation on political issues. Prior studies indicate the connection between the spread of misinformation and online echo chambers. In this paper, to help users escape from an echo chamber, we propose a novel news-analysis platform that provides a panoramic view of stances towards a particular event from different news media sources. Moreover, to help users better recognize the stances of news sources which published these news articles, we adopt a news stance classification model to categorize their stances into “agree”, “disagree”, “discuss”, or “unrelated” to a relevant claim for specified events with political stances. Finally, we proposed two ways showing the echo chamber effects: 1) visualizing the event and the associated pieces of news; and 2) visualizing the stance distribution of news from news sources of different political ideology. By making the echo chamber effect explicit, we expect online users will become exposed to more diverse perspectives toward a specific event. The demo video of our platform is available on youtube1.
ATJ-Net: Auto-Table-Join Network for Automatic Learning on Relational Databases

A relational database, consisting of multiple tables, provides heterogeneous information across various entities, widely used in real-world services. This paper studies the supervised learning task on multiple tables, aiming to predict one label column with the help of multiple-tabular data. However, classical ML techniques mainly focus on single-tabular data. Multiple-tabular data refers to many-to-many mapping among joinable attributes and n-ary relations, which cannot be utilized directly by classical ML techniques. Besides, current graph techniques, like heterogeneous information network (HIN) and graph neural networks (GNN), are infeasible to be deployed directly and automatically in a multi-table environment, which limits the learning on databases. For automatic learning on relational databases, we propose an auto-table-join network (ATJ-Net). Multiple tables with relationships are considered as a hypergraph, where vertices are joinable attributes and hyperedges are tuples of tables. Then, ATJ-Net builds a graph neural network on the heterogeneous hypergraph, which samples and aggregates the vertices and hyperedges on n-hop sub-graphs as the receptive field. In order to enable ATJ-Net to be automatically deployed to different datasets and avoid the ”no free lunch” dilemma, we use random architecture search to select optimal aggregators and prune redundant paths in the network. For verifying the effectiveness of our methods across various tasks and schema, we conduct extensive experiments on 4 tasks, 8 various schemas, and 19 sub-datasets w.r. t. citing prediction, review classification, recommendation, and task-blind challenge. ATJ-Net achieves the best performance over state-of-the-art approaches on three tasks and is competitive with KddCup Winner solution on task-blind challenge.
“Is it a Qoincidence?”: An Exploratory Study of QAnon on Voat

Online fringe communities offer fertile grounds to users seeking and sharing ideas fueling suspicion of mainstream news and conspiracy theories. Among these, the QAnon conspiracy theory emerged in 2017 on 4chan, broadly supporting the idea that powerful politicians, aristocrats, and celebrities are closely engaged in a global pedophile ring. Simultaneously, governments are thought to be controlled by “puppet masters,” as democratically elected officials serve as a fake showroom of democracy. This paper provides an empirical exploratory analysis of the QAnon community on Voat.co, a Reddit-esque news aggregator, which has captured the interest of the press for its toxicity and for providing a platform to QAnon followers. More precisely, we analyze a large dataset from /v/GreatAwakening, the most popular QAnon-related subverse (the Voat equivalent of a subreddit), to characterize activity and user engagement. To further understand the discourse around QAnon, we study the most popular named entities mentioned in the posts, along with the most prominent topics of discussion, which focus on US politics, Donald Trump, and world events. We also use word embeddings to identify narratives around QAnon-specific keywords. Our graph visualization shows that some of the QAnon-related ones are closely related to those from the Pizzagate conspiracy theory and so-called drops by “Q. ” Finally, we analyze content toxicity, finding that discussions on /v/GreatAwakening are less toxic than in the broad Voat community.
Efficient Computation of Semantically Cohesive Subgraphs for Keyword-Based Knowledge Graph Exploration

A knowledge graph (KG) represents a set of entities and their relations. To explore the content of a large and complex KG, a convenient way is keyword-based querying. Traditional methods assign small weights to salient entities or relations, and answer an exploratory keyword query by computing a group Steiner tree (GST), which is a minimum-weight subgraph that connects all the keywords in the query. Recent studies have suggested improving the semantic cohesiveness of a query answer by minimizing the pairwise semantic distances between the entities in a subgraph, but it remains unclear how to efficiently compute such a semantically cohesive subgraph. In this paper, we formulate it as a quadratic group Steiner tree problem (QGSTP) by extending the classical minimum-weight GST problem which is NP-hard. We design two approximation algorithms for QGSTP and prove their approximation ratios. Furthermore, to improve their practical performance, we present heuristics including pruning and ranking strategies.
Extracting Contextualized Quantity Facts from Web Tables

Quantity queries, with filter conditions on quantitative measures of entities, are beyond the functionality of search engines and QA assistants. To enable such queries over web contents, this paper develops a novel method for automatically extracting quantity facts from ad-hoc web tables. This involves recognizing quantities, with normalized values and units, aligning them with the proper entities, and contextualizing these pairs with informative cues to match sophisticated queries with modifiers. Our method includes a new approach to aligning quantity columns to entity columns. Prior works assumed a single subject-column per table, whereas our approach is geared for complex tables and leverages external corpora as evidence. For contextualization, we identify informative cues from text and structural markup that surrounds a table. For query-time fact ranking, we devise a new scoring technique that exploits both context similarity and inter-fact consistency. Comparisons of our building blocks against state-of-the-art baselines and extrinsic experiments with two query benchmarks demonstrate the benefits of our method.
IFSpard: An Information Fusion-based Framework for Spam Review Detection

Online reviews, which contain the quality information and user experience about products, always affect the consumption decisions of customers. Unfortunately, quite a number of spammers attempt to mislead consumers by writing fake reviews for some intents. Existing methods for detecting spam reviews mainly focus on constructing discriminative features, which heavily depend on experts and may miss some complex but effective features. Recently, some models attempt to learn the latent representations of reviews, users, and items. However, the learned embeddings usually lack interpretability. Moreover, most of existing methods are based on single classification model while ignoring the complementarity of different classification models. To solve these problems, we propose IFSpard, a novel information fusion-based framework that aims at exploring and exploiting useful information from various aspects for spam review detection. First, we design a graph-based feature extraction method and an interaction-mining-based feature crossing method to automatically extract basic and complex features with consideration of different sources of data. Then, we propose a mutual-information-based feature selection and representation learning method to remove the irrelevant and redundant information contained in the automatically constructed features. Finally, we devise an adaptive ensemble model to make use of the information of constructed features and the abilities of different classifiers for spam review detection. Experimental results on several public datasets show that the proposed model performs better than state-of-the-art methods.
Timing-Driven X-architecture Steiner Minimum Tree Construction Based on Social Learning Multi-Objective Particle Swarm Optimization

The construction of timing-driven Steiner minimum tree is a critical issue in VLSI routing design. Meanwhile, since the interconnection model of X-architecture can make full use of routing resources compared to the traditional Manhattan architecture, constructing a Timing-Driven X-architecture Steiner Minimum Tree (TDXSMT) is of great significance to improving routing performance. In this paper, an efficient algorithm based on Social Learning Multi-Objective Particle Swarm Optimization (SLMOPSO) is proposed to construct a TDXSMT with minimizing the maximum source-to-sink pathlength. An X-architecture Prim-Dijkstra model is presented to construct an initial Steiner tree which can optimize both the wirelength and the maximum source-to-sink pathlength. In order to find a better solution, an SLMOPSO method based on the nearest and best select strategy is presented to improve the global exploration capability of the algorithm. Besides, the mutation and crossover operators are utilized to achieve the discrete particle update process, thereby better solving the discrete TDXSMT problem. The experimental results indicate that the proposed algorithm has an excellent trade-off between the wirelength and maximum source-to-sink pathlength of the routing tree and can greatly optimize the timing delay.
Learning Dynamic User Behavior Based on Error-driven Event Representation

Understanding the evolution of large graphs over time is of significant importance in user behavior understanding and prediction. Modeling user behavior with temporal networks has gained increasing attention in recent years since it allows capturing users’ dynamic preferences and predicting their next actions. Recently, some approaches have been proposed to model user behavior. However, these methods suffer from two problems: they work on static data, which ignores the dynamic evolution, or they model the whole behavior sequences directly by recurrent neural networks and thus suffer from noisy information. To tackle these problems, we propose a dynamic user behavior learning algorithm called LDBR. It views user behaviors as a set of dynamic events and uses recent event embedding to predict future user behavior and infer the current semantic labels. Specifically, we propose a new strategy to automatically learn a good event embedding in behavior sequence by introducing a smooth sampling strategy and minimizing the temporal link prediction error. It is hard to obtain real-world datasets with evolving labels. Thus in this paper, we provide a new dynamic network dataset with evolving labels called Arxiv and make it publicly available. Based on the Arxiv dataset, we conduct a case study to verify the quality of event embedding. Extensive experiments on temporal link prediction tasks further demonstrate the effectiveness of the LDBR model.
Rumor Detection with Field of Linear and Non-Linear Propagation

The propagation of rumors is a complex and varied phenomenon. In the process of rumor dissemination, in addition to rumor claims, there will be abundant social context information surrounding the rumor. Therefore, it is vital to learn the characteristics of rumors in terms of both the linear temporal sequence and the non-linear diffusion structure simultaneously. However, in some existing research, time-dependent and diffusion-related information has not been fully utilized. Accordingly, in this paper, we propose a novel model Rumor Detection with Field of Linear and Non-Linear Propagation (RDLNP) to automatically detect rumors from the above two fields by taking advantage of claim content, social context and temporal information. First, the Rumor Hybrid Feature Learning (RHFL) we designed can extract the correlations between the claims and temporal information, differentiate the hybrid features of specific posts, and generate unified representations for rumors. Second, we proposed Non-Linear Structure Learning (NLSL) and Linear Sequence Learning (LSL) to integrate contextual features along the path of the diffusion structure and temporal engagement variation of responses respectively. Finally, Shared Feature Learning (SFL) models the representation reinforcement and learns the mutual influence between NLSL and LSL, and then highlights their valuable features. Experiments conduct on two public and widely used datasets, i.e. PHEME and RumorEval, demonstrate both the effectiveness and the outstanding performance of the proposed approach.
“Go eat a bat, Chang!”: On the Emergence of Sinophobic Behavior on Web Communities in the Face of COVID-19

The outbreak of the COVID-19 pandemic has changed our lives in unprecedented ways. In the face of the projected catastrophic consequences, most countries have enacted social distancing measures in an attempt to limit the spread of the virus. Under these conditions, the Web has become an indispensable medium for information acquisition, communication, and entertainment. At the same time, unfortunately, the Web is being exploited for the dissemination of potentially harmful and disturbing content, such as the spread of conspiracy theories and hateful speech towards specific ethnic groups, in particular towards Chinese people and people of Asian descent since COVID-19 is believed to have originated from China. In this paper, we make a first attempt to study the emergence of Sinophobic behavior on the Web during the outbreak of the COVID-19 pandemic. We collect two large datasets from Twitter and 4chan’s Politically Incorrect board (/pol/) over a time period of approximately five months and analyze them to investigate whether there is a rise or important differences with regard to the dissemination of Sinophobic content. We find that COVID-19 indeed drives the rise of Sinophobia on the Web and that the dissemination of Sinophobic content is a cross-platform phenomenon: it exists on fringe Web communities like /pol/, and to a lesser extent on mainstream ones like Twitter. Using word embeddings over time, we characterize the evolution of Sinophobic slurs on both Twitter and /pol/. Finally, we find interesting differences in the context in which words related to Chinese people are used on the Web before and after the COVID-19 outbreak: on Twitter we observe a shift towards blaming China for the situation, while on /pol/ we find a shift towards using more (and new) Sinophobic slurs.
TAKCO: A Platform for Extracting Novel Facts from Tables

Web tables contain a large amount of useful knowledge. Takco is a new large-scale platform designed for extracting facts from tables that can be added to Knowledge Graphs (KGs) like Wikidata. Focusing on achieving high precision, current techniques are biased towards extracting redundant facts, i.e., facts already in the KG. Takco aims to find more novel facts, still at high precision. Our demonstration has two goals. The first one is to illustrate the main features of Takco’s novel interpretation algorithm. The second goal is to show to what extent other state-of-the-art systems are biased towards the extraction of redundant facts using our platform, thus raising awareness on this important problem.
Cost-Effective and Interpretable Job Skill Recommendation with Deep Reinforcement Learning

Nowadays, as organizations operate in very fast-paced and competitive environments, workforce has to be agile and adaptable to regularly learning new job skills. However, it is nontrivial for talents to know which skills to develop at each working stage. To this end, in this paper, we aim to develop a cost-effective recommendation system based on deep reinforcement learning, which can provide personalized and interpretable job skill recommendation for each talent. Specifically, we first design an environment to estimate the utilities of skill learning by mining the massive job advertisement data, which includes a skill-matching-based salary estimator and a frequent itemset-based learning difficulty estimator. Based on the environment, we design a Skill Recommendation Deep Q-Network (SRDQN) with multi-task structure to estimate the long-term skill learning utilities. In particular, SRDQN recommends job skills in a personalized and cost-effective manner; that is, the talents will only learn the recommended necessary skills for achieving their career goals. Finally, extensive experiments on a real-world dataset clearly validate the effectiveness and interpretability of our approach.
L3i_LBPAM at the FinSim-2 task: Learning Financial Semantic Similarities with Siamese Transformers

In this paper, we present the different methods proposed for the FinSIM-2 Shared Task 2021 on Learning Semantic Similarities for the Financial domain. The main focus of this task is to evaluate the classification of financial terms into corresponding top-level concepts (also known as hypernyms) that were extracted from an external ontology. We approached the task as a semantic textual similarity problem. By relying on a siamese network with pre-trained language model encoders, we derived semantically meaningful term embeddings and computed similarity scores between them in a ranked manner. Additionally, we exhibit the results of different baselines in which the task is tackled as a multi-class classification problem. The proposed methods outperformed our baselines and proved the robustness of the models based on textual similarity siamese network.
FANCY: Human-centered, Deep Learning-based Framework for Fashion Style Analysis

Fashion style analysis is of the utmost importance for fashion professionals. However, it has an issue of having different style classification criteria that rely heavily on professionals’ subjective experiences with no quantitative criteria. We present FANCY (Fashion Attributes detectioN for Clustering stYle), a human-centered, deep learning-based framework to support fashion professionals’ analytic tasks using a computational method integrated with their insights. We work closely with fashion professionals in the whole study process to reflect their domain knowledge and experience as much as possible. We redefine fashion attributes, demonstrate a strong association with fashion attributes and styles, and develop a deep learning model that detects attributes in a given fashion image and reflects fashion professionals’ insight. Based on attribute-annotated 302,772 runway fashion images, we developed 25 new fashion styles (FANCY dataset 1). We summarize quantitative standards of the fashion style groups and present fashion trends based on time, location, and brand.