Reviewerly/exHarmony huggingface dataset:
PromptORE - A Novel Approach Towards Fully Unsupervised Relation Extraction

Unsupervised Relation Extraction (RE) aims to identify relations between entities in text, without having access to labeled data during training. This setting is particularly relevant for domain specific RE where no annotated dataset is available and for open-domain RE where the types of relations are a priori unknown. Although recent approaches achieve promising results, they heavily depend on hyperparameters whose tuning would most often require labeled data. To mitigate the reliance on hyperparameters, we propose PromptORE, a "Prompt-based Open Relation Extraction" model. We adapt the novel prompt-tuning paradigm to work in an unsupervised setting, and use it to embed sentences expressing a relation. We then cluster these embeddings to discover candidate relations, and we experiment different strategies to automatically estimate an adequate number of clusters. To the best of our knowledge, PromptORE is the first unsupervised RE model that does not need hyperparameter tuning. Results on three general and specific domain datasets show that PromptORE consistently outperforms state-of-the-art models with a relative gain of more than 40% in B3, V-measure and ARI. Qualitative analysis also indicates PromptORE's ability to identify semantically coherent clusters that are very close to true relations.
Domain-Agnostic Contrastive Representations for Learning from Label Proportions

We study the weak supervision learning problem of Learning from Label Proportions (LLP) where the goal is to learn an instance-level classifier using proportions of various class labels in a bag -- a collection of input instances that often can be highly correlated. While representation learning for weakly-supervised tasks is found to be effective, they often require domain knowledge. To the best of our knowledge, representation learning for tabular data (unstructured data containing both continuous and categorical features) are not studied. In this paper, we propose to learn diverse representations of instances within the same bags to effectively utilize the weak bag-level supervision. We propose a domain agnostic LLP method, called "Self Contrastive Representation Learning for LLP" (SelfCLR-LLP) that incorporates a novel self--contrastive function as an auxiliary loss to learn representations on tabular data for LLP. We show that diverse representations for instances within the same bags aid efficient usage of the weak bag-level LLP supervision. We evaluate the proposed method through extensive experiments on real-world LLP datasets from e-commerce applications to demonstrate the effectiveness of our proposed SelfCLR-LLP. In this paper, we propose to learn diverse representations of instances within the same bags to effectively utilize the weak bag-level supervision. We propose a domain agnostic LLP method, called "Self Contrastive Representation Learning for LLP" (SelfCLR-LLP) that incorporates a novel self--contrastive function as an auxiliary loss to learn representations on tabular data for LLP. We show that diverse representations for instances within the same bags aid efficient usage of the weak bag-level LLP supervision. We evaluate the proposed method through extensive experiments on real-world LLP datasets from e-commerce applications to demonstrate the effectiveness of our proposed SelfCLR-LLP.
M <scp>entor</scp> GNN

Graph pre-training strategies have been attracting a surge of attention in the graph mining community, due to their flexibility in parameterizing graph neural networks (GNNs) without any label information. The key idea lies in encoding valuable information into the backbone GNNs, by predicting the masked graph signals extracted from the input graphs. In order to balance the importance of diverse graph signals (e.g., nodes, edges, subgraphs), the existing approaches are mostly hand-engineered by introducing hyperparameters to re-weight the importance of graph signals. However, human interventions with sub-optimal hyperparameters often inject additional bias and deteriorate the generalization performance in the downstream applications. This paper addresses these limitations from a new perspective, i.e., deriving curriculum for pre-training GNNs. We propose an end-to-end model named MentorGNN that aims to supervise the pre-training process of GNNs across graphs with diverse structures and disparate feature spaces. To comprehend heterogeneous graph signals at different granularities, we propose a curriculum learning paradigm that automatically re-weighs graph signals in order to ensure a good generalization in the target domain. Moreover, we shed new light on the problem of domain adaption on relational data (i.e., graphs) by deriving a natural and interpretable upper bound on the generalization error of the pre-trained GNNs. Extensive experiments on a wealth of real graphs validate and verify the performance of MentorGNN.
Approximate and Interactive Processing of Aggregate Queries on Knowledge Graphs: A Demonstration

This paper demonstrates AGQ [26] - our system for approximate and interactive processing of aggregate queries on knowledge graphs (KGs), e.g., "what is the average price of cars produced in Germany?" One can support aggregate queries based on factoid queries, e.g., "find all cars produced in Germany", by applying an aggregate operation on factoid queries' answers. However, this straightforward method is problematic since both the accuracy and efficiency of factoid query processing would impact the performance of aggregate queries. Moreover, returning a one-time, exact result might add computation overhead and hinder users' engagement and interactivity.
Hierarchical Representation for Multi-view Clustering

Multi-view clustering (MVC) aims at exploiting the consistent features within different views to divide samples into different clusters. Existing subspace-based MVC algorithms usually assume linear subspace structures and two-stage similarity matrix construction strategies, thereby posing challenges in imprecise low-dimensional subspace representation and inadequacy of exploring consistency. This paper presents a novel hierarchical representation for MVC method via the integration of intra-sample, intra-view, and inter-view representation learning models. In particular, we first adopt the deep autoencoder to adaptively map the original high-dimensional data into the latent low-dimensional representation of each sample. Second, we use the self-expression of the latent representation to explore the global similarity between samples of each view and obtain the subspace representation coefficients. Third, we construct the third-order tensor by arranging multiple subspace representation matrices and impose the tensor low-rank constraint to sufficiently explore the consistency among views. Being incorporated into a unified framework, these three models boost each other to achieve a satisfactory clustering result. Moreover, an alternating direction method of multipliers algorithm is developed to solve the challenging optimization problem. Extensive experiments on both simulated and real-world multi-view datasets show the superiority of the proposed method over eight state-of-the-art baselines.
CAVE: Correcting Attribute Values in E-commerce Profiles

Attribute value extraction from product profiles is essential for many applications such as product retrieval, comparison, and recommendation. While existing techniques focus mainly on the extraction task, none of them deals with the problem of correcting wrong attribute values. In this paper we propose CAVE, a novel system for attribute correction and enrichment using the Question Answering (QA) paradigm. CAVE learns information from both titles and attribute tables, using encoder and language models to correct attribute values. It also has the capability to enrich existing product descriptions with new attribute values extracted from titles. To the best of our knowledge, CAVE is the first system that allows users to experiment with a number of powerful QA models and compare their performances on attribute values correction using real-word datasets.
Decoupled Hyperbolic Graph Attention Network for Modeling Substitutable and Complementary Item Relationships

Modeling substitutable and complementary item relationships is a fundamental and important topic for recommendation in e-commerce online scenarios. In the real world, item relationships are usually coupled, heterogeneous and they also have abundant side information and hierarchical data structures. Recently, to take full advantage of both sides information and topological structure, graph neural networks are widely explored in relationship modeling. However, the existing methods are crude in decoupling heterogeneous relationships. Their model designs lack deep insight of relationships' coupling mode, i.e. neglects the prior knowledge of how relationships affect each other. In addition, many existing graph methods, regardless of how they handle coupled relationships, are deployed in Euclidean spaces, which distorts hierarchical data structure and limits the expressive power due to the non power law characteristic of Euclidean topology. In this paper, we propose a novel Decoupled Hyperbolic Graph Attention Network (DHGAN). The innovations of our DHGAN can be highlighted as two aspects. Firstly, we design metapaths in an adequate way following an algebraic perspective of relationships coupling mode, which helps achieving better model interpretability. Secondly, DHGAN maps heterogeneous relationships into separate hyperbolic spaces, which can better capture the hierarchical information of graph nodes and helps improving model's representational capacity. We conduct extensive experiments on three public real-world datasets, demonstrating DHGAN is superior to the state-of-the-art graph baselines. We release the codes at https://github.com/wt-tju/DHGAN.
HQANN: Efficient and Robust Similarity Search for Hybrid Queries with Structured and Unstructured Constraints

The in-memory approximate nearest neighbor search (ANNS) algorithms have achieved great success for fast high-recall query processing, but are extremely inefficient when handling hybrid queries with unstructured (i.e., feature vectors) and structured (i.e., related attributes) constraints. In this paper, we present HQANN, a simple yet highly efficient hybrid query processing framework which can be easily embedded into existing proximity graph-based ANNS algorithms. We guarantee both low latency and high recall by leveraging navigation sense among attributes and fusing vector similarity search with attribute filtering. Experimental results on both public and in-house datasets demonstrate that HQANN is 10x faster than the state-of-the-art hybrid ANNS solutions to reach the same recall quality and its performance is hardly affected by the complexity of attributes. It can reach 99% [email protected] in just around 50 microseconds On GLOVE-1.2M with thousands of attribute constraints.
ChiQA

Visual question answering is an important task in both natural language and vision understanding. However, in most of the public visual question answering datasets such as VQA, CLEVR, the questions are human generated that specific to the given image, such as 'What color are her eyes?'. The human generated crowdsourcing questions are relatively simple and sometimes have the bias toward certain entities or attributes [1, 55].
Global and Local Feature Interaction with Vision Transformer for Few-shot Image Classification

Image classification is a classical machine learning task and has been widely used. Due to the high costs of annotation and data collection in real scenarios, few-shot learning has become a vital technique to improve image classification performances. However, most existing few-shot image classification methods only focus on modeling the global image feature or image local patches, which ignore the global-local interactions. In this study, we propose a new method, named GL-ViT, to integrate both global and local features to fully exploit the few-shot samples for image classification. Firstly, we design a feature extractor module to calculate the interactions between the global representation and local patch embeddings, where ViT is also adopted to achieve efficient and effective image representation. Then, Earth Mover's Distance is adopted to measure the similarity between two images. Abundant Experimental results on several widely-used open datasets show that GL-ViT outperforms state-of-the-art algorithms significantly, and our ablation studies also verify the effectiveness of both global-local features.
Causal Relationship over Knowledge Graphs

Causality has been discussed for centuries, and the theory of causal inference over tabular data has been broadly studied and utilized in multiple disciplines. However, only a few works attempt to infer the causality while exploiting the meaning of the data represented in a data structure like knowledge graph. These works offer a glance at the possibilities of causal inference over knowledge graphs, but do not yet consider the metadata, e.g., cardinalities, class subsumption and overlap, and integrity constraints. We propose CareKG, a new formalism to express causal relationships among concepts, i.e., classes and relations, and enable causal queries over knowledge graphs using semantics of metadata. We empirically evaluate the expressiveness of CareKG in a synthetic knowledge graph concerning cardinalities, class subsumption and overlap, integrity constraints. Our initial results indicate that CareKG can represent and measure causal relations with some semantics which are uncovered by state-of-the-art approaches.
UDM: A Unified Deep Matching Framework in Recommender Systems

Due to the large-scale users and items, industrial recommender systems usually consist of two stages, the matching stage and the ranking stage. The matching stage is responsible for retrieving a small fraction of relevant items from the large-scale item pool which are further selected by the ranking stage. Most of the existing deep learning-based matching models focus on the problem of modeling user interest representation by using inner product between user representation and item representation to obtain the user-to-item relevance. However, the item-to-item relevance between user interacted item and target item is not considered in the deep matching models which is computationally prohibitive for large-scale applications. In this paper, we propose a unified deep matching framework called UDM for the matching stage to mitigate this issue. UDM can model the user-to-item relevance and item-to-item relevance simultaneously with the help of an interest extraction module and interest interaction module, respectively. Specifically, the interest extraction module is used as the main network to extract users' multiple interests with multiple vectors based on users' behavior sequences, while the interest interaction module is used as an auxiliary network to supervise the learning of the interest extraction module, which can model the interaction between user interacted items and target item. In the experiments conducted on two public datasets and a large-scale industrial dataset, UDM achieves consistent improvements over state-of-the-art models. Moreover, UDM has been deployed in the operational system of Alibaba. Online A/B testing results further reveal the effectiveness of UDM. To the best of our knowledge, UDM is the first deep matching framework which combines the user-to-item relevance modeling and item-to-item relevance modeling in the same model.
Modelling Human Word Learning and Recognition Using Visually Grounded Speech

Abstract Many computational models of speech recognition assume that the set of target words is already given. This implies that these models learn to recognise speech in a biologically unrealistic manner, i.e. with prior lexical knowledge and explicit supervision. In contrast, visually grounded speech models learn to recognise speech without prior lexical knowledge by exploiting statistical dependencies between spoken and visual input. While it has previously been shown that visually grounded speech models learn to recognise the presence of words in the input, we explicitly investigate such a model as a model of human speech recognition. We investigate the time course of noun and verb recognition as simulated by the model using a gating paradigm to test whether its recognition is affected by well-known word competition effects in human speech processing. We furthermore investigate whether vector quantisation, a technique for discrete representation learning, aids the model in the discovery and recognition of words. Our experiments show that the model is able to recognise nouns in isolation and even learns to properly differentiate between plural and singular nouns. We also find that recognition is influenced by word competition from the word-initial cohort and neighbourhood density, mirroring word competition effects in human speech comprehension. Lastly, we find no evidence that vector quantisation is helpful in discovering and recognising words, though our gating experiment does show that the LSTM-VQ model is able to recognise the target words earlier.
Bounding box regression with balance for harmonious object detection

Localization is an essential part of object detection, which is usually accomplished by bounding box regression guided by ℓn-norm-based or IoU-based loss functions, where IoU is known for its scale-invariant characteristics. However, introducing the scale-invariance into regression loss in traditional IoU-based methods may result in a bias in favor of smaller boxes and cause redundancy and unstable oscillations. To make up for these shortages of IoU-based losses, we propose a Scale-Balanced Factor (SF) that stabilizes the regression process via a simple adaptive factor. Furthermore, to compensate for the imbalance of different types of losses caused by SF and other IoU-based loss functions, regression losses are always multiplied by a hyperparameter, which is purely empirical and is hard to find an optimum. To address this issue, a Multi-Task Reinforced Equilibrium (MRE) is proposed to dynamically tweak the learning rate of each task based on reinforcement learning. The MRE can guarantee more balanced parameters and maximize the benefit of SF or other improvement methods for IoU. By incorporating the proposed SF and MRE into the classic detectors (RetinaNet, YOLO, and Faster R-CNN, etc.), we have achieved significant performance gains on MS COCO (0.8 AP∼1.9 AP) and PASCAL VOC (0.6 AP∼2.2 AP).
Contour enhanced image super-resolution

Recently, very deep convolution neural network (CNN) has shown strong ability in single image super-resolution (SISR) and has obtained remarkable performance. However, most of the existing CNN-based SISR methods rarely explicitly use the high-frequency information of the image to assist the image reconstruction, thus making the reconstructed image looks blurred. To address this problem, a novel contour enhanced Image Super-Resolution by High and Low Frequency Fusion Network (HLFN) is proposed in this paper. Specifically, a contour learning subnetwork is designed to learn the high-frequency information, which can better learn the texture of the image. In order to reduce the redundancy of the contour information learned by the contour learning subnetwork during fusion, the spatial channel attention block (SCAB) is introduced, which can select the required high-frequency information adaptively. Moreover, a contour loss is designed and it is used with the ℓ1 loss to optimize the network jointly. Comprehensive experiments demonstrate the superiority of our HLFN over state-of-the-art SISR methods.
Measurements and Correctness Criteria for Determining the Line Length of the Data Image Obtained in the Process of Electromagnetic Infiltration

The protection of information against electromagnetic penetration is one of the most important aspects related to the protection of information against its non-invasive acquisition. Compared to the activities of cybercriminals, the use of electromagnetic emissions in the electromagnetic infiltration process does not leave any traces of activity, and the owner of the information is not aware of its loss. The most common activities of electromagnetic eavesdropping are related to the infiltration of emission sources, graphically revealing the processing of information using both analog and digital methods. This allows for the presentation of reconstructed data in the form of images. Correct display of the acquired information requires knowledge of raster parameters such as line length and the number of lines building the reconstructed image. Due to the lack of direct access to the intercepted device, knowledge in this field does not allow for the correct determination of the aforementioned parameters, and thus, for recreating an image that would contain legible and understandable data. Additionally, incorrect values of the parameters result in failure of further processing of the obtained image, e.g., by using a coherent summation of images. Therefore, it is necessary to propose a solution that will allow not so much to roughly define the raster parameters but to estimate them precisely. Moreover, it should enable the automation of the process after the implementation of an appropriate algorithm. The article proposes an algorithm for estimating the line length of the reconstructed image. The raster parameter estimated with the use of the algorithm allows for summarizing images several dozen times with a significant improvement in the image quality and readability of the data contained in it. The image summation algorithm is very often used as one of the main image processing methods in the electromagnetic infiltration process. Incorrect raster parameters often make coherent summation useless. The proposed algorithm for estimating the line length of the reconstructed image uses three methods of determining the line length of the image for a given accuracy. At the same time, criteria were indicated that must be met to determine the correct length of the image line for the assumed accuracy of estimation. Obtained results confirmed that the proposed methods and criteria are effective in the process of electromagnetic infiltration. These methods allow us to determine the line length of reconstructed images with accuracy up to 10−5.