Towards Zero-shot Relation Extraction in Web Mining: A Multimodal Approach with Relative XML Path (2023)

Abstract: The rapid growth of web pages and the increasing complexity of their structure poses a challenge for web mining models. Web mining models are required to understand semi-structured web pages, particularly when little is known about the subject or template of a new page. Current methods migrate language models to web mining by embedding the XML source code into the transformer or encoding the rendered layout with graph neural networks. However, these approaches do not take into account the relationships between text nodes within and across pages. In this paper, we propose a new approach, ReXMiner, for zero-shot relation extraction in web mining. ReXMiner encodes the shortest relative paths in the Document Object Model (DOM) tree of the web page which is a more accurate and efficient signal for key-value pair extraction within a web page. It also incorporates the popularity of each text node by counting the occurrence of the same text node across different web pages. We use contrastive learning to address the issue of sparsity in relation extraction. Extensive experiments on public benchmarks show that our method, ReXMiner, outperforms the state-of-the-art baselines in the task of zero-shot relation extraction in web mining.

## introduction the internet is a vast repository of semi-structured web pages that are characterized by the use of html/xml markup language. compared to plain text in traditional natural language understanding tasks, these web pages possess additional multimodal features such as the semi-structured visual and layout elements from the html/xml source code. these features can be effectively generalized across different websites and provide a richer understanding of the web pages @xcite @xcite . the dynamic nature of the modern internet poses significant challenges for web mining models due to its rapid pace of updates. it is infeasible to annotate emerging web pages and train targeted models for them. modern web mining models are expected to perform zero-shot information extraction tasks with little prior knowledge of emerging subjects or templates @xcite . in this context, the multimodal features extracted from the html/xml source code as well as the textual contents are crucial for dealing with zeroshot information extraction task on the countless emerging web pages. previous approaches to the problem of zero-shot web mining have primarily focused on creating rich representations through large-scale multimodal pretraining, utilizing xml paths of text nodes 1 (lin et al., 2020; zhou et al., 2021; li et al., 2022b). as shown in figure 1 , xml paths are sequences of tags (e.g., div, span, li) indicating the location of the text node in the dom tree foot_1 of the page. these pre-training approaches extend vanilla language models by embedding the absolute xml paths but fail to take into account the relative local relationship expressed by the relative xml paths. the related nodes tend to be close to each other in the dom tree, which results in a long common prefix in their xml paths, as shown in figure 1 . such local relation is more common than the absolute xml path patterns. therefore, it is easy to transfer the relative xml paths to new web pages, and the relative xml paths serve as a more efficient and meaningful signal in predicting the relation between text nodes. additionally, existing web mining approaches tend to treat each web page separately and focus on memorizing their various templates, ignoring the fact that the relevance across different web pages of the same website is also meaningful to identify the related text nodes @xcite @xcite . intuitively, a text node is more likely to be a key word if it appears frequently in a collection of web pages and its surrounding words are not fixed. for example, in web pages about nba players, the statistics about the height, age are common text fields in the player introduction, so the text nodes, such as "height:" and "age:" should appear more frequently than other text nodes and the surrounding text contents should be different. in light of the aforementioned challenges in web mining, we propose a web mining model with relative xml path, rexminer, for tackling the zero-shot relation extraction task from semistructured web pages. our approach aims to learn the local relationship within each web page by exploiting the potential of the dom tree. specifically, we extract the shortest path between text nodes in the dom tree as the relative xml path, which removes the common prefix in the xml paths. inspired by the relative position embedding in t5 @xcite , we then embed the relative xml paths as attention bias terms in the multi-layered transformer. additionally, we incorporate the popularity of each text node by counting the number of times it occurs across different web pages, and embed the occurrence logarithmically in the embedding layer. furthermore, we address the data sparsity issues in the relation extraction task by adopting contrastive learning during training which is widely used in related works @xcite @xcite . we randomly generate negative cases and restrict their ratio to the positive ones, allowing the model to properly discriminate related node pairs from others. by learning from the relationships between text nodes within and across pages, rexminer is able to effectively transfer knowledge learned from existing web pages to new ones. we validate our approach on web pages from three different verticals from the swde dataset @xcite , including movie, university, and nba. the relation labels are annotated by @xcite . we summarize our contribution as follows. ## related work information extraction in web mining how to efficiently and automatically gathering essential information from the internet is always a hot topic in the academia of natural language processing and data mining due to the enormous scale and vast knowledge within the internet. the open information extraction task in web mining is originally proposed by @xcite and further developed by following works, including @xcite al. (2011); bronzi et al. (2013); mausam (2016 shot relation extraction in web mining proposed by zeroshotceres @xcite where the models are required to transfer relation knowledge from the existing verticals to the unseen ones. ze-roshotceres adopts the graph neural network to understand the textual contents and model the layout structure. it finally produces rich multimodal representation for each text node and conduct binary classification to extract related pairs. layout-aware multimodal transformers the pre-trained language models, such as bert (kenton and toutanova, 2019), xlnet @xcite , gpt @xcite , t5 @xcite , are revolutionary in the academia of natural language processing. it achieves state-of-the-art performance in text-only tasks. to further deal with multimodal scenarios, various features are extracted and incorporated into the transformer framework. recent study has shown that it is beneficial to incorporate multimodal features, such as bounding box coordinates and image features,into pre-trained language models to enhance overall performance in understanding visually-rich documents @xcite @xcite @xcite . similarly, web pages are rendered with html/xml markup language and also represent layout-rich structures. multimodal features from the dom tree or rendered web page images are incorporated in the pretrained language models to solve the tasks in the semi-structured web pages @xcite @xcite . ## problem formulation the zero-shot relation extraction in web mining is to learn knowledge of related pairs in the existing web pages and transfer the knowledge to the unseen ones @xcite . the unseen web pages should be orthogonal to the existing ones with regard to vertical, topic, and template. the zero-shot setting requires the web mining models to extract relevant pairs based on both the textual content and the dom tree structure of web pages. specifically, each web page is denoted as a sequence of text nodes, @xmath0, x 2 , ..., x n ], where n is the number of nodes in the page. each node involves textual contents and the xml path extracted from the dom tree, x @xmath1, xpath i ). the goal of the zero-shot relation extraction task is to train a model using related pairs, (x @xmath2, from a set of web pages, and subsequently extract related pairs from unseen ones. for example, as shown in figure 2 , one of our tasks is to train models with web pages from movie and nba verticals and test the models with web pages from the university vertical. ## methodology we extend the text-only language models with multimodal features and propose a novel framework, rexminer, for zero-shot relation extraction task in web mining. figure 3 shows the components in our framework. we adopt the absolute xml path embedding in markuplm @xcite , and further extend it with popularity embedding and relative xml path attention. to cope with the sparsity issue in the relation extraction task, we adopt the contrastive learning strategy where we conduct negative sampling to control the ratio between positive cases and negative cases. ## experiments we conduct experiments and ablation study of zeroshot relation extraction on the websites of different verticals from the swde dataset following the problem settings proposed in @xcite . ## conclusion and future work in this paper, we present rexminer, a web mining model to solve the zero-shot relation extraction task from semi-structured web pages. it benefits from the proposed features, the relative xml paths extracted from the dom tree and the popularity of text nodes among web pages from the same website. specifically, based on markuplm, and we further incorporate the relative xml paths into the attention layers of transformer framework as bias terms and embed the popularity of text nodes in the embedding layer. to solve the relation ex-traction task, we append the backbone model with the contrastive learning module and use the negative sampling to solve the sparsity issue of the annotation. in this way, rexminer can transfer the knowledge learned from the existing web pages to the unseen ones and extract the related pairs from the unseen web pages. experiments demonstrate that our method can achieve the state-of-the-art performance compared with the strong baselines. for future work, we plan to explore the new problem settings with limited supervision, such as few-shot learning and distant supervision, and further study the topological structure information in the dom tree to explore more meaningful signals in understanding the semi-structured web pages in web mining tasks.