AtTGen: Attribute Tree Generation for Real-World Attribute Joint Extraction
Year: 2023

Abstract:
Attribute extraction aims to identify attribute names and the corresponding values from descriptive texts, which is the foundation for extensive downstream applications such as knowledge graph construction, search engines, and e-Commerce. In previous studies, attribute extraction is generally treated as a classification problem for predicting attribute types or a sequence tagging problem for labeling attribute values, where two paradigms, i.e., closed-world and open-world assumption, are involved. However, both of these paradigms have limitations in terms of real-world applications. And prior studies attempting to integrate these paradigms through ensemble, pipeline, and co-training models, still face challenges like cascading errors, high computational overhead, and difficulty in training. To address these existing problems, this paper presents Attribute Tree, a unified formulation for realworld attribute extraction application, where closed-world, open-world, and semi-open attribute extraction tasks are modeled uniformly. Then a text-to-tree generation model, AtTGen, is proposed to learn annotations from different scenarios efficiently and consistently. Experiments demonstrate that our proposed paradigm well covers various scenarios for real-world applications, and the model achieves state-ofthe-art, outperforming existing methods by a large margin on three datasets. Our code, pretrained model, and datasets are available at https://github.com/lsvih/AtTGen.

heading: 1 Introduction
text: Attribute Extraction (AE) is a practical application of the Information Extraction (IE) task, aiming to identify the attribute name and the corresponding attribute value from unstructured or semistructured text fragments (Ghani et al., 2006; Ravi and Pasca, 2008; More, 2016). Figure 1 shows a typical product profile with extracted attribute tags.
∗Corresponding Author
As the foundation for various downstream applications such as knowledge graph construction, search engines, e-Commerce and recommender systems, AE has attracted extensive research interest in recent years (Zheng et al., 2018; Xu et al., 2019; Zhu et al., 2020; Jain et al., 2021; Zhang et al., 2022; Li and Zou, 2022).
There are two basic subtasks in the research of AE, namely, attribute name extraction and attribute value extraction. And we use the RDF-style triple1 <e, n, v> to denote the entity, attribute name, and attribute value respectively. According to whether the attribute name set is pre-defined, AE can be divided into two paradigms, i.e., the Closed-World Assumption (CWA) and the Open-World Assumption (OWA). For CWA AE, the attribute name n is limited to a finite set of the pre-defined schema, where attribute name extraction is typically modeled as a classification task (Zeng et al., 2014; Zhou et al., 2016), and attribute value extraction models are trained for each target attribute (Zheng et al., 2018; Zhu et al., 2020; Yan et al., 2021). While for OWA AE, which is also known as “New Attribute Discover” (Wong and Lam, 2010; Zhang et al., 2022) and “Open Information Extraction” (Cui et al., 2018), the attribute name is schema-free and can be extracted from the text. Sequence tagging methods are broadly employed to extract those attributes (Xu et al., 2019). Recently, researchers
1https://www.w3.org/TR/n-triples/
2139
also explore novel paradigms such as Question Answering (QA) models (Wang et al., 2020; Shinzato et al., 2022; Yang et al., 2022) and generative models (Roy et al., 2022) to generalize the ability of attribute extraction.
However, AE in the real world is far more complicated. On the one hand, in closely related fields like e-commerce, new types of products with new sets of attributes are so constantly arising that the pre-defined schema is never enough. For example, an analysis in Zhang et al. (2022) has shown that only 30 / 51 attributes are found in existing structured product profiles of Amazon’s 10 product types. On the other hand, however, attribute extraction methods shouldn’t overlook the huge value and commonalities behind known attributes, and it is inherent that not all attributes can be fully identified by open extraction methods due to the lack of literal name mentions, e.g. name and size in Figure 1. It is possible to carry out both CWA and OWA methods when needed, just as Zhang et al. (2021) attempts preliminarily. But apart from the fragmentation of the problem form and the unnecessary computing overhead, a more prominent issue is that such simple integration neglects the natural connections between the CWA vocabulary and the OWA ability in attribute extraction, and thus cannot achieve satisfactory results. In this paper, we, for the first time, explicitly unify the different AE paradigms in the form of Attribute Tree, and present a text-to-tree based generative model called AtTGen to solve the real-world attribute joint extraction task.
Specifically, our proposed AtTGen successfully implements the unification of attribute tagging and classification tasks by generating the Attribute Tree, and congenitally circumvents the problem of “null”-value that troubles pioneers (Xu et al., 2019; Wang et al., 2020). Further, the head entity is optional as the root node on Attribute Tree to meet the actual situation, as well as to enhance the extraction performance with the help of the subject guidance (Yu et al., 2021; Zhang et al., 2021). AtTGen reduces the length of the generated sequence and thus shrinks the search space by conducting the tree generation model. And it can accurately mark out the span of attribute values and extract unseen attributes with the pointer-copy mechanism (Zhou et al., 2018). Moreover, the teacher forcing manner (Williams and Zipser, 1989) and the converted path-generation training objective further reduce the exposure bias (Zhang et al., 2020) to improve
the generalization and effectiveness. In short, the major contributions of this paper can be summarized as follows:
• We are the first to define different attribute extraction paradigms like CWA, OWA and semi-open as the attribute tree generation problem, formally unifying multiple tasks and fully capturing the internal connections.
• We design a novel text-to-attribute tree generation model with a pointer-based copy mechanism for extracting both literal mentions and category labels.
• We evaluate our model on several benchmark datasets. Experimental results show that our method achieves state-of-the-art (SOTA) and outperforms existing works by a large margin in all scenarios including open, semi-open and closedworld attribute extraction.

heading: 2 Preliminary
text: We first formalize the definition of two mainstream paradigms widely used in Attribute Extraction.
Definition 1 (Closed-World Assumption). CWA AE receives a descriptive text T = [t1, t2, ...], e.g. a product title, and a pre-defined schema A which contains a set of attributes (i.e., attribute vocabulary) to extract all attribute pairs <n, v> for a possibly given head entity e, where n ∈ A is the attribute name (also called attribute type), and v ∈ T is the attribute value extracted from the text.
Definition 2 (Open-World Assumption). OWA AE takes a descriptive text T = [t1, t2, ...] as input, and the target is to discover all attribute pairs <n, v> for a possibly given head entity e, where both the attribute name n and the attribute value v are from the given text, i.e. n ∈ T and v ∈ T .
As stated in Section 1, individual one of the above paradigms does not always work well in real-world applications, and the pipeline approach adopted by Zhang et al. (2021) to merge the results of the two paradigms would introduce problems such as cascading errors. Therefore, we propose a formal definition of real-world AE and its solution in the following sections.

heading: 3 Problem Formalization
text: Section 1 has expounded that attribute extraction in real-world applications sometimes needs both the
guidance of the schema and the ability to extract free attributes from texts. It is actually an extensive aggregation covering both CWA and OWA AE, as well as a semi-open scenario where attribute names can be obtained from both. Therefore we formally define the real-world attribute extraction as:
Definition 3 (Real-world Attribute Extraction). Given a text T , and an optional A, “real-world AE” is to fill the explicit slots for the optional category in A, or to dig more free attributes from T , or to capture attributes from both A and T . i.e., the final result of real-world AE is a set of attribute pairs <n, v> where v ∈ T , n ∈ H = {A,∅} ∪ {T ,∅} and H ̸= ∅.
To implement such an extraction paradigm uniformly, we devise a principled structure, Attribute Tree, to formally model the target of all real-world AE circumstances:
Definition 4 (Attribute Tree). An attribute tree T for a descriptive sentence sent is an unweighted tree with a fixed height h = 2. All the branches of the tree T have a determined order (r, v, n), and the root r is the only entry node that can be either empty ∅ or the head entity (also called the subject) subj of the attributes.
Figure 2 visualizes the attribute tree and its instances. The path from the root to the leaves is also the reasoning path of the proposed model. Borrowing the notation from epistemology (Martin-Löf, 1996), there are:
{sent, r} ⊢ v {sent, r, v} ⊢ n r ∈ {∅, subj} (1)
which means the attribute value v is derived from the original sentence sent and the root node r; and the attribute name n, whether coming from the input text or the given schema, can be predicted by the integrated information from the sentence, the attribute value, and the root node. This kind of path order can naturally evade the insignificant
“NULL” value problem pointed out by Shinzato et al. (2022).
Definition 5 (Subject Guidance). Setting the subject subj of a descriptive sentence sent as the root node r of the corresponding attribute tree T when available, i.e. let r = subj in Equation 1, is called enabling the subject guidance.
As attributes typically characterize entities and are strongly bound to the subject, we naturally introduce the subject guidance for AE in such a way and the effectiveness has been preliminarily demonstrated in Yu et al. (2021); Zhang et al. (2021).

heading: 4 Methodology
text: We design a unified tree generative model AtTGen, committing to jointly extracting attribute names and values under various scenarios in the real world. It is partially inspired by the success of Seq2Tree models (Dong and Lapata, 2016; Liu et al., 2019; Zhang et al., 2020) and pointer-copy based spanselector (Zhou et al., 2018; Ma et al., 2022) in other tasks. The overall architecture is shown in Figure 3, and we demonstrate the model details in the following subsections.

heading: 4.1 Encoder
text: We employ the classical BiLSTM-CNN (Chiu and Nichols, 2016) neural network to encode the input text into a continuous latent space2. Given a sequence input [t1, t2, ..., tn], the encoded text representation ht ∈ Rm×n is obtained by:
ht = Encoder(sent)
= Convenc(BiLSTMenc(Emb(sent)) (2)
in which Emb is to gain the embedded vector of tokens from the lookup table and m is the dimension of the embedding, BiLSTMenc is Bidirectional Long Short-Term Memory network (Hochreiter and Schmidhuber, 1997) for modeling the dependencies of the input sequence, and Convenc is Convolutional Network (Collobert et al., 2011) for extracting features from the encoded text representation. Meanwhile, the category labels of attribute names from the given schema also contain useful semantic information for generating the attribute tree, thus we use the same encoder to obtain the label representation of the attribute names as:
hl = Encoder(labels) (3) 2Adapting PLMs to AtTGen is discussed in Section 8
Then we can concatenate the two parts and get the initial root node representation as hr = Encoder([sent||labels]), which allows the successor decoders to uniformly generate nodes from both the input sentence and the category label set.
In addition, the subject of the attribute would be concatenated with the input sentence as [⟨subject⟩, [sep], t1, ..., tn] for the subject guidance, in which [sep] is a separator token.

heading: 4.2 Tree Decoder
text: The decoding target of our method is to generate a structured attribute tree. As a tree can be divided into several paths from the root node to the leaf node, the generation of a tree can also be decomposed into the problem of generating multiple paths. Therefore, the decoder of AtTGen is denoted as:
rs,hrs, st = Decoder(T,hp, st−1) (4)
where rs is the generated result, hrs is the representation of the decoded tokens, st and st−1 are the current and the previous state of the decoder respectively. Each decoding step relies on several inputs: (1) the target space of decoding T, which is to limit the selection range of the final result of the decoder and thus shrinks the search space; (2) the representation of the antecedent path hp; (3) the state of the decoder st, used to determine the currently decoded node is at what level of the attribute tree.
Specifically, given the input hp and the previous decoding state st−1, a unary LSTM is employed for decoding the state st as:
st = LSTMdec(hp, st−1) (5)
The decoding feature hrs for generating results is obtained by a convolutional network Convdec with an attention-based weighted sum like (Bahdanau et al., 2015) as:
hrs = Convdec(Att(ht, st)) (6)
Then the final result as follows is decoded from the pointer-based span copier (Ptr) explained in Section 4.3:
istart, iend = Ptrs(hrs),Ptre(hrs)
rs = T[istart : iend] (7)
The whole decoding process for AtTGen is described in Algorithm 1.
Algorithm 1: Attribute Tree Decoder Input :A descriptive sentence:sent
A category set from flattened schema:labels Output :The attribute tree of sent // Decoding attributes from plain text and
pre-defined schema jointly. 1 hr ←Encoder([sent∥labels]) 2 if use subject guidance then 3 r, hr, sr ←Decoder(sent, hr,∅) 4 root←Tree(r) 5 else 6 sr ← ∅ 7 root←Tree(placeholder) 8 v,hv, sv ←Decoder(sent, hr, sr) 9 for v, hv in v,hv do
10 hv = hr ⊕ hv 11 n,hn, sn ←Decoder([sent∥labels], hv, sv) 12 for n, hn in n,hn do 13 if v /∈ root.children() then 14 root.add_child(v) 15 root.find_child(v).add_child(n) 16 return root
where ∅ is a randomly initialized vector to represent the initial decoding state. r, hr and sr are the
decoder’s output for the root node (the optional subject), representing the generated result, the hidden representation and the current state respectively. Similarly, (v, hv, sv) and (n, hn, sn) are the other two sets of outputs from the decoder, for the decoding process of attribute values and attribute names respectively. Note that if subject guidance is enabled, the decoder will update hr by decoding subject firstly, and construct the root node of the tree (Line 2-4), otherwise the root node is replaced by a placeholder (Line 5-7). The attribute values and attribute names are sequentially decoded in the order of Equation 1 to construct Attribute Tree as shown in Line 8-15 in Algorithm 1.

heading: 4.3 Span Copier
text: We propose to use a unified span copier to ensure the spans are correctly copied from the original sentence or the label set during the decoding process.
Ptrs(h) = σ(Wsh+ bs) Ptre(h) = σ(Weh+ be)
(8)
in which Ws and We are trainable weights, bs and be are trainable bias, h denotes the hidden state of the current decoding step, and σ is the sigmoid active function. The Ptr(·) produces a constant vector that denotes the start/end index of the copied span. For those nodes in the closedworld setting whose mention does not exist in the original text (e.g., name, size, and price in Figure 1), we further add an equality constraint Ptrs = Ptre, restricting the pointers to select only one category label when decoding from the label set, which reduces generative errors and improves the training efficiency.

heading: 4.4 Training Objective
text: In the decoding process, we apply teacher forcing manner (Williams and Zipser, 1989) for efficient training and encourage the model to reduce the distance of all paths between the generated tree and the ground truth:
Lpath = δ ∑
i∈{s,e} BCE(Ptri(hr), y∗i_r)
+ ∑
j∈{v,n}
∑
i∈{s,e} BCE(Ptri(hj), y∗i_j)
where δ ∈ {0, 1} indicates whether to enable the subject guidance; y∗s_(·)/y ∗ e_(·) denotes the golden standard start/end index of either a literal mention
or a category label of the target span; h(·) represents the hidden state of the decoder to distinguish the level it is decoding. BCE is the Binary Cross Entropy loss to optimize the prediction of the index vectors individually for each step:
BCE(y, y∗) = − 1 N
N∑
i=1
y∗i ·ln yi+(1−y∗i )·ln(1−yi)
where N is the length of the input sentence, yi is the predicted probability of the i-th element and y∗i is the corresponding ground truth.

heading: 5 Experiments

heading: 5.1 Experimental Setup
text: Datasets. We conduct our experiments on three publicly available datasets to examine the capacity and the generality of our model over various realworld AE settings:
MEPAVE (Close-World Benchmark)3 (Zhu et al., 2020) is a multimodal e-Commerce product attribute extraction dataset, which contains 87k product description texts (in Chinese) and images, involving 26 types of attributes. We follow the same dataset settings as Zhu et al. (2020), except that we leave the visual information and use the description texts only.
AE-110K (Open-World Benchmark)4 (Xu et al., 2019) is a collection of 110k product triples (in English) from AliExpress with 2,761 unique attributes. It can well measure the open extraction ability and generation performance of different models. We split this dataset via the cleaning script of Shinzato et al. (2022), and remove invalid and “NULL” value attributes following Roy et al. (2022).
Re-CNShipNet (Semi-Open Benchmark) is a revised version of the functional attribute extraction dataset CNShipNet5 (Zhang et al., 2021), where numerical attributes account for the majority to bring new challenges. We manually fix the incorrect annotations in the old version and rebalance the ratio of closed- to open-setting labels (Li et al., 2021). Now it contains about 5k entity-attribute instances (mostly in Chinese), among which 40% obtain attributes from the literal texts and others are within 9 pre-defined attribute types. Baselines. We compare the proposed model with several strong and typical baselines including:
3https://github.com/jd-aig/JAVE 4https://github.com/lanmanok/ACL19_Scaling_Up_
Open_Tagging/blob/master/publish_data.txt 5https://github.com/lsvih/SOAE
1) Sequence Tagging-based methods, a kind commonly adopted in IE which typically uses semantic tags such as BIO to identify the extracted items: RNN-LSTM (Hakkani-Tür et al., 2016), Attn-BiRNN (Liu and Lane, 2016), and BiLSTMCRF (Huang et al., 2015) are all specially designed RNN-based models for modeling the intent of classification and extraction tasks. ScalingUp (Xu et al., 2019) is a BERT-based model to extract attribute values with BiLSTM to perform interaction attention between attribute names and values. 2) PLM-based methods: BERT (Devlin et al., 2019) is a well-known pre-trained language model (PLM) and we follow the vanilla setting of classification and sequence tagging tasks, JointBERT (Chen et al., 2019) is a variant of BERT to solve slot filling and classification jointly. 3) Joint IE-based (JE) methods, which originate from the entity-relation extraction task and typically extract entities and classify relations in a cascading fashion: ETL-Span (Yu et al., 2020) and CasRel (Wei et al., 2020) are two classic JE models for relation extraction and we adapt them to the AE task here. SOAE (Zhang et al., 2021) achieved SOTA on CNShipNet by merging the results of a JE model and a classification model. JAVE (Zhu et al., 2020) is an attention-based attribute joint extraction model and M-JAVE further takes advantage of multimodal information, and they were the best models for MEPAVE. 4) Sequence Generative Model: We also implement the latest word sequence generation method (Roy et al., 2022) based on the large-scale pre-trained BART (Lewis et al., 2020) model.
We conduct the baselines and adapt them to the target datasets accordingly. See Appendix A for implementation details. Metrics. Following previous works (Zheng et al., 2018; Xu et al., 2019; Zhu et al., 2020; Zhang et al., 2021), we use F1 score as the metric and adopt Exact Match criteria (Wei et al., 2020), in which only the full match to the ground truth is considered correct. We report the results of attribute name and value extraction respectively as Zhu et al. (2020).

heading: 5.2 Main Results
text: This section presents the overall results of the models over various AE scenarios in Table 1, 2, and 3. In general, we can observe that our model outperforms the baselines over all three scenarios in real-world AE.
As shown in Table 1, our model achieves a big improvement in the closed-world AE task. Even though the previous SOTA model (M-JAVE BERT version) introduces PLM and takes advantage of extra multimodal information (product images), we still gain a 9.09% improvement in attribute value extraction and 5.79% in attribute name prediction.
In the open setting shown in Table 2, AtTGen consistently performs well in attribute value extraction, with a 6.45% improvement than BART, an elaborate and dedicated PLM-based model. It has a slightly lower result compared with BART when extracting attribute names (0.86%), due to the absence of the semantic knowledge contained in the large-scale PLMs for efficiency issues. We will consider introducing such knowledge in future work, which we believe will further improve the performance. But the current results are still strong enough to demonstrate the open extraction capability of our model.
As for the semi-open scenario displayed in Table 3, our model again outperforms CasRel, a strong joint model in the information extraction field. We
also attain better results than SOAE, which was the SOTA on this dataset by conducting both OWA and CWA models. This can be credited to our unified attribute tree model to naturally capture the intrinsic connections in the partial-closed world.
It can be concluded that, as the first to design a tree generative model in AE, our method can be silkily adapted to different real-world scenarios at a small cost, and achieves remarkable results whether the dataset is in the e-Commerce domain (MEPAVE, AE-110K) or news (Re-CNShipNet), and whether the language of the datasets is English (AE-110K) or Chinese (MEPAVE and ReCNShipNet). Moreover, unlike quite many baselines relying on external knowledge in the largescale language models, we achieve outstanding results by training from scratch, and thus has a dominant advantage in the parameter-efficiency (e.g., BERT has ~110M parameters, BART has ~139M, AtTGen has only ~2M). We hypothesize that the superiority comes from the unified problem formalization as well as the novel tree generation model design. On the one hand, our model keeps the simplicity as a generation model, providing a unified way to capture the semantic associations between open and closed vocabulary, and between attribute names and values. On the other hand, different from traditional Seq2Seq models that decode all triples autoregressively into a linear sequence, our tree structure decomposes the decoding target into several paths of length three, removing the unnecessary order among different triplets and effectively alleviating the exposure bias problem in long-distance generation tasks (Zhang et al., 2020).
Furthermore, we notice that the performance of the models varies across different datasets, which can be attributed to the varying levels of complexity
and quality of the datasets. For example, MEPAVE is a well-annotated benchmark with only a small number of attribute types, hopefully for better results. While AE-110K suffers an inevitable longtail distribution problem, and Re-CNShipNet is limited by the data scale and the uncertain ratio of CWA/OWA labels, posing greater challenges and leading to the results that all models still have a large room for improvement.

heading: 5.3 Ablation Study
text: In this section, we carry out several ablation experiments to study the effectiveness of each subcomponent in AtTGen. The whole results are listed in Table 4 and we can find these phenomenons: 1) The performance reduces by 3.15% on ReCNShipNet dataset without the subject guidance, indicating the usefulness to exploit the constraint semantics of the subject in attribute extraction. Along with the findings in Yu et al. (2021); Zhang et al. (2021), we may conclude that subject guidance is a powerful enhancement in various information extraction situations. 2) We remove the span copier by replacing it with an ordinary token generator to extract values from the whole vocabulary. It can be seen that the performance drops by 8.75% on average, and the degradation is more evident in the open and semi-open settings, where the performances are down to the same level as other sequence tagging-based models. This proves that the advantage of the model largely comes from the copy mechanism to detect boundary information of the spans rather than directly modeling the attributes. We therefore say that span copier can play a prominent role in AE. 3) We also explore the influence of the generation order in Attribute Tree and the results show that changing the path order from (r, v, n) to (r, n, v) slightly reduces the effect (4.7% averagely). Somewhat different from a prior experiment conducted
in (Zhang et al., 2020), which shows that in entityrelation joint extraction task, relations should come first to get the best performance, our conclusion here is that attribute values should be extracted before attribute names, especially in open scenarios. One possible explanation for this difference between relation and attribute extraction is that attribute values typically have more evident patterns to trigger the following attribute name prediction. Besides, the path order of (r, v, n) is able to reduce the confusion of multifarious attribute names and well evades the “NULL” value problem. 4) Removing schema information directly deprives the model’s capacity to learn from the existing ontology, and significantly degrades its performance on the Re-CNShipNet dataset, showing that predefined schema can strengthen models’ applicability in real-world AE applications.
By these ablation studies, we have not only demonstrated that each delicate design in our model plays an important role, but proposed several interesting findings which we believe will shed some light for future research.

heading: 5.4 Case Study
text: We present two case studies from Re-CNShipNet dataset to further illustrate our proposed Attribute Tree and the effectiveness of AtTGen model, as shown in Figure 4. In the first case, the sentence contains an out-of-schema attribute, “sea trialed”, which is ignored by the BERT-based extraction model. While our AtTGen model, starting from a given subject, identifies all attribute pairs including the purely literal one by first listing all possible attribute values and then smoothly corresponding to names based on the value and the context. In the other case, the number “158,700” is misextracted as “700” by the Bert-based extractor due to the interference of the thousands-separator. This roots in the model’s failure to really understand numerical values, which is a unique challenge to deep learning-based techniques (Xue et al., 2022). Nonetheless, AtTGen directly captures the boundary pattern of numbers and successfully retains the complete value with the span copier, showing a possible solution for this challenge.

heading: 6 Related Works
text: Attribute Extraction is a classical IE task with extensive research. In earlier years, heuristic rules and dictionaries were usually used to iden-
tify attributes and extract attribute values from the texts (Tan et al., 1999; Sasaki and Matsuo, 2000; Vandic et al., 2012; More, 2016; Zheng et al., 2018; Yan et al., 2021). With the development of deep learning for NLP, researchers attempt to leverage neural network technology-based model for tagging attributes (Huang et al., 2015; Hakkani-Tür et al., 2016; Mai et al., 2018) or classifying attribute types (Riedel et al., 2010; Zeng et al., 2014; Amplayo, 2019; Iter et al., 2020; Zhao et al., 2021). Beyond CWA AE, researchers also explore AE in OWA scenario, e.g., some prior works try to expand free attributes from plain texts (Wong and Lam, 2010; Zhang et al., 2022; Cui et al., 2018) and extract the values of schema-free attributes (Xu et al., 2019). Recently, more novel frameworks are proposed to generalize the capacity of AE models. AVEQA (Wang et al., 2020; Shinzato et al., 2022) and MAVEQA (Yang et al., 2022) introduce Question Answering framework for AE task, and Roy et al. (2022) tries to employ large-scale PLM to introduce external knowledge. Further, some academics propose multimodal AE tasks and datasets to enrich the research (IV et al., 2017; Zhu et al., 2020). Generative Information Extraction, a rising technique in these two years (Ye et al., 2022), is also an inspiration for proposing this research. A contemporaneous work (Roy et al., 2022) adopts sequence generation models and preliminarily shows the potential of generative models in open-world attribute extraction. Alongside sequence-based generation models, structure generation models are also widely studied and have shown power in other IE tasks. For example, REBEL (Huguet Cabot and Navigli, 2021) introduces a structure-linearized model for relation extraction; Seq2UMTree (Zhang et al., 2020) conducts a sequence-to-unorderedmulti-tree generation model for extracting entities and relations jointly; UIE (Lu et al., 2022) proposes a text-to-structure generation framework that can universally model different IE tasks based on the guidance of the pre-defined schema.
Though both attribute extraction and generative models have been widely explored, we are the first to design a novel tree generation model for AE and demonstrate the effectiveness on our unified real-world paradigm.

heading: 7 Conclusion and Future Work
text: In this paper, we formulate the real-world AE task into a unified Attribute Tree, and propose a simple
but effective tree-generation model to extract both in-schema and schema-free attributes from texts. Experiments on three public datasets demonstrate our prominent performance over various scenarios, and detailed analyses also reveal several interesting findings for attribute extraction.
Several potential directions are left for the future. The first one is that our current approach does not utilize the commonly-provided multimodal information in e-Commerce, which can be naturally introduced into our tree structure as nodes for better results later. Besides, PLM has powerful effects on understanding the semantics of texts and scaling to open-domain AE applications, so incorporating knowledge of different granularity from PLMs is also an attractive extension to be explored.

heading: 8 Limitations
text: Adapting PLMs to our proposed model does not go as smoothly as expected, because there are three different forms of tokenization: the PLM tokenizer, the multilingual tokenizer implemented in our proposed model, and the special annotations of numerical values/entity mentions/long-winded attribute values in the attribute extraction datasets, which are difficult to reconcile simultaneously. Although our model without PLM has outperformed PLMbased ones, this does impose a limitation for future explorations.
Although Re-CNShipNet, one of the datasets used in our experiments, is more accurate with our careful re-annotating, the size of which is still so small that would produce randomness bias during the model training and may affect the final experimental results.
Besides, due to the limitation of computational resources, we did not conduct experiments on large
language models such as T5 (Raffel et al., 2020), LLaMA (Touvron et al., 2023), etc., which may lead to insufficiency of the experiment.
Ethics Statement
This work uses three publicly available datasets, and we respect and adhere to their user agreements and licenses. The content of pre-existing datasets does not reflect our perspectives. We, the in-house authors, re-annotate one of these datasets, i.e., Re-CHShipNet; the purpose of re-annotation is mainly to correct errors and re-balance the ratio of CWA/OWA labels. The annotation may introduce personal judgment and bias, which may bring potential risks. Further, the potential downstream applications of this work include knowledge graph construction, search engine, e-Commerce, recommendation system, etc.; we caution that our proposed method may cause misextraction or false information, and may fail in the case of out-ofdistribution and domain shift, which may harm those applications.

heading: Acknowledgements
text: This work was supported by NSFC under grant 61932001 and U20A20174. Lei Zou is the corresponding author of this paper. We would gratefully appreciate the reviewers for their precious comments that help us to improve this manuscript.

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 8 (Limitations).

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 8 (Limitations) and Ethics Statement.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4 & Section 5.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 5.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Ethics Statement Section.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5.1.

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Ethics Statement Section.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 3.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 5.
C 3 Did you run computational experiments? Section 5 & Appendix A.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix A & Section 5.2
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix A & Section 5.1

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix A & Section 5.1

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix A.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Although our model without PLM has outperformed PLMbased ones, this does impose a limitation for future explorations. Although Re-CNShipNet, one of the datasets used in our experiments, is more accurate with our careful re-annotating, the size of which is still so small that would produce randomness bias during the model training and may affect the final experimental results. Besides, due to the limitation of computational resources, we did not conduct experiments on large
language models such as T5 (Raffel et al. , 2020), LLaMA (Touvron et al. , 2023), etc. , which may lead to insufficiency of the experiment