Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a latent adaptive structure-aware generative language model for information extraction tasks. The proposed model incorporates a latent structure induction module that automatically induces tree-like structures akin to dependency and constituency trees. Experiments in IE benchmarks show that the proposed model outperforms the state-of-the-art models.
strengths_and_weaknesses: Strengths
* I appreciate the extensiveness of the experiments in Section 5. I personally had questions regarding the difference between constituency and dependency structures as well as the difference between internally learned (latent) and externally predicted structures. These are answered well in the paper.
* The paper is written well. The motivation behind the use of latent structures and the intuition behind the design of the model look sound.

Weaknesses
* The idea of using task-specific latent structures started in this paper [1], however this is not mentioned at all in the paper. Moreover, there are many prior work in latent structure induction [2, 3, among others] that were also not mentioned in the paper.
* It would be very helpful for readers to know the tradeoff between the performance and speed. This is crucial especially since the differences in performance between the proposed model and the state-of-the-art models are rather small.
* Please also include significance testing in the results.
* A more extensive analysis on the induced tree structures would also be very useful for readers. For example, are the trees interpretable and useful outside of the model? Can humans make use of such trees to explain predictions? Also, including examples of induced trees would be nice to have.

[1] https://ojs.aaai.org/index.php/AAAI/article/view/11975
[2] https://arxiv.org/abs/1904.02142
[3] https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00019/43445/Do-latent-tree-learning-models-identify-meaningful
questions: * How different/similar are the induced trees and the original dependency/constituency trees? These can be compared by calculating the tree induction accuracy.
limitations: * A more extensive analysis on the induced trees would have improved the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper propose a new framework to improve the unified information extraction by considering syntactic knowledge in an unsupervised way. Specifically, they reuse the constituency tree information and dependency tree information learned by pre-trained language models and do further learning. They also provide structural broadcaster and task-oriented fine-tuning to utilize the learned syntactic features. Experimental results on several information extraction tasks show the potential of the propose LasUIE.
strengths_and_weaknesses: Strength
- They show improvements on several types of information extraction tasks and datasets.
- They provide ablation studies to analyze the influence of each module.

Weakness
- Some technical details are not clear for me (please see the questions below).
- The EE result is weird (please see the questions below).
questions: - The post-training is a bit unclear to me. It seems like you construct trees from the pre-trained language models and treat them as ground truth for training. Am I correct?
- The EE result is weird. The reported score of OneIE is around 56 but it's 48.3 in the paper. Also, OneIE is not SOTA anymore, here is one reference paper
  - Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks, NAACL 2021
- The post-training can be independent to the downstream tasks. What will happen if you fine-tune for each task separately after the post-training (no unified training for every tasks)?
- What will happen if you do not use additional corpora (Wikipedia and BooksCorpus) but only use the downstream texts for the post-training?
- In the paper, you mention for fair comparison, you reimplement UIE with T5-base. Why not train LasUIE with T5-large and directly compare to UIE?
- The UIE paper reported the EE (ACE-05) scores as well. You should list them in the Table 1.
limitations: - The proposed method uses additional corpora (Wikipedia and BooksCorpus) to learn the syntactic information. I suggest the author explicitly mention this in the Table 1 and Table 2.
- Missing related work
  - Structured Prediction as Translation between Augmented Natural Languages, ICLR 2021
  - Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks, NAACL 2021
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed a  structure-aware GLM,  in which they leveraged the syntactic knowledge for UIE. A heterogeneous structure inductor is explored to unsupervisedly induce rich heterogeneous structural representations by post-training an existing GLM.  The authors did experiments over 12 IE benchmarks across 7 tasks and showed significant improvements over the baseline UIE system.
strengths_and_weaknesses: Strength:
* Convert the UIE into a generative LM problem, and designed three modules to do the prediction
* An structure aware training is added before the fine tuning stage. 
* Good experiments design.

Weakness:
* The experimented datasets are quite old, it would be better if there are some results on larger scale IE datasets. 
questions: Apart form the dependency and constituency structure, are there any other structures that can be incorporated into the training process ?
limitations: May need to report the training time/complexity for this method, and the computing resources used.
review_title: null

Meta-review: metareview: This paper proposes a latent adaptive structure-aware generative language model (GLM) to leverage syntactic knowledge for information extraction tasks. The proposed model incorporates a latent structure induction module that automatically induces tree-like structures akin to dependency and constituency trees. Experiments in 12 IE benchmarks across 7 tasks showed significant improvements over the baseline.

Overall, all reviewers feel positively about this paper, even though they mention some aspects which can be improved in the final version. The conversion of information extraction tasks into a problem solvable by a GLM problem with three different prediction modules is original and valuable, and the experiments are well designed and generally convincing (although additional experiments in more recent and larger scale datasets would make the paper stronger). The author response addressed well all the concerns of the reviewers, including the addition of several missing references. I urge the authors to incorporate these in their paper and to report the runtime of their method, to better understand the tradeoff between performance and speed, as well as examples of induced tree structures produced by their method, as suggested by one of the reviewers.
confidence: Certain
Decision: Accept