Paper:
With the recent success and popularity of pre-trained language models (LMs) in natural language processing, there has been a rise in efforts to understand their inner workings. In line with such interest, we propose a novel method that assists us in investigating the extent to which pre-trained LMs capture the syntactic notion of constituency. Our method provides an effective way of extracting constituency trees from the pre-trained LMs without training. In addition, we report intriguing findings in the induced trees, including the fact that pre-trained LMs outperform other approaches in correctly demarcating adverb phrases in sentences.

Review:
I am satisfied with the author's response and I am increasing the score to 6 after the rebuttal.
===
This paper introduces a new and simple method of probing whether syntax information under the form of constituency trees is present in recent pre-trained language models (e.g. BERT, RoBERTa, XLNet and GPT2) without any additional task-specific training. They use the previously proposed concept of "syntactic distance" [1] as a sufficient statistics for the constituency tree of a given sentence. They compute the distance between neighboring words in sentence using a distance function f(g(w_i), g(w_i+1)) where f is a divergence between distributions and g(w_i) is the self-attention distribution of w_i at a given layer: the intuition is that words that have similar self-attention distributions belong to the same constituent and thus their syntactic distance is small. In order to conform with right-skewness of English syntax, they propose an affine transformation of the distances that encourage right-skewed trees, and tune the hyper-parameter directly on the target F1 score wrt ground-truth trees. Results suggest that large, pretrained models capture constituency trees to some extent.
This is an empirical and analysis paper which can be considered as the “twin” of Hewitt et. al (2019, analyzes whether dependency structure can be probed from large pre-trained models). The paper is well-written and easy to understand. The experiments are in general well-organized even if they lack clarity at some point. The related work is comprehensive and covers most of the work in the domain of grammar induction and unsupervised parsing. In its current form however, I feel like the paper misses a "second half" therefore it isn’t quite above the acceptance bar. I'd be happy if the authors could kindly elaborate on the following major weaknesses: (i) unclear scientific motivation of using the right-skewness bias; (ii) lack of clarity in the experimental results; (iii) lack of depth and perspective.
 
1) About right-skewness bias:
I appreciate that the authors clearly guard against using explicit right-skewness biases. However, I am not sure why the authors use of Eq. (2) apart from boosting performance. The authors claim that "the main purpose of explicitly injecting such a bias is examining what changes are made to the resulting tree structures" but the purposed changes are not discussed at length in the text. Therefore:
1.1) What's the scientific motivation (apart from boosting performance) of skewing the trees using Eq. (2) ?
1.2) What can you infer from trees without bias and tree with bias ?
2) About clarity in the experimental results:
2.1) (Shen, 2018) use an "implicit" right-skewness bias. From the text, it seems that your results w/o bias use the same bias as (Shen, 2018) ? Therefore they actually have a bias ? This is a bit confusing. What about using the unbiased parsing algorithm as presented in "Straight to the tree..." ?
2.2) This sentence is unclear: "we select one derived from the best choice of f and g in terms of sentence-level F1 (S-F1)
w.r.t. gold-standard trees as a representative for the LM". Are you using S-F1 on *training* for tuning f and g ?
2.3) This sentence is unclear: "As LMs are not fine-tuned with training sets in our framework, we only use the test
set of the PTB", what do you mean by "using the test set" ? Do you mean evaluate or use it for tuning (cf 2.2) ?
3) About perspective and depth:
3.1) The fact that ADVP , ADJP performance is higher for the tested models is interesting. Do you have any hypothesis for why this is happening?
3.2) Right-skewness seems to mainly help with VP. Can you formulate an hypothesis for why current models underperform on this label or do not show right bias?
3.3) The results on Table 2 are interesting, but imho, they are basically showing that these models cannot do constituency parsing with just a linear probe on top of the representations and using the syntactic distance algorithm?
3.4) Did you consider using other parsing algorithms e.g. chart parser, instead of just the syntactic distance ? That would make a stronger paper.