  Analyzing the readability of articles has been an important sociolinguistic
task. Addressing this task is necessary to the automatic recommendation of
appropriate articles to readers with different comprehension abilities, and it
further benefits education systems, web information systems, and digital
libraries. Current methods for assessing readability employ empirical measures
or statistical learning techniques that are limited by their ability to
characterize complex patterns such as article structures and semantic meanings
of sentences. In this paper, we propose a new and comprehensive framework which
uses a hierarchical self-attention model to analyze document readability. In
this model, measurements of sentence-level difficulty are captured along with
the semantic meanings of each sentence. Additionally, the sentence-level
features are incorporated to characterize the overall readability of an article
with consideration of article structures. We evaluate our proposed approach on
three widely-used benchmark datasets against several strong baseline
approaches. Experimental results show that our proposed method achieves the
state-of-the-art performance on estimating the readability for various web
articles and literature.
