Meaningfulness and unit of Z ipf’s law: evidence from danmu comments (2021)

Abstract: Zipf’s law is a succinct yet powerful mathematical law in linguistics. However the mean-ingfulness and units of the law have remained controversial. The current study usesonline video comments call “danmu comment” to investigate these two questions. Theresults are consistent with previous studies arguing Zipf’s law is subject to topical coher-ence. Specifically it is found that danmu comments sampled from a single video followZipf’s law better than danmu comments sampled from a collection of videos. The resultsalso suggest the existence of multiple units of Zipf’s law. When different units includingwords n-grams and danmu comments are compared both words and danmu commentsobey Zipf’s law and words may be a better fit. The issues of combined n-grams in the literature are also discussed.

## conclusion the current study explored zipf's law with novel datasets from danmu comments, which is an emerging type of online video comments and internet language. specifically, the study has three findings. first, danmu comments also follow zipf's law and danmu comments from topically homogeneous video fit zipf's law better. the findings suggest that zipf's law may be driven by semantic motivations. second, danmu comments, clauses, and character unigrams and character bigrams in danmu comments all follow zipf's law. there seems to be a continuum of fitness in the order of bigrams, clauses, and danmu comment. the results indicate that multiple units for zipf's law may exist even in the same data. finally, different from previous studies, combined n-grams do not always have the best fit for zipf's law. it is argued that combined n-grams may be considered as a statistical smoothing rather than a manifestation of zipf's law. there are two limitations of the datasets. first, danmu comments are cumulative and may change over time. therefore, a temporal analysis of danmu comments is necessary in the future. in addition, the danmu comments from a single video seems to fit zipf's law. however, the dataset is relatively small and larger data from more videos are needed to validate the zipfian distribution in danmu comments. internet has greatly changed the way we use languages and multilingualism is becoming increasingly popular. studies on zipf's law in internet language will help us understand the language phenomena better.