distinct
Browse files- distinct.py +5 -4
distinct.py
CHANGED
@@ -52,15 +52,16 @@ _CITATION = """\
|
|
52 |
|
53 |
_DESCRIPTION = """\
|
54 |
Distinct metric is to calculate corpus-level diversity of language. We provide two versions of distinct score. Expectation-Adjusted-Distinct (EAD) is the default one, which removes
|
55 |
-
the biases of the original distinct score on lengthier sentences. Distinct is the original version.
|
56 |
|
57 |
-
|
58 |
-
<img src="https://huggingface.co/spaces/lsy641/distinct/resolve/main/distinct_compare_pic.jpg" alt="drawing" width="200"/>
|
59 |
-
img[alt=drawing]{}
|
60 |
|
61 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
62 |
|
63 |
Please follow ACL paper https://aclanthology.org/2022.acl-short.86 for motivation and follow the rules of thumb provided by https://github.com/lsy641/Expectation-Adjusted-Distinct/blob/main/EAD.ipynb to determine the vocab_size
|
|
|
|
|
|
|
|
|
64 |
"""
|
65 |
|
66 |
|
|
|
52 |
|
53 |
_DESCRIPTION = """\
|
54 |
Distinct metric is to calculate corpus-level diversity of language. We provide two versions of distinct score. Expectation-Adjusted-Distinct (EAD) is the default one, which removes
|
55 |
+
the biases of the original distinct score on lengthier sentences (see Figure below). Distinct is the original version.
|
56 |
|
|
|
|
|
|
|
57 |
|
58 |
For the use of Expectation-Adjusted-Distinct, vocab_size is required.
|
59 |
|
60 |
Please follow ACL paper https://aclanthology.org/2022.acl-short.86 for motivation and follow the rules of thumb provided by https://github.com/lsy641/Expectation-Adjusted-Distinct/blob/main/EAD.ipynb to determine the vocab_size
|
61 |
+
|
62 |
+
<img src="https://huggingface.co/spaces/lsy641/distinct/resolve/main/distinct_compare_pic.jpg" alt="drawing" width="200"/>
|
63 |
+
img[alt=drawing]{}
|
64 |
+
|
65 |
"""
|
66 |
|
67 |
|