SlowGuess/NeurIPS_2023_OCR huggingface dataset:
3. Prepare a dataset consisting of  $k$  pairs  $D = \{ (\log n_i, \log E(S_i)) \}$  and apply linear regression to approximate this set by a line. Let  $\kappa$  be the slope of the fitted line.
4. Repeat steps 2-3 two more times for different random seeds, thus obtaining three slope values  $\kappa_{1},\kappa_{2},\kappa_{3}$ , and take the final  $\kappa_{F}$  as their average.
5. Estimate the dimension  $d$  as  $\frac{1}{1 - \kappa_F}$
C.1 Choice of parameters in the formula for PHD
As we have mentioned in Section 3, we estimate the value of persistent homology dimension from the slope of the linear regression between  $\log E_{\alpha}^{0}(X_{n_i})$  and  $\log n_i$ . Thus, the exact value of PHD of a text actually depends on the non-negative parameter  $\alpha$ . The theory requires it to be chosen to be less than the intrinsic dimension of the text, and in all our experiments we fix  $\alpha = 1.0$ .
Figure 8 shows how the exact value of the PHD for natural and generated texts (of approximately the same length) depends on the choice of  $\alpha$ . Setting  $\alpha = 1.0$  seems to yield reasonable performance, but further investigation of this issue is needed.
For  $\alpha \in [0.5;2.5]$  our results, in general, lie in line with the experiments by Birdal et al. [2021], where performance of  $\dim_{PH}^{0}$  with  $\alpha$  varying between 0.5 and 2.5 was studied on different types of data.
C.2 Effect of paraphrasing on intrinsic dimension
As we show in Section 4, using paraphrasing tools has little effect on the PHD-based detector's ability to capture the differences between generated and natural texts. Here we show how such tampering
Figure 8: PHD estimates at various  $\alpha$  for a natural and a generated texts.
Table 5: Effect of paraphrasing on the intrinsic dimension of the text. Here we can see an uncommon example of data where MLE and PHD behave differently: paraphrasing does not present much trouble for PHD (its quality even increases marginally), but the performance of the MLE-based detector drops significantly, especially for GPT-3.5.
Method	Original	Lex 20	Lex 20 Order 60	DIPPER parameters
Lex 40	Lex 40 Order 60	Lex 60	Lex 60 Order 60
PHD	7.30	7.33	7.35	7.42	7.45	7.51	7.51
±1.66	±1.69	±1.16	±1.69	±1.78	±1.76	±1.72
MLE	9.68	9.91	9.90	9.97	9.95	10.00	10.01
±1.31	±1.22	±1.24	±1.22	±1.21	±1.18	±1.15
with generated sentences affects their intrinsic dimension. Table 5 presents the mean values of PHD and MLE after applying DIPPER with different parameters to the generation of GPT-3.5 Davinci. Where the value of Order (re-ordering rate) is not specified, it means the it was left at the default value 0. Both parameters, Lex (lexical diversity rate) and Order, can vary from 0 to 100; for additional information we refer to the original paper on DIPPER [Krishna et al., 2023].
Increased lexical diversity entails slight growth in both mean PHD and mean MLE that, in theory, should make our detectors less efficient. In the case of a PHD-based detector we do not observe this decrease in performance, probably due to the mean shift being indeed rather small and caused mostly by the right tail of the distribution — the texts that had a high chance of evading detection even before paraphrasing.
Meanwhile, increasing the re-ordering rate from 0 to 60 has almost no noticeable impact.
C.3 Non-native speaker bias
Here we present full results for our experiments on the bias of ML-based artificial text detectors. Figure 9 presents baseline results for all detectors studied by Liang et al. [2023] that were not included into the main text of the paper.
C.4 Intrinsic dimension of texts in different languages
Figure 10 presents the PHD of natural and generated texts in different languages on Wikipedia data. Embeddings were obtained from the same multilingual model XLM-RoBERTa-base.
Figure 9: Comparison of GPT detectors in a non-standard environment. Left: bias against non-native English writing samples (lower is better). Right: effect of the prompt design on performance (higher is better).
Figure 10: Boxplots of PHD distributions in different languages on Wikipedia data. Embeddings are obtained from XLM-RoBERTa-base (multilingual).
D Examples of generated texts
Table 6 provides examples of original text and text generated by ChatGPT.
E Examples of misclassified texts
Table 7 provides examples of misclassified human-written texts, while Table 8 shows misclassified texts generated by ChatGPT.
F Various Intrinsic Dimension estimators
Table 9 presents average intrinsic dimension estimations obtained by various algorithms for texts of different genres.
Table 7: The most extreme (outlier) examples of misclassified texts from humans
Source, PHD	Text sample
davinci, 31.06	Moorcraft and McLaughlin's comment reflects the difficulty that the Rhodesian airmen had in distinguishing innocent refugees from guerilla fighters, as the camp was a mix of both. Sibanda's description of the camp further emphasizes the tragedy of the raid, as innocent children were among those killed.
davinci, 20.89	The average annual rainfall at Tikal is approximately 1,500 mm (59 inches). However, due to the unpredictable nature of the climate, Tikal suffered from long periods of drought. These droughts could occur at any time before the ripening of the crops, endangering the food supply of the inhabitants of the city.
Table 8: The most extreme (outlier) examples of misclassified texts from davinci-003
Figure 11: PHD of several types of texts: created by sampling 512 random tokens uniformly from the vocabulary; written by humans (Wikipedia); generated by text-davinci-003 with default temperature (Wikipedia domain); created by repeating the same randomly chosen token 512 times. We didn't perform any additional restarts for outliers correction here for the purpose of highlighting the edge cases. For this reason, one can see more outliers here than on Figure 3. The outlier texts are shown in Table 8
Table 9: Intrinsic dimensions of English texts of different genres estimated by different methods.
Intrinsic dimension estimator	Wikipedia articles	Fiction stories (Reddit)	Question answering (Stack Exchange)
CorrInt [Grassberger and Procaccia, 1983]	10.05 ± 0.57	8.30 ± 1.64	9.92 ± 0.71
FisherS [Albergante et al., 2019]	6.70 ± 0.36	6.54 ± 0.71	7.10 ± 0.37
KNN [Carter et al., 2010]	6.70 ± 0.36	6.54 ± 0.71	7.10 ± 0.37
IPCA [Cangelosi and Goriely, 2007]	23.18 ± 3.60	30.59 ± 0.71	7.10 ± 0.37
MADA [Farahmand et al., 2007]	366.3 ± 1410.5	23.2 ± 83.8	420.7 ± 960.2
MOM [Amsaleg et al., 2018]	11.05 ± 1.13	5.48 ± 2.79	12.12 ± 0.47
MLE [Levina and Bickel, 2004]	11.83 ± 0.77	11.55 ± 1.20	12.13 ± 1.00
PHD (proposed method)	9.49 ± 1.01	9.21 ± 1.29	9.59 ± 1.29
TLE [Amsaleg et al., 2019]	10.65 ± 0.49	10.03 ± 1.64	11.08 ± 0.75
TwoNN [Facco et al., 2017]	5.85 ± 1.08	5.70 ± 1.72	5.95 ± 1.19
Table 10: Sizes of data splits (# of natural/generated text pairs) used in our experiments on cross-model and cross-domain performance as well as for comparison of our method with universal detectors.
Source	Train	Validation	Test
Wikipedia	Human/GPT2	2001	275	281
Human/OPT	1925	277	273
Human/GPT3.5	1798	259	260
Reddit	Human/GPT3.5	1677	232	235
StackExchange	Human/GPT3.5	2100	300	300
G Data description
Table 10 shows the sizes of data splits that were used to compare our method with universal detectors as well as for evaluating its cross-model and cross-domain performance.
Data for Wikipedia and Reddit was taken from [Krishna et al., 2023]; as for StackExchange, we assembled the dataset ourselves following the same scheme as Krishna et al. [2023]. For all text generation models (GPT2-XL, OPT, GPT-3.5), we randomly selected 2700 pairs of natural/generated texts from the raw data.
First, we divided the data into train/validation/test splits in proportion 7:1:1 to ensure that no human-written texts and no texts generated from the same prompt belong to the train and test split simultaneously. Then we filtered out all texts that were too short for stable estimation of the intrinsic dimensionality (shorter than 50 tokens) and an equal amount of texts from the opposite class.