SlowGuess/NAACL_2022_OCR huggingface dataset:
CNN: The model consists of one GloVe embedding layer (6B.300d), three 2-dimensional convolutions (Conv2d), and one fully-connected layer. The kernel sizes of the three convolution layers are 3, 4, and 5, respectively. We applied the ReLU activation function and 1-dimensional max pooling after each convolution layer. We also used the SGD optimizer and ran 10 training epochs with cross-entropy loss; the learning rate was 1.5, and the batch size was set to 32.
BERT: We used the Adam optimizer and ran 10 training epochs with cross-entropy loss, a learning rate of 2e-5, a linear schedule with no warmup step, a batch size of 32, and gradient norm clipping of 1.0. We also limited the maximum sequence length to 256 tokens, assuming that the leading part of text is indicative of topics. All the other settings are the same as those used in the original BERT paper.
C Language Distribution
We present the language distribution of CoDA in the following table:
Language	Document count	Language	Document count	Language	Document count
English	8855	Persian	7	Basque	1
Russian	542	Swedish	7	Egyptian Arabic	1
German	129	Ukrainian	7	Georgian	1
French	100	Turkish	6	Greek	1
Spanish	61	Catalan	5	Gujarati	1
Portuguese	54	Hungarian	5	Hindi	1
Chinese	38	Cebuano	3	Ido	1
Italian	28	Esperanto	3	Iloko	1
Japanese	27	Indonesian	3	Kurdish	1
Dutch	14	Latin	3	Marathi	1
Finnish	14	Lithuanian	3	Punjabi	1
Korean	12	Norwegian	3	Slovak	1
Czech	11	Bengali	2	Tamil	1
Polish	10	Galician	2	Thai	1
Bulgarian	9	Romanian	2	Urdu	1
Arabic	7	Serbian	2	Vietnamese	1
Hebrew	7	Slovenian	2	Welsh	1
Table 10: Language distribution of documents in CoDA
D Additional Data Analysis Methods & Results
Some additional details and figures of results collected from various data analyses methods in Section 4 are presented here.
D.1 PoS Distribution & Content Word / Function Word Ratio
Choshen et al. (2019) demonstrated that legal and illegal texts in the Dark Web can be distinguishable through their lexical structure, that is, through part-of-speech (PoS) tags and distribution of content and function words. We analyze the PoS distribution and the distribution of content and function words in each dataset to see if there is a meaningful difference in the lexical structures of Dark / Surface Web contents. To obtain the universal PoS of words in the dataset, we utilize the PoS tagger in spaCy. Following Choshen et al. (2019), we define content words as words tagged by spaCy into one of the following PoS tags:
$$
\left\{\text {A D J}, \text {A D V}, \text {N O U N}, \text {P R O P N}, \text {V E R B}, \text {X}, \text {N U M} \right\}
$$
and define all other words as function words. Since text length varies widely for each document in the Dark Web, we analyze the mean PoS ratio and the mean content word / function word ratio (CF ratio) for each category for both Dark and Surface Webs. The mean CF ratio  $\bar{r}_{cf}(\mathcal{C})$  for some category  $\mathcal{C}$  is given by
$$
\bar {r} _ {c f} (\mathcal {C}) = \frac {1}{| \mathcal {C} |} \sum_ {d \in \mathcal {C}} \frac {N _ {c} (d)}{N _ {f} (d)}
$$
where  $|\mathcal{C}|$  denotes the number of documents (text files) in category  $\mathcal{C}$ , and  $N_{c}(d)$ ,  $N_{f}(d)$  denote the total number of content words and function words in some document  $d$ , respectively.
The results for the PoS distribution and CF ratio are shown in Figures 2 and 3. It is evident that Dark Web categories generally have a much higher CF ratio compared to that of the Surface Web. From our PoS distribution analysis, we find that the Dark Web documents have a very high ratio of proper nouns (PROPN) and numerals (NUM) compared to the Surface Web documents, both of which are PoS tags of content words. Moreover, the Surface Web documents have a higher ratio of determiners $^{20}$  (DET) compared to that of the Dark Web. Since function words serve as critical components of sentence structures, this result implies that language used in the Dark Web may contain a higher proportion of non-sentence structures. For example, texts in the drugs category mostly consist of a list of drugs with their price and weight.
D.2 Discussion
The use of spaCy for the comparison of PoS distribution and CF ratios may raise questions as spaCy has not been pretrained on Dark Web content, which may yield a higher error rate on the Dark Web results. However, this is not an issue for our work as the main purpose of this analysis is to show that there are linguistic differences between the Dark Web and the Surface Web. If the result of the analysis is heavily skewed by the presence of Dark Web content and additional training is necessary for the pretrained spaCy model, then this implies that there are meaningful lexical differences in the Dark Web. On the other hand, if the result is not particularly affected by the Dark Web content, then it shows that there are clear differences between the two domains. In either case, it is observable that the results suggest the existence of lexical differences between the Dark Web and the Surface Web.
However, there is one possible limitation in our analysis in that the aggregate Surface Web dataset may not encompass the complete representation of the Surface Web. This is evident from observing that a significant portion of the aggregate dataset (IMDb, Wikitext) consists of text content that is comprised of complete sentences. For example, the inclusion of marketplace content such as eBay or Amazon could affect the PoS distribution and the CF ratio of the aggregate Surface Web data. For future work, we may attempt additional analysis through boilerplate removal of noisy, non-structural texts in CoDA and the aggregate Surface Web data along with the augmentation of a broader diversity of Surface Web content and examine if this approach significantly affects the results obtained in Section D.1.
20Words that modify nouns or noun phrases such as articles
Figure 2: PoS distribution of categories in CoDA and Surface Web aggregate data
Figure 3: Mean content word / function word (CF) ratio  $(\bar{r}_{cf})$  of categories in CoDA and Surface Web aggregate data
Figure 4: Log-log plot of word frequency distribution of CoDA and Surface Web aggregate data by category. We exclude DUTA as it contains too many categories to be presented in the figure.
Figure 5: Mask token distribution by category (excluding ID_NUMBER)
E TF-IDF Measurements
A table of TF-IDF measurements (as mentioned in Section 4.2.4) showing the relevant words and phrases of selected categories in CoDA is listed here.
Rank	Crypto Term	TF-IDF	Drugs Term	TF-IDF	Electronics Term	TF-IDF	Financial Term	TF-IDF
1	ID_NUMBER	0.684	ID_NUMBER	0.863	ID_NUMBER	0.799	ID_NUMBER	0.906
2	ID_CRYPTO_MONEY	0.418	IDWeight	0.255	ID_general_MONEY	0.291	ID_general_MONEY	0.220
3	bitcoin	0.354	IDGENERAL_MONEY	0.249	iphone	0.279	card	0.215
4	ID_TIME	0.233	buy	0.112	ID_FILESIZE	0.184	ID_TIME	0.130
5	IDBTC_ADDRESS	0.227	weed	0.101	apple	0.142	ID_EMAIL	0.067
6	btc	0.110	pot	0.089	pro	0.111	buy	0.065
7	use	0.071	pill	0.084	ipad	0.110	account	0.051
8	wallet	0.069	online	0.080	macbook	0.092	credit	0.048
9	address	0.068	cocaine	0.073	imac	0.089	baypal	0.048
10	buy	0.068	cannabis	0.068	airpod	0.082	transfer	0.044
11	transaction	0.061	lsd	0.057	ID_TIME	0.079	order	0.039
12	get	0.060	mdma	0.051	buy	0.073	get	0.038
13	blockchain	0.057	drug	0.050	card	0.071	dump	0.038
14	invest	0.049	aderall	0.050	watch	0.058	cc	0.037
15	ID GENERAL MONEY	0.048	viagra	0.049	ipod	0.055	good	0.036
16	coin	0.046	xanax	0.049	case	0.052	money	0.036
17	service	0.046	ID_PERCENTAGE	0.046	gopro	0.051	new	0.036
18	make	0.045	product	0.045	product	0.051	use	0.035
19	ID_PERCENTAGE	0.042	order	0.044	xs	0.051	cvv	0.034
20	double	0.041	quality	0.042	order	0.047	shop	0.034
Rank	Gambling Term	TF-IDF	Hacking Term	TF-IDF	Pornography Term	TF-IDF	Violence Term	TF-IDF
1	ID_NUMBER	0.593	ID_NUMBER	0.792	porn	0.605	ID_NUMBER	0.915
2	casino	0.396	hack	0.364	porn	0.557	ID_TIME	0.173
3	game	0.256	facebook	0.262	video	0.272	kill	0.074
4	br	0.190	ID_TIME	0.167	free	0.220	anonymous	0.066
5	online	0.177	account	0.162	sex	0.154	hitman	0.066
6	slot	0.173	hacker	0.092	girl	0.125	ID_FILENAME	0.065
7	play	0.161	password	0.087	teen	0.123	murder	0.064
8	ID_TIME	0.103	use	0.070	ID_NUMBER	0.105	people	0.063
9	bet	0.098	IDgeneral_MONEY	0.068	boy	0.093	like	0.056
10	win	0.096	service	0.059	fuck	0.091	one	0.054
11	poker	0.094	email	0.058	child	0.083	get	0.052
12	page	0.094	software	0.052	cock	0.077	file	0.052
13	get	0.093	ransomware	0.050	cp	0.070	post	0.047
14	free	0.089	download	0.045	young	0.068	site	0.043
15	money	0.087	get	0.044	pussy	0.053	comment	0.043
16	time	0.087	free	0.040	pedo	0.053	ID_NORMAL_URL	0.042
17	card	0.083	instagram	0.039	say	0.051	hire	0.042
18	player	0.081	attack	0.039	mom	0.050	make	0.041
19	good	0.077	hacking	0.039	gay	0.050	say	0.040
20	roulette	0.075	online	0.038	get	0.049	use	0.040
Table 11: Terms with the highest TF-IDF for selected categories in CoDA
F Forum & Marketplace Benchmark Dataset
Category	Website title	Onion URL	# of pages
Drugs	Ang*****	ang*****	377
Drugs	Glo*****	ny4*****.onion	307
Drugs	Opi*****	opi*****.onion	215
Drugs	Pot***	pot*****	19
Drugs	Eu*****	wge*****	13
Drugs	Kam*****	bep*****	5
Financial	Wal*****	z2h*****.onion	241
Financial	Tor*****	tor*****	71
Financial	Cov*****	cov*****	62
Financial	Fin*****	fin*****.onion	51
Financial	Cas*****	hss*****	35
Financial	Car*****	car*****.onion	25
Financial	Imp*****	srw*****	16
Financial	Lig*****	sw3*****	14
Financial	Kin*****	kin*****	13
Financial	Cou*****	cou*****.onion	12
Financial	Cas*****	maf*****	11
Financial	Kry*****	kry*****.onion	10
Financial	The*****	nar*****	9
Financial	Pre*****	hbl*****	8
Financial	Hor*****	hor*****.onion	8
Financial	Ban***	ban*****.onion	7
Financial	Tor*****	vrm*****	7
Financial	net***	net*****.onion	6
Financial	Bla*****	bla*****.onion	6
Financial	LAR*****	fiw*****	5
Financial	eas*****	eas*****.onion	3
Financial	CHE*****	o6k*****.onion	3
Financial	PAY*****	ity*****.onion	3
Weapons	Glo*****	glo*****.onion	266
Weapons	Alp*****	alp*****.onion	219
Weapons	Exe*****	5zk*****.onion	181
Weapons	Eur*****	hyj*****	4
Weapons	UK*****	tuu*****.onion	4
Total Drugs	936
Total Finance	626
Total Weapons	674
All	2236
Table 12: Source of our forum & marketplace benchmark dataset as described in Section 6. To follow ethical guidelines, we mask the website titles and onion addresses.