Shedding New Light on the Language of the Dark Web (2022)

Abstract: The hidden nature and the limited accessibility of the Dark Web, combined with the lack of public datasets in this domain, make it difficult to study its inherent characteristics such as linguistic properties. Previous works on text classification of Dark Web domain have suggested that the use of deep neural models may be ineffective, potentially due to the linguistic differences between the Dark and Surface Webs. However, not much work has been done to uncover the linguistic characteristics of the Dark Web. This paper introduces CoDA, a publicly available Dark Web dataset consisting of 10000 web documents tailored towards text-based Dark Web analysis. By leveraging CoDA, we conduct a thorough linguistic analysis of the Dark Web and examine the textual differences between the Dark Web and the Surface Web. We also assess the performance of various methods of Dark Web page classification. Finally, we compare CoDA with an existing public Dark Web dataset and evaluate their suitability for various use cases.

## introduction the world wide web contains a vast, non-indexed part of the internet (known as the deep web) which is hidden from traditional web search engines. the dark web, which refers to the small portion of the non-indexed pages that require specific routing protocols such as tor 1 for access, has become a safe haven for users wanting to conceal their identity and preserve their anonymity. a consequence of the properties of the dark web (limited methods of access and the volatility of its onion services) is that it is difficult to grasp the general topology and the overall content of the dark web. a number of past academic studies have tried to unravel the dark web through methods such as page classification @xcite @xcite and content analysis @xcite . however, not much work has been done on the linguistic analysis of the dark web @xcite . in addition, the dark web has been studied and analyzed in the security research community to uncover malicious activities including phishing @xcite , illicit online marketplace activity @xcite , terrorism @xcite , cryptocurrency abuse @xcite , and ransomware ecosystems @xcite . we believe that the lack of a comprehensive work on the language of the dark web from the nlp community mainly stems from the lack of dark web datasets publicly available for research. therefore, a new dataset on the dark web may prove to be very useful not only for the nlp community, but also for other research communities devoted to cybersecurity and cybercrime investigation through methods such as page classification and malicious activity detection. to the best our knowledge, the only currently publicly available dark web dataset is duta @xcite , which has been extended to hold over ten-thousand unique onion addresses as duta-10k @xcite . duta foot_0 has become a baseline dataset for many dark web related works such as @xcite , which investigates the characteristics of language used in various illegal and legal onion services. nevertheless, duta has its shortcomings. for example, the category distribution of duta is highly skewed, with some categories such as human trafficking accounting for only 3 out of 10367 total onion services. in addition, the us e of duta as a means of language analysis may not be ideal as it contains many duplicate data, with only 51% of the texts being unique @xcite . to provide a better understanding of the dark web (and thus motivate more research on the dark web), we introduce coda foot_1 (comprehensive darkweb annotations), a text corpus of 10,000 web documents from the dark web (primarily in english) which have been manually classified according to their topic into ten categories. to ensure that the quality of the classification is not overlooked, we develop detailed annotation / tagging guidelines (section 3) to guide our annotators. using coda, we conduct a thorough text-based data analysis (section 4) to uncover some of the linguistic properties of the dark web, and gain insight into differences in how language is used in the surface web and the dark web. we build several text classifier models and train them using coda, and verify which classification methods perform particularly well with the dark web (section 5). finally, to evaluate the use of coda compared to duta, we introduce use cases and compare the performances of classifiers trained on each dataset (section 6). ## related work the dark web is commonly crawled using tor, which relies on onion routing to enable encrypted communications over a computer network @xcite . several works use dark web search engines such as ahmia 4 and web directories such as the hidden wiki to recursively search for content on the dark web @xcite @xcite . this method of crawling works surprisingly well as the visible part of the dark web is suggested to be well-connected via hyperlinks @xcite . to facilitate the research on dark web content analysis, a text-based, manually labeled dataset collected from the active domains in the tor network called duta was made publicly available by al @xcite . in a subsequent work, the original duta dataset was extended to 10367 unique domains with minor changes to the labeling procedure @xcite . to the best of our knowledge, duta is the first and only publicly available dark web text dataset. past works have analyzed the dark web through topical classification of texts in onion services. many have approached text-based page classifica-tion with machine learning methods such as svm (support vector machine), nb (naïve bayes), and lr (logistic regression) @xcite @xcite @xcite using various information retrieval weighting schemes like tf-idf (term frequency-inverse document frequency) and bow (bag-of-words) @xcite @xcite . @xcite have suggested that deep neural models may fare poorly with dark web classification as language in the dark web is lexically and syntactically different compared to that of the surface web. their work demonstrated that representation methods such as glove @xcite and contextualized pre-trained language representations such as elmo @xcite resulted in a subpar performance compared to traditional machine learning methods, suggesting that the small size of training data and the specialized vocabulary in the dark web domain may not be suitable with such methods. nevertheless, transformer-based pre-trained language models like bert @xcite showed promising results in text classification tasks, although it is not often the case that such models adapt with ease in the dark web domain @xcite . ## the coda corpus in this section, we introduce our categorization approach and the methods used to construct our dark web dataset, coda. ## data analysis to assess the linguistic properties of the dark web, illustrate the characteristics of textual content in each category, and better understand the differences in the use of language in the dark / surface web, we conduct an in-depth exploratory data analysis. we analyze the text data of coda and compare measurements with that of the other datasets (see section 4.1) as shown in ## use cases in this section, we elaborate on the use cases of our corpus and the classifiers trained on coda and duta. toyota) are often used in drug-related documents in the dark web to refer to synthetic drugs with brand logos imprinted on each pill. we test the above scenario by training two simple word2vec models @xcite , one using coda and another using duta. for each model, we query tesla and toyota and retrieve the most similar words to the queried terms. in this case, both models output drug-related words such as methoxphenidine, testosterone, and alprazolam. we also query another word, wasabi, which originally refers to a plant but is also used to refer to a bitcoin wallet service. in the dark web, wasabi is more likely to be used as a cryptocurrency term rather than the plant itself. when wasabi is queried, the model trained on coda returns cryptocurrency-related words, while the model trained on duta does not have the word in its vocabulary. we list the top 10 most similar words to wasabi as reported by the model trained on coda, most of which are related to cryptocurrency services: mustard, electrum, samourai, pools, trustless, mycelium, rpc, xapo, hijacker, converter. ## conclusion in this work, we introduced coda, a dark web text corpus collected from various onion services divided into topical categories. using coda, we conducted a thorough analysis of the linguistic properties of the dark web and found that there are clear lexical differences from the surface web including abbreviations and lexical structure such as pos distribution. we also found lexical characteristics of categories through mask id distribution and tf-idf. our text classification results showed that svm and bert perform well in the dark web domain, even with the language differences that the dark web exhibits compared to that of the surface web. finally, we have demonstrated the practicality of coda through two use cases with nlp methods. we speculate that the lack of duplicate content in coda compared to duta may aid in the performance of such applications. we hope that our dataset and our work motivates further research in the field of language-based dark web analysis.