Problematic Webpage Identification: A Trilogy of Hatespeech, Search Engines and GPT (2023)

Abstract: In this paper, we introduce a fine-tuned transformer-based model focused on problematic webpage classification to identify webpages promoting hate and violence of various forms. Due to the unavailability of labelled problematic webpage data, first we propose a novel webpage data collection strategy which leverages well-studied short-text hate speech datasets. We have introduced a custom GPT-4 few-shot prompt annotation scheme taking various webpage features to label the prohibitively expensive webpage annotation task. The resulting annotated data is used to build our problematic webpage classification model. We report the accuracy (87.6% F1-score) of our webpage classification model and conduct a detailed comparison of it against other state-of-the-art hate speech classification model on problematic webpage identification task. Finally, we have showcased the importance of various webpage features in identifying a problematic webpage.

## introduction since the advent of the internet, there has been a rapid rise of content being generated by both users and organisations, which has also expedited the rise of hateful and violent content. in this paper, we focus on the identification of such content within webpages on the internet. we define webpages promoting hate and violence against individuals and communities as problematic webpages. these problematic webpages often affect various search engines, which index such webpages resulting in them showing up in the search results. problematic webpages can be indexed by the search engine crawlers when crawling the web. the ranking models executing at the back-end of these search engines can end up showing these problematic webpages, when a user queries for something similar. this is not just limited to user queries, which are themselves having a problematic intent. such problematic webpages can also show up in information seeking innocuous queries on sensitive topics as well. for example, there is a possibility that a hateful webpage towards black community ends up as a search result for a query: data on black population in us. this can lead to a bad experience for the end user, as well as spread of targeted hate against certain individuals and communities. thus, problematic webpage classification has its applications in search engine indexing, and ranking to filter out such webpages in search engine results and stop spread of such problematic content on the internet. problematic webpages also contribute to hate speech texts in social media as part of a post, comment. @xcite have estimated that 21% of tweets in general have urls shared within them. hence, problematic webpage classification can also be additionally applied to improve hate speech detection as the urls shared within the tweet can be an important feature to classify the underlying text. a lot of research has happened on automatic hate speech detection, with several hate speech datasets @xcite @xcite @xcite , and models @xcite @xcite . these data sets and models primarily focus on identifying hate and violence in short-text data in the form of posts, comments on various social media platforms. the existing hate speech models can be leveraged to identify user queries which might lead to problematic webpages showing up in the top results of a search engine. however, these model cannot be used to identify problematic webpages and remove them from the top results for such queries. many hate speech detection models @xcite @xcite replace urls with a placeholder url from the short-text before conducting hate speech detection. this indicates that the underlying information in the url shared along with the corresponding piece of text is lost when classifying that speech as hateful. hence the current hate speech detection techniques are focused on a part of the boarder scope of online hate and abuse. these hate speech models do not focus on classifying webpages which are prominent in spreading hate and violence in two main forms. firstly, problematic webpages can show up as results in search engine queries, and secondly, as part of posts, comments in popular user generated content sharing platforms. therefore, identifying a problematic webpage is very much crucial to stop online hate. to the best of our knowledge, there has been little to no research in identifying webpages which promote hate and violence of various forms. a webpage is a very complex object as compared to short-texts and contains a variety of associated features which includes url, title, body, links, ads. existing state-of-the-art hate speech detection models often are limited towards detecting shorter text-based hateful content (e.g. tweets, social media posts, reviews etc.). in this paper, we show that these existing sota hate speech detection models are not effective in solving the problem from the perspective of detecting problematic webpages. this is due to complex structures of webpages, large amount of context present within them, and the nature of the data used to train these hate speech models. some of these issues can be addressed with a new classification model dedicated to identifying problematic webpages. we show that such a model trained on data created from webpages containing important features, and annotated with gpt-4 does much better than state-of-the-art hate speech detection models. there exist a lot of challenges to build such a dataset of webpages, both collecting and annotating. hateful, violent webpages cannot be easily discovered and mined. webpage is also not something that can be generated synthetically, rather can only be mined from the world wide web. a lot of work has happened in website classification with respect to phishing, e-commerce website classification such as @xcite . these works primarily focus on the developing classification models and often ignore the process of mining candidate webpages to build such classifiers. this calls for a strategy to discover and mine webpages on the internet to build a comprehensive data set, and eventually build a classification model. annotating webpages is also a very challenging problem which requires to consider various webpage features such as url, title, headings, subheadings, body, links, ads. the problematic webpages are ever-evolving, and are very subtle while promoting hate. to address these issues, sometimes it is required to look at the entire page content which can be very long. in the webpages curated as part of this work, we observed that the webpage body contains a large number of tokens (5350 ± 205). some webpages discussing sensitive topics in the form of news and information can be misinterpreted as problematic. similarly, some webpages which are very subtle when promoting problematic content such as political propaganda and spreading hate against a community can be misinterpreted as non-problematic. hence, it is a complex task that requires a lot of time and attention for a human judge to annotate these webpages. this is a major challenge making it very difficult to build a large scale annotated problematic webpage data set. therefore, we plan to use gpt-4 (ope-nai, 2023) to annotate these webpages at larger scale, as it has been observed that it exhibits strikingly close to human-level performance on complex benchmark tasks @xcite . it is difficult to use gpt-4 as a classifier on its own due the scalability issue towards annotating billions of webpages. thus, we use gpt-4 to annotate a significant amount of data to train a reasonably accurate classifier which further can be used in scale for labelling large volume of webpages. therefore in this paper, we focus on creating a fine-tuned transformer-based @xcite webpage classification model focused on webpage text features: url, title, headings and paragraph texts to identify problematic webpages promoting harmful content. we present a novel webpage data collection and annotation strategy, and use that to create the training, validation, and measurement set which can help future research in this area. we will release all the data publicly upon the acceptance of the paper. the main contribution of the paper are as follows: ## data collection and annotation process as mentioned in the previous section, there are many challenges in creating an annotated problematic webpages dataset. some such problems include: discovering potential candidate webpages from the internet, annotating webpages which are often comprised of large volume of text and has rich meta information, and feature extraction to appropriately represent the webpage. we propose a novel solution to discover candidate webpages by leveraging popular search engines. for feature extractions, we have used popular web scraping tools, and also processed the input data to create important features (cf. section 2.1). the webpage annotation (cf. section 2.2) using gpt-4 takes care of the complex structure and longer token sequence. algorithm 1 illustrates the data curation and annotation process. the details of the steps are described in the following subsections. ## experiments & results as mentioned in section 1, latency and cost are the major challenges to leverage gpt-4 to annotate webpages at scale. this is important point to consider during our experimentation as there are billions of webpages in a search engine index, and millions of webpages embedded in social media posts and comments. thus, to solve the problem of identifying problematic webpage classification, we need a lighter model. ## conclusion this paper presents a novel way of collecting and annotating problematic webpages which is important for building a problematic webpage classification model. we have shown that easily available short-text data along with the knowledge of sota generative models (gpt-4) can help in building annotated datasets for a complex task such as problematic webpage classification. we also report and re-establish the fact that writing precise prompt along with a few examples is effective and achieve very high quality annotation. we compare different pre-trained models and fine-tune them with our dataset and report comparative results. we also report an ablation study and show that the different features used in our experiment are together effective for webpage classification. finally, we show empirically that our data set is effective for building a problematic webpage classifier. the work can be further extended by leveraging additional features for webpage classification such as ads, link connections to other webpages, authority of the domain. the work can also be extended towards creating multilingual dataset for problematic webpage classification and subsequently build a model for the same.