# Are Strong Baselines Enough? False News Detection with Machine Learning
## 2 Materials and Methods
### Datasets
#### 2.2.4 Novel ChatGPT-Generated Dataset  
For experimental purposes, We created a novel dataset that includes news articles generated by ChatGPT to test how well classifiers would handle artificially generated text. The dataset consists of an ID, title, text, author/source, and the labels false and true. Additionally, the texts labeled as true are provided with a link to the original article. We created 200 false news articles for the dataset and 100 true articles. Creating false news articles with ChatGPT was a very specific and unusual task. By default, ChatGPT refuses the creation of false news articles. This, however, can be easily manipulated with wordplay and tricking ChatGPT into writing articles that contain misleading information. In Figure 3 we can observe our initial failure in creating false news with ChatGPT, and in Figure 1, we can observe how we succeeded in manipulating ChatGPT to generate false news for us. The text generated by ChatGPT is believable, and creative, and does not contain language usually found in false news, such as exaggerated language. The example of how to generate false news articles with ChatGPT shows how easily it can be used to generate untruthful information in general. The so-called truthful news for this dataset was created by first summarizing actual news articles from credible news sources, mainly from Reuters ([[https://www.reuters.com/](https://www.reuters.com/)]([https://www.reuters.com/](https://www.reuters.com/)), accessed on 12 August 2024), Helsinki Times ([[https://www.helisniktimes.fi/](https://www.helisniktimes.fi/)]([https://www.helisniktimes.fi/](https://www.helisniktimes.fi/)), accessed on 12 August 2024), The Kyiv Independent ([[https://kyivindependent.com/](https://kyivindependent.com/)]([https://kyivindependent.com/](https://kyivindependent.com/)), accessed on 12 August 2024), and NHK WORLD-JAPAN ([[https://www3.nhk.or.jp/nhkworld/](https://www3.nhk.or.jp/nhkworld/)]([https://www3.nhk.or.jp/nhkworld/](https://www3.nhk.or.jp/nhkworld/)), accessed on 12 August 2024), then making ChatGPT write the news article again using the summary that was created. We also made sure that each generated article was truthful towards its \begin{table}
\begin{tabular}{c c} \hline \hline
**Elements** & **Value of Elements** \\ \hline ID & politifact182 \\ News URL & [[http://www.gao.gov/new.items/d071195.pdf](http://www.gao.gov/new.items/d071195.pdf)]([http://www.gao.gov/new.items/d071195.pdf](http://www.gao.gov/new.items/d071195.pdf)) (accessed on 12 August 2024). \\ Title & US Government Accountability Office Report to Congressional Committees \\ Tweet IDs & 956894522511736832 \\ Label & real \\ \hline ID & politifact14944 \\ News URL & [[http://thehill.com/homenews/senate/369928-who-is-affected-by-the-government-shutdown](http://thehill.com/homenews/senate/369928-who-is-affected-by-the-government-shutdown)]([http://thehill.com/homenews/senate/369928-who-is-affected-by-the-government-shutdown](http://thehill.com/homenews/senate/369928-who-is-affected-by-the-government-shutdown)) (accessed on 12 August 2024) \\ Title & Who is affected by the government shutdown? \\ Tweet IDs & 954602090462146560 954602093171609600 954650329668349954 \\ Label & false \\ \hline ID & gossipcop-897603 \\ News URL & [[https://www.teenvogue.com/story/selena-gomez-not-changing-blonde-hair](https://www.teenvogue.com/story/selena-gomez-not-changing-blonde-hair)]([https://www.teenvogue.com/story/selena-gomez-not-changing-blonde-hair](https://www.teenvogue.com/story/selena-gomez-not-changing-blonde-hair)) (accessed on 12 August 2024) \\ Title & [PERSON] Is Going To Keep Her Blonde Hair \\ Tweet IDs & 936830208857878528 \\ Label & real \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Examples of randomly chosen false and real news articles from both FakeNewsNet’s datasets.