## 3 Digital News Story Preservation Initiative
### Enhancing \"Digital News Story Extractor (DNSE)\"  
The digital news story extractor (DNSE) is a Java-based tool for extracting digital news stories from online news websites using JSOUP and POI libraries. Initially, the DNSE was developed for English news sources [26], then enhanced for Urdu news articles, and since  
Figure 3: Archival information package (AIP) of DNSA [1].  
Figure 2: High-Level System Architecture.  
further enhanced by including Arabic news sources and some features for quantification. The DNSE extracts news stories from online sources, extracts meta information, i.e., metadata, and normalizes news content and related metadata into XML format for preservation in the DNSA. However, these enhancements have encountered the following problems:  
* _Non-uniform Web structure_: There are many platforms and technologies for developing Web-based applications. Front-end technologies include HTML, CSS, JAVA, and JAVASCRIPT and its frameworks, and back-end logic creation technologies include PHP, ASP.net, and XML, among many others. Due to the use of different technologies, the Web structure varies; hence, extracting the desired information is challenging.
* _Recency or maintenance of fresh content_: The Web contents of dynamic Web applications, such as blogs and news websites, update instantly and frequently. The recency of news content is very important to maintain efficiently, considering access frequency and network traffic issues.
* _Rise of anti-scraping tools_: The biggest challenge in extracting news content is the rise of anti-scraping tools, e.g., Captcha, which differentiates between bots and humans. The extractor got stuck when anti-scraping tools were implemented.
* _Unknown host issue_: An unreliable Internet connection leads to an unknown host issue; the extraction of news can be restarted, but the interruption is time-consuming.
* _Socket timeout_: Most websites temporarily block or suspend their services when frequently accessing the contents for a specific period during preservation. The websites consider a bot unnecessary to send requests, overload the server, and start blocking access.
* _Garbage collection_: The inconsistency in development approaches leads to erroneous extraction by collecting unwanted data, such as in-text links, tags, or other code, during news extraction.
* _Identifying and preprocessing of low-resource languages_: The DNSE tool deploys different libraries for the identification and preprocessing of low-resource languages, and the preprocessing is computationally expensive.
* _Firewall blocking_: Few online news sources are protected from extraction using a firewall.  
The developed extractor somewhat manages problems such as firewall blocking, the rise of anti-scraping applications, preprocessing, garbage collection, and dealing with non-uniform Web structure. The extractor handled the maintenance of fresh news articles, unknown host issues, and socket timeout issues using different techniques and Web APIs. Extraction is important for any digital archive and challenging when preserving low-resource contents. The enhanced DNSE is enabled to deal with the above challenges efficiently.