# Leveraging Transfer Learning for Article Segmentation in Historical Newspapers

Nancy Girdhar, Deepak Sharma, Mickaël Coustaty, Antoine Doucet

► To cite this version:

Nancy Girdhar, Deepak Sharma, Mickaël Coustaty, Antoine Doucet. Leveraging Transfer Learning for Article Segmentation in Historical Newspapers. The 28th International Conference on Theory and Practice of Digital Libraries, Sep 2024, Ljubljana, Slovenia. hal-04676213

HAL Id: hal-04676213

https://hal.science/hal-04676213v1

Submitted on 23 Aug 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Leveraging Transfer Learning for Article Segmentation in
Historical Newspapers

Nancy Girdhar¹*[0000-0002-1009-3875], Deepak Sharma²[0000-0002-6132-1719], Mickaël Coustaty¹[0000-0002-0123-439X], and Antoine Doucet¹[0000-0001-6160-3356]

¹ L3i, University of La Rochelle, La Rochelle, 17000, France

² Department of Computer Science, Christian-Albrechts-Universität zu Kiel, Kiel, Schleswig-Holstein,
24118, Germany

nancy.girdhar@univ-lr.fr, des@informatik.uni-kiel.de, mickael.coustaty@univ-lr.fr,
antoine.doucet@univ-lr.fr

**Abstract.** Historical newspapers serve as invaluable resources for understanding past societies and preserving cultural heritage. However, digitizing these newspapers presents challenges due to their complex layouts and vast content. Article segmentation, involving the identification and extraction of individual articles from scanned newspaper images, is crucial for efficient information mining and retrieval. While some rule-based algorithms have been proposed, the applicability of deep neural networks (DNNs) for this task has recently gained attention. In this work, we explore the applicability of transfer learning to segment articles from historical newspaper images. For this, we employed nine pre-trained backbone architectures, specifically selected from the ResNet family, and proposed a bounding-box approximation based article segmentation module designed specifically for the task. Furthermore, we introduced a mean estimated article coverage metric that computes the segmentation capability of a model on an article-level. Experiments were conducted on the NAS dataset (NewsEye Article Separation), ensuring the relevance of our approach to historical data. Our study evaluates the performance of various pre-trained models, achieving a mean estimated article coverage of 0.956, 0.969, and 0.995 on the ONB, NLF, and BNF datasets, respectively. These findings underscore the effectiveness of transfer learning in adapting to historical layout analysis tasks, particularly article segmentation. Moreover, these results reaffirm the significance of transfer learning and pre-trained models as efficient tools for handling complex historical newspaper layouts.

**Keywords:** article segmentation · pre-trained models · transfer learning · article separation · historical newspapers.

# 1 Introduction

Digital libraries serve as invaluable repositories for data mining and information retrieval, offering access to extensive collections of digitized documents. Among these resources, historical newspapers hold particular significance due to their rich historical content and cultural value. However, the digitization and preservation of historical newspapers present unique challenges. These challenges encompass various physical factors such as *paper defects* (e.g., stains, wrinkles), *printing imperfections* (e.g., ink bleeding, baseline variations), *facsimile distortions* (e.g., blur, skew), *typographic variations* (e.g., typeface, font style), and *environmental factors* (e.g., infestation, humidity) [1].

<footnote>* Corresponding author: Nancy Girdhar, nancy.girdhar@univ-lr.fr</footnote>
Efficiently segmenting articles within historical newspapers is crucial for enabling effective information retrieval and analysis. However, this task is hindered by numerous challenges, including *complex layouts* [2], *diverse fonts*, *language evolution* [3,4], *degraded print quality*, *irregular column structures*, and *degradation over time* [5,6,7], as well as *annotation challenges* [8]. Figure 1 illustrates various layouts featured in the Newseye Article Separation (NAS) dataset [9] of historical newspaper images.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Sample images of various historical newspaper layouts included in the NAS dataset. German *Freie Presse* newspaper (Left), Finnish *Suometar* newspaper (Middle), and French *Le Gaulois* newspaper (Right).</figcaption></figure>

Addressing these challenges necessitates the development of robust segmentation techniques capable of handling the complexities inherent in historical newspaper documents. While existing research in this domain is limited and primarily based on analyses of private datasets [10,11,2], some authors have explored rule-based techniques [10] or utilized visual features extracted through fully convolutional neural networks (FCNs) [2]. However, these approaches are often tailored to contemporary newspaper datasets [12,13] and may not generalize well to historical newspapers. Moreover, few studies have incorporated semantic textual features into segmentation algorithms [8], indicating a gap in the research landscape. Leveraging visual features such as layout cues, font characteristics, and spatial relationships presents a promising approach to effectively identifying and separating individual articles within complex newspaper layouts. Despite its importance, article segmentation in historical newspapers remains relatively unexplored in the literature.

In this work, our focus is on leveraging transfer learning and visual features for article segmentation in historical newspapers. Building upon the success of pixel-by-pixel segmentation approaches demonstrated in [14], as well as the effectiveness of transfer learning highlighted in [15], our research
aims to adapt and enhance these methodologies for the task of article segmentation in historical newspapers. Specifically, we adopt the methodology proposed by [14] for pixel-by-pixel segmentation in historical documents and tailor it for article segmentation. We implement this methodology using a diverse array of pre-trained models, focusing on those derived from the ResNet family. Our aim is to assess the efficacy of transfer learning with pre-trained models on a curated collection of historical newspaper image datasets, such as the one made publicly available by [9].

The remainder of the paper is organized as follows. Section 2 provides background on the concepts related to article segmentation. Section 3 presents a brief overview of similar works in this field. Section 4 elaborates on the methodology, detailing the approach for leveraging visual features and pre-trained models for article segmentation in historical newspapers. To evaluate our methodology, section 5 describes the experimental setup and presents the evaluation results, followed by a discussion on the implications of our work. Finally, section 6 concludes the paper and outlines potential avenues for future research in this domain.

# 2 Preliminaries

This section provides an essential foundation by discussing transfer learning, pre-trained models, and key concepts related to article segmentation in historical newspapers.

## 2.1 Transfer Learning

Transfer learning is a machine learning technique that improves the performance of a model on a target task by leveraging knowledge gained from training on a related source task [16]. In this approach, a pre-trained model developed for a specific domain is fine-tuned on a new dataset or task, allowing it to adapt and learn task-specific features. Transfer learning is particularly valuable when labeled data for the target task is limited, as it enables the model to utilize knowledge acquired from the source task to improve performance in the target domain [17]. The transfer learning process typically involves two stages: *pre-training* and *fine-tuning*. During the *pre-training* stage, a neural network model is trained on a large-scale dataset for a specific task, such as image classification. In this stage, the model learns general features and patterns from the data, which are then transferred to the fine-tuning stage. During *fine-tuning*, the pre-trained model is adapted to the target task by adjusting its parameters using a smaller dataset specific to the task. This fine-tuning process allows the model to specialize in the target domain while retaining the knowledge gained during pre-training. In this study, we opted to utilize transfer learning due to its ability to address the challenge of limited labeled data in new tasks [18,19]. By fine-tuning pre-trained models on historical newspaper datasets, we can adapt them to the specific characteristics of the data, enhancing both performance and efficiency. This approach is particularly beneficial for tasks where labeled data are scarce or costly to obtain, such as historical newspaper analysis.

## 2.2 Pre-trained Models

Pre-trained models are neural network architectures that have been trained on large-scale datasets for specific tasks. These models are trained using vast amounts of labeled data and computational resources, resulting in models that have learned rich representations of the data [20]. Pre-trained models are widely used in transfer learning, where they serve as a starting point for training models on new tasks or datasets. These models are typically trained on tasks such as image classification,
object detection, language modeling, or speech recognition [21,22]. Once trained, these models can be fine-tuned or adapted to new tasks by retraining them on smaller, task-specific datasets. We chose to utilize pre-trained models to leverage the benefit from the knowledge and features learned by the model during pre-training, saving time and computational resources compared to training from scratch. This approach allows us to adapt state-of-the-art models to the specific characteristics of historical newspaper data, enhancing the overall performance of our segmentation task. Some popular pre-trained models include ImageNet-trained convolutional neural networks (CNNs) such as ResNet, ResNeXt, and MobileNetV2 for image-related natural language processing tasks.

## 2.3 Concepts and Definitions

**Article:** Within the context of newspaper analysis, an *article* represents a cohesive unit comprising text regions, images, or a combination of both. These text regions consist of groups of text lines and may extend across multiple columns or even onto subsequent pages. The task of logically identifying and delineating these articles is referred as *article segmentation*. There are two primary approaches to article segmentation [8]:

- **Visual cues-based Approaches:** These methodologies leverage visual features such as *layout cues*, *font characteristics*, and *spatial relationships* to identify and segment articles within newspaper page images.

- **Textual cues-based Approaches:** These strategies exploit semantic textual features, such as *text embeddings similarity*, to identify and segment articles in newspaper page images.

## 3 Related Work

Effective article segmentation in historical newspapers is crucial for digital preservation and efficient information retrieval, which requires the development of advanced segmentation techniques capable of handling complex layouts. Recent literature has explored diverse approaches to address this challenge. In a study that focused on rule-based methods that use visual cues for article separation, the researchers proposed a methodology applied to historical Greek newspapers [10]. While achieving notable precision and recall, this approach did not fully exploit semantic cues relevant to article segmentation tasks. Another study employed a combination of conditional random field (CRF) with rule-based techniques for article segmentation in French historical newspapers, yielding promising accuracy. However, its evaluation was limited by the small size of the dataset [11,23]. In contrast, an investigation utilized visual features with a 2D markov model (2D MM) to segment articles, applying their approach to contemporary English newspaper data [13]. However, limitations were noted due to the oversight of certain visual cues and intricate dependencies, potentially affecting efficacy in historical newspaper archives.

On the contrary, a separate study introduced a novel approach that takes advantage of textual features to separate articles in historical newspapers [8]. Although demonstrating initial promising results in tests conducted on French and Finnish newspapers from the 19th and early 20th centuries, this method solely explored textual features and did not incorporate visual cues that could potentially enhance its performance. Another approach introduced a fully convolutional network (FCN) for article segmentation, tested on contemporary datasets [2]. However, limitations were observed in its inability to differentiate between article text and ad images and its reliance on recognizing only rectangular shapes. Moreover, its generalizability to historical newspapers remains unexplored.
Similarly, a fixed point method proposed for article segmentation in contemporary English newspapers achieved notable accuracy [12], but faced constraints in terms of dataset size and applicability to historical archives.

Since the advent of deep learning methods, particularly convolutional neural networks (CNNs) [24], significant advancements have been made in various pattern recognition tasks, including document segmentation. Despite their effectiveness, such methods often require substantial computational resources during training [25]. Hence, to address the challenges, and identified gaps in the existing literature, our study investigates the efficacy of transfer learning and pre-trained models utilizing visual cues for the task of article segmentation in historical newspapers. Our focus lies on employing visual features for article segmentation, utilizing transfer learning principles, and leveraging pre-trained models to mitigate computational resource requirements and time constraints. Additionally, we aim to assess the performance of these methods on historical newspaper images comprising limited dataset sizes. Table 1 presents an overview of the existing literature investigating the identification and segmentation of articles from newspapers.

<table><caption>Table 1: Article segmentation in historical newspapers (VF: Visual Features; TF: Textual Features; Lang: Language; HN: Historical Newspapers; CN: Contemporary Newspapers; CRF: Conditional Random Field; MM: Markov Model; FCN: Fully Convolutional Neural Network).</caption><thead><tr><th rowspan="2">[Ref]</th><th rowspan="2">Year</th><th rowspan="2">Approach</th><th colspan="2">Method</th><th colspan="2">Dataset</th><th colspan="2">Type</th></tr><tr><th>VF</th><th>TF</th><th>Name</th><th>Lang</th><th>Pages</th><th>HN</th><th>CN</th></tr></thead><tbody><tr><td>[10]</td><td>1999</td><td>Rule-based</td><td>✓</td><td></td><td>TO VIMA</td><td>Greek</td><td>100</td><td>✓</td><td></td></tr><tr><td>[11]</td><td>2012</td><td>CRF+Rule-based</td><td>✓</td><td></td><td>Journal De Rouen</td><td>French</td><td>42</td><td>✓</td><td></td></tr><tr><td>[12]</td><td>2014</td><td>Fixed point</td><td>✓</td><td></td><td>-</td><td>English</td><td>45</td><td></td><td>✓</td></tr><tr><td>[2]</td><td>2017</td><td>FCN</td><td>✓</td><td></td><td>ARGUS DATA IN-SIGHTS Schweiz AG</td><td>-</td><td>426</td><td></td><td>✓</td></tr><tr><td>[13]</td><td>2019</td><td>2D MM</td><td>✓</td><td></td><td>Fairfax Media</td><td>English</td><td>3332</td><td></td><td>✓</td></tr><tr><td>[8]</td><td>2023</td><td>Rule-based</td><td></td><td>✓</td><td>NewsEye Article Separation</td><td>French, Finnish</td><td>383</td><td>✓</td><td></td></tr></tbody></table>

# 4 Methodology

This section provides a detailed description of the proposed methodology for extracting articles from historical newspaper pages. Our approach to article segmentation is based on transfer learning and utilizes visual features. We build upon the pixel-by-pixel segmentation framework proposed by [14] for historical document analysis, tailoring its generic methodology to the specific task of article segmentation. We introduce modifications to adapt the approach effectively to this task. To summarize, the overall methodology consists of the following steps, as we describe in further detail in this section:

- To capture the complex visual features present in historical newspaper pages and facilitate article segmentation, we employ a diverse array of pre-trained convolutional neural network (CNN) architectures. Specifically, we selected nine pre-trained models based on residual networks including ResNet [26], ResNeXt [27], WideResNet [28], and MobileNetV2 [29]. These models are chosen based on their proven effectiveness in image classification and feature extraction tasks, as well as their compatibility with the requirements of our segmentation task [14,30].
- During the training phase, we utilize annotated labels to generate corresponding masks for historical newspaper images. These masks serve as input data for training the selected CNN models. The annotations encompass information regarding article boundaries and relevant attributes necessary for accurate segmentation. In addition to the comprehensive image masks covering entire newspaper images, per-article masks are generated. These masks precisely delineate the boundaries of individual articles within the newspaper image, which are further utilized to compute the article coverage with the predicted mask.

- Following the generation of probability maps by the CNN models, we designed article segmentation module tailored specifically to segment articles from the predicted mask. This involves overlaying the per-article ground truth mask onto the predicted mask generated by the model. By pixel-wise mapping of the individual article mask and the corresponding regions covered by the model's predictions, we estimated the article region to be segmented. Our implementation leverages the PyTorch framework [31], a widely utilized deep learning toolkit known for its flexibility and efficiency, to facilitate model development and experimentation.

## 4.1 Network Architecture

The network architecture, illustrated in Figure 2, comprises two primary pathways:

- **Contracting Path (Encoder)**: The framework integrates a contracting path that follows the architecture of the chosen pre-trained model. This path consists of five sequential steps, each step reduces the size of feature maps while retaining essential information for subsequent processing. Pre-trained weights from a general image classification task [32] are utilized within this pathway to enhance robustness and aid in generalization.

- **Expanding Path (Decoder)**: The expanding path complements the contracting path by mapping low-resolution encoder feature maps to full input resolution feature maps. This pathway consists of five blocks and a final convolutional layer, facilitating the assignment of article segmentation classes to individual pixels. Each deconvolutional step involves upscaling feature maps from the preceding block, followed by concatenation with corresponding feature maps from the contracting path. Subsequent convolutions and ReLU activations [33] refine the segmentation output. To optimize computational efficiency and memory utilization, a reduction in feature channels is employed via 1x1 convolutions before concatenation in specific steps, with upsampling achieved through bilinear interpolation.

## 4.2 Models Description

Our network architecture incorporates pre-trained models<fnref n="3" /> for the encoder component, selected from a variety of widely used CNN architectures. Below are descriptions of the chosen pre-trained models:

- **ResNet18**, **ResNet34**, **ResNet50**, **ResNet101**: These models are based on the concept of residual learning, which facilitates the training of deeper networks. ResNet architectures reformulate the layers as residual functions with reference to the layer inputs, leading to improved training efficiency and performance [26].

<footnote n="3">³ https://pytorch.org/vision/stable/models.html</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2: The network architecture.</figcaption></figure>

- **ResNeXt50_32x4d and ResNeXt101_32x8d**: These models employ an aggregated residual transformation approach, presenting a highly modularized network architecture. By repeating a building block that aggregates transformations with the same topology, ResNeXt networks achieve effective feature learning and classification [27].

- **WideResNet50 and WideResNet101**: These models are based on the idea of wide residual networks, where the network architecture involves decreasing depth and increasing width. Wide residual networks demonstrate superior performance compared to their thin and very deep counterparts, making them suitable for various image classification tasks [28].

- **MobileNetV2**: This architecture is built on an inverted residual structure, featuring thin bottleneck layers at the input and output of the residual block. MobileNetV2 uses lightweight depthwise convolutions to filter features in the intermediate expansion layer [29].

Table 2 provides specific details regarding the feature maps channel size of the last convolutional layer in these pre-trained models.

<table><caption>Table 2: Feature map details of pre-trained models.</caption><thead><tr><th>[Ref]</th><th>Model</th><th>Feature Map</th></tr></thead><tbody><tr><td rowspan="4">[26]</td><td>ResNet18</td><td>512</td></tr><tr><td>ResNet34</td><td>512</td></tr><tr><td>ResNet50</td><td>2048</td></tr><tr><td>ResNet101</td><td>2048</td></tr><tr><td rowspan="2">[27]</td><td>ResNeXt50</td><td>2048</td></tr><tr><td>ResNeXt101</td><td>2048</td></tr><tr><td rowspan="2">[28]</td><td>WideResNet50</td><td>2048</td></tr><tr><td>WideResNet101</td><td>2048</td></tr><tr><td>[29]</td><td>MobileNetV2</td><td>1280</td></tr></tbody></table>
## 4.3 Model Training

We train each CNN model on the pre-processed dataset using the Adam optimizer. To prevent overfitting, we apply L2 regularization with weight decay ($10^{-6}$). Additionally, we utilize a learning rate schedule with an exponential decay rate of 0.95, initially set within the range of $[10^{-5}, 10^{-4}]$. We employ Xavier initialization for efficient weight initialization. Addressing batch diversity issues, we implement batch renormalization with parameters ($rmin = 0.1, rmax = 100, dmax = 1$). We resize the images to $1024 \times 688$ to accommodate memory constraints and facilitate batch training. While most training process parameters and choices are applicable to various experiments, we carefully consider resizing the input image to ensure an adequately large receptive field for the network's task. All training and inference processes are conducted on Nvidia GPUs to optimize computational efficiency. Utilizing pre-trained weights in the contracting path of the network significantly reduces training time and enhances regularization by reducing sensitivity to outliers.

## 4.4 Post-processing and Article Segmentation Module

For post-processing, we adopt a minimalist approach, focusing on standard operations to refine the predictions output by the network. We employ thresholding to generate a binary map from the network's predictions, with the threshold value fixed within the range of $[0, 1]$. Fundamental morphological operators are exclusively applied to binary images. Connected components analysis is then employed to filter out small connected components that may persist after thresholding or morphological operations, refining the segmentation output by eliminating extraneous noise.

To segment articles from the post-processed predicted mask generated by the network, we employed a method based on bounding box approximation. The goal was to accurately estimate the spatial extent or span of each article within the predicted mask. To achieve this, we first generated individual masks corresponding to each article present in the input image. These per-article masks were obtained through segmentation techniques applied specifically to isolate each article from the background and other articles within the image. Once we had obtained the per-article masks, we proceeded to overlay the bounding box of each article onto the predicted mask generated by the network. The bounding box serves as a rectangular footprint that encloses the entire extent of the article within the predicted mask. This approach allowed us to effectively localize and separate individual articles within the predicted mask. Figure 3 illustrates the input image (3a), per-article ground truth mask (3b), predicted mask (3c), and the respective post-processed segmented article region (3d).

# 5 Experimental Setup and Results

This section outlines the experimental setup and presents the results of leveraging CNN-based pre-trained models to explore the applicability of transfer learning for visual feature-based article separation in historical newspapers. Prior to model training, the dataset undergoes preprocessing to ensure compatibility with CNN architectures, which includes resizing the images to a standard input size. In our experiments, we compared various ResNet architectures; however, a comparison with baseline or state-of-the-art methods was not conducted due to the proprietary nature and limited accessibility of the datasets used by these methods.
<figure><img src="image_4.png" /><figcaption>Fig. 3: Illustrations of (a) an input image, (b) per-article ground truth mask (c) predicted output mask generated by the network, (d) segmented article after post-processing.</figcaption></figure>

## 5.1 Datasets and Evaluation Metrics

For our experiments, we utilized the multilingual NewEye Article Separation (NAS) dataset [9], consisting of scanned newspaper page images in German, Finnish, and French languages. These newspaper pages, published during the 19th and early 20th centuries, originate from the archives of three prominent European National Libraries: the Austrian National Library (ONB), the National Library of Finland (NLF), and the National Library of France (BNF). Importantly, this dataset is publicly available through the NewsEye<fnref n="4" /> project [34], which offers researchers a valuable resource for historical document analysis. Table 3 presents the detailed dataset description.

<table><caption>Table 3: Statistical description of dataset (# denotes number of pages).</caption><thead><tr><td>Dataset</td><td>Language</td><td>#Train</td><td>#Test</td><td>#Total</td></tr></thead><tbody><tr><td>ONB<sup>a</sup></td><td>German</td><td>184</td><td>46</td><td>230</td></tr><tr><td>NLF<sup>b</sup></td><td>Finnish</td><td>150</td><td>50</td><td>200</td></tr><tr><td>BNF<sup>c</sup></td><td>French</td><td>147</td><td>36</td><td>183</td></tr></tbody></table>

For evaluating our proposed article segmentation task, we chose to utilize *Intersection over Union* (IoU), a widely used metric in image segmentation tasks. IoU quantifies the degree of overlap between predicted and ground-truth masks of an image, providing a measure of their similarity. A higher IoU value indicates greater overlap between the predicted and ground-truth masks, with values closer to 1 indicating higher similarity and values closer to 0 indicating less overlap.

For a given article in a page *i*, let $P_i$ denote the set of predicted pixels representing the article, and $G_i$ denote the set of ground-truth pixels for the same article. The IoU for the article in page *i* is defined as in Eq.(1):

<footnote>a https://www.onb.ac.at</footnote>

<footnote>b https://www.kansalliskirjasto.fi</footnote>

<footnote>c https://www.bnf.fr</footnote>

<footnote n="4">⁴ https://newseye.eu</footnote>
$$IoU_i = \frac{|P_i \cap G_i|}{|P_i \cup G_i|} \qquad (1)$$

where:

- $P_i \cap G_i$ represents the intersection of predicted and ground-truth pixel sets.

- $P_i \cup G_i$ represents the union of predicted and ground-truth pixel sets.

- $|P_i|$ denotes the cardinality of set $P_i$.

- $|G_i|$ denotes the cardinality of set $G_i$.

To compute the article coverage, we derive the *Estimated Article Coverage* (EAC) metric from the IoU method. The EAC calculates the pixel-wise overlap between the predicted article masks and corresponding ground truth article masks within a page. Mathematically, it can be expressed as Eq.(2), where *n* is the total number of articles in a page:

$$EAC = \frac{\sum_{i=0}^{n} IoU_i}{n} \qquad (2)$$

Given *p* is the number of pages, the total mean EAC (mEAC) considering all pages is calculated as Eq.(3):

$$mEAC = \frac{\sum_{i=0}^{p} EAC_i}{p} \qquad (3)$$

## 5.2 Results and Discussion

Our investigation into article segmentation using various pre-trained models has yielded insightful findings regarding applicability of transfer learning in historical newspaper images. Table 4 presents the results of various pre-trained models for article segmentation on the NAS dataset. The mean Intersection over Union (mIoU) values demonstrate the alignment between ground truth and predicted masks generated by different pre-trained models. Additionally, the mean Estimated Article Coverage (mEAC) metrics illustrate the extent of estimated segmented article coverage based on these predicted masks.

Figures 4, 5, and 6 illustrate the predicted newspaper image mask results obtained from various models on ONB, NLF, and BNF datasets, respectively. The following highlights key findings of this study:

**Model Performance within Datasets:** On the ONB dataset, ResNet18 demonstrated the highest performance, with a training mIoU of 0.769 and a testing mIoU of 0.712, along with an mEAC of 0.956. Other ResNet models, including ResNet101 and WideResNet101, also performed competitively, achieving notable mEAC scores ranging from 0.905 to 0.952. For the NLF dataset, ResNet101 emerged as the top-performing model, with a testing mIoU of 0.594 and an mEAC of 0.969. Similarly, on the BNF dataset, ResNet18 outperformed other models, achieving a testing mIoU of 0.629 and an mEAC of 0.995.

**Model Performance Across Datasets:** Examination of the results obtained from various pre-trained models reveals significant variability in performance across datasets. While certain models, such as ResNet18, demonstrate exceptional performance on specific datasets, their efficacy on others appears more modest. This observation suggests that different model architectures within the ResNet family may exhibit varying degrees of suitability to the characteristics of individual datasets.
Particularly, the disparities in performance may stem from variations in network layers and implemented functions, highlighting the nuanced impact of architectural choices on model outcomes.

<table><caption>Table 4: Article segmentation results of pre-trained models on the NAS dataset.</caption><thead><tr><th>Dataset</th><th>Models</th><th>Train mIoU</th><th>Test mIoU</th><th>Test mEAC</th></tr></thead><tbody><tr><td rowspan="7">ONB</td><td>MobileNetV2</td><td>0.406</td><td>0.392</td><td>0.620</td></tr><tr><td>ResNet18</td><td><strong>0.769</strong></td><td><strong>0.712</strong></td><td><strong>0.956</strong></td></tr><tr><td>ResNet34</td><td>0.727</td><td>0.685</td><td>0.942</td></tr><tr><td>ResNet50</td><td>0.695</td><td>0.612</td><td>0.915</td></tr><tr><td>ResNet101</td><td>0.715</td><td>0.687</td><td>0.905</td></tr><tr><td>ResNeXt50</td><td>0.680</td><td>0.640</td><td>0.896</td></tr><tr><td>ResNeXt101</td><td>0.632</td><td>0.585</td><td>0.930</td></tr><tr><td rowspan="5">NLF</td><td>WideResNet50</td><td>0.659</td><td>0.613</td><td>0.767</td></tr><tr><td>WideResNet101</td><td>0.746</td><td>0.698</td><td>0.952</td></tr><tr><td>MobileNetV2</td><td>0.612</td><td>0.568</td><td>0.965</td></tr><tr><td>ResNet18</td><td>0.604</td><td>0.582</td><td>0.932</td></tr><tr><td>ResNet34</td><td>0.601</td><td>0.543</td><td>0.947</td></tr><tr><td rowspan="7">BNF</td><td>ResNet50</td><td>0.555</td><td>0.492</td><td>0.981</td></tr><tr><td>ResNet101</td><td><strong>0.625</strong></td><td><strong>0.594</strong></td><td><strong>0.969</strong></td></tr><tr><td>ResNeXt50</td><td>0.577</td><td>0.541</td><td>0.912</td></tr><tr><td>ResNeXt101</td><td>0.583</td><td>0.543</td><td>0.981</td></tr><tr><td>WideResNet50</td><td>0.620</td><td>0.570</td><td>0.982</td></tr><tr><td>WideResNet101</td><td>0.605</td><td>0.562</td><td>0.986</td></tr><tr><td>MobileNetV2</td><td>0.617</td><td>0.561</td><td>0.979</td></tr><tr><td rowspan="7">BNF</td><td>ResNet18</td><td><strong>0.699</strong></td><td><strong>0.629</strong></td><td><strong>0.995</strong></td></tr><tr><td>ResNet34</td><td>0.617</td><td>0.585</td><td>0.988</td></tr><tr><td>ResNet50</td><td>0.587</td><td>0.526</td><td>0.964</td></tr><tr><td>ResNet101</td><td>0.682</td><td>0.633</td><td>0.991</td></tr><tr><td>ResNeXt50</td><td>0.626</td><td>0.598</td><td>0.978</td></tr><tr><td>ResNeXt101</td><td>0.576</td><td>0.527</td><td>0.994</td></tr><tr><td>WideResNet50</td><td>0.588</td><td>0.539</td><td>0.985</td></tr><tr><td>WideResNet101</td><td>0.564</td><td>0.512</td><td>0.967</td></tr></tbody></table>

**Robustness of ResNet101 Across Datasets:** ResNet101 from ResNet series demonstrated consistent performance across all three datasets, suggesting its robustness and adaptability. It achieved competitive mIoU scores and exhibited superior article coverage on the NLF dataset, making it a promising choice for diverse historical newspaper collections.

**Influence of Model Depth and Complexity:** Our analysis highlights the influence of model depth and complexity on segmentation performance. While deeper models generally exhibited superior performance, lighter deep neural networks architectures such as MobileNetV2 also showcased competitive results on specific datasets. This highlights the importance of empirical evaluation and judicious model selection, considering factors such as computational resources availability and trainable parameters. Remarkably, lightweight models may present a practical option in scenarios where resource efficiency is paramount.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Results on the ONB dataset. (a)-(b) show the input image and its ground truth mask, respectively, (c)-(k) demonstrate the detection outcomes achieved by different pre-trained models.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Results on the NLF dataset. (a)-(b) show the input image and its ground truth mask, respectively, (c)-(k) demonstrate the detection outcomes achieved by different pre-trained models.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 6: Results on the BNF dataset. (a)-(b) show the input image and its ground truth mask, respectively, (c)-(k) demonstrate the detection outcomes achieved by different pre-trained models.</figcaption></figure>

Overall, the observed variations in performance across different datasets and models can be attributed to several factors, including the complexity of historical newspaper layouts, variations in image quality, and the inherent challenges associated with article segmentation tasks. While certain models, such as ResNet18 and ResNet101, consistently performed well across all datasets, the choice of the optimal model may depend on specific dataset characteristics and application requirements. Our findings highlight the importance of leveraging pre-trained models and transfer learning techniques in addressing the challenges of article segmentation in historical newspapers. However, the effectiveness of these models depends on the quality and completeness of dataset annotations.

# 6 Conclusions and Future Outlook

In this work, we investigated the applicability of transfer learning in document layout analysis, focusing specifically on the challenging task of article segmentation in historical newspapers. Leveraging pre-trained models from the ResNet family, we conducted extensive experiments to assess the performance of various architectures in segmenting articles from complex historical layouts. Our methodology involved visual feature extraction using pre-trained models to predict article masks, followed by estimation of article coverage based on bounding box approximation using ground truth article annotations. The experiments, conducted on ONB, NLF, and BNF historical newspaper datasets obtained from the NewsEye project, yielded substantial initial results. Additionally, introducing the mean estimated article coverage metric provided a novel perspective on assessing a model's segmentation capability at the article level. Our findings underscore the variability in segmentation results across different datasets and ResNet architectures, emphasizing the nuanced influence of architectural choices on prediction accuracy. While certain models, such as ResNet18 and ResNet101, consistently performed well across datasets, the optimal choice of architecture may
depend on specific dataset characteristics and application requirements. These findings highlight the significance of transfer learning and pre-trained models as potential solutions for article segmentation tasks in historical newspapers. Despite the dependence of our approach on the quality and completeness of dataset annotations, pre-trained models offer computational efficiency and benefit from extensive training data. Our approach leverages publicly available pre-trained models to ensure reproducibility and transparency, enhancing scientific accessibility and verifiability. Future work will aim to incorporate more extensive comparisons as new datasets become available and computational resources permit. Additionally, research directions may include further investigation of approaches that combine visual and textual features to address the complexities of historical newspaper analysis, as well as exploration of multi-page article segmentation.

## Acknowledgements

This work has been supported by the ANNA (2019-1R40226), TERMITRAD (AAPR2020-2019-8510010), Pypa (AAPR2021-2021-12263410), and Actuadata (AAPR2022-2021-17014610) projects funded by the Nouvelle-Aquitaine Region, France.

## References

1. Girdhar, N., Coustaty, M., Doucet, A.: Digitizing history: Transitioning historical paper documents to digital content for information retrieval and mining—a comprehensive survey. IEEE Transactions on Computational Social Systems pp. 1–30 (2024)

2. Meier, B., Stadelmann, T., Stampfli, J., Arnold, M., Cieliebak, M.: Fully convolutional neural networks for newspaper article segmentation. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 1, pp. 414–419. IEEE (2017)

3. Oberbichler, S., Boroş, E., Doucet, A., Marjanen, J., Pfanzelter, E., Rautiainen, J., Toivonen, H., Tolonen, M.: Integrated interdisciplinary workflows for research on historical newspapers: Perspectives from humanities scholars, computer scientists, and librarians. Journal of the Association for Information Science and Technology 73(2), 225–239 (2022)

4. González-Gallardo, C.E., Boros, E., Giamphy, E., Hamdi, A., Moreno, J.G., Doucet, A.: Injecting temporal-aware knowledge in historical named entity recognition. In: Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part I. pp. 377–393. Springer (2023)

5. Gatos, B., Pratikakis, I., Perantonis, S.J.: Efficient binarization of historical and degraded document images. In: 2008 The Eighth IAPR International Workshop on Document Analysis Systems. pp. 447–454. IEEE (2008)

6. Le, D.S., Thoma, G.R., Wechsler, H.: Automated page orientation and skew angle detection for binary document images. Pattern Recognition 27(10), 1325–1344 (1994)

7. Sezgin, M., Sankur, B.I.: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic imaging 13(1), 146–168 (2004)

8. Girdhar, N., Coustaty, M., Doucet, A.: Stras: A semantic textual-cues leveraged rule-based approach for article separation in historical newspapers. In: International Conference on Asian Digital Libraries. pp. 89–105. Springer (2023)

9. Girdhar, N., Coustaty, M., Doucet, A.: Benchmarking nas for article separation in historical newspapers. In: International Conference on Asian Digital Libraries. pp. 76–88. Springer (2023)

10. Gatos, B., Mantzaris, S., Chandrinos, K., Tsigris, A., Perantonis, S.J.: Integrated algorithms for newspaper page decomposition and article tracking. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR’99 (Cat. No. PR00318). pp. 559–562. IEEE (1999)
11. Palfray, T., Hebert, D., Nicolas, S., Tranouez, P., Paquet, T.: Logical segmentation for article extraction in digitized old newspapers. In: Proceedings of the 2012 ACM symposium on Document engineering. pp. 129–132 (2012)

12. Bansal, A., Chaudhury, S., Roy, S.D., Srivastava, J.: Newspaper article extraction using hierarchical fixed point model. In: 2014 11th IAPR International Workshop on Document Analysis Systems. pp. 257–261. IEEE (2014)

13. Naoum, A., Nothman, J., Curran, J.: Article segmentation in digitised newspapers with a 2d markov model. In: 2019 International conference on document analysis and recognition (ICDAR). pp. 1007–1014. IEEE (2019)

14. Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach for document segmentation. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)

15. Liebl, B., Burghardt, M.: An evaluation of dnn architectures for page segmentation of historical newspapers. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 5153–5160. IEEE (2021)

16. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22(10), 1345–1359 (2009)

17. Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. Journal of Big data 3, 1–40 (2016)

18. Chowdhury, A.G., Schut, N., Atzmueller, M.: A hybrid information extraction approach using transfer learning on richly-structured documents. In: LWDA. pp. 13–25 (2021)

19. De Nardin, A., Zottin, S., Colombi, E., Piciarelli, C., Foresti, G.L.: Is imagenet always the best option? an overview on transfer learning strategies for document layout analysis. In: International Conference on Image Analysis and Processing. pp. 489–499. Springer (2023)

20. Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence 44(7), 3523–3542 (2021)

21. Marcelino, P.: Transfer learning from pre-trained models. Towards data science 10(330), 23 (2018)

22. Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X.: Pre-trained models for natural language processing: A survey. Science China Technological Sciences 63(10), 1872–1897 (2020)

23. Hebert, D., Palfray, T., Nicolas, S., Tranouez, P., Paquet, T.: Automatic article extraction in old newspapers digitized collections. In: Proceedings of the First International Conference on Digital Access to Textual Cultural Heritage. pp. 3–8 (2014)

24. Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision. pp. 1520–1528 (2015)

25. Fakhry, A., Zeng, T., Ji, S.: Residual deconvolutional networks for brain electron microscopy image segmentation. IEEE transactions on medical imaging 36(2), 447–456 (2016)

26. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

27. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1492–1500 (2017)

28. Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016)

29. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4510–4520 (2018)

30. Barman, R., Ehrmann, M., Clematide, S., Oliveira, S.A., Kaplan, F.: Combining visual and textual features for semantic segmentation of historical newspapers. Journal of Data Mining & Digital Humanities (HistoInformatics) (2021)

31. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019)
32. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

33. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10). pp. 807–814 (2010)

34. Doucet, A., Gasteiner, M., Granroth-Wilding, M., Kaiser, M., Kaukonen, M., Labahn, R., Moreux, J.P., Muehlberger, G., Pfanzelter, E., Thérenty, M.È., et al.: Newseye: A digital investigator for historical newspapers. In: 15th Annual International Conference of the Alliance of Digital Humanities Organizations, DH 2020 (2020)