Hugging Face's logo

SBB
/
sbb_binarization

Image-to-Image
TF-Keras
pixelwise-segmentation
Model card Files Community
2
sbb_binarization / README.md
Jrglmn's picture
Jrglmn
Complete Update of the Model Card
2cb3dc1
preview code
|
raw
history blame
No virus
13.1 kB
metadata
tags:
  - keras
  - image-to-image
  - pixelwise-segmentation
datasets:
  - DIBCO
  - H-DIBCO
license: apache-2.0

Model Card for sbb_binarization

This is a pixelwise segmentation model for document image binarization. The model is a CNN encoder-decoder model (Resnet50-Unet). It can be used to convert all pixels in a color or grayscale document image to only black or white pixels. The main aim is to improve the contrast between foreground (text) and background (paper) for purposes of OCR.

Table of Contents

Model Details

Model Description

Document image binarization is one of the main pre-processing steps for text recognition in document image analysis. Noise, faint characters, bad scanning conditions, uneven light exposure or paper aging can cause artifacts that negatively impact text recognition algorithms. The task of binarization is to segment the foreground (text) from these degradations in order to improve optical character recognition (OCR) results. Convolutional neural networks (CNNs) are one popular method for binarization, and the sbb_binarization model is one of the. We have applied a CNN encoder-decoder model architecture.

Uses

Document image binarization is the main use case of this model. The architecture of this model alongside with training techniques like model weights ensembling can reach or outperform state-of-the-art results on standard Document Binarization Competition (DIBCO) datasets in the both machine-printed and handwritten documents.

Direct Use

The intended use is limited to the binarization of images of historical documents, understood as one of the main pre-processing steps necessary for text recognition.

Downstream Use [Optional]

A possible downstream use of this model might lie with the binarization of illustrative elements contained in document images such as digitized newspapers, magazines or books. In such cases, binarization might support analysis of creator attribution, artistic style (e.g., in line drawings), or analysis of image similarity. Furthermore, the model can be used / be trained for any other image enhancement use cases too.

Out-of-Scope Use

This model does NOT perform any optical character recognition (OCR).

Bias, Risks, and Limitations

The aim of the development of this model was to improve document image binarization as a necessary pre-processing step. Since the content of the document images is not touched, ethical challenges cannot be identified. The endeavour of developing the model was not undertaken for profit; though a product based on this model might be developed in the future, it will be openly accessible without any commercial interest. This algorithm performs a pixelwise segmentation which is done in patches. Therefore, one limitation of this model is that it is unable to capture and see long range dependencies.

Recommendations

The application of machine learning models to convert a document image into a binary output is a process which can still be improved. New model structures like Transformers or Hybrid CNN-Transformers may be applied. The transformers would support the model in capturing long range dependencies in image patches. Alongside with a CNN which increases the input features, this could improve image enhancement performance. In addition, we have used many pseudo-labeled images to train our model, so any improvement or ground truth extension would probably lead to better results.

Training Details

Training Data

The dataset used for training is a combination of training sets from previous DIBCO binarization competitions alongside with the Palm Leaf dataset and the Persian Heritage Image Binarization Competetion PHIBC dataset, with additional pseudo-labeled images from the Berlin State Library (SBB; datasets to be published). Furthermore, a dataset for very dark or very bright images has been produced for training.

Training Procedure

We have used a batch size of 8 with learning rate of 1e − 4 for 20 epochs. A soft dice is applied as loss function. In the training we have taken advantage of dataset augmentation. The augmentation includes flip, scaling and blurring. The best model weights are chosen based on some problematic documents from the SBB dataset. The final model results out of the ensemble of best weights.

Preprocessing

In order to use this model for binarization no preprocessing is needed for input image.

Speeds, Sizes, Times

More information needed

Training hyperparameters

In the training process the hyperparameters were patch size, learning rate, number of epochs and depth of encoder part.

Training results

See the two papers listed below in the evaluation section.

Evaluation

In the DocEng’2021 [Time-Quality Binarization Competition] (https://dib.cin.ufpe.br/docs/DocEng21_bin_competition_report.pdf), the model ranked twelve times under the top 8 of 63 methods, winning 2 tasks.

In the ICDAR 2021 Competition on [Time-Quality Document Image Binarization] (https://dib.cin.ufpe.br/docs/papers/ICDAR2021-TQDIB_final_published.pdf), the model ranked two times under the top 20 of 61 methods, winning 1 task.

Testing Data, Factors & Metrics

Testing Data

The testing data are the ones used in the Time-Quality Binarization Competition and listed in the paper on Time-Quality Document Image Binarization

Factors

More information needed

Metrics

The model has been evaluated both based on OCR and pixelwise segmentation results. The metrics which have been used in the case of visual evaluation are pixel proportion error and Cohen's Kappa value, and Levenshtein distance error in the case of OCR.

Results

See the two papers listed above in the evaluation section.

Model Examination

More information needed

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • Hardware Type: More information needed
  • Hours used: More information needed
  • Cloud Provider: More information needed
  • Compute Region: More information needed
  • Carbon Emitted: More information needed

Technical Specifications [optional]

Model Architecture and Objective

The proposed model is a CNN encoder-decoder model. The encoder part consists of a ResNet-50 model. The ResNet-50 includes convolutional neural networks and is responsible for extracting as many features as possible from the input image. After that the input image goes through the CNN part, then the output undergoes upsampling convolutional layers until the same output size as image input is rebuilt.

Compute Infrastructure

More information needed

Hardware

More information needed

Software

More information needed

Citation

BibTeX:

More information needed

APA:

More information needed

Glossary [optional]

More information needed

More Information [optional]

More information needed

Model Card Authors [optional]

Vahid Rezanezhad, Clemens Neudecker, Konstantin Baierer

Model Card Contact

Questions and comments about the model can be directed to Clemens Neudecker at clemens.neudecker@sbb.spk-berlin.de, questions and comments about the model card can be directed to Jörg Lehmann at joerg.lehmann@sbb.spk-berlin.de

How to Get Started with the Model

Use the code below to get started with the model.

sbb_binarize
-m <from_pretrained_keras("sbb_binarization")>

How to get started with this model is explained in the Read Me-file of the GitHub repository [over here](https://github.com/qurator-spk/sbb_binarization).