README_EN.md

360LayoutAnalysis

github

Chinese

I. Background

In today's digital age, document layout analysis is one of the key steps in information extraction and document understanding. Document layout analysis, also known as document image analysis or document layout analysis, refers to the process of identifying and extracting text, images, tables, and other elements from scanned document images. This technology has extensive applications in fields such as automated document processing, electronic data exchange, and digitization of historical documents. Traditional document layout analysis models often find it difficult to accurately distinguish paragraphs and other layout elements in documents, which limits the further processing and utilization of document information. The development of deep learning and pattern recognition technologies has brought new opportunities for document layout analysis. By training datasets, the model's understanding of document structure can be improved. However, high-quality annotated datasets are the foundation for training effective models. In document layout analysis, refined annotation is very necessary, among which paragraph annotation is particularly crucial because it directly affects the semantic understanding and information extraction of the text. Currently, in the field of layout analysis, as far as we know, in paper scenarios, previous open-source datasets such as CDLA (A Chinese document layout analysis) lack annotation of paragraph information; The layout analysis model in the research report scenario is still relatively lacking. Therefore, in order to solve this problem, we manually annotated the paper documents for fine-grained label transformation and data optimization, and constructed a fine-grained layout analysis dataset for research report scenarios. It is best to use these annotated datasets to train multiple new Chinese document layout analysis models, which performed well on the closed test set. In this open source project, we have prioritized the development of lightweight model weights and corresponding label systems for page analysis in two scenarios: paper and research report. The aim is to identify paragraph boundaries and other information in documents, accurately distinguish text, images, tables, formulas, and other elements, and ultimately promote industrial development.

II. Usage

• Weights download link: 🤗LINK

• Usage:

  The open-source weights are trained with yolov8, and the prediction method is as follows:

  from ultralytics import YOLO
  
  image_path = ''  # Path to the image to be predicted
  model_path = ''  # Path to the weights
  model = YOLO(model_path)
  
  result = model(image_path, save=True, conf=0.5, save_crop=False, line_width=2)
  print(result)
  
  print(result[0].names)         # Output the id to label map
  print(result[0].boxes)         # Output all detected bounding boxes
  print(result[0].boxes.xyxy)    # Output the top-left and bottom-right coordinates of all detected bounding boxes
  print(result[0].boxes.cls)     # Output the id corresponding to the class of all detected bounding boxes
  print(result[0].boxes.conf)    # Output the confidence of all detected bounding boxes
  

III. Layout Analysis

3.1 Academic Paper Scenario

• Label Categories

  Element	Name
  Text	Main Text (Paragraph)
  Title	Title
  Figure	Image
  Figure caption	Image Caption
  Table	Table
  Table caption	Table Caption
  Header	Header
  Footer	Footer
  Reference	Reference
  Equation	Equation

• Example

3.2 Research Report Scenario

• Label Categories

  Element	Name
  Text	Main Text (Paragraph)
  Title	Title
  Figure	Image
  Figure caption	Image Caption
  Table	Table
  Table caption	Table Caption
  Header	Header
  Footer	Footer
  Toc	Table of Contents

• Example

3.3 publaynet

• Label Categories

  Element	Name
  text	Main Text (Paragraph)
  title	Title
  figure	Image
  list	List
  table	Table

• Example

3.4 General Layout

• Label category

  Element	Name
  Text	Main text
  Title	Title
  Figure	List
  Table	Table
  Equation	Image
  Caption	Image Caption/Table Caption

License

This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses. The content of this project itself is licensed under the Apache license 2.0.

License

The source code of this repository follows the open-source license Apache 2.0. The 360LayoutAnalysis model open-source model supports commercial use. If you need to use this model and its derivative models for commercial purposes, please apply through the email (360ailab-nlp@360.cn), and see the specific license agreement in "360LayoutAnalysis Model Open Source Model License".