|
--- |
|
license: mit |
|
base_model: microsoft/Florence-2-large-ft |
|
tags: |
|
- image-text-to-text |
|
- generated_from_trainer |
|
model-index: |
|
- name: Florence-2-large-TableDetection |
|
results: [] |
|
datasets: |
|
- ucsahin/pubtables-detection-1500-samples |
|
pipeline_tag: image-text-to-text |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# Florence-2-large-TableDetection |
|
|
|
This model is a fine-tuned version of [microsoft/Florence-2-large-ft](https://huggingface.co/microsoft/Florence-2-large-ft) on [ucsahin/pubtables-detection-1500-samples](https://huggingface.co/datasets/ucsahin/pubtables-detection-1500-samples) dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.7601 |
|
|
|
[microsoft/Florence-2-large-ft](https://huggingface.co/microsoft/Florence-2-large-ft) can detect various objects in zero-shot setting with the task prompt "\<OD\>". Please check [Florence-2-large sample inference](https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb) for how to use Florence-2 model in inference. However, the ft-base model is not able to detect tables on a given image. |
|
|
|
The following Colab notebook showcases how you can finetune the model with your custom data to detect objects. |
|
|
|
[Florence2-Object Detection-Finetuning-HF-Trainer.ipynb](https://colab.research.google.com/drive/1Y8GVjwzBIgfmfD3ZypDX5H1JA_VG0YDL?usp=sharing) |
|
|
|
## Model description |
|
|
|
- This model is a multimodal language model fine-tuned for the task of detecting tables in images given textual prompts. The model utilizes a combination of image and text inputs to predict bounding boxes around tables within the provided images. |
|
- The primary purpose of this model is to assist in automating the process of table detection within images. It can be utilized in various applications such as document processing, data extraction, and image analysis, where identifying tables within images is essential. |
|
|
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-06 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 10 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:----:|:---------------:| |
|
| 1.3199 | 1.0 | 169 | 1.0372 | |
|
| 0.7922 | 2.0 | 338 | 0.9169 | |
|
| 0.6824 | 3.0 | 507 | 0.8411 | |
|
| 0.6109 | 4.0 | 676 | 0.8168 | |
|
| 0.5752 | 5.0 | 845 | 0.7915 | |
|
| 0.5605 | 6.0 | 1014 | 0.7862 | |
|
| 0.5291 | 7.0 | 1183 | 0.7740 | |
|
| 0.517 | 8.0 | 1352 | 0.7683 | |
|
| 0.5139 | 9.0 | 1521 | 0.7642 | |
|
| 0.5005 | 10.0 | 1690 | 0.7601 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.42.0.dev0 |
|
- Pytorch 2.3.0+cu121 |
|
- Datasets 2.20.0 |
|
- Tokenizers 0.19.1 |
