|
--- |
|
tags: |
|
- image-to-text |
|
- image-captioning |
|
license: apache-2.0 |
|
widget: |
|
- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/savanna.jpg |
|
example_title: Savanna |
|
- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/football-match.jpg |
|
example_title: Football Match |
|
- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/airport.jpg |
|
example_title: Airport |
|
base_model: |
|
- distilbert/distilgpt2 |
|
- google/vit-base-patch16-224-in21k |
|
--- |
|
|
|
This model is a variation of https://huggingface.co/nlpconnect/vit-gpt2-image-captioning |
|
|
|
- Read the blog post here https://ziade.org/2024/03/17/distilvit-image-captioning-model |
|
- The training code is here: https://github.com/tarekziade/distilvit |
|
|
|
Results after after 3 epochs (and ~45 hours of training) |
|
|
|
- eval_loss: 0.19939416646957397 |
|
- eval_rouge1: 43.006 |
|
- eval_rouge2: 16.9939 |
|
- eval_rougeL: 38.8923 |
|
- eval_rougeLsum: 38.8877 |
|
- eval_gen_len: 11.327256736227712 |
|
- eval_runtime: 1816.5255 |
|
- eval_samples_per_second: 13.77 |
|
- eval_steps_per_second': 1.721 |
|
- train_runtime: 46263.3695 |
|
- train_samples_per_second: 38.373 |
|
- train_steps_per_second: 4.797 |
|
- train_loss: 0.05974134062104816 |
|
|
|
|
|
|
