|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
base_model: |
|
- HuggingFaceTB/SmolLM2-135M-Instruct |
|
- facebook/dinov2-small |
|
pipeline_tag: visual-question-answering |
|
tags: |
|
- tinyllava |
|
--- |
|
|
|
Finetuned, 500 epochs |
|
|
|
[MMMU](https://tinyllava-factory.readthedocs.io/en/latest/Evaluation.html#mmmu) |
|
|
|
| Category | # Samples | Accuracy | |
|
|---------------------------------|-----------|----------| |
|
| Overall | 900 | 0.250 | |
|
| Overall-Art and Design | 120 | 0.217 | |
|
| Art | 30 | 0.167 | |
|
| Art Theory | 30 | 0.267 | |
|
| Design | 30 | 0.300 | |
|
| Music | 30 | 0.133 | |
|
| Overall-Business | 150 | 0.300 | |
|
| Accounting | 30 | 0.367 | |
|
| Economics | 30 | 0.300 | |
|
| Finance | 30 | 0.300 | |
|
| Management | 30 | 0.267 | |
|
| Marketing | 30 | 0.267 | |
|
| Overall-Science | 150 | 0.227 | |
|
| Biology | 30 | 0.200 | |
|
| Chemistry | 30 | 0.167 | |
|
| Geography | 30 | 0.267 | |
|
| Math | 30 | 0.267 | |
|
| Physics | 30 | 0.233 | |
|
| Overall-Health and Medicine | 150 | 0.293 | |
|
| Basic Medical Science | 30 | 0.400 | |
|
| Clinical Medicine | 30 | 0.167 | |
|
| Diagnostics and Laboratory Med. | 30 | 0.267 | |
|
| Pharmacy | 30 | 0.300 | |
|
| Public Health | 30 | 0.333 | |
|
| Overall-Humanities and Soc. Sci.| 120 | 0.267 | |
|
| History | 30 | 0.167 | |
|
| Literature | 30 | 0.233 | |
|
| Sociology | 30 | 0.367 | |
|
| Psychology | 30 | 0.300 | |
|
| Overall-Tech and Engineering | 210 | 0.210 | |
|
| Agriculture | 30 | 0.200 | |
|
| Architecture and Engineering | 30 | 0.200 | |
|
| Computer Science | 30 | 0.200 | |
|
| Electronics | 30 | 0.167 | |
|
| Energy and Power | 30 | 0.167 | |
|
| Materials | 30 | 0.267 | |
|
| Mechanical Engineering | 30 | 0.267 | |
|
|
