Spaces:

RaviNaik
/

MultiModal-Phi2

Sleeping

App Files Files Community

MultiModal-Phi2 / README.md

ravi.naik

Fixed relative import issues

667ae00 5 months ago

|

raw history blame contribute delete

No virus

1.44 kB

A newer version of the Gradio SDK is available: 4.36.1

Upgrade

metadata

title: MultiModal Phi2
emoji: 🚀
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 3.35.2
app_file: app.py
pinned: false
license: mit

Phi2 : Multimodal Finetuning

Details

LLM Backbone: Phi2
Vision Tower: clip-vit-large-patch14-336
Audio Model: Whisper
Pretraining Dataset: LAION-CC-SBU dataset with BLIP captions(200k samples)
Finetuning Dataset: Instruct 150k dataset based on COCO

Design

Pretraining

Training Loss Curve

Learing Rate

Training Logs

Finetuning

Training Loss Curve

Learing Rate

Training Logs

Results