Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
---
|
2 |
title: CAPSTONE-PROJECT
|
3 |
-
emoji:
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
@@ -9,36 +9,3 @@ app_file: app.py
|
|
9 |
pinned: false
|
10 |
license: mit
|
11 |
---
|
12 |
-
## Phi2 : Multimodal Finetuning
|
13 |
-
### Details
|
14 |
-
1. LLM Backbone: Phi2
|
15 |
-
2. Vision Tower: clip-vit-large-patch14-336
|
16 |
-
3. Audio Model: Whisper
|
17 |
-
4. Pretraining Dataset: LAION-CC-SBU dataset with BLIP captions(200k samples)
|
18 |
-
5. Finetuning Dataset: Instruct 150k dataset based on COCO
|
19 |
-
|
20 |
-
### Design
|
21 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/56df24cd-2681-4e17-ab64-9652f609b15f)
|
22 |
-
|
23 |
-
### Pretraining
|
24 |
-
#### Training Loss Curve
|
25 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/b6c37a95-0a56-4b52-8719-3ff56dc1b703)
|
26 |
-
|
27 |
-
#### Learing Rate
|
28 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/44d9a11b-b28d-47e1-ba1d-d6dc22ebe748)
|
29 |
-
|
30 |
-
#### Training Logs
|
31 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/76543d98-d9fe-4c1a-ac47-3d06e48053ad)
|
32 |
-
|
33 |
-
### Finetuning
|
34 |
-
#### Training Loss Curve
|
35 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/45ef40bd-fae5-4cfe-a522-c0eed2833230)
|
36 |
-
|
37 |
-
#### Learing Rate
|
38 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/df60ee62-a537-4e36-a7f7-f7111e101162)
|
39 |
-
|
40 |
-
#### Training Logs
|
41 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/2747acce-bc99-4c37-a05a-d5e81cb9aa9d)
|
42 |
-
|
43 |
-
### Results
|
44 |
-
![image](https://github.com/RaviNaik/ERA-CAPSTONE/assets/23289802/f12a9f04-df32-413e-b957-774c30381b2b)
|
|
|
1 |
---
|
2 |
title: CAPSTONE-PROJECT
|
3 |
+
emoji: 🤖
|
4 |
colorFrom: blue
|
5 |
colorTo: red
|
6 |
sdk: gradio
|
|
|
9 |
pinned: false
|
10 |
license: mit
|
11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|