Yongxin-Guo
commited on
Commit
•
4a52ef6
1
Parent(s):
7d39f30
Update README.md
Browse files
README.md
CHANGED
@@ -45,25 +45,55 @@ In this work
|
|
45 |
| Youcook2 (Zero-Shot) | CIDER | METEOR | SODA_c | F1 |
|
46 |
| --- | --- | --- | --- | --- |
|
47 |
| TRACE | 8.1 | 2.8 | 2.2 | 22.4 |
|
48 |
-
| TRACE-
|
|
|
49 |
|
50 |
| Charades-STA (Zero-Shot) | 0.3 | 0.5 | 0.7 | mIOU |
|
51 |
| --- | --- | --- | --- | --- |
|
52 |
| TRACE | 58.6 | 40.3 | 19.4 | 38.7 |
|
53 |
| TRACE-retrieval | 57.9 | 37.4 | 17.3 | 37.4 |
|
|
|
54 |
|
55 |
| QVHighlights (Zero-Shot) | mAP | Hit@1 |
|
56 |
| --- | --- | --- |
|
57 |
| TRACE | 26.8 | 42.7 |
|
58 |
| TRACE-retrieval | 27.9 | 44.3 |
|
|
|
59 |
|
60 |
|
61 |
| ActivityNet-DVC | CIDER | METEOR | SODA_c | F1 |
|
62 |
| --- | --- | --- | --- | --- |
|
63 |
| TRACE | 25.9 | 6.0 | 6.4 | 39.3 |
|
64 |
| TRACE-retrieval | 25.7 | 5.9 | 6.5 | 40.1 |
|
|
|
65 |
|
66 |
| ActivityNet-MR | 0.3 | 0.5 | 0.7 | mIOU |
|
67 |
| --- | --- | --- | --- | --- |
|
68 |
| TRACE | 54.0 | 37.7 | 24.0 | 39.0 |
|
69 |
| TRACE-retrieval | 54.4 | 39.8 | 24.9 | 40.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
| Youcook2 (Zero-Shot) | CIDER | METEOR | SODA_c | F1 |
|
46 |
| --- | --- | --- | --- | --- |
|
47 |
| TRACE | 8.1 | 2.8 | 2.2 | 22.4 |
|
48 |
+
| TRACE-retrieal | 8.3 | 2.9 | 2.3 | 24.1 |
|
49 |
+
| TRACE-uni | 8.6 | 2.9 | 2.3 | 22.4 |
|
50 |
|
51 |
| Charades-STA (Zero-Shot) | 0.3 | 0.5 | 0.7 | mIOU |
|
52 |
| --- | --- | --- | --- | --- |
|
53 |
| TRACE | 58.6 | 40.3 | 19.4 | 38.7 |
|
54 |
| TRACE-retrieval | 57.9 | 37.4 | 17.3 | 37.4 |
|
55 |
+
| TRACE-uni | 63.7 | 43.7 | 21.0 | 41.5 |
|
56 |
|
57 |
| QVHighlights (Zero-Shot) | mAP | Hit@1 |
|
58 |
| --- | --- | --- |
|
59 |
| TRACE | 26.8 | 42.7 |
|
60 |
| TRACE-retrieval | 27.9 | 44.3 |
|
61 |
+
| TRACE-uni | 27.5 | 43.9 |
|
62 |
|
63 |
|
64 |
| ActivityNet-DVC | CIDER | METEOR | SODA_c | F1 |
|
65 |
| --- | --- | --- | --- | --- |
|
66 |
| TRACE | 25.9 | 6.0 | 6.4 | 39.3 |
|
67 |
| TRACE-retrieval | 25.7 | 5.9 | 6.5 | 40.1 |
|
68 |
+
| TRACE-uni | 29.2 | 6.9 | 6.4 | 40.4 |
|
69 |
|
70 |
| ActivityNet-MR | 0.3 | 0.5 | 0.7 | mIOU |
|
71 |
| --- | --- | --- | --- | --- |
|
72 |
| TRACE | 54.0 | 37.7 | 24.0 | 39.0 |
|
73 |
| TRACE-retrieval | 54.4 | 39.8 | 24.9 | 40.2 |
|
74 |
+
| TRACE-uni | 53.2 | 38.2 | 24.7 | 39.4 |
|
75 |
+
|
76 |
+
| MVBench | Avg | AS | AP | AA | FA | UA | OE | OI | OS | MD | AL | ST | AC | MC | MA | SC | FP | CO | EN | ER | CI |
|
77 |
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
78 |
+
| TRACE | 48.1 | 61.2 | 56.5 | 72.5 | 46.5 | 61.0 | 48.0 | 69.5 | 40.0 | 22.0 | 31.0 | 86.5 | 37.5 | 37.0 | 51.0 | 45.0 | 40.5 | 39.0 | 31.0 | 43.5 | 44.5 |
|
79 |
+
| TRACE-uni | 53.8 | 68.1 | 58.5 | 72.5 | 41.5 | 73.5 | 55.1 | 71.5 | 40.5 | 25.0 | 53.0 | 88.5 | 63.5 | 38.5 | 51.0 | 52.5 | 49.0 | 59.5 | 33.5 | 49.5 | 32.5 |
|
80 |
+
|
81 |
+
|
82 |
+
| VideoMME (w/o subtitle) | Short | Midium | Long | Avg |
|
83 |
+
| --- | --- | --- | --- | --- |
|
84 |
+
| TRACE | 49.5 | 42.5 | 39.3 | 43.8 |
|
85 |
+
| TRACE-uni | 58.2 | 48.1 | 42.3 | 49.6 |
|
86 |
+
|
87 |
+
#### Bibliography
|
88 |
+
If you find this repository helpful for your project, please consider citing:
|
89 |
+
```
|
90 |
+
@misc{guo2024tracetemporalgroundingvideo,
|
91 |
+
title={TRACE: Temporal Grounding Video LLM via Causal Event Modeling},
|
92 |
+
author={Yongxin Guo and Jingyu Liu and Mingda Li and Xiaoying Tang and Qingbin Liu and Xi Chen},
|
93 |
+
year={2024},
|
94 |
+
eprint={2410.05643},
|
95 |
+
archivePrefix={arXiv},
|
96 |
+
primaryClass={cs.CV},
|
97 |
+
url={https://arxiv.org/abs/2410.05643},
|
98 |
+
}
|
99 |
+
```
|