quchenyuan commited on
Commit
fa9f8ec
1 Parent(s): 6323fb8

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ ---
4
+
5
+ # <i>360+x</i> Dataset
6
+
7
+ For more information, please feel free to check our [project page](https://x360dataset.github.io/).
8
+
9
+ ## Overview
10
+
11
+ 360+x dataset introduces a unique panoptic perspective to scene understanding, differentiating itself from traditional
12
+ datasets by offering multiple viewpoints and modalities, captured from a variety of scenes
13
+
14
+ ### Key Features:
15
+
16
+ - **Multi-viewpoint Captures:** Includes 360° panoramic video, third-person front view video, egocentric monocular
17
+ video, and egocentric binocular video.
18
+ - **Rich Audio Modalities:** Features normal audio and directional binaural delay.
19
+ - **2,152 multi-model videos** captured by 360 cameras and Spectacles camera (8579k frames in total) Captured in 17
20
+ cities across 5 countries, covering 28 scenes ranging from Artistic Spaces to Natural Landscapes.
21
+ - **Action Temporal Segmentation:** Provides labels for 38 action instances for each video pair.
22
+
23
+ ## About This Repo
24
+
25
+ This repository stores the pretrained models of the 360+x dataset. For more code information, please check our [official
26
+ code repository](https://github.com/chqwer2/360x_Video_Experiments).
27
+
28
+ ## Dataset Details
29
+
30
+ ### Project Description
31
+
32
+ - **Developed by:** Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiaohan Hong, Jianbo Jiao
33
+ - **Funded by:** the Ramsay Research Fund, and the Royal Society Short Industry Fellowship
34
+ - **License:** Creative Commons Attribution-NonCommercial-ShareAlike 4.0
35
+
36
+ ### Sources
37
+ - **Repository:** Coming Soon
38
+ - **Paper:** https://arxiv.org/abs/2404.00989
39
+
40
+ ## Dataset Statistics
41
+
42
+ - **Total Videos:** 2,152, split between 464 videos captured using 360 cameras and 1,688 with Spectacles cameras.
43
+ - **Scenes:** 15 indoor and 13 outdoor, totaling 28 scene categories.
44
+ - **Short Clips:** The videos have been segmented into 1,380 shorter clips, each approximately 10 seconds long, totaling
45
+ around 67.78 hours.
46
+ - **Frames:** 8,579k frames across all clips.
47
+
48
+ ## Dataset Structure
49
+
50
+ Our dataset offers a comprehensive collection of panoramic videos, binocular videos, and third-person videos, each pair
51
+ of videos accompanied by annotations. Additionally, it includes features extracted using I3D, VGGish, and ResNet-18.
52
+ Given the high-resolution nature of our dataset (5760x2880 for panoramic and binocular videos, 1920x1080 for
53
+ third-person front view videos), the overall size is considerably large. To accommodate diverse research needs and
54
+ computational resources, we also provide a lower-resolution version of the dataset (640x320 for panoramic and binocular
55
+ videos, 569x320 for third-person front view videos) available for download.
56
+
57
+ <b>In this repo, we provide the lower-resolution version of the dataset. To access the high-resolution version, please
58
+ visit the <a href="https://x360dataset.github.io/">official website</a>.</b>
59
+
60
+
61
+
62
+
63
+ ## BibTeX
64
+ ```
65
+ @inproceedings{chen2024x360,
66
+ title={360+x: A Panoptic Multi-modal Scene Understanding Dataset},
67
+ author={Chen, Hao and Hou, Yuqi and Qu, Chenyuan and Testini, Irene and Hong, Xiaohan and Jiao, Jianbo},
68
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
69
+ year={2024}
70
+ }
71
+ ```