Dionyssos commited on
Commit
7184f5f
1 Parent(s): 23acb9e
README.md CHANGED
@@ -44,59 +44,27 @@ virtualenv --python=python3 ~/.envs/.my_env
44
  source ~/.envs/.my_env/bin/activate
45
  cd shift/
46
  pip install -r requirements.txt
47
- ```
48
-
49
- Start API
50
 
51
- ```
52
  CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py
53
  ```
54
 
55
- The following need `api.py` to be already running `on a tmux session`.
56
-
57
  ## Landscape 2 Soundscape
58
 
 
 
59
  ```python
60
  # TTS & soundscape - overlay to .mp4
61
  python landscape2soundscape.py
62
  ```
63
 
64
- # YouTube Videos / Examples
65
-
66
- Substitute Native voice via TTS
67
 
68
- [![Native voice ANBPR video](assets/native_video_thumb.png)](https://www.youtube.com/watch?v=tmo2UbKYAqc)
69
 
70
- ##
71
-
72
- Same video where Native voice is replaced with English TTS voice with similar emotion
73
 
74
 
75
  [![Same video w. Native voice replaced with English TTS](assets/tts_video_thumb.png)](https://www.youtube.com/watch?v=geI1Vqn4QpY)
76
 
77
-
78
- <details>
79
- <summary>
80
-
81
- Video dubbing from subtitles `.srt`
82
-
83
- </summary>
84
-
85
- ## Video Dubbing
86
-
87
- [![Review demo SHIFT](assets/review_demo_thumb.png)](https://www.youtube.com/watch?v=bpt7rOBENcQ)
88
-
89
- Generate dubbed video:
90
-
91
-
92
- ```python
93
- python tts.py --text assets/head_of_fortuna_en.srt --video assets/head_of_fortuna.mp4
94
-
95
- ```
96
-
97
-
98
- </details>
99
-
100
  ## Joint Application of D3.1 & D3.2
101
 
102
  <a href="https://youtu.be/wWC8DpOKVvQ" rel="Subtitles to Video">![Foo4](assets/caption_to_video_thumb.png)</a>
@@ -128,6 +96,8 @@ For SHIFT demo / Collaboration with [SMB](https://www.smb.museum/home/)
128
 
129
  [![02](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____02_Constable_AI555_001.jpg)](https://youtu.be/2YjxAPkdXIc)
130
 
 
 
131
  [![10](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____10_Boecklin_967648_NG2-80_001_rsz.jpg)](https://www.youtube.com/watch?v=Y8QyYUgLaCg)
132
 
133
 
 
44
  source ~/.envs/.my_env/bin/activate
45
  cd shift/
46
  pip install -r requirements.txt
 
 
 
47
 
 
48
  CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py
49
  ```
50
 
 
 
51
  ## Landscape 2 Soundscape
52
 
53
+ The following needs `api.py` to be already running `on a tmux session`.
54
+
55
  ```python
56
  # TTS & soundscape - overlay to .mp4
57
  python landscape2soundscape.py
58
  ```
59
 
60
+ # Videos / Examples
 
 
61
 
 
62
 
63
+ Video where Native voice is replaced with English TTS voice
 
 
64
 
65
 
66
  [![Same video w. Native voice replaced with English TTS](assets/tts_video_thumb.png)](https://www.youtube.com/watch?v=geI1Vqn4QpY)
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  ## Joint Application of D3.1 & D3.2
69
 
70
  <a href="https://youtu.be/wWC8DpOKVvQ" rel="Subtitles to Video">![Foo4](assets/caption_to_video_thumb.png)</a>
 
96
 
97
  [![02](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____02_Constable_AI555_001.jpg)](https://youtu.be/2YjxAPkdXIc)
98
 
99
+ [![03](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____03_Schinkel_WS200-002.jpg)](https://youtu.be/BhMh02knkco)
100
+
101
  [![10](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____10_Boecklin_967648_NG2-80_001_rsz.jpg)](https://www.youtube.com/watch?v=Y8QyYUgLaCg)
102
 
103
 
landscape2soundscape.py CHANGED
@@ -62,7 +62,7 @@ DESCRIPTIONS = [
62
  '03_Schinkel_WS200-002.jpg',
63
  '03_Schinkel_WS200-002.txt',
64
  'Arriving at the valley on galloping horses',
65
- 'KF. Schinkel Gotische Kirche Auf Einem Felsen 1815',
66
  'en_US/hifi-tts_low#6097',
67
  ],
68
  #
@@ -146,7 +146,7 @@ SILENT_VIDEO = '_silent_video.mp4'
146
  # SILENT CLIP
147
 
148
 
149
- for img, text, scene, title, voice in DESCRIPTIONS[2:20]:
150
 
151
 
152
 
 
62
  '03_Schinkel_WS200-002.jpg',
63
  '03_Schinkel_WS200-002.txt',
64
  'Arriving at the valley on galloping horses',
65
+ 'K. Schinkel Gotische Kirche Auf Einem Felsen 1815',
66
  'en_US/hifi-tts_low#6097',
67
  ],
68
  #
 
146
  # SILENT CLIP
147
 
148
 
149
+ for img, text, scene, title, voice in DESCRIPTIONS[2:4]:
150
 
151
 
152
 
uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____03_Schinkel_WS200-002.jpg ADDED