---
license: mit
language:
- en
pipeline_tag: text-to-speech
tags:
- audiocraft
- audiogen
- styletts2
- audio
- synthesis
- shift
- audeering
- dkounadis
- sound
- scene
- acoustic-scene
- audio-generation
---


# Affective TTS / SoundScapes

  - [SHIFT TTS tool](https://github.com/audeering/shift) 
  - Analysis of emotionality [#1](https://huggingface.co/dkounadis/artificial-styletts2/discussions/2)
  - Soundscape `e.g. trees, water` via [AudioGen](https://huggingface.co/dkounadis/artificial-styletts2/discussions/3)
  - `landscape2soundscape.py` shows how to overlay TTS & sound to image and create video

## Available Voices

<a href="https://audeering.github.io/shift/">Listen to available voices!</a>

## Flask API

Clone this repo

```
git clone https://huggingface.co/dkounadis/artificial-styletts2
```
Build env

```
virtualenv --python=python3 ~/.envs/.my_env
source ~/.envs/.my_env/bin/activate
cd shift/
pip install -r requirements.txt

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py
```

## Landscape 2 Soundscape

The following needs `api.py` to be already running `on a tmux session`. 

```python
# TTS & soundscape - overlay to .mp4
python landscape2soundscape.py
```

# Videos / Examples


Video where Native voice is replaced with English TTS voice


[![Same video w. Native voice replaced with English TTS](assets/tts_video_thumb.png)](https://www.youtube.com/watch?v=geI1Vqn4QpY)

## Joint Application of D3.1 & D3.2

<a href="https://youtu.be/wWC8DpOKVvQ" rel="Subtitles to Video">![Foo4](assets/caption_to_video_thumb.png)</a>


From an image and text create a video:

```python

python tts.py --text sample.txt --image assets/image_from_T31.jpg
```

## Landscape 2 Soundscape





```python
# Loads image & text & sound-scene text and creates .mp4
python landscape2soundscape.py
```

For SHIFT demo / Collaboration with [SMB](https://www.smb.museum/home/)
  - YouTube Videos


[![01](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____01_Schick_AII840_001.jpg)](https://youtu.be/SSi3gUO4GtY)

[![02](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____02_Constable_AI555_001.jpg)](https://youtu.be/2YjxAPkdXIc)

[![03](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____03_Schinkel_WS200-002.jpg)](https://youtu.be/BhMh02knkco)

[![10](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____10_Boecklin_967648_NG2-80_001_rsz.jpg)](https://www.youtube.com/watch?v=Y8QyYUgLaCg)


# Live Demo - Paplay

Flask

```python
CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python live_api.py
```

Client (Ubutu)

```python
python live_demo.py  # will ask text input & play soundscape
```