Fix Silence Sampling Algorithm for ASR Multi-speaker Data Simulator
State: closed | Pull request: yes
Labels: name: ASR

Signed-off-by: stevehuang52 <heh@nvidia.com>

# What does this PR do ?

Replace the previous silence insertion method with a new one that guarantees close approximation to specified `mean_silence`


**Collection**: [ASR]

# Adding Silence in ASR Data Simulator

## Requirements:
1. Sentence durations in each session follow a negative-binomial (NB) distribution
2. Silence ratios in all sessions follows a Beta distribution (range in [0.1])
3. Per silence length should look like a long tail distribution, thus Gamma distribution is used.
4. Silence Uniformity: each speech sentence (overlaps combined) should be followed by some minimum silence
5. Silence Variablity: per-silence durations and sentence durations should be approximately independent (i.e., p-value close to 0) 

## Parameters: 
- `NUM_SESSIONS`: number of sessions
- `MAX_SESS_DUR`: maximum session duration
- `SAMPLING_RATE`: sampling rate for audio
- `[NB_COUNT, NB_PROB]`: parameters for per sentence duration distribution
- `SILENCE_RATIO_MEAN`: mean for target silence ratio in all sessions, in (0,1)
- `SILENCE_RATIO_VAR`: std for target silence ratio in all sessions, set small values (e.g., 0.1) for better approximation to mean, set larger (e.g., 2.0) for more diversity in silence.
- `PER_SILENCE_VAR`: std for individual silence length, default to 20 for achieving p-value=0.1 to de-correlate speech and silence lengths
- `[PER_SILENCE_MIN,PER_SILENCE_MAX]`: mix and max of per silence duration in seconds, max=-1 for no constraint

## Algorithm:

```python
MAX_SESSION_LEN = MAX_SESS_DUR * SAMPLING_RATE
MIN_SILENCE_LEN = SILENCE_RATIO_MIN * SAMPLING_RATE
MAX_SILENCE_LEN = min(SILENCE_RATIO_MAX * SAMPLING_RATE, MAX_SESSION_LEN)

sessions = []
for i in range(NUM_SESSIONS):
    curr_session = []

    curr_sess_len = 0
    curr_speech_len = 0
    curr_silence_len = 0

    a = SILENCE_RATIO_MEAN ** 2 * (1 - SILENCE_RATIO_MEAN) / SILENCE_RATIO_VAR - SILENCE_RATIO_MEAN
    b = SILENCE_RATIO_MEAN * (1 - SILENCE_RATIO_MEAN) ** 2 / SILENCE_RATIO_VAR - (1 - SILENCE_RATIO_MEAN)
    sess_silence_mean = Beta(a, b).rvs()

    while curr_sess_len < MAX_SESSION_LEN:
        speech_len = NB(NB_COUNT, NB_PROB).rvs()
        sentence = build_sentence(speech_len,curr_sess_len,  MAX_SESSION_LEN)
        
        curr_session += sentence
        curr_sess_len += len(sentence)
        curr_speech_len += len(sentence)

        if curr_sess_len >= MAX_SESSION_LEN:
            break
        
        # dynamically adjust silence mean to achieve the overall mean
        silence_mean = max(1, MIN_SIL1ENCE_LEN, (sess_silence_mean * curr_sess_len - curr_silence_len) / (1 - sess_silence_mean))

        # sampling with large std to de-correlate with previous sentence length
        silence_len = Gamma(a=silence_mean ** 2 / PER_SILENCE_VAR, scale=PER_SILENCE_VAR / silence_mean).rvs()  

        # enforce valid length
        silence_len = min(max(MIN_SILENCE_LEN, silence_len), MAX_SILENCE_LEN, max_session_len - curr_sess_len)  
        
        silence = add_silence(silence_len)
        
        curr_session += silence
        curr_sess_len += silence_len
        curr_silence_len += silence_len

    sessions.append(curr_session)

```

## Notes
- Harder to get desired silence ratio with shorter session length
  - E.g., sess_len=20s, mean_silence=0.35, 500 hours -> actual ratio 0.24
  - E.g., sess_len=120s. mean_silence=0.3, 100 hours -> actual ratio ~0.3
- Silence lengths distribution is approximately exponential, and reducing silence ratio can reduce the avg silence length
  - E.g., mean_silence=0.1, std=0.05 -> mean per-silence length 0.2s
  - E.g., mean_silence=0.2, std=0.5 -> mean per-silence length 2.4s