Repository: PaddlePaddle/PaddleSpeech
Pull request: add filter for double punctuation in sentences; add homonym, test=tts
fix https://github.com/PaddlePaddle/PaddleSpeech/issues/2232

Problem statement:
when process english text with !" , get error
English text: The girls stared at their father. Mrs. Bennet said only, "Nonsense, nonsense!"

remove !  then the error disappeared.

Error Log:

Traceback (most recent call last):
  File "readText_Paddle.py", line 1175, in <module>
    englishTxt2WavbyPaddle(mp3filepath,textforSpeak)
  File "readText_Paddle.py", line 147, in englishTxt2WavbyPaddle
    wav_file = tts_executor(	\
  File "/home/grow/PaddleSpeech/paddlespeech/cli/utils.py", line 328, in _warpper
    return executor_func(self, *args, **kwargs)
  File "/home/grow/PaddleSpeech/paddlespeech/cli/tts/infer.py", line 538, in __call__
    self.infer(text=text, lang=lang, am=am, spk_id=spk_id)
  File "/home/grow/.conda/envs/paddleconda/lib/python3.8/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/home/grow/.conda/envs/paddleconda/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 331, in _decorate_function
    return func(*args, **kwargs)
  File "/home/grow/PaddleSpeech/paddlespeech/cli/tts/infer.py", line 405, in infer
    mel = self.am_inference(part_phone_ids)
  File "/home/grow/.conda/envs/paddleconda/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 914, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2.py", line 812, in forward
    normalized_mel, d_outs, p_outs, e_outs = self.acoustic_model.inference(
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2.py", line 709, in inference
    _, outs, d_outs, p_outs, e_outs = self._forward(
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2.py", line 514, in _forward
    x_masks = self._source_mask(ilens)
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2.py", line 783, in _source_mask
    x_masks = make_non_pad_mask(ilens)
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/modules/nets_utils.py", line 249, in make_non_pad_mask
    return paddle.logical_not(make_pad_mask(lengths, xs, length_dim))
  File "/home/grow/PaddleSpeech/paddlespeech/t2s/modules/nets_utils.py", line 148, in make_pad_mask
    seq_range_expand = seq_range.unsqueeze(0).expand([bs, maxlen])
  File "/home/grow/.conda/envs/paddleconda/lib/python3.8/site-packages/paddle/tensor/manipulation.py", line 1869, in expand
    return _C_ops.expand_v2(x, 'shape', shape)
ValueError: (InvalidArgument) The 1th element of 'shape' for expand_v2 op must be greater than 0, but the value given is 0.
  [Hint: Expected expand_shape[i] > 0, but received expand_shape[i]:0 <= 0:0.] (at /paddle/paddle/fluid/operators/expand_v2_op.cc:78)
  [operator < expand_v2 > error]

Hints from issue discussion:
大概知道问题所在，因为 !" 是 2 个连续的标点，中间其实被拆出来一个为空的句子，所以报错了, 应该在某个位置加个判空就行了, 欢迎开发者提交 pr 修改
<img width="310" alt="image" src="https://user-images.githubusercontent.com/24568452/183379668-fd479fcc-0323-4138-b1e3-6b201bf75319.png">

Commit message: add filter for double punctuation in sentences; add homonym, test=tts

Patch to review:
diff --git a/paddlespeech/t2s/frontend/phonectic.py b/paddlespeech/t2s/frontend/phonectic.py
index 8e9f11737d1..ff8c149afb5 100644
--- a/paddlespeech/t2s/frontend/phonectic.py
+++ b/paddlespeech/t2s/frontend/phonectic.py
@@ -99,7 +99,11 @@ def get_input_ids(self, sentence: str,
                 if (phn in self.vocab_phones and phn not in self.punc) else "sp"
                 for phn in phones
             ]
-            phones_list.append(phones)
+            # when sentences have double punctuations in the end, like "!\"",or "?\"", will get null list like []
+            # add judge to filter it 
+            if len(phones) != 0:
+                # replace unk phone with sp
+                phones_list.append(phones)
 
         if merge_sentences:
             merge_list = sum(phones_list, [])
diff --git a/paddlespeech/t2s/frontend/zh_frontend.py b/paddlespeech/t2s/frontend/zh_frontend.py
index 143ccbc15d4..06e7dca2668 100644
--- a/paddlespeech/t2s/frontend/zh_frontend.py
+++ b/paddlespeech/t2s/frontend/zh_frontend.py
@@ -94,10 +94,12 @@ def __init__pypinyin(self):
         large_pinyin.load()
 
         load_phrases_dict({u'开户行': [[u'ka1i'], [u'hu4'], [u'hang2']]})
+        load_phrases_dict({u'俩个':[[u'lia3ng'],[u'ge4']]})
         load_phrases_dict({u'发卡行': [[u'fa4'], [u'ka3'], [u'hang2']]})
         load_phrases_dict({u'放款行': [[u'fa4ng'], [u'kua3n'], [u'hang2']]})
         load_phrases_dict({u'茧行': [[u'jia3n'], [u'hang2']]})
         load_phrases_dict({u'行号': [[u'hang2'], [u'ha4o']]})
+        load_phrases_dict({u'行政': [[u'xing2'], [u'zhen4g']]})
         load_phrases_dict({u'各地': [[u'ge4'], [u'di4']]})
         load_phrases_dict({u'借还款': [[u'jie4'], [u'hua2n'], [u'kua3n']]})
         load_phrases_dict({u'时间为': [[u'shi2'], [u'jia1n'], [u'we2i']]})


Code review comments:
diff_hunk: @@ -94,10 +94,12 @@ def __init__pypinyin(self):
         large_pinyin.load()
 
         load_phrases_dict({u'开户行': [[u'ka1i'], [u'hu4'], [u'hang2']]})
+        load_phrases_dict({u'俩个':[[u'lia3ng'],[u'ge4']]})
         load_phrases_dict({u'发卡行': [[u'fa4'], [u'ka3'], [u'hang2']]})
         load_phrases_dict({u'放款行': [[u'fa4ng'], [u'kua3n'], [u'hang2']]})
         load_phrases_dict({u'茧行': [[u'jia3n'], [u'hang2']]})
         load_phrases_dict({u'行号': [[u'hang2'], [u'ha4o']]})
+        load_phrases_dict({u'行政': [[u'xing2'], [u'zhen4g']]})
path: paddlespeech/t2s/frontend/zh_frontend.py
text: @user1:
'zhen4g' 的标注是否正确？
```python
>>> from pypinyin import pinyin, lazy_pinyin, Style
>>> pinyin('行政', style=Style.TONE2, heteronym=True)
[['xi2ng'], ['zhe4ng']]
>>> pinyin('行政', style=Style.TONE3, heteronym=True)
[['xing2'], ['zheng4']]
```

@author:
done

diff_hunk: @@ -94,10 +94,12 @@ def __init__pypinyin(self):
         large_pinyin.load()
 
         load_phrases_dict({u'开户行': [[u'ka1i'], [u'hu4'], [u'hang2']]})
+        load_phrases_dict({u'俩个':[[u'lia3ng'],[u'ge4']]})
path: paddlespeech/t2s/frontend/zh_frontend.py
text: @user1:
这两个多音字似乎发音没什么问题，具体 badcase 的输入文本是什么？按照汉语拼音标准，"俩" == "两个"，"俩个" 这种用法本身就是错的

@author:
removed this line