JAIST Repository
https://dspace.jaist.ac.jp/

Title

Study on semi‑scramble method for speech signals
based on phonemic restoration

Author(s)Yamamoto, Katsuhiko; Zhu, Zhi; Unoki, Masashi;
Aoki, Naofumi

Citation

Journal of Signal Processing, 18(4): 205‑208

Issue Date

2014

Type

Journal Article

Text version

publisher

URL

http://hdl.handle.net/10119/12892

RightsCopyright (C) 2014 信号処理学会. Katsuhiko
Yamamoto, Zhi Zhu, Masashi Unoki, and Naofumi
Aoki, Journal of Signal Processing, 18(4), 2014,
205‑208. http://dx.doi.org/10.2299/jsp.18.205

Description

Japan Advanced Institute of Science and Technology

Journal of Signal Processing, Vol.18, No.4, pp.205-208, July 2014

SELECTED PAPER AT NCSP'14

Study on Semi-scramble Method for Speech Signals Based on Phonemic Restoration
Katsuhiko Yamamoto1 , Zhi Zhu1 , Masashi Unoki1 and Naofumi Aoki2
School of Information Science, Japan Advanced
Institute of Science and Technology
1–1 Asahidai, Nomi, Ishikawa 923–1292, Japan
E-mail: {kyamamoto, zhuzhi, unoki}@jaist.ac.jp

1

Graduate School of Information Science andTechnology, Hokkaido University
Kita–ku, Kita–14 Nishi–9, Sapporo 060–0814, Japan
E-mail: aoki@ime.ist.hokudai.ac.jp
2

Abstract
Speech scrambling methods are widely used for copyrightprotection and encrypting digital speech signals in order toguarantee the confidentiality of the original signals. They
are very important methods for preventing eavesdropping andunauthorized copying. However, it seems to be impossible to completely recover a scrambled speech signal into theoriginal signal. Moreover, nobody can comprehend the partial speech content from speech signals scrambled with thesemethods. In this paper, we propose a semi-scramble method
for speech signals based on phonemic restoration. By using
a speech scrambling method based on the random-bit shift ofquantization bits, speech signals are converted to scrambled
signals in partial intervals. We evaluated the confidentiality
and efficiency of the proposed method by using two objective
measures, signal-to-error ratio (SER) and perceptual evaluation of speech quality (PESQ). As a result, we confirmedthat the proposed method can play a role in copyright protection for an original signal and recover a semi-scrambled
speech signal into the original one. Finally, we indicated thatthe acoustic characteristics of signal semi-scrambled with theproposed method enable the listener to understand the speech
information.
1. Introduction
Recently, encryption and information protection for digital data have become more important. Speech scrambling
methods are widely used for copyright protection and encrypting digital speech signals in order to guarantee the confidentiality of an original signal [1]. These methods arevery important for preventing eavesdropping and unauthorized copying. There are, for example, typical methods suchas speech codecs and reverse scrambling in the frequency
domain. However, it seems to be impossible to completely
recover a scrambled speech signal into the original signal.
Moreover, nobody can make out the partial speech content
from speech signals scrambled with these methods.
A semi-scramble speech method is one way to encrypt content to the extent that people can understand most of it. Although semi-scrambled data cannot be encrypted completely,
it plays a role in copyright protection. In previous studies,
there have been typical methods such as semi-scrambling a
digital speech signal through coding by changing MDCT co-

Journal of Signal Processing, Vol. 18, No. 4, July 2014(127)10=(0111 1111)2
127
Ͳ128

c(m)=3x(m)bitshift:cm

Sample(leftͲcirculationshift)
Original(1111 1011)2=(Ͳ5)10127
Ͳ5Samplexs(m)Scramble

Figure 1: Example of scrambling with speech scrambling
method based on random-bit shift of quantization (quantization bit length: n = 8, shifting frequency: c(m) = 3)

efficients of the original signal [2], and reversing the order ofthe samples in the time domain. However, the same as withthe previous scramble methods, it seems to be impossible tocompletely recover a semi-scrambled speech signal into theoriginal signal.
This study aims to achieve a semi-scramble method thatplays a role in copyright protection for original speech signals
and recovers a semi-scrambled speech signal into the original
one completely. In this paper, we propose a semi-scramble
method for speech signals based on phonemic restoration.
Here, a speech scramble method based on the random-bit
shift of quantization bits [3] is used to solve the problem.
2. Key Techniques
2.1 Speech scramble method based on random-bit shiftof quantization bits
Figure 1 shows an example of scrambling by using the
speech scramble method based on random-bit shift of quantization bits that the author proposed [3]. In this example,
the numerical representation is binary or decimal, and thequantization bit rate is n = 8. When the value x(m) of theoriginal speech signals is represented as the decimal form of(127)10 , it is represented as the binary form of (01111111)2
by the quantization bits. This scrambling method cyclicallyshifts the arrangement of the bits. The shift amount of the
bit shift is based on the random number sequence c generated
from a seed number as the public key. When the value of theshift amount is c(m) = 3, the value of the x(m) is cyclically205

original speech signal, speech information is partially encrypted. In the proposed method, the partial scrambling isrepeated in the intervals of the semi-scrambled sound. How0
ever, the listener can restore each of the encrypted pieces−32768of information of the semi-scrambled signal perceptually byphonemic restoration. Therefore, the proposed method is able(b)tLength:n32767
to semi-scramble the information of speech sounds.
0
Figure 2 shows an example of semi-scrambling with theproposed method. In this example, the quantization bit length−32768Periods:T
is n = 16, and the scrambling method is applied to the speech(c)signal at two parameters, the length tn = 60 ms and the pe32767
riod T = 200 ms of the scrambling. Figure 2(a) is the wave0
form of the original speech. The semi-scramble method is−32768applied to the original speech signal, and the semi-scrambled
0
0.5
1
1.5
2
2.5
speech signal that has the intervals of noise is converted, asTime (s)shown in Fig. 2(b). The descrambled speech signal (Fig.
2(c)) is converted from the semi-scrambled speech signal. If
Figure 2: Examples of semi-scrambling with the proposed
the semi-scrambled speech is converted with the correct seedmethod: (a) Original speech, (b) Semi-scrambled speech, and number of the random sequence number, the descrambled(c) Descrambled speechspeech corresponds to the original speech completely.
(a)Amplitude

Amplitude

Amplitude

32767

4. Evaluations
shifted 3 bits to the left. As a result, the value of the x(m) istransformed to xs (m) = (11111011)2 , and it is representedas (−5)10 by considering two’s complement. These procedures are applied to all samples of the original speech signal
and convert it into a scrambled speech signal. Moreover, if
the seed number is correct, the scrambled speech signal canbe completely descrambled to the original one by applying
the inverse process.

4.1 Semi-scrambled speech signal

Two objective tests, signal-to-error ratio (SER) and perceptual evaluation of speech quality (PESQ) [6] of semiscrambled and descrambled speech, were carried out to evaluate the confidentiality and efficiency of the proposed method.
SER is represented as the error ratio of the target signal
x1 (m) to the standard signal x2 (m) and is defined by thefollowing
equation,
2.2 Phonemic restoration∑M−1 2
When the partial intervals of a speech signal are replaced
m=0 x1 (m)
(dB)
SER
=
10
log
10 ∑M−1by gaps of silence, the speech’s intelligibility is drastically2m=0 (x1 (m) − x2 (m))
reduced. However, when the gaps are filled with white noisethat is louder than the recorded voice, the utterance sounds where m = 0, 1, . . . , M − 1, and M is the total number ofmore natural and continuous. This phenomenon is referred samples of signal. When the value of SER is positive, theto as the phonemic restoration effect. This effect occurs not error between the target and standard signal is small. Moreonly with speech sounds but also any sounds such as pure over, when the value is ∞ dB, the result indicates that thetones. These perceptual restorations are collectively called target signal is completely the same as the standard signal.
the “auditory continuity effect” [4].
When the value of SER is negative, the error between the tarWhen the effect of phonemic restoration occurs, the nec- get and standard signal is high. PESQ shows an objective
essary conditions for the effect of auditory continuity are sat- difference grade (ODG) represented as a value from −0.5 toisfied. Apart from that, co-articulation (temporal distribution 4.5. These values correspond to mean opinion score (MOS)
of information for a phonetic segment) helps the listener to represented as a value from 1 (Bad; Very annoying) to 5 (Exincrease the intelligibility [5]. In addition, it is revealed that cellent; Imperceptible). 20 stimuli were selected from theby using the semantic information from the context, speech Japanese multi-emotion single speaker Fujitsu database procontinuity is kept from being interrupted any longer.
duced and recorded by Fujitsu Laboratory [7]. The sampling
frequency was 22.05 kHz, the quantization bit rate was 16 bit,and the length of the stimuli was about 3 sec.
3. Proposed Method
In this study, to evaluate the variations of SER and PESQ
We propose a semi-scramble method for speech signals of the proposed method numerically, the stimuli, the semibased on phonemic restoration. The speech scrambling scrambled and descrambled speech signals, were generated
method based on the random-bit shift of quantization bits by varying two parameters, the length tn and the period T
converts an original speech sounds into an unintelligible ones. of the scrambling (Fig. 2). The periods of the scramble T s
By using the scrambling method on partial intervals of the were set to 100, 200, 300, 400, and 500 ms. The length tn was

206

Journal of Signal Processing, Vol. 18, No. 4, July 2014

5

T=100 (ms)
T=200 (ms)
T=300 (ms)
T=400 (ms)
T=500 (ms)

T=100 (ms)
T=200 (ms)
T=300 (ms)
T=400 (ms)
T=500 (ms)SER (dB)

SER (dB)

0

∞

−5

−10

0

−15
0

100

200

tn (ms)

300

400

0

500

100

1.5

1.5

0.5

−0.5

−0.5
100

200

t (ms)

300

400

500

n

500

T=100 (ms)
T=200 (ms)
T=300 (ms)
T=400 (ms)
T=500 (ms)

2.5

0.5

0

400

3.5

PESQ (ODG)

PESQ (ODG)

2.5

300

4.5

T=100 (ms)
T=200 (ms)
T=300 (ms)
T=400 (ms)
T=500 (ms)3.5

tn (ms)Figure 5: SER of descrambled speech

Figure 3: SER of semi-scrambled speech
4.5

200

0

100

200

tn (ms)

300

400

500

Figure 4: PESQ of semi-scrambled speech

Figure 6: PESQ of descrambled speechincreased from 20 ms to T in steps of 20 ms. When the length
tn is 0 ms, the scramble process is not applied. Therefore, thesemi-scrambled speech is the same as the original one. In
addition, when the length tn equals the period T , scrambled
speech encrypted in all sections is made by our scrambling
method [3].
Figures 3 and 4 show the results of SER and PESQ for
semi-scrambled speech. Each of the values is represented as
an average, and each of the error bars is represented as a standard deviation of the result for each parameter. When tn was
0 ms, SER was ∞ dB and PESQ was 4.5 ODG in the entire
period of the scramble T . As the length tn became longer,
SER and PESQ reduced gradually. When tn reached T , theSER and PESQ of the semi-scrambled speech were the sameas that of the scrambled speech [3]. These results could confirm that the proposed method could scramble the information of the original signal in partial intervals. and control the
degree of the scrambling.
Figures 5 and 6 show the results of SER and PESQ for descrambled speech. For every length T and the period tn , SER
was ∞ dB, and PESQ was 4.5 ODG. These results confirmedthat the proposed method could recover the semi-scrambled
speech signal into the original one.

Journal of Signal Processing, Vol. 18, No. 4, July 2014

4.2 Semi-scrambled audio signal
An objective test, the perceptual evaluation of audio quality (PEAQ) [8], was carried out to investigate applying the
proposed method to an audio signal. PEAQ shows ODG
represented as a value from −4 (Bad; Very annoying) to 0
(Excellent; Imperceptible). 100 stimuli were selected from
the Music Genre Database in the RWC Music Database [9].
These stimuli were processed to the monaural signals of theleft channel in advance, and the length was set to 10 s (interval from 60 to 70 s in the original audio). As with the evaluation of the semi-scrambled speech signal, the stimuli of thesemi-scrambled audio signal were generated by varying two
parameters, the length tn and the period T of the scrambling.
The period of the scramble T was set to 100, 200, 300, 400,
and 500 ms. The length tn was increased from 50 ms to T insteps of 50 ms.
Figure 7 shows the results of PEAQ for the semi-scrambled207

T=100 (ms)
T=200 (ms)
T=300 (ms)
T=400 (ms)
T=500 (ms)0−0.5

PEAQ (ODG)

−1
−1.5
−2
−2.5
−3
−3.5
−4
0

100

200

t (ms)

300

400

n

Figure 7: PEAQ of semi-scrambled audio

500

and audio, we will consider a subjective evaluation of the proposed method through hearing tests.
6. Conclusion
We proposed a semi-scramble method for speech signals
based on phonemic restoration. Two objective measures, SER
and PESQ, were used to evaluate the confidentiality and efficiency of the proposed method. These results confirmed thatthe proposed method plays a role in copyright protection for
an original signal and recovers a semi-scrambled speech signal into the original one. An objective measurement, PEAQ,
was used to investigate applying the proposed method to audio signals. These results confirmed that the proposed method
deteriorated the sound quality of the audio signal. In addition,
we also confirmed that the signal scrambled with the proposed method is a white Gaussian noise. Finally, we indicatedthat the acoustic characteristic of the signal semi-scrambled
with the proposed method enables the listener to comprehend
speech and music information.

Acknowledgments
audio. Each of the values is represented as an average, andeach of the error bars is represented as a standard deviation
This work was supported by a Grant-in-Aid for Scientificof the result for each parameter. For the parameter used for
Research (B) (No. 23300070) and the Fostering ICT Global
the semi-scrambled speech sound, the values of PEAQ were
Leader Program.
distributed from −1.5 to −2.5. These results could confirm
that the proposed method deteriorated the sound quality ofReferences
the audio signal.
5. Discussion[1] N. S. Jayant: Analog scramblers for speech privacy, Computers& Security, Vol.1, pp. 275–289, 1982.
[2] Y. Nishihara, N. Iwakami, H. Fujii and K. Kushima: A researchon semi-disclosure method of audio-music contents, IPSJ SIG
As an additional consideration, we investigated the acousNotes, Vol. 97, No. 28, pp. 31–36, 1997 (in Japanese).
tic characteristics of the scrambled signal in semi-scrambled
speech. In this study, we hypothesized that the scrambled [3] Z. Zhu, K. Yamamoto, M. Unoki and N. Aoki: Study on scramble method for speech signal based by using random-bit shift ofsignal is white noise, and then analyzed the histogram of thequantization, Proc. NCSP’14, pp. 109–112, 2014.
amplitude, the spectrogram, and autocorrelation [10]. The results showed that each sample of the scrambled signal had a [4] R. M. Warren: Auditory Perception: A New Analysis and Synnormal distribution with zero mean and a specific variance,thesis, Cambridge University Press, Cambridge, 1999.
flat spectrum over the range of frequencies, and its value at[5] M. Kashino: Phonemic restoration: The brain creates missing
one time was uncorrelated with any other time. Therefore
speech sounds, Acoustical Science and Technology, Vol. 27,
we confirmed that the scrambled signal with the proposed
No. 6, pp. 318–321, 2006.
method was a white Gaussian noise. The characteristics satis[6]H. Yi and C. L. Philipos: Evaluation of objective measures forfied the necessary conditions for the auditory continuity effectspeech enhancement, Proc. Interspeech2006, pp. 1447–1450,[5].
Pittsburgh, Pennsylvania, 2006.
Thus, phonemic restoration enables the listener to comprehend the speech information even though the sound quality of [7] R. Elbarougy and M. Akagi: Improving speech emotion dimensions estimation using a three-layer model for human percepthe semi-scrambled speech is deteriorated. Moreover, whention, Acoustical Science and Technology, Vol. 35, No. 2, pp.
the lengths of the scrambling intervals are sufficiently short,
86–98, 2014.
the semi-scrambled speech can be perceived as the original
one, as shown in our preliminary subjective tests. In addi- [8] P. Kabal: An examination and interpretation of ITU-R BS.1387:
Perceptual evaluation of audio quality, TSP Lab. Technical Retion, the continuity effect enables the listener to listen to muport, Dept. Electrical & Computer Engineering, McGill Universic more naturally and continuously even though the soundsity, pp. 1–89, 2002.
quality of the semi-scrambled audio is deteriorated. However,these results indicate that the ideal length of the scrambling [9] M. Goto: Development of the RWC music database, Proc. thefor the semi-scrambled audio, which does not have linguistic
18th International Congress on Acoustics (ICA 2004), pp. I–
information, becomes shorter than that of the scrambling for
553–556, 2004.
semi-scrambled speech.
[10] K. Yamamoto, Z. Zhu, M. Unoki and N. Aoki: Study on semiIn a future study, to reveal the relationship between scramscramble method for speech signals based on phonemic restorabling length and the ease of hearing semi-scrambled speechtion, IEICE Technical Report, Vol. 113, No. 290, pp. 59–64,
2013 (in Japanese).

208

Journal of Signal Processing, Vol. 18, No. 4, July 2014