Issue 9
April 2015

In This Issue
We introduce you APSIPA newsletter issue-9 with many interesting articles andreports in addition to the regular columns. We first present two reports from thetechnical committees of Wireless Communications and Networking (WCN) and
Signal and Information Processing Theory and Methods (SIPTM). We intend to
introduce the technical committee reports from the other technical committees in
the coming APSIPA newsletter issues. We are aiming toward promoting APSIPA
technical committees to attract more members as well as help channeling paper
submissions to APSIPA conferences and Journal.
The articles presented in APSIPA newsletter are to prosper knowledge in important researchareas in a quick and condensed mode. There are four articles in this issue sequenced asfollow:
 Matlab real-time tools for speech and signal processing education
 Affective Analysis of Music Signals using Acoustic Emotion Gaussians: A Brief
Overview
 Post-Filter Using Modulation Spectrum as a Metric to Quantify Over-Smoothing
Effects in Statistical Parametric Speech Synthesis
 Interference Suppression Schemes for Radio over Fiber Simultaneously Transmittedwith 10 Gbps On-Off Keying
We also have in this issue an important announcement about the release of the first publicly
released Mandarin-English code-switching
speech database for automatic speech recogniWCN Technical Committee Report
Page 2tion research which is available through LDC.
SIPTM Technical Committee Report
Page 3
We hope you find this issue useful and pleasefeel free to send us comments and suggestions
Page 5
MATLAB Real-time Tools for Speech
to improve our newsletter to benefit as much
Page 11Affective Analysis of Musicas possible our APSIPA members.
Waleed Abdulla
APSIPA Newsletter Editor-in-Chief

Post-filter Using Modulation Spectrum

Page 14

Interference Suppression Schemes

Page 17

SEAME Speech Database

Page 21

P a ge 2

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

Wireless Communications and Networking (WCN)
Technical Committee Report
Tomoaki Ohtsuki

It is our great pleasure to introduce Wireless Communications and Networking(WCN) Technical Committee (TC). Current officers are TC Chair: Tomoaki
Ohtsuki, Keio University (Japan), Vice-Chair: Sumei Sun, I2R (Singapore), andSecretary: Takahiko Saba, Chiba Institute of Technology.
WCN-TC sponsors papers, participates in organization of conferences, andpromotes technical workshops on those aspects of communications that pertain
to the innovation, development and application of algorithms and electronic and photonic
devices or subsystems for generation, processing, storage, transmission, recovery, andpresentation of communications signals. In addition, the committee has professional development goals for committee members and other practitioners working in the above areas.
1. Fields of Interest
The fields of interest of the WCN TC shall be, but not limited to, the following:

Signal processing in wireless communications, including physical layer, medium
access, and networking functions



Signal processing architecture and circuits for wireless communications and networks



Signal processing for applications of wireless mobile communications and networks



Signal representation, theory, and processing
communications and networks, and their applications

towardinnovative

wireless

2. TC MembershipThe TC members shall be researchers in the fields of interest with good standing, andbe elected by current members of the TC.



The WCN TC shall consist of maximum 50 members plus the Chair, the Vice Chair,
the Secretary, and the Past-Chair. All TC members are voting members. Each member
shall ordinarily serve two consecutive three-year terms, but membership may beterminated at any time if the Chair agrees that the member has not fulfilled his/her TC
responsibilities. A member whose two three-year terms expire will be eligible forre-election only after a one-year pause.

Each member of the TC is expected to participate in at least 75% of the votes.
Members not meeting their obligations will not be considered for renewal of their first
three-year term. Renewal of the first three-year term is subject to a vote by the TC
members, and is conducted along with the election of new members as outlined in
WCN Bylaw 5, that is available on APSIPA WCN website.

Memberships expire at the end of the third year.

P a ge 3

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

3. Member Election Procedure:The TC Chair and Vice-Chair are responsible to publicize the rules of nomination andprepare a ballot for election.

Nominations are open to all researchers in the field of interest with good standing. The
nominator is responsible for contacting the nominee before the nomination andprovides a succinct case for the candidate.



The candidates who receive the highest number of votes will be elected. In the event ofa tie, a swift election is conducted. The quorum for a winning election is 50% of the
TC membership.

The election will be conducted via e-mail, unless another means is proposed andapproved by the TC.

4. Supporting Activities
As appropriate, WCN-TC will be active in all of APSIPA’s activities. This will include
APSIPA hosting conferences (such as APSIPA Annual Summit and Conference) byproviding representatives from their respective Technical Program Committees, bysoliciting assistance from its membership to provide professional review of submitted
papers, and by organizing mini-conferences, symposia, panels, short courses, tutorials, etc.,
as deemed appropriate by the Society and WCN-TC. Further, WCN-TC can individuallyorganize workshops and conferences. Also due to the broad nature of committee activities,
collaborative sessions with other committees will be sponsored and heartily encouraged.
WCN-TC will further support APSIPA Transaction and standards activities by soliciting
volunteers as authors and editors, submitting proposals, and identifying committee members
from its membership. WCN-TC will seek ways to increase active participation of its
members in information exchange related to the charter of this Committee, such as:
stimulating Feature Topics and Special Issues of APSIPA Transaction; and by sponsoringworkshops, tutorials, short courses, panel sessions, etc. on special topics.

APSIPA Signal and Information Processing Theory and Methods
Technical Committee Report
Anthony Kuh and Akira Hirabayashi
This is a note of activities by the Signal and Information Processing Theory and Methods(SIPTM) Technical Committee (TC). The SIPTM TC is an active TC promoting the richarea of signal and information processing theory and methods. This area has grown inrecent years with much research and education in the areas of theory, algorithms, andapplications. Areas range from traditional topics such as digital filters, statistical signal
processing and adaptive signal processing to newer topics in applying signal andinformation processing to big data and smart grids. The SIPTM TC promotes activities
through organization of special and regular sessions at the APSIPA Annual Summit andConference (ASC), through publication of journal articles in the APSIPA Transactions on
Signal and Information Processing, through articles such as this one in the APSIPAP a ge 4

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

Newsletter discussing activities, and encouraging researchers and educators in signal andinformation processing to join our community.
The last APSIPA ASC was held in Siem Reap, Cambodia. We held a SIPTM TC meetingon December 10th. The meeting was attended by Oscar Au, Mrityunjoy Chakraborty,
Akira Hirabayashi, Anthony Kuh, and Yili Xia. At the meeting we discussed recent
activities of the SIPTM TC and also ways we could contribute to APSIPA including
publicizing SIPTM TC activities. There was also a succession of leadership with Akira
Hirabayashi succeeding Anthony Kuh as the new chair of the SIPTM TC. SIPTM had
several special and regular sessions at the 2014 APSIPA ASC including special sessions on``Signal and Information Processing Applications of Smart Grids and Energy’’ and``Machine Learning Algorithms and Applications in Signal Processing’’.
In the APSIPA Transactions on Signal and Information Processing we currently have a
special ongoing issue on ``Signal and Information Processing for the Smart Grid’’. In 2014
three articles from this special issue appeared and there are currently more papers in thereview process. We look forwards to more contributions from the SIPTM community forboth special issues and journal submissions to the APSIPA Transactions on Signal andInformation Processing.
We welcome researchers and educators in signal and information processing to join our TC
by contacting either Akira at akirahrb@media.ritsumei.ac.jp or Anthony at
kuh@hawaii.edu.

Science Quotes
Science cannot solve the ultimate mystery of nature. And that is because, in the last
analysis, we ourselves are part of nature and therefore part of the mystery that we are trying
to solve.
— Max Planck (Bor n 23 Apr 1858; died 4 Oct 1947)
From: Where is Science Going, Max Planck (1933).
There is a story that once, not long after he came to Berlin, Planck forgot which room had
been assigned to him for a lecture and stopped at the entrance office of the university to findout. Please tell me, he asked the elderly
man in charge, 'In which room does
Professor Planck lecture today?' The oldman patted him on the shoulder 'Don't go
there, young fellow,' he said 'You are much
too young to understand the lectures of ourlearned Professor Planck'.
From: Men Who Made a New Physics:
Physicists and the Quantum Theory,
Barbara Lovett Cline (1987).

From left to right: W. Nernst, A. Einstein, M. Planck, R.A. Millikan and von Laueat a dinner given by von Laue in Berlin on 11 November 1931

Page 5APSIPA Newsletter

April 2015

Matlab realtime tools
for speech and signal processing education
Hideki Kawahara∗∗ Faculty of Systems Engineering, Wakayama University, Wakayama, Wakayama, Japan

E-mail: kawahara@sys.wakayama-u.ac.jp Tel: +73-457-8461

Abstract—Interactive realtime tools for education were developed making use of Matlab GUI functions. They are a) a realtime spectrogram with narrowband, wideband and perceptual
frequency resolution axes, b) a realtime spectrum monitoring
followed by an interactive spectrogram display with soundplayback of a rectangular time-frequency region and scrubbing
inspection, c) realtime vocal tract shape display of input sounds,
and d) realtime F0 extraction with event detection and display.
In addition to these tools, animation generator functions areprepared for understanding fundamentals of Fourier transformand digital signal processing.

I. I NTRODUCTION
Extraordinary advancement in computational power of personal machines makes it possible to use these machines asinteractive learning tools for understanding not-very-intuitive
mathematical concepts underlying speech, hearing and digital
signal processing. In addition to this computational power,
introduction of modern high-level software development systems greatly reduces necessary time an d effort to implement
such tools. This article introduces tools we have developed
for training undergraduate and graduate students. The tools,
with their Matlab source codes, are freely available from theauthor’s web page [1].
II. B EFORE INTRODUCING REALTIME TOOLS
Students have to have some acquaintance to the concepts
such as time, frequency and waveform before introducing
tools described in this article. In the very beginning of
the introduction lecture of media technologies for the firstyear undergraduate students, I prepared animation materials
shown in Fig. 1. The top left movie introduces the relation
between complex exponential function and sinusoids, Euler’s
equation. The top right movie introduces Fourier transform of
periodic signals as a Ptolemaic system by morphing settingsof harmonic amplitude and phase shown in the lower partof the movie. In addition to these movies, a movie withsound, shown in the bottom left panel, is presented to makestudents acquire/experience relation between pitch, timbre,
harmonic structure and waveform. For graduate students, I
prepared one additional movie, shown in the bottom rightpanel, for introducing relative phase of harmonic signals on
timbre perception and waveform, using a cyclic transition fromcosine phase, sine phase, alternating phase [2], and Schroeder
phase [3]. This snap shot shows Schroeder phase.

Fig. 1. Introduction movies to (top left) complex exponent, (top right) Fourier
series, (bottom left) harmonic structure and timber, and (bottom right) relative
phase on timber and waveform.

All these movies were made using Matlab [4] scripts and
QuicTime Player 7 pro. The moves shown in Fig. 1 and the
Matlab scripts archive are accessible from my site. But please
note that this cumbersome procedure for making movies withsounds is already outdated today. It can be replaced by Matlab
GUI-based programming. The following sections also showsuch examples.
III. R EALTIME M ATLAB TOOLSThis section introduces each Matlab realtime tool with
examples of basic operation and function. It also provides
practical hints for implementing such tools using Matlab
GUI functions. These tools only require the signal processing
toolbox and do not use platform dependent codes.
A. Realtime spectrum analysis and recording toolThis tool has two modes of operation; realtime monitoring
and recording mode and interactive inspection mode.

Page 6APSIPA Newsletter

Fig. 2. Screen capture of the realtime spectrum analyser mode when monitoring the background noise in my office. This illustrates realtime monitoringof the waveform segment, the time windowing function and the calculated
power spectrum using log-frequency axis.

1) Realtime monitoring and recording mode: Figure 2
shows a screen capture of this tool in its realtime monitoring
and recording mode. The bottom left panel updates waveformin realtime and the bottom right panel also updates thepower spectrum of the waveform segment shaped by the time
windowing function that is also displayed in the bottom left
panel. In this mode, top right panel displays the peak level ofeach analysis segment. The clipping level is set 0 dB in thismode.
This GUI uses default toolbar icons shown in the top left
corner of the tool. They are zoom-in, zoom-out, panning anddata cursor. In this snapshot, a data cursor is attached to oneof a salient peak of the power spectrum plot and updating theattached coordinate values in realtime. The peak was causedby the noise from the desiccator for measuring microphonestorage. These icon tools can be toggled by clicking while
monitoring realtime.
The available windowing functions are Blackman, Hamming, Hanning, Bartlett, Rectangular [5] and Nuttall [6]windows. The preset window lengths are 10, 30, 100 and
300 ms. The frequency axis can be selected from the following
list; Logarithmic, Linear full scale, Linear 1000 Hz, Linear
8000 Hz and Linear 4000 Hz. This figure shows Linear fullscale. These settings can also be changed on the fly.
In this mode, only Stop and Quit buttons are enabled.
Clicking Stop button terminates data acquisition and displays
the spectrogram of the acquired data. Displaying spectrogram
initiates interactive inspection mode.
2) Interactive inspection mode: Figure 3 shows a snap shotof inspection mode. Top right panel shows the spectrogram of
the all acquired data in the time domain and 0 Hz to fs /2
in the frequency domain, where fs represents the sampling
frequency. The analysis window is a 15 ms Blackman window
with 2 ms frame update. The pseudo colour dynamic range is
80 dB from the peak power spectrum level. In this figure theApril 2015

Fig. 3. Initial display of interactive inspection mode. Three segments ofJapanese vowel sequences and two whistling sounds were acquired. Note thatthe zoom-in tool is highlighted in the tool bar. The second whistling region
is being selected using rubberband selector around 5.6 s. The rubberband ismade visible by magnifying the region of this spectrogram.

second whistling region around 5.6 s is being selected using
a rubberband selector. Figure 4 also shows the zoomed region(This display show scrubbing mode. This is explained later.).
The GUI toolbar icons are also usable in this mode. Spectrogram, waveform and power spectrum display panels canbe zoomed, panned and by using the data cursor coordinate
values can be monitored.
In this mode, buttons other than “Stop” is enabled. The“Play all” button plays back all data acquired in the input
buffer. The “Play visible” button plays back the data, which isdisplayed in the zoomed spectrogram. The “Play visible timefrequency region” plays back the sound which corresponds tothe selected time-frequency region by zooming operation. This
operation is implemented using FIR filter design based on thevisible frequency axis information of the spectrogram display
panel. The “Save all” button saves whole acquired data to awave file. The “Save visible” button saves only the visible
portion of the data. The “Load” button replaces the wholeacquired data with the content of the file selected by Matlab
GUI’s file input dialogue.
This “Play visible time-frequency region” function is useful for demonstrating timbre differences between different
frequency regions and age related hearing loss. It can be
instructive to playback so-called “mosquito sound” using thisfunction. The majority of the sound is located around 16 kHz.
Zooming the region around 16 kHz sounds like silence for(most of the) teachers but not for students. It sounds irritating
tweets for teenagers, majority of the first year undergraduate
students.
3) Scrubbing of spectrogram display: When the “Scrubbingmode” radio button is on, the spectrum slice, the waveform
segment and the corresponding sound of a specific portion of
the spectrogram can be inspected by scrubbing the spectrogram using the scrubbing cursor. As mentioned before, thePage 7APSIPA Newsletter

Fig. 4. Zoomed spectrogram in scrubbing mode ON. The zoomed region
shows a transitional mode of whistling, where two oscillation modes are coexisting and produce beating in the waveform.

zoomed whistling display in Fig. 4 shows a snap shot in thismode.
The user can drag the scrubbing cursor (white thick verticalline in the spectrogram) to view any part of the displayed
spectrogram. Two green lines flanking the scrubbing cursor
indicates segment boundaries currently analysed. The waveform panel updates the display synchronised with the dragmovement. The power spectrum display also is updated.
Similar to the realtime monitoring mode, the window shapeand the length, and frequency axis can also be changed on
the fly. (Figure 5 shows a snap shot with linear frequency
axis up to 4000 Hz with 30 ms analysis window length.
This setting is relevant for demonstrating vowel and formant
relations.) Scrubbing radio button is enabled to be selected
only when all GUI toolbar icons are deselected. By setting“with audio” radio button ON, sound can also be playedback while scrubbing. Please use this “with audio” function
carefully in classroom situation, because using this mode forspeech makes it sound like stuttering. One of students reported
after my lecture that it hurt him, a stutterer.
B. Realtime spectrogram tool
Figure 6 shows a snap shot of the realtime spectrogram tool
in its “wide” band spectrogram display mode. The durationof the displayed spectrogram is one second. The spectrogram
is continuously scrolling from right to left. The windowing
function is Nuttall window with 10 ms in length and 0.5 ms
frame update interval. The display update interval is 100 ms
in this implementation. In “narrow” band spectrogram display
mode, the window length and the frame update interval are set
80 ms and 7 ms respectively. The duration of the displayed
spectrogram is increased to two seconds. In wide band spectrogram, the frequency rage is set from 0 Hz to 8000 Hz.
In narrow band spectrogram, the frequency rage is set from
0 Hz to 4000 Hz. These modes are designed for introducing
basics of acoustics phonetics and the wide band display is used

April 2015

Fig. 5. Zoomed spectrogram in scrubbing button ON. The scrubbing cursor
is located at the initial part of a Japanese vowel sequence /aiueo/ spoken bya male speaker. The power spectrum display uses the linear frequency axis
from 0 Hz to 4000 Hz.

Fig. 6. Realtime spectrogram in “wide band” display mode. The spectrogram
is continuously scrolling from right to left. A Japanese vowel sequence /aiueo/
spoken by a male and a click sound are shown.

mainly for showing formant trajectories and the narrow band
display is used mainly showing prosody, in other words, F0(fundamental frequency) trajectories and harmonic structure.
This tool also has several types of simulated filter bankmode. They are a) ERB N number [7] frequency axis with
1 ERB N filter bank, b) Bark [8] filter bank and c) onethird octave filter bank. Figure 7 shows the ERB filter bank.
Please note that these simulated filter bank analyses arecalculated from the narrow band spectrogram and the temporal
resolution of these displays are highly smeared in the higher
frequency region. Also the group delay caused by filtering
is not represented. The latter problem is severe in the lower
frequency region. Even with these limitations, simulated filterbank mode is useful for demonstrating differences between
engineering representations, such as wide band, narrow band

Page 8APSIPA Newsletter

April 2015

Fig. 7. Realtime spectrogram in “ERB N number” display in simulated filterbank analysis mode. The spectrogram is continuously scrolling from right toleft. Two segments of Japanese vowel sequence /aiueo/ spoken by a male andtwo segments of whistling are shown.

and one third octave, and perceptual representations, suchas ERB and Bark. Please note that ERB N number closely
matches with the frequency allocation on the basilar membranein the cochlea of human [9].
Display modes selection menu in the bottom left corneris enabled when “Stop” button is clicked to suspend running spectrogram mode. After selection of the display mode,running spectrogram starts. The vertical slider is used to sed
the dynamic range of pseudo colour display. The gain of theautomatic gain controller is reset to prevent clipping of thesignal and linearly slowly increases while no clipping happens.
C. Event display tool
Figure 8 shows snap shots of the event display tool. This
tool is mainly designed to serve as a template of realtime
programming using Matlab GUI functions. This tool calculates
one third octave band levels, modified correlation based F0 andmaximum correlation (a variant of frequency domain methods [10], dividing power spectra by their frequency-smoothedversions followed by base-band weighting and inverse Fouriertransform to yield modified autocorrelations, 1970s’ technology), segmental power, and segmental kurtosis. It also detects
acoustic event based on the maximum correlation, the power
jump and the kurtosis value. These are calculated simultaneously triggered by timer interruption (interval is 50 ms in thistool) associated with the audio recording object in Matlab.
The central plot of each panel displays F0 values as green
dots. The frequency axis is represented in either the musicalnote scale or the logarithmic frequency. These display scales
can be changed on the fly by clicking the radio button shownin the left side of the GUI.
The top plots show the dB power, acoustic events detectedby the kurtosis, events detected by the onset of segmental
power, and periodic events such as voiced sounds. The threebars in the bottom left represents the power, the kurtosis andFig. 8. Realtime event detection tool. The initial part shows a musical scalesang in vowel /a/ sound and the latter part shows segments of vowel sequence/aiueo/ by a male. The top panel shows F0 trajectories using musical noteaxis. The bottom panel shows them using the logarithmic frequency axis. The
frequency axes mode can be changed on the fly.

the maximum correlation of each analysis frame. As shown inthe bottom panel of Fig. 8, increase in the colour saturationlevel indicate event detection by the corresponding bar(s). In
this example, the event is detected by the high kurtosis level.
The one third octave band level display and the waveform
display are updated in the same interval, every 50 ms. The facecolour of the left rectangular box is determined by mapping
relative power levels of lower, middle and higher frequency
bands to RGB colour levels.
D. Vocal tract shape tool
Figure 9 shows the realtime display of the vocal tract shapeof the input signal. The top right plot shows the logarithmic
area function. The horizontal axis represents the distancefrom the lip opening. The three dimensional vocal tract shapein the bottom right plot is drawn using a constant volume

Page 9APSIPA Newsletter

April 2015

Fig. 10. Animation movies for introducing group delay. (Left plot) unwrapped
phase of a windowed pulse train. (Right plot) group delay of a windowed pulse
train.

Fig. 9. Realtime display of the vocal tract area function.

constraint on the vocal tract cavity. The vocal tract areafunction is derived from LPC [11], [12] coefficients throughreflection coefficients [13], [14], in other words PARCOR
coefficients [15]. A pre-processing stage consisting of highfrequency emphasis (differentiation) and equalisation using
first order FFT cepstrum coefficient is used for compensating
effects of glottal source spectral shape and the radiation
characteristics from lip opening to the open space.
The three dimensional view can be rotated in three dimensional space on the fly in any mode. This rotation functionality
is default for Matlab graphics objects. The bottom left plotshows the short term power spectrum and the spectral envelope
based on LPC analysis results. The window type, the lengthand the frame shift are Blackman, 30 ms and 8 ms respectively.
The sampling frequency was set 8000 Hz. The LPC analysis
results using the analysis order 14 is used to display thespectrum envelope. For vocal tract shape analysis, the analysis
order 10 is used, since needs of extra analysis order is removedby the pre-processing stage.
IV. I MPLEMENTATIONThe open GUI layout editor of Matlab [4], “guide” wasused in GUI design and programming of these tools. Realtime
update of displays are written as event handlers which are
triggered by the timer interruption associated with the Matlab
audio input object. Interactive inspections such as scrubbing
are also written as event handlers. Button up, button down, motion handlers are dynamically assigned or released dependingon events for enabling interactive scrubbing and synchronised
update of displayed plots. These handlers are assigned to someof graphic objects such as the scrubbing cursor.
V. D ISCUSSION
The audio output object of Matlab also can define timer interruption and “guide” provides means to write event handlers
for it. Using this timer interruption, animation movies withsound output introduced in Fig. 1 can easily be implemented
as interactive realtime tools. It is generally painful to inspect

Fig. 11. Vocal tract-based STRAIGHT parameter manipulation GUI.

30 sheets of graph plot and to find underlying phenomena but
it is easy or sometimes fun to observe a scientific visualisation
movie of one second long with 30 frames per second update
to find what kind of change is happening. In this respect,interactive realtime animation tools are promising for assisting
speech and signal processing education.
Figure 10 shows one such example introducing the conceptof group delay. In both panels, the top left plot shows the pulse
train and windowing function. The bottom left plot shows thephase spectrogram. The horizontal axis of each plot is aligned.
In the left panel, the bottom right plot shows the unwapped
phase of the calculated short term Fourier transform. The sameplot in the right panel shows the group delay. Comparing these
two movies help students acquire an physically meaningfulinterpretation of group delay as a centroid of energy in eachfrequency.
The Matlab source codes provide templates for designing tools for serious research. Figure 11 shows one
such example. This GUI enables parameter modification of
STRAIGHT, speech analysis, modification and resynthesis
system [16], [17], based on an intuitive representation, vocal
tract shape [18]. This GUI is implemented in two days bymodifying the code of the vocal tract shape tool. The top

Page 10APSIPA Newsletterright spectrogram display, the bottom center power spectrum
and LPC spectral envelope display, the top left logarithmic
area function display, and three dimensional vocal tract shapedisplay are updated synchronously to manipulation of vocal
tract shape by using tools in the left side of the GUI. Detailsof this tool will be presented elsewhere.
VI. C ONCLUSIONSA set of realtime tools implemented using GUI functionsof Matlab is introduced. Also a set of introduction movies formedia technologies is introduced. Their Matlab source codes
are freely accessible from the author’s web page [1].
ACKNOWLEDGMENT
The author appreciates to the APSIPA newsletter editor Prof.
Abdulla for suggesting this submission for sharing codes andpractice in signal processing education. This article serves asa subtext of one of the topics in my lecture as a 2015-2016
Distinguished Lecturer of APSIPA.
R EFERENCES[1] H. Kawahara. Hideki kawahara, professor wakayama university. [Online]. Available: http://www.wakayama-u.ac.jp/˜kawahara/index e.html[2] R. D. Patterson, “A pulse ribbon model of monaural phase perception,”
J. Acoust. Soc. Am., vol. 82, no. 5, pp. 1560–1586, 1987.
[3] M. R. Schroeder, “Synthesis of low-peak-factor signals and binary
sequences with low autocorrelation,” IEEE Trans. Information Theory,
vol. 16, no. 1, pp. 85–89, 1970.
[4] Matlab, 8.4.0.150421 (R2014b).
Natick, Massachusetts, USA: The
MathWorks Inc., 2013.
[5] F. J. Harris, “On the use of windows for harmonic analysis with thediscrete Fourier transform,” Proceedings of the IEEE, vol. 66, no. 1, pp.
51–83, 1978.
[6] A. H. Nuttall, “Some windows with very good sidelobe behavior,” IEEE
Trans. Audio Speech and Signal Processing, vol. 29, no. 1, pp. 84–91,
1981.

April 2015[7] B. C. Moore and B. R. Glasberg, “Suggested formulae for calculating
auditory-filter bandwidths and excitation patterns,” The Journal of the
Acoustical Society of America, vol. 74, no. 3, pp. 750–753, 1983.
[8] E. Zwicker and E. Terhardt, “Analytical expressions for critical-bandrate and critical bandwidth as a function of frequency,” The Journal of
the Acoustical Society of America, vol. 68, no. 5, pp. 1523–1525, 1980.
[9] D. D. Greenwood, “A cochlear frequency-position function for several
species—29 years later,” The Journal of the Acoustical Society ofAmerica, vol. 87, no. 6, pp. 2592–2605, 1990.
[10] W. Hess, Pitch Determination of Speech Signals: Algorithms and Devices. Springer-Verlag, 1983.
[11] F. Itakura and S. Saito, “A statistical method for estimation of speech
spectral density and formant frequencie,” Electro. Comm. Japan, vol.
53-A, no. 1, pp. 36–43, 1970, [in Japanese].
[12] B. S. Atal and S. L. Hanauer, “Speech analysis and synthesis by linear
prediction of the speech wave,” J. Acoust. Soc. Am., vol. 50, no. 2B, pp.
637–655, 1971.
[13] H. Wakita, “Estimation of vocal-tract shapes from acoustical analysisof the speech wave: The state of the art,” Acoustics, Speech and Signal
Processing, IEEE Transactions on, vol. 27, no. 3, pp. 281 – 285, jun
1979.
[14] J. Schroeter and M. Sondhi, “Techniques for estimating vocal-tract
shapes from the speech signal,” Speech and Audio Processing, IEEE
Transactions on, vol. 2, no. 1, pp. 133–150, Jan 1994.
[15] F. Itakura and S. Saito, “On the optimum quantization of feature parameters in the parcor speech synthesizer,” in Conference Record, 1972
International Conference on Speech Communication and Processing,
Boston, MA, 1972, pp. 434–437.
[16] H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné, “Restructuring
speech representations using a pitch-adaptive time-frequency smoothing
and an instantaneous-frequency-based F0 extraction,” Speech Communication, vol. 27, no. 3-4, pp. 187–207, 1999.
[17] H. Kawahara, M. Morise, T. Takahashi, R. Nisimura, T. Irino, and
H. Banno, “TANDEM-STRAIGHT: a temporally stable power spectral
representation for periodic signals and applications to interference-free
spectrum, F0 and aperiodicity estimation,” in Proc. ICASSP 2008, 2008,
pp. 3933–3936.
[18] A. Arakawa, Y. Uchimura, H. Banno, F. Itakura, and H. Kawahara, “Highquality voice manipulation method based on the vocal tract area function
obtained from sub-band LSP of STRAIGHT spectrum,” in Proc. ICASSP
2010, March 2010, pp. 4834–4837.

Page 1 1

APSIPA Newsletter

April 2015

Affective Analysis of Music Signals using Acoustic
Emotion Gaussians: A Brief Overview
Ju-Chiang Wang,? Yi-Hsuan Yang,?† and Hsin-Min Wang?†
Institute of Information Science, Academia Sinica, Taipei, Taiwan†
Research Center for IT Innovation, Academia Sinica, Taipei, Taiwan
Emails: {asriver, yang, whm}@iis.sinica.edu.tw?

Fig. 1.

A2

z2

q2

AK-1

zK-1

qK-1

G1

G2
^

G
G3
Weighted
Combination

G4

ArousalFeatureVectors

q1

…

k=1

…

z1

…

T HE AEG M ODELAs Figure 1 shows, AEG involves the generative process
of VA emotion distributions from audio signals. To start thegenerative process of AEG, we first learn an acoustic GMM
as the bases to represent a music clip,
XK
p(x) =
πk Ak (x | mk , Sk ) ,(1)

Music Clip

A1…

I NTRODUCTION
We provide in this letter a brief overview of a recently
proposed generative model called Acoustic Emotion Gaussians(AEG) for emotion-based music signal processing and information retrieval [3]–[8]. The idea is to present the possible
affective responses to a music clip by a probability distribution, so as to account for the subjective nature of emotion
perception. The term affective response in this letter refers to
the emotion people perceive as being expressed in a musicclip. Moreover, we describe emotions in terms of valence(or pleasantness; positive or negative affective states) andarousal (or activation; energy level), the two most importantdimensions of emotion [2], [9], [10]. For example, happiness isan emotion state associated with a positive valence and a higharousal, while sadness is an emotion state associated with anegative valence and a low arousal. The valence-arousal (VA)
space is viewed as an Euclidean space and any point in the
VA space can be considered as a specific emotion state.
The name of the AEG model comes from its use of multiple
Gaussian distributions to model the affective content of music.
The algorithmic part of AEG has been first introduced in [6],along with the preliminary evaluation of AEG for emotionbased music annotation (aka music emotion recognition, or
MER). More details about the analysis part of the model
learning of AEG can be found in a recent article [8]. Dueto the parametric nature of AEG, model adaptation techniques
have also been proposed to personalize an AEG model in an
online, incremental fashion, rather than learning from scratch[1], [7]. The application of AEG to emotion-based music
video generation and emotion-based music retrieval can befound in [4] and [5], respectively. Moreover, as shown in [3],the AEG model can be extended to explore the connection
between emotion dimensions and discrete emotion categories
for visualization and retrieval purposes.
In what follows, we briefly describe the AEG model, andthen introduce its five possible applications. The source codes
for implementing AEG can be obtained from the link: http://slam.iis.sinica.edu.tw/demo/AEG/.

Valence

GK-1

AK

zK

qK

Acoustic
GMM

Latent Topics

Topic
Posterior Prob.

GK
Affective GMM

A Single, Representative
Affective Gaussian

Illustration of the generative process of the AEG model.

where Ak (·) is the k-th component Gaussian, and πk , mk ,and Sk are its corresponding prior weight, mean vector, andcovariance matrix, respectively. Suppose that we have an emotion annotated corpus X consisting of N music clips {si }Ni=1 .
iGiven a set of short-time feature vectors {xi,t }Tt=1
extracted
from a clip si , we first compute the posterior probability foreach feature vector,
Ak (xi,t | mk , Sk )p(Ak | xi,t ) = PK.
h=1 Ah (xi,t | mh , Sh )(2)

Then, the clip-level topic posterior probability θi,k of si canbe approximated by
1 XTip(Ak | xi,t ) .
(3)
θi,k ← p(zk | si ) ≈t=1
TiKWe put the values of {θi,k }K
k=1 in a single vector θi ∈ Rcalled topic posterior vector to represent the audio content.
In view of the subjectivity of emotion perception, a musicclip in the labeled dataset is usually annotated by multiple
subjects. Let ei,j ∈ R2 (a vector indicating the VA values)
denote the annotation of si given by the j-th subject, and Ui
denotes the number of subjects who have annotated si . We
assume that each annotation ei,j can be generated from anaffective GMM weighted by its topic posterior vector θi ,XKp(ei,j | θi ) =θi,k Gk (ei,j | µk , Σk ) ,(4)
k=1

where Gk (·) is a bivariate Gaussian, and µk and Σk are themean vector and covariance matrix of the k-th component (i.e.,
latent topic). Then, we obtain the log-likelihood function of
total annotations over X :X X X
L = log
θi,k Gk (ei,j | µk , Σk ) .
(5)
i

j

k

To learn the parameters of the affective GMM, we adopt the
EM algorithm to maximize Eq. 5 with respect to {µk , Σk }K
k=1 .

Page 1 2

APSIPA Newsletter

April 2015

In the E-step, we compute the posterior probabilities:
θi,k Gk (ei,j | µk , Σk ).
p(zk | ei,j ) = PK
h=1 θi,h Gk (ei,j | µh , Σh )In the M-step, we obtain the updating forms,
P Pij p(zk | ei,j )ei,j0µk ← P P,ij p(zk | ei,j )
P P
0
0 T
i
j p(zk | ei,j )(ei,j − µk )(ei,j − µk )
0
P P
Σk ←
.
i
j p(zk | ei,j )(6)adjust shape(8)

AEG predicts the emotion distribution of an unseen clip byusing its topic posterior θ̂ = {θ̂k }K
k=1 on the learned affective
GMM. We can also use a single, representative GaussianG(µ̂, Σ̂) to approximate the weighted affective GMM [6]:

Σ̂ =

Xk
k

θ̂k µk ,Tθ̂k Σk + (µk − µ̂) (µk − µ̂)
.

2 sec

(a) Point-based

M USIC E MOTION R ECOGNITIONX<1 sec(7)

A thorough analysis of the learning process of the AEG model
can be found in [8].

µ̂ =

3 sec

(9)
(10)

The accuracy of MER can be measured by computing the
KL divergence between the groundtruth (i.e., human labeled)
and the predicted ones, or by comparing the mean VA values.
Evaluations on two emotion-labeled datasets have validated
the effectiveness of AEG over prior arts for MER [8].

P ERSONALIZED M USIC E MOTION R ECOGNITION
Personalization is important for practical emotion-based
music applications. As AEG is a probabilistic model, it can
incorporate personal information of a particular user via model
adaptation techniques to make custom predictions. In light ofthe cognitive load for annotating music emotion, we cannot
assume that a user is willing to provide a sufficient amountof personal annotations at once to make the system reach an
acceptable performance level. On the contrary, a user may
provide annotations sporadically in different listening sessions.
An online learning strategy is therefore desirable. When the
annotations of a target user are scarce, a good online learning
method needs to prevent over-fitting to the personal data inorder to keep certain model generalizability.
Motivated by the GMM-universal background model developed for speaker verification, we can first treat the affective
GMM learned from broad subjects as a background model, andthen employ, for example, maximum likelihood linear regression (MLLR) [1] or maximum a posteriori (MAP) methods[7] to update the parameters of the background model in an
online fashion using the personal annotations. The resulting
personalized model should find a good trade-off between thetarget user’s annotations and the background model.

(b) Trajectory-based

Fig. 2. The stress-sensitive user interface for emotion-based music retrieval.
Users can (a) specify a point or (b) draw a trajectory, while specifying thevariance with different levels of duration.

E MOTION - BASED M USIC R ETRIEVALThe VA space creates a straightforward visualization means
for music collection browsing [9]. Moreover, it offers a readycanvas for music retrieval through the specification of a pointin the emotion space. For example, users can retrieve musicclips of certain emotions without specifying the song titles
or artist names, or draw a trajectory to indicate the desired
emotion changes across a list of songs (e.g. from tenderness
to anger) [11]. An interesting retrieval approach enabled by
AEG is to use a Gaussian-based query by specifying thedesired variances (or the confidence level at the center point)of emotion by pressing a point in the VA space with different
levels of duration or strength. As Figure 2 illustrates, thevariance of the Gaussian gets smaller as one increases theduration or strength of pressing. Larger variances indicate lessspecific emotion around the center point. After specifying thesize of a circular variance shape, one can even pinch fingers
to adjust the variance shape. For a trajectory-based query
input, similarly, the corresponding variances are determinedaccording to the dynamic speed when drawing the trajectory.
Fast speed corresponds to a less specific query and the system
will return pieces whose variances of emotion are larger. If
songs with more specific emotions are desirable, one can slow
down the speed when drawing the trajectory. Such queries canbe handled by AEG by various ways, as shown in [5].
E MOTION - BASED M USIC V IDEO G ENERATION
Nowadays, everyone can easily create a video sequence by aconsumer camcorder and broadcast it over the Internet throughvideo sharing websites such as YouTube. To enhance theentertaining and aesthetic qualities of the video sequences, it
is useful to accompany a video sequence with a piece of musicthat goes well together. For example, people like to accompany
sports video by exciting music, and cheerful music for home
video. We can formulate machine-based automatic generationof music videos as a cross-modality retrieval problem andtackle it based on an extension of AEG. Specifically, this canbe done by jointly learning the tripartite relationship amongmusic, video, and emotion from an emotion-annotated corpus
of music videos. Figure 3 is an illustration of this Acousticvisual Emotion Gaussians (AVEG) model. For a music clip (ora video sequence), the AVEG model is applied to predict its
emotion distribution in the VA space from the corresponding

Page 1 3

APSIPA Newsletter

ArousalVisual
GMM
1
2

2
3

Valence

Valence3
4AK-1 K-1

J-1

K

J

AK
Music Affective GMM

V2

VJ-1
-1

VJ

Video Affective GMM…

GMM KL
Divergence

V1

…

Audio
Segments

A2…

…

Transform & Summarize

Music Signal

VEG Model

ArousalA1

1Transform & Summarize

AEG Model

Acoustic
GMM

Video
Segments

Fig. 3. System diagram of the Acousticvisual Emotion Gaussians (AVEG)
model for emotion-based music video generation.
ATB Model

1

2

2

AEG Model
G1

G2
^

K-1

…

…

Angry

G
G3
G4

Valence

K-1

K

K

Latent Bernoulli
Distribution

Topic Posterior
Probability

Arousal

1

GK-1
GK
Affective GMM

Representative VA GaussianFig. 4. Illustration of the generation flow between tag-based and VA-based
emotion semantics of music. Two component models, namely Acoustic Tag
Bernoullis (ATB) and AEG, are shown in the left and right panels, respectively.
The representative VA Gaussian of a tag can be generated by following theblack dashed arrows.

low-level acoustic (resp. visual) features. Finally, music andvideo are matched by measuring the distance between thetwo corresponding emotion distributions, based on a distancemeasure such as the KL divergence. This model won the firstprize in the 2012 ACM SIGMM Multimedia Grand Challenge.
Please refer to [4] for more details.
C ONNECTING E MOTION D IMENSIONS AND C ATEGORIESIn addition to describing emotions by dimensions, emotions
can also be described in terms of discrete labels, or tags.
While the dimensional approach offers a simple means forconstructing a 2-D user interface, the categorical approach
offers an atomic description of music that is easy to beincorporated into conventional text-based retrieval systems.
Being two extreme scenarios (discrete/continuous), the two
approaches actually share a unified goal of understanding theemotion semantics of music. As the two approaches are functionally complementary, it is therefore interesting to investigate
the relationship between them and combine their advantages
to enhance the performance of emotion-based music retrieval
systems. For example, as a novice user may be unfamiliar withthe essence of the valence and activation dimensions, it would
be helpful to display emotion tags in the emotion space togive the user some cues. This can be achieved if we have themapping between the emotion tag space and the VA space.
Based on AEG, we can unify the two semantic modalities
under a unified probabilistic framework, as illustrated in Fig. 4.
Specifically, we establish a probabilistic framework consistingof two component models, the Acoustic Tag Bernoullis (ATB)
model and the AEG model, to computationally model thegenerative processes from acoustic features to the perceptionsof an emotion tag and a pair of valence-activation values,
respectively. The latent topics {zk }K
k=1 can act as a bridge
between the two spaces, so that the ATB and AEG models

April 2015

can share and transit the semantic information to each other.
The latent topics are learned directly from acoustic feature
vectors, and thus the training datasets for learning the ATB and
AEG models can be totally separate, relieving the requirement
for a jointly-annotated dataset for the two emotion modalities.
Interested readers are referred to [3] for details.
C ONCLUSIONSIn this letter, we have presented the main ideas of a novel
generative model called AEG for music emotion applications.
We have also presented five applications of AEG, including MER, personalized MER, emotion-based music retrieval,emotion-based music video generation, and connecting emotion dimensions and categorical tags. As AEG is a genericframework, it can be easily extended to other multimedia datasuch as speech, image and video. It is hoped that the lettercan bring more attention to affective-based applications for
multimedia retrieval and recommendation.
R EFERENCES[1] Y.-A. Chen, J.-C. Wang, Y.-H. Yang, and H.-H. Chen, “Linear
regression-based adaptation of music emotion recognition models for
personalization,” in Proc. IEEE ICASSP, 2014, pp. 2149–2153.
[2] J. A. Russell, “A circumplex model of affect,” J. Personality and Social
Science, vol. 39, no. 6, pp. 1161–1178, 1980.
[3] J.-C. Wang, Y.-H. Yang, K.-C. Chang, H.-M. Wang, and S.-K. Jeng,“Exploring the relationship between categorical and dimensional emotion semantics of music,” in Proc. ACM Int. Works. Music Information
Retrieval with User-centered & Multimodal Strategies, 2012, pp. 63–68.
[4] J.-C. Wang, Y.-H. Yang, I. Jhuo, Y.-Y. Lin, and H.-M. Wang, “Theacousticvisual emotion Gaussians model for automatic generation ofmusic video,” in Proc. ACM Multimedia, 2012, pp. 1379–1380.
[5] J.-C. Wang, Y.-H. Yang, and H.-M. Wang, “Affective music information retrieval,” in Emotions and Personality in Personalized Services,
M. Tkalk̂ik̂ et al., Eds. Springer, 2015.
[6] J.-C. Wang, Y.-H. Yang, H.-M. Wang, and S.-K. Jeng, “The acoustic
emotion Gaussians model for emotion-based music annotation andretrieval,” in Proc. ACM Multimedia, 2012, pp. 89–98.
[7] J.-C. Wang, Y.-H. Yang, H.-M. Wang, and S.-K. Jeng, “Personalizedmusic emotion recognition via model adaptation,” in Proc. APSIPA
Annual Summit & Conference, 2012.
[8] J.-C. Wang, Y.-H. Yang, H.-M. Wang, and S.-K. Jeng, “Modeling theaffective content of music with a Gaussian mixture model,” IEEE Trans.
Affective Computing, 2015, in press.
[9] Y.-H. Yang and H. H. Chen, Music Emotion Recognition. CRC Press,
2011.
[10] Y.-H. Yang and H. H. Chen, “Machine recognition of music emotion:
A review,” ACM Trans. Intell. Syst. Technol., vol. 3, no. 4, 2012.
[11] Y.-H. Yang, Y.-C. Lin, H.-T. Cheng, and H. H. Chen, “Mr. Emo: Music
retrieval in the emotion plane,” in Proc. ACM Multimedia, 2008, pp.
1003–1004.

Page 1 4

APSIPA Newsletter

April 2015

Post-Filter Using Modulation Spectrum
as a Metric to Quantify Over-Smoothing Effects
in Statistical Parametric Speech Synthesis
Shinnosuke Takamichi∗† , Tomoki Toda∗ , Alan W Black† and Satoshi Nakamura∗∗ Graduate School of Information Science, Nara Institute of Science and Technology (NAIST), Japan

E-mail: shinnosuke-t@is.naist.jp† Language Technologies Institute, Carnegie Mellon University (CMU), U. S. A

Abstract—Recently, we have found a Modulation Spectrum(MS) is capable of sensitively quantifying over-smoothing effects,
which is one of the main factors causing quality degradationin synthetic speech. This letter briefly describes our proposed
approach based on a post-filter to modify the MS to alleviate theover-smoothing effects.

I. I NTRODUCTION
There is no doubt that speech-based systems have beenchanging our lifestyle. Speech synthesis techniques, such as
Text-To-Speech (TTS) synthesis [1] or Voice Conversion (VC)[2], are one of the key technologies to develop these systems.
In 1990s, the basic idea of the statistical parametric approaches
to speech synthesis was originally proposed [3], [2], and recently many attempts, such as Hidden Markov Model (HMM)[4], Gaussian Mixture Model [5], kernel regression [6], anddeep neural nets-based approaches [7], have been studied to
deploy the speech-based systems.
One of the common issues of the statistical parametricspeech synthesis is quality degradation of the synthetic speech[8], [9]. The synthetic speech parameters predicted from the
input information are often overly smoothed, and this oversmoothing effect causes muffled sounds in synthetic speech.
There have been various approaches to alleviate this oversmoothing effect, e.g., modification of the synthetic speechparameters to revise their features clearly different from thosein natural speech parameters. However, this issue still remains
to be addressed.
Recently, we have found a Modulation Spectrum (MS)of the synthetic speech parameters as a novel feature tosensitively quantify the over-smoothing effect and have proposed several post-filter based implementations to modify it
to improve synthetic speech quality [10], [11], [12]. Our
approach has the following merits.
Supported by previous work: The effectiveness of theMS have been reported on speech perception and speech
recognition [13], [14]. Furthermore, the MS is also regarded as
the extension of the Global Variance (GV) [5], [15] known asa conventional feature to quantify the over-smoothing effect.
Application to various speech components: Though someconventional post-filters, such as cepstral emphasis [16] orpeak-to-valley emphasis [17], can be applied to only spectralfeatures, our proposed post-filter is applicable to not only thespectral component but also excitation and duration components.
High portability: Because our proposed post-filter lessdepends on the speech synthesis procedure, it is available invarious speech synthesizers. In Section IV, its effectivenesswill be verified in HMM-based TTS [4] and GMM-based VC[15].
This paper briefly describes the concept of the MS-based
post-filter. The post-filter is trained in advance using thenatural and synthetic speech parameters, and then, it is applied
to the synthetic speech parameters in a synthesis stage. Notethat a part of our work has been implemented in HTS (HMMbased Speech Synthesis System) version 2.3 beta [18].
II. S TATISTICAL PARAMETRIC S PEECH S YNTHESIS AND
OVER -S MOOTHNESS A NALYSIS
A. Speech Parameter Prediction from Input Information
In training, a specific statistical model, e.g., HMM or GMM,
models a probability density function of the natural speechparameters corresponding to input information. The synthetic
speech parameters are predicted based on the maximum likelihood criterion using the trained statistical models. The averaging process in these frameworks often excessively removes
the detailed characteristics of the natural speech parameters,
and causes the muffled effects in synthetic speech. Although
the GV is well incorporated in the parameter generation stage[5], [15], its quality gains are limited.
B. Over-Smoothness Analysis Using Modulation Spectrum
Here, we analyze the natural and generated speech parameter sequences based on the MS. The MS is defined as
the power spectrum of the parameter sequence; i.e., temporalfluctuation of the parameter sequence is decomposed into
individual modulation frequency components and their power
values are represented as the MS. Figure 1 illustrates theaveraged MS of the natural (“natural”) and generated (“HMM”
and “HMM+GV”) mel-cepstral coefficients in HMM-based
TTS. We can observe that the generated MS is markedly
degraded at the higher modulation frequency. This is because
high temporal fluctuation observed in the natural speech parameter sequence is usually lost in the parameter generation

Page 1 5

APSIPA Newsletterquences generated by a statistical parametric speech synthesis
framework.
2) Conversion: The following filter is applied to the generated speech parameter sequence:





 
 










April 2015

s′d (f ) =(1 − k)sd (f )
]
[ (N)
)
σd,f ((G)(N)
sd (f ) − µd,f + µd,f ,
(1)
+ k
(G)
σd,f
where k is a post-filter emphasis coefficient valued between
0 and 1. The finally filtered parameter trajectory is calculated
from the filtered MS and phase components extracted fromthe parameter sequence before filtering.



 


GIHKJ LNMOHQP



@BA:A




!"$#&%(')+*+$,.-0/2143$#516,578:9<;>=$?

@BA:AC5DFE

B. Various Implementations

Fig. 1. Averaged modulation spectra of 1st, 9th, and 15th mel-cepstralcoefficient sequences in HMM-based TTS.
Training

Parameter sequence

Statistical modelPower
calculation

Generation
Synthesis

Modulation spectrumPowercalculation

TrainingPowercalculation

Generation

Postfilter
Phase

Fig. 2. A schematic diagram of the MS-based post-filter in the case of HMMbased TTS.

frameworks. We can also observe that the degradation tends
to be larger in higher order of the mel-ceptral coefficients.
III. M ODULATION S PECTRUM -BASED P OST-F ILTER
Our proposed post-filter is designed to make the MS of thegenerated speech parameters close to that of the natural ones.
Figure 2 shows the schematic diagram of the post-filter inthe case of HMM-based TTS. The post-filter is automaticallytrained using natural and generated parameter sequences in the
training data.
A. Basic Process
1) Training: The training stage estimates the statistics ofthe natural and generated MS. Let sd (f ) be the f -th MS of(N)
the d-th dimension of the parameter sequence. The mean µd,f()2
(N)
and variance σd,fof sd (f ) are estimated using the natural
(
)2
(G)
(G)
parameter sequences. The mean µd,f and variance σd,f
are estimated in the same manner using the parameter se-1) Application to F0 and Duration: The post-filtering process is available in various speech features. In the case of
the F0 contour [10], the post-filter cannot be directly applied
because the observed F0 contour is not a continuous sequence.
To address this issue, continuous F0 contour calculated fromthe observed F0 contour is firstly filtered, and then, the U/Vregion is restored. For duration [12] in HMM-based TTS,
the post-filter is effectively applied to not HMM-state but
phoneme duration. The modified HMM-state is calculated by
maximizing the duration likelihood given the filtered phoneme
duration.
2) Segment-Level Filtering: Because the original post-filter[10] performed the utterance-level conversion, the MS calculation is not accurately calculated when the length of anutterance to be synthesized is longer than the previously
determined DFT length. To address this problem, we proposed
another post-filter implementation that modifies the MS of thesegmented parameter sequence [12]. The filtered parameter sequence is generated by overlapping and adding the segmented
sequence.
IV. E XPERIMENTAL E VALUATION
We conducted several subjective evaluations in order toconfirm the quality gain by the post-filter. The experimental
condition is described in [10], [11], [12]. Figures. 3-6 illustrate
the results of the preference AB test on speech quality. The
error bar is 95% confidence interval. “HMM” or “GMM”
performed the ML-based parameter generation [19], [15],“*+GV” considered the GV in the parameter generation [5],[15], “*+MSPF” applied the post-filter. The post-filter emphasis coefficients were determined by objective evaluations [10],[11], [12]. In these figures, the post-filter have achieved betterquality than the conventional approaches, and the quality
gain is the largest in the spectral features. Even when we
consider the GV in parameter generation, the additional quality
gain is observed by the post-filter. This is because the MS
compensation by the GV-based approaches is insufficient asshown in Figure 1.
Figure 7 illustrates an example of the mel-ceptral coefficientsequences. We can see that the post-filter makes the sequence

Page 1 6

APSIPA Newsletter

April 2015





 





 

'!" 

'!" 

 

&%

&%

$%"

$%"

# !""  
!"
 

*

# !""  
!"
 

       
 

/ %0
../ '

        

Fig. 3. Preference scores (mel- Fig. 4. Preference scores (F0 contour
cepstrum in HMM-based TTS)
in HMM-based TTS)









 ! 








  









 -% " &'* +,) !  %(
& ' ! " #$ %  3" !  13 1   
 
 

   
 




!  





 
 #



12

46587:9 ;=<?>"@1;BA

3"1

32

Fig. 7. Examples of 1st, 9th, and 15th mel-cepstral coefficient sequences in
HMM-based TTS.


 


 
 

Fig. 5. Preference scores (duration in Fig. 6. Preference scores (melcepstrum in GMM-based VC)
HMM-based TTS)

fluctuated, and and such a fluctuation tends to be larger inhigher order mel-cepstral coefficient.
V. C ONCLUSION
This letter presented our approach using Modulation Spectrum (MS) for high-quality statistical parametric speech synthesis. The MS represents the fluctuation of the speech parameter sequence, and the MS-based post-filter improves thesynthetic speech quality by recovering the MS of the generated
parameter sequence.
Acknowledgements: Part of this work was supported by
JSPS KAKENHI Grant Number 26280060 and Grant-in-Aid
for JSPS Fellows Grant Number 26 · 10354, and part of thiswork was executed under ”JSPS Strategic Young Researcher
Overseas Visits Program for Accelerating Brain Circulation.”
R EFERENCES[1] Y. Sagisaka. Speech synthesis by rule using an optimal selection of
non-uniform synthesis units. In Proc. ICASSP, pp. 679–682, New York,U.S.A, Apr. 1988.
[2] Y. Stylianou, O. Cappe, and E. Moulines. Continuous probabilistic transform for voice conversion. IEEE Trans. Speech and Audio Processing,Vol. 6, No. 2, pp. 131–142, Mar. 1988.
[3] K. Tokuda, T. Kobayashi, and S. Imai. Speech parameter generationfrom HMM using dynamic features. In Proc. ICASSP, pp. 660–663,
Detroit, U.S.A, May 1995.
[4] K. Tokuda, Y Nankaku, T. Toda, H. Zen, J. Yamagishi, and K. Oura.
Speech synthesis based on hidden markov models. Proceedings of theIEEE, Vol. 101, No. 5, pp. 1234–1252, 2013.
[5] T. Toda and K. Tokuda. A speech parameter generation algorithm
considering global variance for HMM-based speech synthesis. IEICE
Trans., Vol. E90-D, No. 5, pp. 816–824, 2007.

[6] E. Helander, T. Virtanen H. Silen, and M. Gabbouj. Voice conversion
using dynamic kernel partial least squares regression. IEEE Transactions
on Audio, Speech, and Language Processing, Vol. 20, No. 3, pp. 806–
817, Mar. 2012.
[7] H. Zen and A. Senior. Deep mixture density networks for acoustic
modeling in statistical parametric speech synthesis. In Proc. ICASSP,
pp. 3872–3876, Florence, Italy, May 2014.
[8] S. King and V. Karaiskos. The blizzard challenge 2011. In Proc. Blizzard
Challenge workshop, Turin, Italy, Sept. 2011.
[9] Y. Stylianou. Voice transformation: A survey. In Proc. ICASSP, pp.
3585–3588, Taipei, Taiwan, Apr. 2009.
[10] S. Takamichi, T. Toda, G. Neubig, S. Sakti, and S. Nakamura. A postfilter to modify modulation spectrum in HMM-based speech synthesis.
In Proc. ICASSP, pp. 290–294, Florence, Italy, May 2014.
[11] S. Takamichi, T. Toda, A. W. Black, and S. Nakamura. Modulation
spectrum-based post-filter for gmm-based voice conversion. In Proc.
APSIPA ASC, Siem Reap, Cambodia, Dec. 2014.
[12] S. Takamichi, T. Toda, A. W. Black, and S. Nakamura. Modified
modulation spectrum-based post-filter for hmm-based speech synthesis.
In Proc. GlobalSIP, pp. 710–714, Atlanta, United States, Decc. 2014.
[13] R. Drullman, J .M. Festen, and R. Plomp. Effect of reducing slow
temporal modulations on speech reception. J. Acoust. Soc. of America,
Vol. 95, pp. 2670–2680, 1994.
[14] S. Thomas, S. Ganapathy, and H. Hermansky. Phoneme recgnition usng
spectral envelop and modulation frequency features. In Proc. ICASSP,
pp. 4453–4456, Taipei, Taiwan, April 2009.
[15] T. Toda, A. W. Black, and K. Tokuda. Voice conversion based on
maximum likelihood estimation of spectral parameter trajectory. IEEE
Transactions on Audio, Speech and Language Processing, Vol. 15, No. 8,
pp. 2222–2235, 2007.
[16] T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura.
Incorporation of mixed excitation model and postfilter into HMM-based
text-to-speech synthesis. IEICE Trans. Inf. Syst., Vol. J87-D-II, No. 8,
pp. 1563–1571, 2004.
[17] F. Eyben and Y. Agiomyrgiannakis. A frequency-weighted post-filteringtransform for compensation of the over-smoothing effect in HMM-based
speech synthesis. In Proc. ICASSP, pp. 275–279, Florence, Italy, May
2014.
[18] HMM-based speech synthesis system (HTS) http://hts.sp.nitech.ac.jp/.
[19] K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura.
Speech parameter generation algorithms for HMM-based speech synthesis. In Proc. ICASSP, pp. 1315–1318, Istanbul, Turkey, June 2000.

Page 1 7

APSIPA Newsletter

April 2015

Interference Suppression Schemes for Radio overFiber Simultaneously Transmitted with 10 Gbps
On-Off Keying
Yuya Kaneko, Takeshi Higashino, and Minoru Okada
Graduate School of Information Science, Nara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara, 630-0192 JAPAN
Email: kaneko.yuya.kr2@is.naist.jp
I.

I NTRODUCTION

The radio over fiber (RoF) offers small base stations,centralized operation for heterogeneous wireless service, cooperative distributed antenna system, and large transmission
capacity. The typical RoF link is configured by the intensity
modulation/direct detection (IM/DD) link. Radio frequency(RF) signal directly modulates the laser diode (LD) or externally modulates the light source. The RF signal is detected bya photodetector (PD) after the transmission over optical fiber
channel. Meanwhile, optical On-Off Keying (OOK) modulation for baseband (BB) transmission is employed in 10 Gbps
Ethernet PHY layer. The 10 Gbps Ethernet infrastructure is
widely used in in-building local area network (LAN). The
optical fiber communication technology is employed as various
broadband transmission such as Ethernet, mobile backhaul,
and mobile fronthaul. However, these provide single communication service. The realization for providing multiservice
motivates the simultaneous transmission for baseband andRF signals[1]-[4]. It enables independent different types of
networks to share infrastructure and realizes quick and costeffective implementation.
In previous papers[1]-[3], the frequency band of RF signal
is higher than the main lobe of the OOK spectrum. Since
the frequency channels of major cellular system and Wireless
LAN (WLAN) such as the Long-Term Evolution (LTE) andthe WiFi are assigned at lower than 10 GHz, investigation forcoexistence of the radio systems with frequency lower than
10 GHz and 10 Gbps optical BB signal is required. Chenet al.[4] investigated simultaneous transmission of 10 Gbps
OOK signal and RF signal at 2 GHz. The 8B10B channel coding is employed in their proposal in order to make
spectrum notch with specific interval. The RF signal canbe allocated at the notch frequency band due to suppressedinterference. This scheme cannot be applicable to standard
ethernet because 64B66B channel coding is employed, there isno spectrum notch over the OOK spectrum. Previous study[5]proposed another configuration of RoF link with optical OOK
signal. The proposed system is called as radio on optical
OOK (RoOOOK). The RoOOOK is easy to construct overthe existing fiber link, however, the experiment reveals thatsimultaneously transmitted OOK signal interferes with RF
signal[5].
In this paper, interference suppression schemes in the
RoOOOK are proposed. At first, a theoretical analysis of its

power spectrum is presented and it is compared with theexperiment. In the analysis, the stochastic process of optical
OOK stream is re-modulated by the RF signal. Signal tonoise power ratio (SNR) and error vector magnitude (EVM)of the RF signal are derived. An experimental evaluation using
1.9 GHz RF signal is conducted to investigate coexistencewith 10 Gbps OOK signal. Experimental results agree withthe theory. The experiment shows that the simultaneous transmission of the RF and BB signals without serious performancedegradation is possible without any interference suppression.
However, interference is still a problem, therefore we propose
interference suppression schemes to improve EVM and widen
dynamic range of the system. The improvement of SNR
and dynamic range is discussed from theoretical analysis andcomputer simulation.
II.

S YSTEM DESCRIPTION

Fig. 1 shows the configuration of RoOOOK using 10 Gbps
Ethernet. A pair of 10 Gbps Ethernet switches is connectedby optical fibers. The transmitting optical stream is re-used
as a carrier for RF modulation. The OOK is employed asa modulation format of 10 Gbps Ethernet. The amplitude of
RF signal is mapped onto the intencity of the optical pulse
using an external optical modulator (EOM). At the receivingside, optical signal is divided into two branches using optical
coupler (OC). One branch is led to the OOK demodulator in
the Ethernet receiver. The other signal is fed into the opticto-electric (O/E) converter. Since the obtained photo current
is still waveform of pulsed RF signal, the electric band-passfilter (BPF) is used to regenerate the original RF signal.
In the viewpoint of RF signal transmission, it can be considered as the band-pass sampling and regeneration, because
the RF signal is sampled by the random pulse sequence.
Therefore, the spectrum analysis with stochastic process canbe applicable.
III.

P OWER SPECTRUM ANALYSIS

Fig. 2 shows the external re-modulation of the OOK signal.
Consider p(t) as the rectangular pulse waveform taking 1 or
0 with its duration of 1/fp , sr (t) as the RF signal, B as theamplitude of pulses, and 0 ≤ m ≤ 1 as the modulation index.
The re-modulated OOK signal is expressed as,v(t) = {1 + msr (t)}Bp(t).

(1)

APSIPA Newsletter

April 2015

Power Spectral Density [dBm/MHz]

Page 1 8

Fig. 1: Radio on optical OOK (RoOOOK) using 10 Gbps
Ethernet.

0
← DC

−10
−20

← RF

−30

↓ OOK

−40
−50

↓ Alias

−60
−70
−80
−90
−20

−10

0
frequency [GHz]1020

Fig. 3: Power spectral density of RoOOOK ( fp = 10 Gbps,
fr = 1.9 GHz, W = 384 [kHz], B = 0.05 volt, andm = 0.1).

Optical intensity
mB

Bias
point

Rectangular pulse
waveform, p(t)Transfer curve of
external opticalmodulator

TABLE I: Specification of the equipments.

Re-modulated signal,
Bp(t) + mBp(t)sr(t)RF signal, sr(t)Fig. 2: OOK signal externally re-modulated by RF signal.

XFP specification
PHY Std.
10GBASE-ER
Tx. power
0 dBm
Rx. sensitivity-16 dBm
RF specification
Carrier freq.
1.9 GHzBit rate
384 [kbps]
Modulation
π/4 QPSK
LPF
RNYQ, α = 0.5
Optical system specification
λ1550 nm
fiber
SMF
Vπ
5 volt

The autocorrelation function of v(t), Rv (τ ), is expressed as,
Rv (τ )

= B 2 Rp (τ ) + m2 B 2 Rp (τ )Rs (τ ),(2)where τ is the time lag, Rp (τ ) is the autocorrelation functionof p(t), and Rs (τ ) is the autocorrelation function of sr (t).
Assumiing that the OOK signal is non return-to-zero OOK
bit sequence, any bits are statistically independent, and theprobability of occurrence for mark (p(t) = 1) is ρ, then On
the basis of Wiener-Khinchin theorem, the Fourier transform of(2), Gv (f ) = F [Rv (τ )], is the power spectral density (PSD)of the transferred signal.
 nρ(1 − ρ) 2f
B sinc2+ m2 B 2
Gv (f ) = B 2 ρ2 δ(f ) +
fp
f
  p
o
ρ(1
−
ρ)
f
ρ2 Gs (f ) +sinc2∗ Gs (f ) , (3)fpfp

where Gs (f ) is the Fourier transform of Rs (τ ) and theoperator ∗ means convolution. The first term on the right-handside of (3) is the DC component. The second term is thefrequency component of the OOK signal. The third term is
the RF signal. The fourth term is the noise caused by the
sampling of the RF signal by the OOK signal thus we callthis term alias. Fig. 3 shows an example of PSD.
The peak amplitude of sr (t) is normalized by 1 and thepower of RF signal is considered as a combination of m, B,
and Ss which is the power of Gs (f ). Ss equals 1/2 when sr (t)is a sinusoidal wave. Assuming that the occupied bandwidthof Gs (f ) is the center frequency fr and the bandwidth W , S
which is the signal power after through BPF is,
Z −fr +W/2 Z fr +W/2 !
S=+
Gv (f )df.
(4)
−fr −W/2

fr −W/2

The SNR of the RF signal, γ, is expressed as a function of mand ρ,
γ(m, ρ) =m 2 B 2 ρ2 S s
W,
S − m 2 B 2 ρ2 S s + N 0 W f s(5)where fs is the symbol rate of the RF signal, and N0 is theone-sided noise power spectral density.
IV.

E XPERIMENT

Tab. I shows the specification of the equipments. Fig. 4
shows the experimental setup. IM/DD RoF with external modulation (EM) using continuous wave light source is comparedwith the proposal. The average optical power of the LD is
3 dB higher than the output of the optical Ethernet switch. Thissetting equalizes the instantaneous maximum incident opticalpower to the Mach-Zehnder modulator which is used as an
EOM. The measured total optical power loss between Ethernet
switches including connector loss, fiber loss, insertion loss,
biasing of modulator, and splitting loss is 11 dB.
Fig. 5 shows the PSDs of both RoF with EM andRoOOOK. The RF signal power in RoOOOK is less than RoF
with EM by 6 dB. It is caused by the 3 dB difference of theincident optical power to EOM. The spectral components of
OOK interfere with the RF signal. It corresponds to the resultof the theoretical analysis shown in (3) and Fig. 3.
Theoretical EVM is obtained fromp of average√ the relation
EVM and SNR[6], EV M ≃ 1/ SN R = 1/ γ(m, ρ). A
typical value of ρ is considered as 1/2 because the occurrence
for mark (p(t) = 1) and space (p(t) = 0) are equiprobable.
The incident RF power to EOM is related to m by depending

Page 1 9APSIPA Newsletter

April 201530

theoryexperiment

EVM for RF [%]25
20

Dynamic range ofthe system (12 dB)

Critical point of stable
Ethernet link

15

Required EVM

10
5
0
−15

−10

−5
0
5
Incident RF power to EOM [dBm]10

15

Fig. 6: Relationship between EVM and incident RF power to
EOM.

Fig. 4: Experimental setup.

V.

I NTERFERENCE SUPPRESSION USING RECTIFICATIONIn this section, we propose interference suppression
schemes using rectification. The proposal outperforms theconventional in terms of EVM.
A. Description of proposed scheme
As shown in Figs. 3 and 5, a spectral component of OOK
interferes with an RF signal and deteriorates SNR. Half-wave
rectification can reduce the spectral component of OOK. Fig. 7
shows a proposed interference suppression scheme using halfwave rectification. The DC component is blocked, then theobtained signal is rectified. The half-wave rectified signal isexpressed as mh = 2m and Bh = B/2 for m and B in (1),respectively. Therefore, the half-wave rectification is equivalent
to doubling the modulation index.
As the development of this scheme, biased half-wave
rectification is more effective when the modulation index isknowable at the receiver side. Fig. 8 shows a proposed interference suppression scheme using biased half-wave rectification.
The received signal is biased by (1 − m)B and then it is halfwave rectified. The half-wave rectified signal is expressed as
mh = 2m and Bh = B/2 for m and B in (1), respectively.
The biased half-wave rectification is equivalent to making themodulation index 1.
Fig. 5: Power spectral density of IM-DD RoF with external
modulation ( top ) and RoOOOK ( bottom ), RBW= 3 MHz, RLV = −20 dBm, 5 dB/div, RF in = 0 dBm.

Undesirable physical properties of rectifier (e.g., reverse
recovery time and nonlinearity of diode) are ignored in thispaper.
B. Simulation result√
on m = 2Vrms /Vπ /2 , where Vrms is the root mean squarevoltage of RF signal and Vπ is the half-wave voltage of theEOM. Fig. 6 shows a comparison between the theory and theexperiment in EVM. The theoretical curve corresponds to theexperimental result. Although deep superimposing can improve
EVM, it yields service outage in Ethernet link. When incident
RF power exceeds over 6 dBm, the throughput of Ethernet
link significantly decreases and the link is in a service outage.
The required EVM for RF signal and the service outage of
10 Gbps Ethernet link can define the dynamic range of thesystem. In this case, required EVM is 12.5 %, then the minimal
required incident RF power is found to be −6 dBm. Therefore,
the dynamic range is 12 dB between −6 dBm and 6 dBm of
incident RF power.

A theoretical curve of EVM versus incident RF power to
EOM is drawn in the same way as section IV. Figs. 9 and 10
show theoretical EVM curves and simulation results of conventional RoOOOK and proposal with noise power densities
N0 = −60 and −70 dBm/MHz, respectively. Typically, N0 is
determined by the thermal noise of a receiver. These figures
show improvement of EVM. Fig. 9 shows that the half-wave
rectification improves EVM by about 6 dB and the biased halfwave rectification improves EVM by about 20 dB. Therefore,
the required minimal incident RF power can be reduced by
6 dB or 20 dB compared with the conventional. This power
reduction contributes improvement for the system dynamic
range.
The improvement effect due to use of the half-wave rectification in Fig. 10 is not different from the result of Fig. 9.

Page 2 0

APSIPA Newsletter

Fig. 7: The waveform of half-wave rectified signal.

April 2015

RoOOOK, the dominant noise source changes depending on
the modulation index m which relates to incident RF power.
In a range of low incident RF power which means low m,
the dominant noise is the thermal noise. By the use of thebiased half-wave rectification, the required minimal incident
RF power is determined by the thermal noise of the receiver.
In a range of high incident RF power which means high m, thedominant noise is the spectrum of OOK component in eachcase. Therefore, the EVM curve has the lower limit. However,the lower limit presented in Fig. 9 is sufficiently low to satisfy
the requirement. The proposed schemes improve EVM andwiden the dynamic range.
VI.

Fig. 8: The waveform of biased half-wave rectified signal.

30

theory conventional
theory half−wave rectification
theory biased half−wave rectification
simulation conventional
simulation half−wave rectification
simulation biased half−wave rectification

EVM for RF [%]

25
20
15
10
5
0
−30

−25

−20

−15

−10
−5
0
5
10
Incident RF power to EOM [dBm]

15

20

25

Fig. 9: The simulation results of relationship between EVM
and incident RF power to EOM at N0 = −60 dBm/MHz.
30

theory conventional
theory half−wave rectification
theory biased half−wave rectification
simulation conventional
simulation half−wave rectification
simulation biased half−wave rectification

EVM for RF [%]25
20This paper proposed interference suppression schemes for
RoF simultaneously transmitted with 10 Gbps optical OOK.
After theoretical power spectrum analysis, theoretical EVM
performance is compared with the experiment on radio signaltransmission over 10 Gbps optical Ethernet link. The theoretical EVM corresponds to the experimental results. The external
re-modulation of the optical OOK signal yields impact on BB
and RF resulting in an service outage for 10 Gbps Ethernet
link, and less dynamic range compared with an independent
RoF link. Experimental results show the both two signals
can coexist and share single fiber channel with the dynamic
range of 12 dB. To improve EVM and system dynamic
range, interference suppression schemes are proposed. The
improvement is discussed from the theoretical analysis andsimulation results. They imply that the proposed scheme using
biased half-wave rectification improves the dynamic range
by about 20 dB. The proposed schemes are expected to be
implemented by analog electrical circuits. An empirical proofof the proposed scheme is a future work.
R EFERENCES
[1]

15
10

[2]
5
0
−30

−25

−20

−15

−10
−5
0
5
10
Incident RF power to EOM [dBm]

15

20

25

Fig. 10: The simulation results of relationship between EVM
and incident RF power to EOM at N0 = −70 dBm/MHz.
In contrast, The improvement effect due to use of the biased
half-wave rectification in Fig. 10 is better than the resultof Fig. 9. In the conventional RoOOOK and the half-wave
rectified RoOOOK, the dominant noise source is the spectral
component of OOK signal, because it is generally higher
than the thermal noise. Therefore, the difference of N0 makes
no change of EVM curve. In the biased half-wave rectified

C ONCLUSION

[3]

[4]

[5]

[6]

C.T. Lin, et al., “Hybrid Optical Access Network Integrating Fiber-tothe-Home and Radio-Over- Fiber Systems,” IEEE Photonics Technology
Letters, Vol.19, no.8, Apr.2007.
C.W. Chow, C.H. Yeh, C.H. Wang, F.Y. Shih and S. Chi, “SignalRemodulated Wired/Wireless Access Using Reflective Semiconductor
Optical Amplifier With Wireless Signal Broadcast,” Photonics Technology Letters, IEEE , Vol.21, no.19, pp.1459,1461, Oct. 2009.
S. Yaakob, et al., “Minimal optimization technique for radio over fiber
WLAN transmission in IM-DD optical link,” Microwave and Optical
Technology Letters, Vol.52, issue 4, pp.812-815, 2010.
C. Chen, R. Penty, I. White, and M. Crisp, “Transmission of Simultaneous 10Gb/s Ethernet and Radio-over-Fiber Transmission using In-band
Coding,“ OFC/NFOEC 2013, OSA Technical Digest (online) (OpticalSociety of America, 2013), paper OM3D.1, 2013.
Y. Kaneko, T. Higashino, and M. Okada, “An Empirical Performance
Evaluation of Radio on Optical OOK System,” IEICE Tech. Rep.,
vol. 113, no. 397, MWP2013-63, pp. 51-56, Jan. 2014.
R. A. Shafik, S. Rahman, R. Islam, and N. S. Ashraf, “On the Error
Vector Magnitude as a Performance Metric and Comparative Analysis,”International Conference on Emerging Technologies (ICET), pp.27-31,
2006.

P a ge 2 1

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

SEAME: South-East Asia Mandarin-English Speech Database
Ho Thi Nga, Chng Eng Siong, Haizhou Li
In this newsletter, we introduce the SEAME (South-East Asia Mandarin-English) database,the first publicly released Mandarin-English code-switching speech database for automatic
speech recognition (ASR) research. The database was jointly developed by the SpeechGroups in Temasek Laboratories@Nanyang Technological University, Singapore and in
Universiti Sains Malaysia, Malaysia, and will be released by LDC (LDC2015S04) in Q2,
2015 [5]. SEAME includes 192 hours of Mandarin-English code-switching and monolingual utterances in conversation and interview settings.
What is code-switching speech?
Code-switching occurs when a speaker alternates between two or more languages in a conversation. It has become very common in bilingual societies, people code-switch to convey
a thought, and to fit in a social context among many other reasons [1]. In Singapore andMalaysia, the population consists of multiple races who can speak their respective native
languages well along with the common language, English. Hence, it is quite common tohave speakers embedding English words or phrases when they are speaking their own native
languages and vice versa.
Developing a code-switching ASR system is considerably more difficult than amono-lingual one. This is because the code-switching system will need to accommodate
within one sentence the various languages pronunciation and language model
simultaneously. Furthermore, there are only very few code-switching corpora available
publicly, and are small in scale [1].
Why SEAME corpus?
The SEAME corpus is designed to support many aspects of code-switching studies related
to ASR as follows, among others.







Pronunciation modelling
Language modelling
Acoustic modelling
Automatic speech recognition
Language identification [2]Language turn detection [3][4]Database description
SEAME (LDC2015S04) consists of 192 hours of speech, of which 63 hours are manuallylabelled with word level transcription. Most of the 63 hours of speech are code-switchingutterances. The un-transcribed speech data, mostly mono-lingual segments, will be releasedin the future.
The voices are recorded in two speaking styles: conversational (about 92 hours) andinterview (about 100 hours). In conversational setting, the voices from both interlocutors
were recorded, while in interview setting, the interviewee's voice was recorded. The corpus

P a ge 2 2

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

was recorded in two sites (Singapore and Malaysia) with the same recording setup during
2009 to 2010, with a close-talk microphone in a quiet environment. Each recording session
is about one hour long for either interview or conversation. There are 156 distinct speakers.
They are balanced in gender with age between 19 and 33.
The transcriptions are given with sentence boundary. In addition to lexical words, thediscourse particles are also provided in either Mandarin or English, for example [啊], [喔]in Mandarin and [ah], [oh] in English. The other meta-information when they occur, suchas the appearance of other languages or non-speech events, is also labelled.
Where to get the database?
Visit LDC website: https://catalog.ldc.upenn.edu/LDC2015S04. The database will beavailable on the website from 15 April 2015 onward.
References[1] Dau-Cheng Lyu, Tien Ping Tan, Engsiong Chng, Haizhou Li: SEAME: a
Mandarin-English code-switching speech corpus in south-east asia. INTERSPEECH 2010:
1986-1989[2] Ngoc Thang Vu, Dau-Cheng Lyu, Jochen Weiner, Dominic Telaar, Tim Schlippe, Fabian
Blaicher, Engsiong Chng, Tanja Schultz, Haizhou Li: A first speech recognition system for
Mandarin-English code-switch conversational speech. ICASSP 2012: 4889-4892[3] Dau-Cheng Lyu, Eng-siong Chng, Haizhou Li: Language diarization for code-switch
conversational speech with pronunciation dictionary adaptation. ChinaSIP 2013[4] Dau-Cheng Lyu, Tien-Ping Tan, Eng-Siong Chng, and Haizhou Li: An Analysis of a
Mandarin-English Code-switching Speech Corpus: SEAME. OCOCOSDA 2010.
[5] https://catalog.ldc.upenn.edu/LDC2015S04

P a ge 2 3

AP SIP A Ne ws l et t er

Ap r i l 2 01 5

P a ge 2 4

Ap r i l 2 01 5

AP SIP A Ne ws l et t er

Call for Proposals for APSIPA 2015 Tutorial Sessions
You are cordially invited to submit your proposals for 2015 APSIPA ASC tutorial
sessions. We would like to enrich APSIPA conference with diverse knowledge
through a set of tutorial sessions of 3 hours each. The tutorials are looked at as steppingstones to new researchers to undertake research in a certain field, boost the knowledge ofexperienced researchers, or direct the attention of the research community to an importantfield. The topics could be in any field within, but not limited to, the APSIPA themes:
•
•
•
•
•
•

Biomedical/Biological Signals and Systems
Circuits, Design and Implementation
Information Processing Theory and Applications
Image, Video, and Multimedia
Speech, Language, and Audio
Wireless Communications and Networking

As a reward APSIPA conference organisers dedicated an honorarium of $US1000 paid as alump sum to the presenters of each tutorial session.

Please email your interest with the following information to the APSIPA Tutorial Sessions
Co-Chair Waleed H. Abdulla w.abdulla@auckland.ac.nz
1. Full name(s) of the tutorial presenter (s)2. Affiliation(s) (University, Organisation, Institutes)
3. Contact details (Postal Address, Phone, Fax, E-mail)4. Title and abstract of the tutorial
5. Resume

IMPORTANT DATES

Submission of Proposals for Tutorial Sessions

May 8, 2015

Notification of Proposals Acceptance:

June 8, 2015

Tutorial Session Date

December 16, 2015

Looking forward to receiving your valuable proposals
Tutorial Co-Chairs: Waleed Abdulla, Woon-Seng Gan, Wing-Kuen Ling, Lee Tan
Conference Tutorials webpage: http://www.apsipa2015.org/ then click Tutorials tab

7 th

APSIPAASC 2015
Honorary General Chair
Wan-Chi Siu, Hong Kong Polytechnic University
General Co-Chairs
Kenneth Lam, Hong Kong Polytechnic University
Oscar Au, Hong Kong Univ. of Science & Tech.
Helen Meng, Chinese University of Hong Kong
Technical Program Co-Chairs
Changchun Bao, Beijing Univ. of Technology
Akira Hirabayashi, Ritsumeikan University
Jiwu Huang, Shenzhen University
Gwo Giun Lee, National Cheng Kung University
Daniel Lun, Hong Kong Polytechnic University
Tomoaki Ohtsuki, Keio University
Tomasz M. Rutkowski, University of Tsukuba
Sumei Sun, I2R, A*STAR
Finance Chair
Chris Chan, Hong Kong Polytechnic University
Secretary
Bonnie Law, Hong Kong Polytechnic University
Forum Co-Chairs
Wai-Kuen Cham, Chinese Univ. of Hong Kong
Homer Chen, National Taiwan University
King N. Ngan, Chinese University of Hong Kong
Ming-Ting Sun, University of Washington
Panel Session Co-Chairs
Shing-Chow Chan, University of Hong Kong
Dominic K.C. Ho, University of Missouri
Yo-Sung Ho, Gwangju Inst. of Science & Tech.
Yoshikazu Miyanaga, Hokkaido University
Special Session Co-Chairs
Mrityunjoy Chakraborty, India Inst. of Technology
Yui-Lam Chan, Hong Kong Polytechnic University
Lap-Pui Chau, Nanyang Technological University
Hsueh-Ming Hang, National Chiao-Tung University
Hitoshi Kiya, Tokyo Metropolitan University
Tutorial Co-Chairs
Waleed Abdulla, University of Auckland
Woon-Seng Gan, Nanyang Technological Univ.
Wing-Kuen Ling, Guangdong Univ. of Tech.
Lee Tan, Chinese University of Hong Kong
Registration Co-Chairs
Man-Wai Mak, Hong Kong Polytechnic University
Lai-Man Po, City University of Hong Kong
Publication Chair
Zheru Chi, Hong Kong Polytechnic University
Publicity Co-Chairs
Kiyoharu Aizawa, University of Tokyo
Yui-Lam Chan, Hong Kong Polytechnic Univ.
Hing Cheung So, City University of Hong Kong
Mark Liao, IIS, Academia Sinica
Thomas Fang Zheng, Tsinghua University
Local Arrangement Co-Chairs
Edward Cheung, Hong Kong Polytechnic Univ.
Frank Leung, Hong Kong Polytechnic University
Advisory Committee
Chairs:
Sadaoki Furui, Toyota Technological Instituteat Chicago
Wen Gao, Peking University
C.-C. Jay Kuo, University of Southern California
Haizhou Li, Inst. for Infocomm Research, A*STAR
Ray Liu, University of Maryland
Members:
Thierry Blu, Chinese University of Hong Kong
Shih-Fu Chang, Columbia University
Liang-Gee Chen, National Taiwan University
Li Deng, Microsoft ResearchTakeshi Ikenaga, Waseda University
Kebin Jia, Beijing Univ. of Technology
Anthony Kuh, University of Hawaii
Antonio Ortega, University of Southern California
Soo-Chang Pei, National Taiwan University
Susanto Rahardja, National Univ. of Singapore
Yodchanan Wongsawat, Mahidol University
Chung-Hsien Wu, National Cheng Kung Univ.

apsipa2015.org
CALL FOR PAPERS
APSIPA ASC 2015 will be the seventh annual conference
organized by the Asia-Pacific Signal and Information Processing
Association (APSIPA). Founded in 2009, APSIPA aims to promoteresearch and education in signal processing, information technology
and communications. Annual conferences have previously been heldin Japan (2009), Singapore (2010), China (2011), the USA (2012),
Taiwan (2013), and Cambodia (2014). The field of interest of APSIPA
concerns all aspects of signal and information including processing,recognition, classification, communications, networking, computing,system design, security, implementation, and technology withapplications to scientific, engineering and social areas. Accepted papers
in regular sessions and special sessions will be published in
APSIPA ASC 2015 proceedings which will be submitted for inclusion
into IEEE Xplorer as well as other Abstracting and Indexing (A&I)
databases.
The technical program includes the following tracks, but are not limited to:
1. Biomedical Signal Processing and Systems (BioSiPS)
Biomedical Signal and Information: Theory and Methods, Medical Information
and Telecare Systems, Neural Systems and Applications, Bio-inspired Signal
Processing and System, Biomedical Circuits and Systems.
2. Signal Processing Systems: Design and Implementation (SPS)
Nanoelectronics and Gigascale Systems, VLSI Systems and Applications,Embedded Systems, 3D Video Processing and Coding, High Efficiency Video Coding.
3. Image, Video and Multimedia (IVM)
Image/Video Processing, Coding and Analysis, Image/Video Storage, Retrieval andAuthentication, Computer Vision and Graphics, Multimedia Systems and Applications.
4. Speech, Language and Audio (SLA)
Audio Processing, Speech Information Processing: Recognition, Synthesis,
Understanding and Translation, Natural Language Processing: Language Modeling,
Natural Language Generation/Understanding, Machine Translation.
5. Signal and Information Processing Theory and Methods (SIPTM)
Modern Signal Processing Theory and Method, Detection and Parameter Estimation,
Array Processing and Multi-channels, Signal and Information Processing inApplications.
6. Wireless Communications and Networking (WCN)
Information and Network Security, Wireless Communications
Standards and Emerging Technology, RF and Antennas.

andNetworking,

PAPER SUBMISSION
Prospective authors are invited to submit either full papers, up to 10 pages in length,or short papers up to 4 pages in length, where full papers will be for the single-track
oral presentation and short papers will be mostly for poster presentation.
IMPORTANT DATES
Submission of Proposals for Special Session,
Forum, Panel & Tutorial Sessions:
Submission of Full and Short Papers:
Submission of Papers in Special Sessions:Notification of Papers Acceptance:
Submission of Camera-Ready Papers:
Author Registration Deadline:
Tutorial Session Date:
Summit and Conference Dates:

May 8, 2015
June 8, 2015
June 8, 2015
September 1, 2015
October 1, 2015
October 1, 2015
December 16, 2015
December 17-19, 2015

Organizer/Sponsor: The Hong Kong Polytechnic University
Sponsor: Asia-Pacific Signal and Information Association (APSIPA)
Technical Co-Sponsor: IEEE Signal processing SocietyIEEE Hong Kong Chapter of Signal Processing
Email: secretary@apsipa2015.org

P a ge 2 6

Ap r i l 2 01 5

AP SIP A Ne ws l et t er

APSIPA Who’s Who
President: Haizhou Li, Institute for Infocomm Resear ch, A*STAR, Singa- Homer Chen, National Taiwan Univer sity, Taiwan
Woon-Seng Gan, Nanyang Technological Univer sity, Singapor e

porePresident-Elect:

Wan-Chi Siu, The Hong Kong Polytechnic Univer sity, Wen Gao, Peking Univer sity, China

Hong Kong

Hsueh-Ming Hang, National Chiao-Tung University, Taiwan

Past President: C.-C. Jay Kuo, Univer sity of Souther n Califor nia, USA

Anthony Kuh, Univer sity of Hawaii at Manoa, USA

VP - Conferences: Susanto Rahardja, National Univer sity of Singapor e,
Singapore
VP - Industrial Relations and Development: Haohong Wang, TCL
Research America, USA
VP - Institutional Relations and Education Program:
Yo-Sung Ho, Gwangju Institute of Science and Technology, Korea
VP - Member Relations and Development: Kenneth Lam, The Hong Kong
Polytechnic University, Hong Kong
VP - Publications: Tatsuya Kawahara, Kyoto Univer sity, J apan

K.J. Ray Liu, Univer sity of Mar yland, USA
Tomoaki Ohtsuki, Keio Univer sity, J apan
Ming-Ting Sun, Univer sity of Washington, USA
Headquarters
Address:
Asia Pacific Signal and Information Processing Association,
Centre for Signal Processing,
Department of Electronic and Information Engineering,
The Hong Kong Polytechnic University,
Hung Hom, Kowloon, Hong Kong.

VP - Technical Activities: Oscar C. Au, Hong Kong Univer sity of Science Officers:
and technology, Hong Kong
Director: Wan-Chi Siu, email: enwcsiu@polyu.edu.hk
Members-at-Large:
Waleed H. Abdulla, The Univer sity of Auckland, New ZealandKiyoharu Aizawa, The Univer sity of Tokyo, J apan
Mrityunjoy Chakraborty, Indian Institute of Technology, India
Kosin Chamnongthai, King Mongkut' s University of Technology Thonbur i,Manager: Kin-Man Lam, Kenneth,
email: enkmlam@polyu.edu.hk
Secretary: Ngai-Fong Law, Bonnie,
email: ennflaw@polyu.edu.hk
Treasurer: Yuk-Hee Chan, Chris,email: enyhchan@polyu.edu.hk

ThailandAPSIPA Newsletter Editorial Board Members
Waleed H. Abdulla (Editor -in-Chief), The University of Auckland, New Zealand.
Iman T. Ardekani, Unitec Institute of Technology, New Zealand.
Oscar C. Au, Hong Kong University of Science and technology, Hong Kong
Takeshi Ikenaga, Waseda University, Japan
Yoshinobu Kajikawa, Kansai Univer sity, J apan
Hitoshi Kiya, Tokyo Metr opolitan Univer sity, J apan
Anthony Kuh, University of Hawaii, USA
Kenneth Lam, The Hong Kong Polytechnic Univer sity, Hong Kong
Bonnie Law (Deputy EiC), The Hong Kong Polytechnic Univer sity, Hong Kong
Haizhou Li, Institute for Infocomm Resear ch, A*STAR, Singapor e
Tomoaki Ohtsuki, Keio University, Japan
Woon-Seng Gan, Nanyang Technological Univer sity, Singapor e
Yodchanan Wongsawat, Mahidol University, Thailand
Chung-Hsien Wu, National Cheng Kung University, Taiwan
Thomas Zheng, Tsinghua Univer sity, China

Are you an APSIPA member?
If not, then register online at
http://www.apsipa.org