2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

October 22-25, 2023, New Paltz, NY

IMPROVED PORTABLE MULTIPLE SOUND SPOT SYNTHESIS SYSTEM
WITH A BAFFLED CIRCULAR ARRAY OF 16 LOUDSPEAKERS
Takuma Okamoto1∗ , Katsushi Ueno2 , Tsukasa Okabe2 , Kentaro Tani2 , Yasuhiko Yoshikata2 ,Miyuki Sudo1 , Manae Kuwahara1 , Keita Hikita1
1

National Institute of Information and Communications Technology, Japan, 2 Kitanihon Onkyo, Japan
134.4 mm<latexit sha1_base64="1M00/pveEBR7jSORblLG9avKnK4=">AAACa3ichVHLLgRBFD3T3uM12AgWHZMRq86d1pnBahIbS69hEkS6W6GjX+numYSJH7C0sWBDIiI+w8YPWPgEESsSGwt3elrEArdSVbdO3XPr1L2Gb1thRPSYklpa29o7OrvS3T29ff2ZgcHV0KsGpiibnu0FFUMPhW25ohxZkS0qfiB0x7DFmrE/17hfq4kgtDx3JTrwxaaj77rWjmXqEUOV/JSmaLLjbGWypMwUiqSpMikUGzuaWtCKqpxPkCwSW/Ay19jANjyYqMKBgIuIfRs6Qh7ryIPgM7aJOmMBe1Z8L3CENHOrHCU4Qmd0n9ddPq0nqMvnRs4wZpv8is0zYKaMHD3QDb3SPd3SE338mqse52hoOeDdaHKFv9V/PLz8/i/L4T3C3jfrT80RdjAda7VYux8jjV+YTX7t8PR1eXYpV5+gS3pm/Rf0SHf8A7f2Zl4tiqUzpLkBX1WWf3dWVSVfULRFNVsqJa3oxCjGMcn1LqKEeSygHNf5BGc4T71IQ9KINNYMlVIJZwg/TMp9AmI1i/Q=</latexit>

ABSTRACT
To make multiple sound spot synthesis technology widely available,
we have previously implemented a portable demonstration system
based on mode-matching method using a compact circular array of
16 loudspeakers. However, the previous system has two criticalproblems for practical applications. 1) The maximum output soundpressure level is only 72 dB, which is too small for demonstration
in a large conference venue. 2) The output sound quality is nothigh because low frequency components below 500 Hz cannot be
produced. Additionally, 3) a large space is required because 4) participants must basically walk around the array to listen to the synthesized sound field. To improve the previous system, we developed
an improved system using a baffled circular array of 16 loudspeakers combined with an electric turntable. The improved system can1) produce a sound pressure level of 80 dB, which is sufficient for
demonstration in a large conference venue, and 2) drastically improve the synthesis sound quality with low frequency components
above 200 Hz. Additionally, it can be demonstrated 3) in a small
space because 4) participants do not have to walk around the array by the electric turntable. It can realize eight-language soundspot synthesis for eight directions. The improved system including
the turntable can be carried out with a single suitcase. In WASPAA 2023 Demonstrations, we demonstrate both four- and eightlanguage sound spot synthesis for four and eight directions using
the improved system.
Index Terms— Circular loudspeaker array, localized soundspot synthesis, multiple sound spot synthesis, portable demo system, text-to-speech
1. INTRODUCTION
Compared to parametric arrays of ultrasonic loudspeakers [1], localized sound spot synthesis [2–13], which can realize audible andinaudible areas using multiple loudspeakers, is superior in termsof the synthesis sound quality and produced sound pressure level.
Additionally, multiple sound spot synthesis [4–6, 8, 14], which cansimultaneously present different sounds in different areas by superposing multiple localized sound spots, is also an important soundpresentation technology for multilingual communication, museums,
and other speech and audio applications.
Typical sound field synthesis methods, such as wave field synthesis [15–17], spectral division method [18, 19], and higher-order
Ambisonics (HOA) [20–22], introduce planar, linear and surrounding arrays of loudspeakers. However, these arrays are difficult to∗ This study was partly supporred by JSPS KAKENHI Grant Number
JP23K11177, and JST A-STEP Tryout Grant Number JPMJTM22D8.

25 mm<latexit sha1_base64="IdPmBurCXRyDw5Ey+qyq0M24MtA=">AAACaHichVHLLgRBFD3T3uM1WCA2HRNiNbnTGTNYTWJj6TVIEOluZVT0K909kzDxAzaWiBWJiPgMGz9g4ROwJLGxcKenRSxwK1X31ql77j1VZXiWDEKix4TS1NzS2tbekezs6u7pTfX1rwRuxTdFyXQt118z9EBY0hGlUIaWWPN8oduGJVaNvdn6+WpV+IF0neVw3xObtl525I409ZChkjap2vZWKk2Z6XyBcppKGYqMg5yWzxU0NRsjacQ276ausYFtuDBRgQ0BByHHFnQEPNaRBcFjbBM1xnyOZHQucIgkcyucJThDZ3SP1zLv1mPU4X29ZhCxTe5i8fSZqWKMHuiGXumebumJPn6tVYtq1LXsszcaXOFt9R4NLb3/y7LZh9j9Zv2pOcQOpiKtkrV7EVK/hdngVw9OXpdmFsdq43RJL6z/gh7pjm/gVN/MqwWxeI4kf8DXK6u/BytaJpvP5Ba0dLEYf0U7RjCKCX7vAoqYwzxK3FfiGKc4SzwrKWVQGW6kKomYM4Afpox+Ashfi0M=</latexit>

4m<latexit sha1_base64="oaYDscO8xp4TSRT2GqoG0KvHrjU=">AAACZnichVFNLwNRFD0d3/VVREhsJhpi1dxOJi1WwsaSVpEgMjOempivzEyb0PADJLYsrEhExM+w8Qcs+g+IJYmNhdvpiFjgvrx37zvvnnvPe0/3LDMIieoJqaW1rb2jsyvZ3dPb158aGFwN3IpviJLhWq6/rmuBsExHlEIztMS65wvN1i2xpu8vNM7XqsIPTNdZCQ88sWVrZcfcNQ0tZKioHtvbqTRlZnJ5UhWZMhQZB6qSU/OKnI2RNGJbclM32MQOXBiowIaAg5BjCxoCHhvIguAxtoUaYz5HZnQucIQkcyucJThDY3Sf1zLvNmLU4X2jZhCxDe5i8fSZKWOCHumWXumB7uiZPn6tVYtqNLQcsNebXOFt95+MFt//ZdnsQ+x9s/7UHGIX05FWk7V7EdK4hdHkVw/PX4uzhYnaJF3RC+u/pDrd8w2c6ptxvSwKF0jyB3y9svx7sKpksrmMuqyk5+bjr+jEGMYxxe+dxxwWsYQS9y3jFGc4TzxJfdKwNNJMlRIxZwg/TJI/Afjyiu8=</latexit>

4m<latexit sha1_base64="oaYDscO8xp4TSRT2GqoG0KvHrjU=">AAACZnichVFNLwNRFD0d3/VVREhsJhpi1dxOJi1WwsaSVpEgMjOempivzEyb0PADJLYsrEhExM+w8Qcs+g+IJYmNhdvpiFjgvrx37zvvnnvPe0/3LDMIieoJqaW1rb2jsyvZ3dPb158aGFwN3IpviJLhWq6/rmuBsExHlEIztMS65wvN1i2xpu8vNM7XqsIPTNdZCQ88sWVrZcfcNQ0tZKioHtvbqTRlZnJ5UhWZMhQZB6qSU/OKnI2RNGJbclM32MQOXBiowIaAg5BjCxoCHhvIguAxtoUaYz5HZnQucIQkcyucJThDY3Sf1zLvNmLU4X2jZhCxDe5i8fSZKWOCHumWXumB7uiZPn6tVYtqNLQcsNebXOFt95+MFt//ZdnsQ+x9s/7UHGIX05FWk7V7EdK4hdHkVw/PX4uzhYnaJF3RC+u/pDrd8w2c6ptxvSwKF0jyB3y9svx7sKpksrmMuqyk5+bjr+jEGMYxxe+dxxwWsYQS9y3jFGc4TzxJfdKwNNJMlRIxZwg/TJI/Afjyiu8=</latexit>

Figure 1: Previous demo system.

transport easily compared with applications for speech communication technologies, such as automatic speech recognition (ASR) [23]and text-to-speech (TTS) [24], that can be easily demonstrated elsewhere using a smartphone.
2. PREVIOUS SYSTEM
2.1. Overview of previous system
To make sound field control technology, especially multiple soundspot synthesis technology widely available, we have implemented
a portable multiple sound spot synthesis system based on modematching method [5] using a compact circular array of 16 loudspeakers as small as possible (Fig. 1) [25, 26].12 Each loudspeaker driver is 25 mm, and the diameter of the circular arrayis only 134.4 mm. The spatial Nyquist frequency is then about
6.5 kHz. The implemented system, constructed from the compact
loudspeaker array, an amplifier for 16 loudspeakers including D/A,
a loudspeaker stand, a laptop, a tablet and cables, can be carried out
with a single suitcase. The demo system is implemented with PureData (Pd) [27] and controlled by the tablet via open sound control.
The synthesized sound field can be interactively rotated by the tablet
in real-time with HOA-based panning implemented in Pd. The
previous demo system realized four-language sound spot synthesis for four directions combined with multilingual neural TTS [28](Fig. 3(a)), where neural TTS models for English and Japanese aretrained using Hi-Fi-CAPTAIN corpus [29].
2.2. Problems of previous system
Although the previous array is compact, it has two critical problems
for practical applications because the size of the array is too small.
1) The maximum output sound pressure level is 70 dB, which is too
1 https://ast-astrec.nict.go.jp/en/MultipleSoundSpotSynthesis/
2 https://youtu.be/In8AfVcoTC4

2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

October 22-25, 2023, New Paltz, NY

13.44 cm

こんにちは

178.6 mm

Japanese

English<latexit sha1_base64="3mnCzfZuRum8KzhRNQJdPCKAsRk=">AAACa3ichVG7SgNBFD1ZXzE+ErURtVgMEavlZglJtArYWPqKBlRkdx11cV/sbgIa/AFLGwttFETEz7DxByz8BBErBRsLbzYrYqHeYWbunLnnzpl7dc8yg5DoMSF1dHZ19yR7U339A4PpzNDwauDWfUNUDddy/ZquBcIyHVENzdASNc8Xmq1bYk3fn2vdrzWEH5iusxIeeGLT1nYdc8c0tJChWr5UVoqybW9lsqTMFEtUUGVSKDJ2CmqxUFLlfIxkEduCm7nGBrbhwkAdNgQchOxb0BDwWEceBI+xTTQZ89kzo3uBI6SYW+cowREao/u87vJpPUYdPrdyBhHb4Fcsnj4zZeTogW7ole7plp7o49dczShHS8sB73qbK7yt9PHo8vu/LJv3EHvfrD81h9hBOdJqsnYvQlq/MNr8xuHp6/LsUq45RZf0zPov6JHu+AdO4824WhRLZ0hxA76qLP/urKpKvqgUFtVspRK3IolxTGKa611CBfNYQDWq8wnOcJ54kUakMWmiHSolYs4IfpiU+wR2Z4v+</latexit>

Multiple<latexit sha1_base64="PCfbaAEOApi6F5ICC9BFn+ejKts=">AAACa3ichVG7SgNBFD1ZXzG+ojaiFsEQsQoTEQ1Wgo2NYKIxAZWwu446ONlddmcDGvwBSxsLbRRExM+w8Qcs/AQRKwUbC+9uFkSDepfZOXPmnjtn7hiOFJ5i7DGmtbV3dHbFuxM9vX39A8nBoXXP9l2Tl0xb2m7F0D0uhcVLSijJK47L9ZohednYXwz2y3XuesK21tSBw7dq+q4ldoSpK6Iqy75UwpG8mkyzLAsj1QpyEUgjihU7eY1NbMOGCR81cFhQhCV0ePRtIAcGh7gtNIhzCYlwn+MICdL6lMUpQyd2n/67tNqIWIvWQU0vVJt0iqThkjKFDHtgN+yV3bNb9sQ+fq3VCGsEXg5oNppa7lQHjkdW3/9V1WhW2PtS/elZYQf50Ksg707IBLcwm/r64enr6nwx05hkl+yZ/F+wR3ZHN7Dqb+ZVgRfPkKAHyP1sdytYn87mZrMzhen0Qj56ijjGMIEp6vccFrCEFZTCPp/gDOexF21YG9XGm6laLNIM41tomU+vF40P</latexit>

Speechrecognition<latexit sha1_base64="ORyMU4FVLI65qiqbOCljJt63Yu8=">AAACaXichVFNLwNRFD0dX1UfLTYNm0ZTsZrcNmXUqomNZatakhKZGQ/DdGYyM21SjT9gZSdYkYiIn2HjD1j4CWJJYmPhdloRC9yX9+59591z73nvaY5peD7RU0jq6e3rHwgPRoaGR0ajsbHximfXXV2Uddu03XVN9YRpWKLsG74p1h1XqDXNFGvawVL7fK0hXM+wrVW/6YjNmrprGTuGrvoMVUqOEPreVixJcjqjKEQJknO5nKLMdQLKKom0TIEl0bWCHbvBBrZhQ0cdNQhY8Dk2ocLjUUUaBIexTbQYczkygnOBI0SYW+cswRkqowe87vKu2kUt3rdregFb5y4mT5eZCaTokW7plR7ojp7p49daraBGW0uTvdbhCmcrehwvvf/LqrH3sffN+lOzjx0sBFoN1u4ESPsWeoffODx9LS2upFozdEUvrP+Snuieb2A13vTroli5QIQ/4OuVE78HlYycnpezxUwyv9D9ijCmMI1Zfm8FeSyjgDL33ccJznAeepHGpLg02UmVQl3OBH6YlPwEOSGMbA==</latexit>

Hello

Hello

Machine<latexit sha1_base64="FNVkEfkP6n/3dyWXpNJmTMlVe9I=">AAACanichVG7SgNBFD1Z3/EVtVFsxKhYhdkgKlaCjY2QhzGBGGR3nSSDm91ldxKIwR+wsxJMpSAifoaNP2CRTxDtFGwsvNksiAb1DjNz5sw9d87M6I4pPMlYK6T09Pb1DwwOhYdHRsfGIxOTe55ddQ2eMWzTdnO65nFTWDwjhTR5znG5VtFNntWPttr72Rp3PWFbu7Lu8EJFK1miKAxNEpXd0YwyKQ8iURZjfsx1AzUAUQSRsCM32MchbBioogIOC5KwCQ0etTxUMDjEFdAgziUk/H2OE4RJW6UsThkasUc0lmiVD1iL1u2anq826BSTukvKOSyyR3bLXtkDu2NP7OPXWg2/RttLnWa9o+XOwfjpdPr9X1WFZonyl+pPzxJFrPteBXl3fKZ9C6Ojrx2fv6Y3UouNJXbFnsn/JWuxe7qBVXszrpM81USYPkD9+dzdYC8eU1djK8l4dHM9+IpBzGIey/Tea9jENhLI+O7OcIFm6EWZVGaU2U6qEgo0U/gWysInWAeMbg==</latexit>你好

Speech<latexit sha1_base64="n/sFK17WD+yAsr2/nWlICl6LV9U=">AAACanichVFNS8NAEH2N3/Wr6kXxEqwVT2Fa1FhPghePam0raJEkbttgmoQkLWjxD3jzJOhJQUT8GV78Ax78CaK3Cl48OE0r4qE6y+7Mvp03+3ZWdy3TD4ieI1JXd09vX/9AdHBoeGQ0Njae852qZ4is4ViOt6NrvrBMW2QDM7DEjusJraJbIq8frjXP8zXh+aZjbwdHrihUtJJtFk1DCxjKZ1whjLK8H4uTkkypKpFMSjqdVtXFVkALqpxUKLQ42rbhxG6xhwM4MFBFBQI2Ao4taPB57CIJgstYAXXGPI7M8FzgBFHmVjlLcIbG6CGvJd7ttlGb982afsg2+BaLp8dMGQl6ojtq0CPd0wt9dqxVD2s0tRyx11tc4e6Pnk5mPv5lVdgHKP+w/tQcoIjlUKvJ2t0Qab7CaPFrx+eNzMpWoj5H1/TK+q/omR74BXbt3bjZFFuXiPIHfHdZ7hzkUkpySVnYTMVXl9tf0Y9pzGCe+61iFevYQDZUd4YLXEbepHFpSppupUqRNmcCv0ya/QKg44yW</latexit><latexit sha1_base64="ppRPBb1yb00s29ydE3zAOFRn8sI=">AAACbXichVHLSgMxFD0d3/VVFUFQpFh8rEoqouKq4MZltfaBD2RmTOvgTGaYSQta/AHXggtRUBARP8ONP+CinyAuXFRw48I70wFRUW9IcnJyz81Jojmm4UnG6hGlpbWtvaOzK9rd09vXHxsYzHt2xdV5TrdN2y1qqsdNQ/CcNKTJi47LVUszeUHbX/b3C1XueoYt1uWBw7cttSyMkqGrkqgNqiN2455jy51YgiVZEPGfIBWCBMLI2LEbbGEXNnRUYIFDQBI2ocKjtokUGBzitlEjziVkBPscR4iStkJZnDJUYvdpLNNqM2QFrf2aXqDW6RSTukvKOCbZI7tlDfbA7tgTe/+1Vi2o4Xs5oFlrarmz0388kn37V2XRLLH3qfrTs0QJi4FXg7w7AePfQm/qq4enjezS2mRtil2xZ/J/yersnm4gqq/69SpfO0OUPiD1/bl/gvxsMjWfnFudTaQXw6/oxCgmMEPvvYA0VpBBjs4VOME5LiIvyrAypow3U5VIqBnCl1CmPwCHLY3m</latexit><latexit sha1_base64="UTwI+1KZzIuf5gzkll0k7hA4DAM=">AAACbHichVHLSiNBFD1pH6Nx1PhYCDIQJiiumpvg2MaV4Malj4k6OCLdbUUL+0VXJRCDP+DWhQt1oSAifoYbf2AWfsIguFFw42JuOpHBhXqLqnvr1D33nqpyIk8qTXSXMtraOzq/dHWne7729vVnBgZXVFiJXVFyQy+M1xxbCU8GoqSl9sRaFAvbdzyx6uzONc5XqyJWMgx+6lokNnx7O5Bl6dqaoV+qFugdoaTazOTIzBcsiyhLZrFYtKwfzYAmrWzepMRyaNlCmLnEb2whhIsKfAgE0Bx7sKF4rCMPQsTYBuqMxRzJ5FxgH2nmVjhLcIbN6C6v27xbb6EB7xs1VcJ2uYvHM2ZmFmP0h67okW7pmv7Sy7u16kmNhpYae6fJFdFm/8HI8vOnLJ+9xs5/1oeaNcqYTrRK1h4lSOMWbpNf3Tt6XJ5ZGquP0znds/4zuqMbvkFQfXIvFsXSMdL8Aa+vnH0/WCmY+SlzcrGQm51ufUUXRvEdE/zeFmYxjwWUuK+PQ5zgNPVgDBujxrdmqpFqcYbwxozxf9e0jhw=</latexit>

translation<latexit sha1_base64="IbEEyCqW6i4gGG8Vq+Yzsiwo5Zo=">AAACbnichVHLSsNAFD2N7/qqCiKIKBbFVZkUUXFVcOOyD6tCFUniqINpEpJpoRZ/wL24EBQFEfEz3PgDLvoJ4kZQcOPCm2lAVNQbJnPmzD13ztwxPVsEkrFGTGtpbWvv6OyKd/f09vUnBgZXA7fiW7xoubbrr5tGwG3h8KIU0ubrns+NsmnzNXN/Kdxfq3I/EK6zImse3ywbu47YEZYhiSpJ33ACW+GtRJKlmIqJn0CPQBJRZN3ENTawDRcWKiiDw4EkbMNAQF8JOhg84jZRJ84nJNQ+xyHipK1QFqcMg9h9+u/SqhSxDq3DmoFSW3SKTcMn5QSm2AO7YS/snt2yR/b+a626qhF6qdFsNrXc2+o/Gim8/asq0yyx96n607PEDhaUV0HePcWEt7Ca+urByUthMT9Vn2aX7In8X7AGu6MbONVX6yrH86eI0wPo39v9E6ymU/pcajaXTmYWoqfoxCgmMUP9nkcGy8iiqDp2jDOcx561YW1MG2+marFIM4Qvoc18APHLjpA=</latexit>안녕하세요Circulararraysound spot

synthesis<latexit sha1_base64="KXaqadr9nNXa/84E913yRm9v4eU=">AAACbnichVHLSsNQED2N7/poVRBBxGJRXIVpqca6Ety4bNWqUEWSeFtD0yQkaUGLP+BeXAiKgoj4GW78ARd+grgRFNy4cJJWxIU6Iblnzp0zOXeu5piG5xM9RqS29o7Oru6eaG9f/0AsPji07tk1VxcF3TZtd1NTPWEalij4hm+KTccValUzxYZWWQr2N+rC9QzbWvP3HbFdVcuWUTJ01Weq6Ard5jzAO/Ekyam0ohAlSM5ms4oy2wSUURIpmcJIohU5O36NLezCho4aqhCw4DM2ocLjp4gUCA5z22gw5zIywn2BQ0RZW+MqwRUqsxX+ljkrtliL86CnF6p1/ovJr8vKBKbogW7ole7plp7o49dejbBH4GWfV62pFc5O7Gh09f1fVZVXH3vfqj89+yhhPvRqsHcnZIJT6E19/eDkdXVhZaoxTZf0zP4v6JHu+ARW/U2/youVU0T5Ar6mnPgdrKfl1JycyaeTi/Otq+jGGCYxw/NWsIhl5FAIJ3aMM5xHXqQRaVyaaJZKkZZmGD9CmvkEl8qO5w==</latexit>

synthesis<latexit sha1_base64="SgORWuu0vOXZv+TziFw4UHWM2Ik=">AAACbHichVHLSgMxFD0dX7W+6mMhiCAWxVVJRbS4Krhx2VbrgyplZowanBeTtFCLP+DWhQt1oSAifoYbf8BFP0EENxXcuPB2OiAq6g1JTk7uuTlJDM8SUjFWj2ht7R2dXdHuWE9vX/9AfHBoTbpl3+QF07Vcf8PQJbeEwwtKKItveD7XbcPi68bBUnN/vcJ9KVxnVVU9vm3re47YFaauiNqUVUftcylkKZ5gSRbExE+QCkECYWTd+A22sAMXJsqwweFAEbagQ1IrIgUGj7ht1IjzCYlgn+MIMdKWKYtThk7sAY17tCqGrEPrZk0ZqE06xaLuk3ICU+yR3bIGe2B37Im9/1qrFtRoeqnSbLS03CsNHI+uvP2rsmlW2P9U/elZYRfpwKsg717ANG9htvSVw9PGymJ+qjbNrtgz+b9kdXZPN3Aqr+Z1jufPEKMPSH1/7p9gbTaZmk/O5WYTmXT4FVGMYRIz9N4LyGAZWRToXBsnOMdF5EUb0ca08VaqFgk1w/gS2vQHFTSNtw==</latexit>..
.
<latexit sha1_base64="gh+W4rzsvfaL/oqzOUUfo9erAUk=">AAACaXichVHLSsNAFD2N7/qquim6KQbFVZmIaHEluHHZh20FFUniqNG8SKaBWvwBV+5EXSmIiJ/hxh9w0U+QLiu4ceFNGhAV9Q4zc+bMPXfOzGiuafiCsWZC6uru6e3rH0gODg2PjKbGxiu+U/N0XtYd0/E2NNXnpmHzsjCEyTdcj6uWZvKqdrQa7lcD7vmGY6+Lusu3LXXfNvYMXRVEVbaCXUf4OymZZVkUmZ9AiYGMOPJO6g5b2IUDHTVY4LAhCJtQ4VPbhAIGl7htNIjzCBnRPscJkqStURanDJXYIxr3abUZszatw5p+pNbpFJO6R8oMZtgzu2dt9sQe2At7/7VWI6oReqnTrHW03N0ZPU2X3v5VWTQLHHyq/vQssIdc5NUg727EhLfQO/rg+LxdWi7ONGbZDWuR/2vWZI90Azt41W8LvHiFJH2A8v25f4LKfFZZzC4UFuSVXPwV/ZjCNObovZewgjXkUaZzD3GGC1wmWtKYlJYmO6lSItZM4EtI8gffmYw9</latexit>

ChineseKorean<latexit sha1_base64="nscgLCchdVwgfkjWHH9Vg91nLNY=">AAACqHichVFBTxQxGH0MILiirHox8TJhA0ETN11C1Hgi4aK3ZXFZDJJNZ/hgGzqdybSzZtn4B/wDHjxpYgzxZ3jxxomE/QmGIyZePPjt7CREifA1bb++vvf1tQ0SrawTYjDmjU9MXpuavl66MXPz1mz59p0NG2dpSM0w1nG6GUhLWhlqOuU0bSYpySjQ1Ar2V4f7rS6lVsXmpesltB3JPaN2VSgdQ+3y6qJ84DdI6kdOReTbhCjs+C6Vxuqc4r9RruNHmXYq0UyIM7PDtNj5tmdch6yy7XJFVEUe/sWkViQVFFGPy1/wGjuIESJDBIKB41xDwnLbQg0CCWPb6DOWcqbyfcJblFibMYuYIRnd53GPV1sFang9rGlzdcinaO4pK33Mi2NxKM7Ed/FV/BC//1urn9cYeunxHIy0lLRn391b/3WlKuLZoXOuutSzwy6e5l4Ve09yZHiLcKTvHrw/W3/WmO8viE/ilP1/FAPxjW9guj/Dz2vU+IASf0Dt3+e+mGwsVWuPq8trS5WVF8VXTOM+5rDI7/0EK3iOOpp87iGOcIKB99Crey3v1YjqjRWau/grvOAPt7Gkjg==</latexit>(b)(a) Real-time speech translation with multiple(a)sound spot synthesis

32 mm<latexit sha1_base64="kVry3qE5FmxkRBsyJQPLK2TpxOU=">AAACaHichVHLLgRBFD3T3uM1WCA2HRNiNbnTJjNYTWJj6TVIEOluZVT0K909kzDxAzaWiBWJiPgMGz9g4ROwJLGxcKenRSxwK1X31ql77j1VZXiWDEKix4TS1NzS2tbekezs6u7pTfX1rwRuxTdFyXQt118z9EBY0hGlUIaWWPN8oduGJVaNvdn6+WpV+IF0neVw3xObtl525I409ZCh0qSm2vZWKk2Z6XyBcppKGYqMg5yWzxU0NRsjacQ276ausYFtuDBRgQ0BByHHFnQEPNaRBcFjbBM1xnyOZHQucIgkcyucJThDZ3SP1zLv1mPU4X29ZhCxTe5i8fSZqWKMHuiGXumebumJPn6tVYtq1LXsszcaXOFt9R4NLb3/y7LZh9j9Zv2pOcQOpiKtkrV7EVK/hdngVw9OXpdmFsdq43RJL6z/gh7pjm/gVN/MqwWxeI4kf8DXK6u/BytaJpvP5Ba0dLEYf0U7RjCKCX7vAoqYwzxK3FfiGKc4SzwrKWVQGW6kKomYM4Afpox+AsRai0E=</latexit>

Improved multiple sound spot synthesis system<latexit sha1_base64="A2AQJVAGUcSUIxJiWqClzyV1UP4=">AAACknichVG7bhNBFD1ZXsE8Yh4FEs0KK4gG63pl2UmqRKEAiSIPnEQKkbW7vklGmd0d7YwtGYsfoE1BQQUSQogvoCUNP0CRT0CUQaKh4Hq9CFEE7mhmzpy5586ZmchoZR3R8ZR35uy58xemL1YuXb5ydaZ67fqGzfp5zJ0401m+FYWWtUq545TTvGVyDpNI82Z0sDze3xxwblWWPnFDwztJuJeqXRWHTqhu9f6jxOTZgHt+0tdOGc2+1E57vjWZ8+0wdftslRVkHSfdao3q8602NQOf6lSEgGbQarYDv1EyNZSxklXf4Sl6yBCjjwSMFE6wRggrbRsNEIxwOxgJlwtSxT7jOSqi7UsWS0Yo7IGMe7LaLtlU1uOatlDHcoqWnovSxyx9ofd0Qp/pA32ln6fWGhU1xl6GMkcTLZvuzItb6z/+q0pkdtj/o/qnZ4ddzBVelXg3BTO+RTzRD569PFlfWJsd3aU39E38v6ZjOpIbpIPv8dtVXnuFinzA71f2TwcbQb3RqjdXg9rig/IrpnEbd3BP3ruNRTzECjpy7iE+4hOOvJvegrfkLU9SvalScwN/hff4F24PnHg=</latexit>

Englishnativespeaker<latexit sha1_base64="lfQ9J/ioUQBNg9bgHxPxyOU8lUM=">AAACanichVFNS8NAEH2N3/Wr6kXxUqwVT2Fqq2k9FUTwqNZaQUWSuK3BNAlJWtDiH/DmSdCTgoj4M7z4Bzz4E0RvFbx4cJpWxEN1lt2ZfTtv5u2u5piG5xM9h6SOzq7unt6+cP/A4NBwZGR007Mrri7yum3a7pamesI0LJH3Dd8UW44r1LJmioJ2uNQ4L1SF6xm2teEfOWK3rJYso2joqs9QYdkqcZODvUiMZFKU5HwqSvJ8RlGUdBBklHQympApsBhatmpHbrGDfdjQUUEZAhZ8jk2o8HhsIwGCw9guaoy5HBnBucAJwsytcJbgDJXRQ15LvNtuoRbvGzW9gK1zF5Ony8wo4vREd1SnR7qnF/psW6sW1GhoOWKvNbnC2Rs+Hc99/Msqs/dx8MP6U7OPItKBVoO1OwHSuIXe5FePz+u5xfV4bYau6ZX1X9EzPfANrOq7frMm1i8R5g/4fuVo+2BzTk4syKm1uVg23fqKXkxiCrP83gqyWMEq8oG6M1zgMvQmjUoT0mQzVQq1OGP4ZdL0F1Y6jPE=</latexit>

Chinesenativespeaker<latexit sha1_base64="1Wl4BNMI2TaQ0/cEPliwIFL0gos=">AAACanichVE9S8NQFD2NX7V+1booLsVacSo3RWrrVHBxbNVaQUtJ4tMG0yQkaaEW/4Cbk6CTgoj4M1z8Aw7+BNFNwcXBm7QiDtUbXt55595z33nvqbahux7RU0jq6x8YHAoPR0ZGx8YnopOxLddqOJooaZZhOduq4gpDN0XJ0z1DbNuOUOqqIcrq4aqfLzeF4+qWuem1bFGpKwemvq9risdUebXGOldUowlKEUcmE/eBnCWZQS6XTadzcTlIESXQjYIVvcEu9mBBQwN1CJjwGBtQ4PK3AxkEm7kK2sw5jPQgL3CMCGsbXCW4QmH2kP8HvNrpsiav/Z5uoNZ4F4OHw8o4kvRIt/RGD3RHz/TZs1c76OF7afGsdrTCrk6cTG98/Kuq8+yh9qP607OHfWQDrzp7twPGP4XW0TePzt42VtaT7QW6ohf2f0lPdM8nMJvv2nVRrF8gwg/wfcvx3mArnZIzqaViOpHPdp8ijFnMYZHvexl5rKGAUuDuFOe4CL1KMWlGmu2USqGuZgq/Qpr/AuDmjLU=</latexit>

2m<latexit sha1_base64="L+dKGHPg6RNrN5Yd9Y/qY0CTaF0=">AAACZnichVFNLwNRFD0d30UVERKbiYZYNbeTpsVK2FiiWhJEZsZTE/OVmWkTGn6AxJaFFYmI+Bk2/oCFf0AsK7GxcDsdEYtyX9679513z73nvae5puEHRM8xqa29o7Oruyfe29efGEgODpV8p+Lpoqg7puNtaKovTMMWxcAITLHhekK1NFOsaweLjfP1qvB8w7HXgkNXbFtq2Tb2DF0NGCooJ9ZOMkXp2VyesopMaQqNg6ySy+YVORMhKUS27CRvsYVdONBRgQUBGwHHJlT4PDaRAcFlbBs1xjyOjPBc4Bhx5lY4S3CGyugBr2XebUaozftGTT9k69zF5OkxU8YkPdEd1emR7umVPlvWqoU1GloO2WtNrnB3Bk7HCh//siz2AfZ/WH9qDrCHmVCrwdrdEGncQm/yq0cX9cLc6mRtiq7pjfVf0TM98A3s6rt+syJWLxHnD/h+Zbl1UFLSmVw6u6Kk5heir+jGOCYwze+dxzyWsIwi9y3jDOe4iL1ICWlEGm2mSrGIM4xfJslf9O6K7Q==</latexit><latexit sha1_base64="npo1j+sBCKoBL/3AuxTkcndPY1k=">AAACaXichVE9SwNBEH05v2L8SNQmaBMMilWYCxITq4CNpSYmCirh7tzE08vdcXcJxOAfsLITtVIQEX+GjX/Awp8glgo2Fs5dImIRnWV33r6ZNzu7q9qG7npEzyGpr39gcCg8HBkZHRuPxiYmy67VcDRR0izDcrZUxRWGboqSp3uG2LIdodRVQ2yqhyt+fLMpHFe3zA2vZYvdulIz9aquKR5TZZNdU1RiSUoRWyaT8IGcJZlBLpdNp3MJOQgRJdG1NSt2ix3swYKGBuoQMOExNqDA5bENGQSbuV20mXMY6UFc4BgR1jY4S3CGwuwhrzXebXdZk/d+TTdQa3yKwdNhZQJz9ER39EaPdE8v9NmzVjuo4ffSYq92tMKuRE/ixY9/VXX2HvZ/VH/27KGKbNCrzr3bAePfQuvom0dnb8Xlwlx7nq7plfu/omd64BuYzXftZl0ULhHhD/h+5URvUE6n5ExqcT2dzGe7XxHGDGaxwO+9hDxWsYYSn3uAU5zjIvQqTUhxabqTKoW6min8Min5BUoFjHM=</latexit><latexit sha1_base64="NeHbcsQWkeA09iVYwbd19mvGULw=">AAACaXichVE9SwNBEH05v+NXok3QJhgUq2Pi10WrgI1loiYRNIS7c42nl7vj7hKIwT9gZSdqpSAi/gwb/4CFP0EsI9hYOLlExEKdZXdm386bfTurOabh+UTPIamru6e3r38gPDg0PDIaiY7lPbvq6iKn26btbmmqJ0zDEjnf8E2x5bhCrWimKGiHq63zQk24nmFbm37dEcWKWraMPUNXfYbyFruaKEUSJJOizC8uxEleXFYUJRUEy0pqPp6UKbAEOpaxI7fYwS5s6KiiAgELPscmVHg8tpEEwWGsiAZjLkdGcC5wjDBzq5wlOENl9JDXMu+2O6jF+1ZNL2DrfIvJ02VmHNP0RHfUpEe6pxf6+LVWI6jR0lJnr7W5wimNnsQ23v9lVdj72P9m/anZxx5SgVaDtTsB0nqF3ubXjs6aGyvr040ZuqZX1n9Fz/TAL7Bqb/pNVqxfIswf8NXl+O9Bfk5OLskL2blEOtX5in5MYgqz3G8Faawhgxzfe4BTnOMi9CpFpZg00U6VQh3OOH6YlPgEqSaMpA==</latexit><latexit sha1_base64="7HwupCdMwDSqHYSxA/GK9gpAs9g=">AAACanichVHLSsNAFD2Nr1pftW4UN8VacVVuitTWVcGNS1+1hVpKEkcNTZOQpAUt/oA7V4JdKYiIn+HGH3DhJ4juFNy48CatiIvqDZM5c+49d87MqLahux7RU0jq6x8YHAoPR0ZGx8YnopOxHddqOJooaJZhOSVVcYWhm6Lg6Z4hSrYjlLpqiKJaW/XzxaZwXN0yt70jW1TqyoGp7+ua4jFVdG2h1IRTjSYoRRyZTNwHcpZkBrlcNp3OxeUgRZRAN9at6A12sQcLGhqoQ8CEx9iAApe/MmQQbOYqaDHnMNKDvMAJIqxtcJXgCoXZGv8PeFXusiav/Z5uoNZ4F4OHw8o4kvRIt/RGD3RHz/TZs1cr6OF7OeJZ7WiFXZ04nd76+FdV59nD4Y/qT88e9pENvOrs3Q4Y/xRaR988Pn/bWtlMthboil7Y/yU90T2fwGy+a9cbYrONCD/A9y3He4OddErOpJY20ol8tvsUYcxiDot838vIYw3rKATuznCBduhVikkz0mynVAp1NVP4FdL8FzoEjOE=</latexit>

2m<latexit sha1_base64="uKBNsLwZlh3K2rVKVb0bqxyQUHg=">AAACanichVE9SwNBEH05v+NXjI1iE4wRq2NiEi+xEmwsNZoP0CB356pHLnfH3SWgwT9gZyVopSAi/gwb/4CFP0G0U7CxcHKJiIU6y+7Mvp03+3ZWc0zD84keQ1JXd09vX/9AeHBoeGQ0MhYtenbd1UVBt03bLWuqJ0zDEgXf8E1Rdlyh1jRTlLTqcuu81BCuZ9jWhn/giEpN3bOMXUNXfYZKniPUqnC3I3GSSVFSmXSM5ExOUZRsEOSUbCqWlCmwODq2akeusYUd2NBRRw0CFnyOTajweGwiCYLDWAVNxlyOjOBc4Ahh5tY5S3CGymiV1z3ebXZQi/etml7A1vkWk6fLzBgS9EA39Er3dEtP9PFrrWZQo6XlgL3W5gpne/R4Yv39X1aNvY/9b9afmn3sIhtoNVi7EyCtV+htfuPw9HV9MZ9oztIlPbP+C3qkO36B1XjTr9ZE/hxh/oCvLsd+D4rzcnJBTq/Nx5eyna/oxxSmMcf9VrCEFayiEKg7wRnOQy9SVJqUptqpUqjDGccPk2Y+AZkljRI=</latexit><latexit sha1_base64="L+dKGHPg6RNrN5Yd9Y/qY0CTaF0=">AAACZnichVFNLwNRFD0d30UVERKbiYZYNbeTpsVK2FiiWhJEZsZTE/OVmWkTGn6AxJaFFYmI+Bk2/oCFf0AsK7GxcDsdEYtyX9679513z73nvae5puEHRM8xqa29o7Oruyfe29efGEgODpV8p+Lpoqg7puNtaKovTMMWxcAITLHhekK1NFOsaweLjfP1qvB8w7HXgkNXbFtq2Tb2DF0NGCooJ9ZOMkXp2VyesopMaQqNg6ySy+YVORMhKUS27CRvsYVdONBRgQUBGwHHJlT4PDaRAcFlbBs1xjyOjPBc4Bhx5lY4S3CGyugBr2XebUaozftGTT9k69zF5OkxU8YkPdEd1emR7umVPlvWqoU1GloO2WtNrnB3Bk7HCh//siz2AfZ/WH9qDrCHmVCrwdrdEGncQm/yq0cX9cLc6mRtiq7pjfVf0TM98A3s6rt+syJWLxHnD/h+Zbl1UFLSmVw6u6Kk5heir+jGOCYwze+dxzyWsIwi9y3jDOe4iL1ICWlEGm2mSrGIM4xfJslf9O6K7Q==</latexit>

Japanese<latexit sha1_base64="NKdad53RWsiDW7GXYrSE1Wf04zE=">AAACa3ichVFNS8NAEH2N3/Wr6kXUQ7BUPIVpKK16EryIJ7VWCyqSxG0NpklI0oIW/4BHLx70oiAi/gwv/gEP/gQRTxW8eHCaRsRDdZbdnX07b/btjO5aph8QPcekjs6u7p7evnj/wODQcGJkdNN3qp4hCoZjOV5R13xhmbYoBGZgiaLrCa2iW2JLP1xq3m/VhOebjr0RHLlit6KVbbNkGlrAUHFFczVb+GIvkSRlPpujjCqTQqGxk1GzmZwqpyMkichWncQtdrAPBwaqqEDARsC+BQ0+j22kQXAZ20WdMY89M7wXOEGcuVWOEhyhMXrIa5lP2xFq87mZ0w/ZBr9i8fSYKSNFT3RHDXqke3qhz7a56mGOppYj3vUWV7h7w6fj+Y9/WRXeAxz8sP7UHKCEuVCrydrdEGn+wmjxa8fnjfzCeqo+Q9f0yvqv6Jke+Ad27d24WRPrF4hzA76rLLd3NlUlnVUya2pycS5qRS8mMY1ZrncOi1jGKgphnc9wgcvYmzQmTUhTrVApFnHG8Muk1BfK5I0f</latexit>

English area<latexit sha1_base64="KM2F1BRcXPVLM61tEg9clyVWHVY=">AAACb3ichVHLSsNAFD2Nr1ofrbpQECRYKropkyJaXBVEcOmrKthSkjhtQ9MkJGmhFn/AD9CFCx8gIn6GG3/ART9BXEkFNy68SQOiRb0hM2fO3HPvmRnF0jXHZawVEnp6+/oHwoORoeGR0WhsbHzXMWu2yrOqqZv2viI7XNcMnnU1V+f7ls3lqqLzPaWy6u3v1bntaKax4zYsnq/KJUMraqrsEpVbM0rUpCzKpCnE4izJ/BC7gRSAOILYMGO3yOEQJlTUUAWHAZewDhkOfQeQwGARl0eTOJuQ5u9zHCNC2hplccqQia3QWKLVQcAatPZqOr5apS46/TYpRSTYE7tjbfbI7tkz+/i1VtOv4Xlp0Kx0tNwqRE+mtt//VVVpdlH+Uv3p2UURad+rRt4tn/FOoXb09aOz9vbKVqI5x67ZC/m/Yi32QCcw6m/qzSbfOkeEHkD6ed3dYDeVlJaSi5upeCYdPEUY05jFPN33MjJYxway1NfCKS5wGXoVJoUZQeykCqFAM4FvISx8AsCDjm4=</latexit>

nativespeaker<latexit sha1_base64="K5tLCx00dQ/BLkCRK+AJfRhPYes=">AAACaXichVE9SwNBEH05v+NHEm2CNsEQsQqTIyRqJdhYxo9EQUXuzk08vdwdd5dADP4BKztRKwUR8WfY+Acs/AmSMoKNhZPLiVhEZ9md2bfzZt/Oqrahux7Ra0jq6x8YHBoeCY+OjU9EorHJkmvVHE0UNcuwnG1VcYWhm6Lo6Z4htm1HKFXVEFvq8UrnfKsuHFe3zE2vYYu9qlIx9bKuKR5DJZNdXexHk5RezOUpKycoTb5xkJVz2bycyARIEoEVrOg9dnEACxpqqELAhMexAQUujx1kQLAZ20OTMYcj3T8XOEWYuTXOEpyhMHrMa4V3OwFq8r5T0/XZGt9i8HSYmUCKXuiB2vRMj/RGnz1rNf0aHS0N9mqXK+z9yFl84+NfVpW9h8Mf1p+aPZSx4GvVWbvtI51XaF1+/eSivbG0nmrO0S21WP8NvdITv8Csv2t3a2L9GmH+gO8uJ3oHJTmdyaWza3JyeSH4imHMYBbz3O88lrGKAop87xHOcYmrUEuKSXFpupsqhQLOFH6ZlPwCO8aMaw==</latexit>

Chinese area<latexit sha1_base64="ArzqLJcNU6Mx+iBGSg/FuY1dQfo=">AAACb3ichVHLSgMxFD0dX7U+WnWhIMhgqeimpEVUXBXcuPRVW2hLmRmjBufFTFrQ4g/4Abpw4QNExM9w4w+48BPElSi4ceHtdEC0qDckOTm55+Yk0V1T+JKxx4jS0dnV3RPtjfX1DwzGE0PDm75T8wyeNxzT8Yq65nNT2DwvhTR50fW4ZukmL+h7S839Qp17vnDsDbnv8oql7dhiWxiaJKq8tEs6n6saaaqJJEuzINR2kAlBEmGsOIlrlLEFBwZqsMBhQxI2ocGnVkIGDC5xFTSI8wiJYJ/jEDHS1iiLU4ZG7B6NO7QqhaxN62ZNP1AbdIpJ3SOlihR7YDfsld2zW/bEPn6t1QhqNL3s06y3tNytxo/G1t//VVk0S+x+qf70LLGNhcCrIO9uwDRvYbT09YOT1/XFtVRjil2yZ/J/wR7ZHd3Arr8ZV6t87RQx+oDMz+duB5vZdGYuPbuaTeYWwq+IYhyTmKb3nkcOy1hBns51cYwznEdelFFlQlFbqUok1IzgWygzn6oojmM=</latexit><latexit sha1_base64="deAIqC+jcGChSfw6ML8/57eWB10=">AAACanichVG7SgNBFD1ZXzG+ojaKTTBGrMLNsiRqFbCx9BUjxBB211GX7IvdTUCDP2BnJWilICJ+ho0/YOEniHYKNhbebFbEQr3DzL1z5p47Z+5ormn4AdFjTOrq7unti/cnBgaHhkeSo2ObvtPwdFHSHdPxtjTVF6Zhi1JgBKbYcj2hWpopylp9qX1ebgrPNxx7IzhwRdVS92xj19DVgKGy7wq1LrxaMk3ZhXyBFDlFWQqNA0XOKwU5lYuQNCJbcZLX2MYOHOhowIKAjYBjEyp8HhXkQHAZq6LFmMeREZ4LHCHB3AZnCc5QGa3zuse7SoTavG/X9EO2zreYPD1mppChB7qhV7qnW3qij19rtcIabS0H7LUOV7i1keOJ9fd/WRb7APvfrD81B9jFfKjVYO1uiLRfoXf4zcPT1/XFtUxrli7pmfVf0CPd8Qvs5pt+tSrWzpHgD/jqcur3YFPO5vJZZVVOF+ejr4hjCtOY434XUMQyVlAK1Z3gDOexF2lMmpSmOqlSLOKM44dJM58rxYzZ</latexit>

Japanese area<latexit sha1_base64="KdzJQNIzbMMUDlkzaWNS+fYOjQM=">AAACcHichVHLSgMxFD0d3/XRqpuCC6tFERclFdHiquBGXGm1KmiRzBhr6LycmRZq8Qf8gS7cWEFE/Aw3/oCLfoK4U8GNC+9MB0RFvSHJyck9NyeJauvS9RhrRZSOzq7unt6+aP/A4FAsPjyy5VoVRxMFzdItZ0flrtClKQqe9HSxYzuCG6outtXysr+/XRWOKy1z06vZomjwkikPpcY9ooqr3OamcEWSk2g/nmJpFkTyJ8iEIIUw1qz4NfZwAAsaKjAgYMIjrIPDpbaLDBhs4oqoE+cQksG+wCmipK1QlqAMTmyZxhKtdkPWpLVf0w3UGp2iU3dImcQUe2A37IXds1v2yN5/rVUPavheajSrba2w92NniY23f1UGzR6OPlV/evZwiGzgVZJ3O2D8W2htffWk8bKxlJ+qT7NL9kT+m6zF7ugGZvVVu1oX+XNE6QMy35/7J9iaS2cW0vPrc6lcNvyKXoxhEjP03ovIYQVrKNC5x2jgAs3Is5JQxpWJdqoSCTWj+BLK7Aekd47V</latexit>

Korean area<latexit sha1_base64="9TNOAw5NwDaKS+h8EK5XDFIiGMY=">AAACbnichVHLSsNAFD2Nr1ofrQoiiFgsFVdlWkSLq4IbwU0f9gG1lCROa2iahCQt1OIPuBcXgqIgIn6GG3/ART9B3AgV3LjwNg2IFvUOmTlz5p6bM3ckQ1Usm7GORxgaHhkd8477Jianpv2BmdmcpTdMmWdlXdXNgiRaXFU0nrUVW+UFw+RiXVJ5Xqpt987zTW5aiq7t2S2Dl+piVVMqiizaRBV3dUrWgiLN5UCIRZgTwUEQdUEIbiT1wC32cQAdMhqog0ODTViFCItGEVEwGMSV0CbOJKQ45xzH8JG2QVmcMkRiazRXaVd0WY32vZqWo5bpLyp9JimDCLMndse67JHds2f28WuttlOj56VFq9TXcqPsP1nIvP+rqtNq4/BL9adnGxXEHa8KeTccpncLua9vHp11M1vpcHuVXbMX8n/FOuyBbqA13+SbFE+fw0cPEP3Z7kGQi0WiG5H1VCyUiLtP4cUiVrBG/d5EAjtIIut07BQXuPS8CvPCkrDcTxU8rmYO30JY+wTCuY36</latexit>(c)

Korean<latexit sha1_base64="i7A3956p6+FvvP353+BxVuwxA/s=">AAACaXichVG7SgNBFD1Z3/GRqI1oEwyKVbgb31aCjWBjEvMAFdldR13dF7ubgAZ/wMpO1EpBRPwMG3/AIp8gKSPYWHh3ExEL9Q4zc+bMPXfOzKiOoXs+US0itbV3dHZ190R7+/oHYvHBoYJnl11N5DXbsN2SqnjC0C2R93XfECXHFYqpGqKoHq0E+8WKcD3dtjb8Y0dsm8q+pe/pmuIzVVizOdnaiScptRjEdKIJ5gJAMzLJswk5RWEk0Yp1O36PLezChoYyTAhY8BkbUOBx24QMgsPcNqrMuYz0cF/gFFHWljlLcIbC7BGP+7zabLEWr4OaXqjW+BSDu8vKBCbohR6oQc/0SK/08Wutalgj8HLMs9rUCmcndjaSe/9XZfLs4+Bb9adnH3tYCL3q7N0JmeAWWlNfOblo5JayE9VJuqU6+7+hGj3xDazKm3aXEdlrRPkDvl458TsopFPyXGomk04uL7S+ohtjGMcUv/c8lrGKdeT53EOc4xJXkbo0KI1Io81UKdLSDONHSMlPUOaMdw==</latexit>

Figure 2: Improved demo system with an electric turntable. It canbe carried out with a single suitecase.

native<latexit sha1_base64="Yfy3OdZ765o0voUOwEPZAJ2/gXU=">AAACaXichVHLSsNAFD2Nr1pfrW5EN8VScVVufNTHSnDj0lZbhVokidMaTZOQpIFa/AFX7kRdKYiIn+HGH3DhJ4hLBTcuvEkr4kK9IXPPnLnnzpkZ1TZ01yN6ikgdnV3dPdHeWF//wOBQPDFcdK26o4mCZhmWs6UqrjB0UxQ83TPElu0IpaYaYlM9WAnWN33huLplbngNW5RrStXUK7qmeEwVTU6+2ImnKLMYxEyyBbIBoFmZ5LmknKEwUmjHmhW/wTZ2YUFDHTUImPAYG1Dg8leCDILNXBlN5hxGergucIQYa+tcJbhCYfaAxyrPSm3W5HnQ0w3VGu9i8O+wMok0PdItvdID3dEzffzaqxn2CLw0OKstrbB3ho5H19//VdU4e9j7Vv3p2UMFC6FXnb3bIROcQmvp/cPT1/WlfLo5SVf0wv4v6Ynu+QSm/6Zd50T+AjF+gK9bTv4OitMZOZuZzU2nlhfaTxHFOCYwxfc9j2WsYg0F3ncfJzjDeeRFSkij0lirVIq0NSP4EVLqE5+AjJ4=</latexit>

speaker<latexit sha1_base64="ZarDnN5zeGk/lj5g5UBOOKfWB/c=">AAACanichVG7SgNBFD1Z3/GRqI1iE4yKVbgb31aCjWU0xgRiCLvrqEv2xe4moMEfsLMSTKUgIn6GjT9gkU8Q7RRsLLy7iYiFeoeZOXPmnjtnZlTH0D2fqBmROjq7unt6+6L9A4NDsfjwyI5nV11N5DTbsN2CqnjC0C2R83XfEAXHFYqpGiKvVtaD/XxNuJ5uW9v+kSNKpnJg6fu6pvhM5T1HKBXhluNJSq0EMZdogcUA0LxM8kJCTlEYSbQjY8dvsIs92NBQhQkBCz5jAwo8bkXIIDjMlVBnzmWkh/sCJ4iytspZgjMUZis8HvCq2GYtXgc1vVCt8SkGd5eVCUzTI93SKz3QHT3Rx6+16mGNwMsRz2pLK5xy7HQs+/6vyuTZx+G36k/PPvaxHHrV2bsTMsEttJa+dnz+ml3dmq7P0BU9s/9LatI938CqvWnXm2KrgSh/wNcrJ34HO+mUvJia30wn15bbX9GLCUxilt97CWvYQAa50N0ZLtCIvEgj0rg00UqVIm3NKH6ENPUJj3+NDA==</latexit><latexit sha1_base64="dhQY95QqzER03vSK+6Ev11mxtK8=">AAACrXichVHLahRBFD1pX3F8ZKIbwU3hEImLTGqCJCGriJu4y8NJAkkcqit3OkWqH3RVD0wGf8AfcOFKQUQEf8KNH6CLLHQvLiO4ceGdngbRoN6mq06duufeU1VhZo3zUh6PBWfOnjt/Yfxi7dLlK1cn6pPXNl1a5JraOrVpvh0qR9Yk1PbGW9rOclJxaGkrPLw/3N/qUe5Mmjz0/Yz2YhUlpmu08kx16ivT4R1xT/tCWWHizFJMiad9wTXsjDcxiS63mrEqiQoVkXAZkT4QPleJs2UN4frOU9ypN2RTliFOg1YFGqhiNa2/wi72kUKjQAxCAs/YQsHxt4MWJDLm9jBgLmdkyn3CY9RYW3AWcYZi9pDHiFc7FZvweljTlWrNXSz/OSsFpuRH+VqeyPfyjfwif/y11qCsMfTS5zkcaSnrTDy5sfH9v6qYZ4+DX6p/evboYrH0ath7VjLDU+iRvnf09GRjaX1qcFu+kF/Z/3N5LN/xCZLeN/1yjdafocYP0Przuk+Dzblma755d22usfygeopx3MQtTPN9L2AZK1hFm/u+xQd8wudgNmgHu8GjUWowVmmu47cIop9z6KaA</latexit>

Jap

ne
pa
h

<latexit sha1_base64="C/+bEJsM/MAN+u8A4ZgWmDCyZ9A=">AAACb3ichVHLSgMxFD0dX7U+WnWhIMhgqdRNyYiouCq4cdmqbQUVmRnTGjqdGeYFWvwBP0AXLnyAiPgZbvwBF36CuBIFNy68nQ6IinpDkpOTe25OEs02hOsx9hCTOjq7unvivYm+/oHBZGpouOxavqPzkm4ZlrOuqS43hMlLnvAMvm47XG1oBq9o9aXWfiXgjissc83bs/lWQ62Zoip01SNqM6tOywWHB8Ly3e1UmuVYGPJPoEQgjSgKVuoKm9iBBR0+GuAw4RE2oMKltgEFDDZxW2gS5xAS4T7HARKk9SmLU4ZKbJ3GGq02ItakdaumG6p1OsWg7pBSRobds2v2wu7YDXtk77/WaoY1Wl72aNbaWm5vJw/HVt/+VTVo9rD7qfrTs4cqFkKvgrzbIdO6hd7WB/vHL6uLK5nmFLtgT+T/nD2wW7qBGbzql0W+coIEfYDy/bl/gvJMTpnLzRZn0vl89BVxjGMSWXrveeSxjAJKdK6NI5ziLPYsjUoTktxOlWKRZgRfQpr+ABVSjiI=</latexit>

lish

Eng
Ch

Figure 4: (a) Real-time multilingual speech translation system combined with multiple sound spot synthesis. (b) Actual implemented
real-time four-language speech translation system combined withimproved multiple sound spot synthesis system.

nch

ine

Fre

se
ne

Ja
is

hi

an

n
rea
Ko

h
nis
Spa

ssi

gl

C

bic

se

Ara

En

n

ea
or

K

Ru

ane
se

(b) Actual implemented real-time four-language speech translation systemseeight directions (Fig. 3(b)). The improved system including theturntable can also be carried out with a single suitcase. The detailed
results of experiments will be submitted to a journal paper.

<latexit sha1_base64="+WuiUG+JCf/BPXi4UiVO2xKjfgE=">AAACb3ichVHLSgMxFD0d3/XRqgsFQYpFqZuSEVFxVXCjO22tFtpSZqZpHZwXM+mAFn/AD9CFCx8gIn6GG3/AhZ8grkTBjQtvpwOiot6Q5OTknpuTRHUM3ROMPUSkjs6u7p7evmj/wOBQLD48suXZDVfjec02bLegKh43dIvnhS4MXnBcrpiqwbfV3ZXW/rbPXU+3rU2x5/CyqdQtvaZriiCqlFJnE2um49o+r1biSZZmQSR+AjkESYSxbsevUEIVNjQ0YILDgiBsQIFHrQgZDA5xZTSJcwnpwT7HAaKkbVAWpwyF2F0a67QqhqxF61ZNL1BrdIpB3SVlAtPsnl2zF3bHbtgje/+1VjOo0fKyR7Pa1nKnEjscz739qzJpFtj5VP3pWaCGpcCrTt6dgGndQmvr/f3jl9xydro5wy7YE/k/Zw/slm5g+a/a5QbPniBKHyB/f+6fYGsuLS+k5zfmkplM+BW9mMAUUvTei8hgFevI07kOjnCKs8izNCZNSol2qhQJNaP4EtLsB+k1jgw=</latexit>(a) Previous
Four spot synthesis<latexit sha1_base64="SXIFQE3oaZDnqOtEBnVpxFnMrG4=">AAACeHichVHLSgMxFD0d3/XRqhvBzWCpj01Ji6i4Kgjisj6qQpUyM0YbOp0ZJmmhFn/AH3DhQhREi5/hxh9w4SeISwVBXHg7HRAV9YYkJyf33JwkpmcLqRh7iGgdnV3dPb190f6BwaFYfHhkU7pV3+J5y7Vdf9s0JLeFw/NKKJtvez43KqbNt8zyUmt/q8Z9KVxnQ9U9vlsxDhyxLyxDEVWMjy5TJV16rtJl3VElLoUsxhMsxYLQf4J0CBIII+fGr7CDPbiwUEUFHA4UYRsGJLUC0mDwiNtFgzifkAj2OY4QJW2VsjhlGMSWaTygVSFkHVq3aspAbdEpNnWflDqS7J412TO7Yzfskb3/WqsR1Gh5qdNstrXcK8aOx9Zf/1VVaFYofar+9Kywj4XAqyDvXsC0bmG19bXDk+f1xbVkY5JdsCfyf84e2C3dwKm9WJerfO0UUfqA9Pfn/gk2M6n0XGp2NZPIZsOv6MU4JjBN7z2PLFaQQ57OreMM12hG3jRdm9Jm2qlaJNSM4ktomQ+Ln5H2</latexit>(b) Improved
Eight spot synthesis<latexit sha1_base64="SyByYztGeus4D1SieHzRJjlOcbY=">AAACeXichVHLSgMxFD0dX7W+6mMhuKkWRVyUVEXFVUEEl221KqiUmTG2wenMMEkLbfEH/AEXblQQqX6GG3/AhZ8gLhVc6MLb6YCoqDckOTm55+YkMVxLSMXYQ0hra+/o7Ap3R3p6+/oHooNDm9IpeybPmY7leNuGLrklbJ5TQll82/W4XjIsvmUcrjT3tyrck8KxN1TV5XslvWCLA2Hqiqh8dGRVFIoqJl2HhqqtilwKmY/GWYL5EfsJkgGII4i0E73CLvbhwEQZJXDYUIQt6JDUdpAEg0vcHurEeYSEv89xhAhpy5TFKUMn9pDGAq12AtamdbOm9NUmnWJR90gZwyS7Zw32zO7YDXtk77/Wqvs1ml6qNBstLXfzA8ej66//qko0KxQ/VX96VjjAku9VkHfXZ5q3MFv6Su3keX05O1mfYhfsifyfswd2SzewKy/mZYZnTxGhD0h+f+6fYHM2kVxIzGdm46lU8BVhjGEC0/Tei0hhDWnk6NwaztDAdehNG9emtZlWqhYKNMP4EtrcB2GsklU=</latexit>

Figure 3: (a) Four-language sound spot synthesis for four directions
by previous system. (b) Eight-language sound spot synthesis foreight directions by improved system.

small for demonstration in a large conference venue. 2) The output sound quality is not high because low frequency components
below 500 Hz cannot be produced. Additionally, 3) a large spaceis required for demonstration because 4) participants must basically walk around the array to listen to different sounds although
the synthesized sound field can be interactively rotated by the tablet(Fig. 1).
3. IMPROVED SYSTEM
3.1. Overview of improved system
To improve the previous system, we developed an improved systemusing a baffled circular array of 16 loudspeakers combined with anelectric turntable (Fig. 2). To improve the total output power andlow frequency response, we developed a slightly larger loudspeaker
driver with a diameter of 32 mm, and the diameter of the improved
circular array is 178.6 mm. The spatial Nyquist frequency is thenabout 4.9 kHz. For avoiding the diffraction of sound from otherdirections, a cylindrical rigid baffle was introduced. Although thespatial Nyquist frequency of the improved system is lower than thatof the previous system, the improved system can 1) produce a soundpressure level of 80 dB, which is sufficient for demonstration ina large conference venue, and 2) drastically improve the synthesissound quality with low frequency components above 200 Hz. Additionally, it can be demonstrated 3) in a small space, and 4) participants do not have to walk around the array by the electric turntable.
Furthermore, it can realize eight-language sound spot synthesis for3.2. WASPAA 2023 Demonstrations
In WASPAA 2023 Demonstrations, we demonstrate both four- andeight-language sound spot synthesis for four and eight directions using the improved system (Fig. 3). The input eight-language speechsounds are also generated by multilingual TTS based on highfidelity and real-time methods [28, 30, 31]. Additionally, copyrightfree music audio signals are also synthesized. The driving signals of
16 loudspeakers are calculated from the input audio signals and FIRfilters for localized sound spot synthesis for each area in real-timeby FFT-based convolution. The demo system is consructed from thebaffled circular array, amplifier for 16 loudspeakers including D/A,
loudspeaker stand, laptop, tablet, cables, and turntable (Fig. 2). All
the equipment can be carried out with a single suitcase. A single
electrical outlet (100 V in USA) is sufficient. The minimum spacerequired for the demonstration is 2 m × 2 m (Fig. 2). If a larger
space is available, it is better. Participants can experience the synthesized eight-spot sound field that they have never heard before,
and they can interactively rotate the synthesized sound field withthe tablet. Additionally, they can understand that demonstration
systems for sound field control can also be carried out elsewhere.
3.3. Actual implemented application (not included in demo)
Finally, we briefly introduce an actual implemented application using the improved system, which has been presented in NICT openhouse 2023. By combining the improved multiple sound spot synthesis system based on acoustic signal processing with real-time
multilingual speech translation system (ASR + machine translation+ TTS) based on machine learning (Fig. 4(a)), we have actually implemented a real-time four-language speech translation system withmultiple sound spot synthesis (Fig. 4(b)). In the system, each native
speaker speaks only his or her own language and listens only to thesynthesized speech sounds translated from other languages into his
or her own language, where each translated speech sound is heard
only by each native speaker through multiple sound spot synthesis.

2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

4. REFERENCES[1] P. J. Westervelt, “Parametric acoustic array,” J. Acoust. Soc.
Am., vol. 35, no. 4, pp. 535–537, Apr. 1963.
[2] J.-W. Choi and Y.-H. Kim, “Generation of an acoustically bright zone with an illuminated region using multiple
sources,” J. Acoust. Soc. Am., vol. 111, no. 4, pp. 1695–1700,
Apr. 2002.
[3] M. Shin, S. Q. Lee, F. M. Fazi, P. A. Nelson, D. Kim, S. Wang,
K. H. Park, and J. Seo, “Maximization of acoustic energy difference between two spaces,” J. Acoust. Soc. Am., vol. 128,
no. 1, pp. 121–131, July 2010.
[4] T. Okamoto, “Generation of multiple sound zones by spatial
filtering in wavenumber domain using a linear array of loudspeakers,” in Proc. ICASSP, May 2014, pp. 4733–4737.
[5] ——, “Analytical methods of generating multiple sound zones
for open and baffled circular loudspeaker arrays,” in Proc.
WASPAA, Oct. 2015.
[6] T. Okamoto and A. Sakaguchi, “Experimental validation of
spatial Fourier transform-based multiple sound zone generation with a linear loudspeaker array,” J. Acoust. Soc. Am., vol.
141, no. 3, pp. 1769–1780, Mar. 2017.
[7] J. Donley, C. H. Ritz, and W. B. Kleijn, “Multizone soundfield
reproduction with privacy and quality based speech masking filters,” IEEE/ACM Trans. Audio, Speech, Lang. Process.,
vol. 26, no. 6, pp. 1041–1055, June 2018.
[8] T. Lee, J. K. Nielsen, and M. G. Christensen, “Signal-adaptive
and perceptually optimized sound zones with variable span
trade-off filters,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 28, pp. 2412–2426, 2020.
[9] D. Wallace and J. Cheer, “Design and evaluation of personal
audio systems based on speech privacy constraints,” J. Acoust.
Soc. Am., vol. 147, no. 4, pp. 2271–2282, Apr. 2020.
[10] L. Shi, T. Lee, L. Zhang, J. K. Nielsen, and M. G. Christensen,“Generation of personal sound zones with physical meaningful constraints and conjugate gradient method,” IEEE/ACM
Trans. Audio, Speech, Lang. Process., vol. 29, pp. 823–837,
2021.
[11] V. Moles-Cases, S. J. Elliott, J. Cheer, G. Pinero, and A. Gonzalez, “Weighted pressure matching with windowed targets
for personal sound zones,” J. Acoust. Soc. Am., vol. 151, no. 1,
pp. 334–345, Jan. 2022.
[12] M. Hu, H. Zou, J. Li, and M. G. Christensen, “Maximizing
the acoustic contrast with constrained reconstruction error under a generalized pressure matching framework in sound zone
control,” J. Acoust. Soc. Am., vol. 151, no. 4, pp. 2571–2759,
Apr. 2022.
[13] T. Abe, S. Koyama, N. Ueno, and H. Saruatari, “Amplitude matching for multizone sound field control,” IEEE/ACM
Trans. Audio, Speech, Lang. Process., vol. 31, pp. 656–669,
2023.
[14] T. Betlehem, W. Zhang, M. Poletti, and T. Abhayapala, “Personal sound zones: Delivering interface-free audio to multiple
listeners,” IEEE Signal Process. Mag., vol. 32, no. 2, pp. 81–
91, Mar. 2015.
[15] S. Spors, R. Rabenstein, and J. Ahrens, “The theory of wave
field synthesis revisited,” in Proc. 124th Conv. Audio Eng.
Soc., May 2008.

October 22-25, 2023, New Paltz, NY[16] A. J. Berkhout, D. de Vries, and P. Vogel, “Acoustic control
by wave field synthesis,” J. Acoust. Soc. Am., vol. 93, no. 5,
pp. 2764–2778, May 1993.
[17] G. Firtha, P. Fiala, F. Schultz, and S. Spors, “Improved
referencing schemes for 2.5D wave field synthesis driving
functions,” IEEE/ACM Trans. Audio, Speech, Lang. Process.,
vol. 25, no. 5, pp. 1117–1127, May 2017.
[18] J. Ahrens and S. Spors, “Sound field reproduction using planar and linear arrays of loudspeakers,” IEEE Trans. Audio,
Speech, Lang. Process., vol. 18, no. 8, pp. 2038–2050, Nov.
2010.
[19] G. Firtha, P. Fiala, F. Schultz, and S. Spors, “On the general relation of wave field synthesis and spectral division method forlinear arrays,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no. 12, pp. 2393–2043, Dec. 2018.
[20] M. A. Poletti, “Three-dimensional surround sound systems
based on spherical harmonics,” J. Audio Eng. Soc., vol. 53,
no. 11, pp. 1004–1025, Nov. 2005.
[21] J. Ahrens and S. Spors, “An analytical approach to sound field
reproduction using circular and spherical loudspeaker distributions,” Acta Acust. Acust., vol. 94, no. 6, pp. 988–999, Nov.
2008.
[22] F. Winter, J. Ahrens, and S. Spors, “On analytic methods for
2.5-D local sound field synthesis using circular distributions ofsecondary sources,” IEEE/ACM Trans. Audio, Speech, Lang.
Process., vol. 24, no. 5, pp. 914–926, May 2016.
[23] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. r. Mohamed,
N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath,and B. Kingsbury, “Deep neural networks for acoustic modeling in speech recognition: The shared views of four researchgroups,” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82–97,
Nov. 2012.
[24] K. Tokuda, Y. Nankaku, T. Toda, H. Zen, J. Yamagishi, andK. Oura, “Speech synthesis based on hidden Markov models,”
Proc. IEEE, vol. 101, no. 5, pp. 1234–1252, May 2013.
[25] T. Okamoto, K. Ueno, T. Okabe, K. Tani, Y. Yoshikata,
M. Sudo, M. Kuwahara, and K. Hikita, “Portable multilingual
sound spot synthesis system with a compact circular array of
16 loudspeakers,” in ICASSP 2023 Show & Tell Demo, June
2023.
[26] T. Okamoto, “Multilingual sound spot synthesis systems,” inProc. Internoise, Aug. 2023.
[27] M. S. Puckette, “Pure data,” in Proc. ICMC, Sept. 1997.
[28] T. Okamoto, T. Toda, and H. Kawai, “Multi-stream HiFi-GAN
with data-driven waveform decomposition,” in Proc. ASRU,
Dec. 2021, pp. 610–617.
[29] T. Okamoto, Y. Shiga, and H. Kawai, “Hi-Fi-CAPTAIN:
High-fidelity and high-capacity conversational speechsynthesis corpus developed by NICT,” https://astastrec.nict.go.jp/en/release/hi-fi-captain/, 2023.
[30] D. Lim, S. Jung, and E. Kim, “JETS: Jointly training FastSpeech2 and HiFi-GAN for end to end text to speech,” in Proc.
Interspeech, Sept. 2022, pp. 21–25.
[31] T. Okamoto, T. Toda, and H. Kawai, “E2E-S2S-VC: End-toend sequence-to-sequence voice conversion,” in Proc. Interspeech, Aug. 2023, pp. 2043–2047.