Updated on May 17, 2024

Yosuke HIGUCHI Personal Website

Google Scholar

LinkedIn  GitHub

Office Address: 27 Waseda-machi, Shinjuku-ku, Tokyo, Japan
Office Phone: (+81) 3-3203-4468
E-mail: higuchi [at] pcl.cs.waseda.ac.jp

Education•

Ph.D. in Computer Science and Engineering

Apr. 2021 – Mar. 2024

Department of Computer Science and Communications Engineering, Waseda University, Japan– Supervisor: Tetsunori Kobayashi, Tetsuji Ogawa– Thesis title: “Incorporating Linguistic Cues for End-to-End Speech Recognition”•M.E. in Computer Science and Engineering

Apr. 2019 – Mar. 2021

Department of Computer Science and Communications Engineering, Waseda University, Japan– Supervisor: Tetsunori Kobayashi, Tetsuji Ogawa– Thesis title: “Non-Autoregressive End-to-End Speech Recognition with CTC and Mask-predict”•B.E. in Computer Science and Engineering

Apr. 2015 – Mar. 2019

Department of Computer Science and Engineering, Waseda University, Japan– And with a minor in Intermedia Art and Science– Supervisor: Tetsunori Kobayashi, Tetsuji Ogawa– Thesis title: “Speaker Invariant Feature Extraction for Zero-Resource Languages”

Experience•Google LLC

Oct. 2022 – Jan. 2023

Research Intern

New York, USA– Mentor: Andrew Rosenberg, Bhuvana Ramabhadran– Worked on text injection in speech recognition– Major publication(s): [Higuchi+, ASRU’23]•Carnegie Mellon UniversityMar. 2022 – June 2022

Visiting Scholar

Pennsylvania, USA– Mentor: Shinji Watanabe– Worked on language modeling in speech recognition– Major publication(s): [Higuchi+, EMNLP Findings’22] [Higuchi+, ICASSP’23b]•Mitsubishi Electric Research Laboratories

Oct. 2020 – Mar. 2021, Apr. 2021 – Dec. 2021

Remote Research Intern, Part-Time Researcher

Massachusetts, USA– Mentor: Takaaki Hori, Niko Moritz, Jonathan Le Roux
– Worked on semi-supervised speech recognition– Major publication(s): [Higuchi+, Interspeech’21] [Higuchi+, JSTSP’22]•Johns Hopkins UniversityVisting Scholar

Dec. 2019 – Mar. 2020
Maryland, USA– Mentor: Shinji Watanabe– Worked on non-autoregressive speech recognition– Major publication(s): [Higuchi+, Interspeech’20] [Higuchi+, ICASSP’21]•IBM Research AI
Research Intern

June 2019 – Oct. 2019
Tokyo, Japan– Mentor: Masayuki Suzuki, Gakuto Kurata– Worked on speaker embeddings for speaker diarization– Major publication(s): [Higuchi+, ICASSP’20]•NTT Communication Science Laboratories
Research Intern– Mentor: Hirokazu Kameoka– Worked on voice conversion based on generative adversarial networks

Aug. 2018 – Sept. 2018Kangawa, JapanExternal Research Funding•

Super Global University (Visit to Carnegie Mellon University) | 1,000,000 JPY

Mar. 2022 – June 2022

from Top Global University Japan Project– In this study abroad program, I have participated in joint research with Prof. Shinji Watanabe at the Language
Technologies Institute, Carnegie Mellon University. The primary goal of our project was to explore the integrationof a pre-trained language model into end-to-end speech recognition.
– Major publication(s): [Higuchi+, EMNLP Finding’22] [Higuchi+, ICASSP’23a] [Yan+, EACL’23]•ACT-X Frontier of Mathematics and Information Science | 4,500,000 JPY

Oct. 2021 – Mar. 2024

from Japan Science and Technology Agency (JST)– This research project aimed to improve the recognition accuracy and functionality of end-to-end speech recognition models, focusing on the development of technology for effectively extracting linguistic information fromspeech information. In particular, by explicitly designing a feature extraction process that incorporates linguisticknowledge, I have established a novel framework for end-to-end speech recognition that does not solely rely on
data-driven approaches. Moreover, the proposed framework has been shown useful for tasks that require semantic
understanding in conjunction with speech recognition.
– Major publication(s): [Higuchi+, ICASSP’22b] [Higuchi+, ICASSP’23b] [Higuchi+, arXiv’23]•Research Fellowship for Young Scientists (DC1) | 2,200,000 JPY

Apr. 2021 – Mar. 2024

from Japan Society for the Promotion of Science (JSPS)– To enhance the practicality of speech-based communication interfaces, this research project aimed to developa speech recognition system that operates with high accuracy and fast speed. To this end, I have constructeda non-autoregressive end-to-end speech recognition system based on masked language models. Compared toconventional autoregressive systems, the proposed model has been shown to achieve significant improvements ininference speed while maintaining a similar level of recognition accuracy.
– Major publication(s): [Higuchi+, ICASSP’21] [Higuchi+, ASRU’21] [Zhao+, EUSIPCO’23]•Super Global University (Visit to Johns Hopkins University) | 1,000,000 JPYDec. 2019 – Mar. 2020

from Top Global University Japan Project– In this study abroad program, I have participated in joint research with Prof. Shinji Watanabe at the Center forLanguage and Speech Processing, Johns Hopkins University. The primary goal of our project was to explore an
end-to-end speech recognition model architecture that can operate with fast inference speed.
– Major publication(s): [Higuchi+, Interspeech’20] [Guo+, ICASSP’21] [Watanabe+, DSLW’21]

Awards•Best Reviewer Award

Dec. 2023

from IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)•ISS Young Researcher’s Award in Speech Field

Sept. 2021

from the Institute of Electronics, Information and Communication Engineers (IEICE)•Repayment Exemption for Graduate Students with Excellent Achievements (full-exemption) Mar. 2021
from Japan Student Services Organization (JASSO)•Best Student Presentation Award

Nov. 2020

from the Acoustical Society of Japan (ASJ)•Yamashita SIG Research Award

Aug. 2020

from Information Processing Society of Japan (IPSJ)•Yahoo! JAPAN Award

Jan. 2020

from Information Processing Society of Japan (IPSJ) SIG-SLP

Reviewing Activity
ACL: 2024 APSIPA: 2020, 2021 ASRU: 2023 ICASSP: 2023 Interspeech: 2023, 2024 SLT: 2022
SPECOM: 2024 TASLP: 2022, 2023, 2024

Skills•Language: Japanese (native), English (fluent)•Programming: Python, C, C++, Java, JavaScript, LaTeX•

Software & Tools: PyTorch, Chainer, TensorFlow, ESPnet, Kaldi, Lingvo

Publications
Journal
1. Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, Takaaki Hori, “Momentum Pseudo-Labeling: SemiSupervised ASR with Continuously Improving Pseudo-Labels,” IEEE Journal of Selected Topics in Signal
Processing, October 2022.
International Conference (First Author)
1. Yosuke Higuchi, Andrew Rosenberg, Yuan Wang, Murali Karthick Baskar, Bhuvana Ramabhadran,“Mask-Conformer: Augmenting Conformer with Mask-Predict Decoder,” 2023 IEEE Automatic Speech
Recognition and Understanding Workshop (ASRU), December 2023.
2. Yosuke Higuchi, Tetsuji Ogawa, Tetsunori Kobayashi, Shinji Watanabe, “BECTRA: Transducer-based
End-to-End ASR with BERT-Enhanced Encoder,” 2023 IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP), June 2023.
3. Yosuke Higuchi, Tetsuji Ogawa, Tetsunori Kobayashi, Shinji Watanabe, “InterMPL: Momentum PseudoLabeling with Intermediate CTC Loss,” 2023 IEEE International Conference on Acoustics, Speech andSignal Processing (ICASSP), June 2023.
4. Yosuke Higuchi, Brian Yan, Siddhant Arora, Tetsuji Ogawa, Tetsunori Kobayashi, Shinji Watanabe,“BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked
Language Model,” Findings of the Association for Computational Linguistics: EMNLP 2022, December
2022.
5. Yosuke Higuchi, Keita Karube, Tetsuji Ogawa, Tetsunori Kobayashi, “Hierarchical Conditional Endto-End ASR with CTC and Multi-Granular Subword Units,” 2022 IEEE International Conference onAcoustics, Speech and Signal Processing (ICASSP), May 2022.
6. Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, Takaaki Hori, “Advancing Momentum Pseudo-Labelingwith Conformer and Initialization Strategy,” 2022 IEEE International Conference on Acoustics, Speechand Signal Processing (ICASSP), May 2022.
7. Yosuke Higuchi, Nanxin Chen, Yuya Fujita, Hirofumi Inaguma, Tatsuya Komatsu, Jaesong Lee, Jumon
Nozaki, Tianzi Wang, Shinji Watanabe, “A Comparative Study on Non-Autoregressive Modelings for
Speech-to-Text Generation,” 2021 IEEE Automatic Speech Recognition and Understanding Workshop(ASRU), December 2021.
8. Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, Takaaki Hori, “Momentum Pseudo-Labeling for SemiSupervised Speech Recognition,” 22nd Annual Conference of International Speech Communication Association (Interspeech), August 2021.
9. Yosuke Higuchi, Shinji Watanabe, Hirofumi Inaguma, Tetsuji Ogawa, Tetsunori Kobayashi, “Improved
Mask-CTC for Non-Autoregressive End-to-End ASR,” 2021 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP), June 2021.
10. Yosuke Higuchi, Naohiro Tawara, Atsunori Ogawa, Tomoharu Iwata, Tetsunori Kobayashi, TetsujiOgawa, “Noise-robust Attention Learning for End-to-End Speech Recognition,” 28th European Signal
Processing Conference (EUSIPCO), January 2021.
11. Yosuke Higuchi, Shinji Watanabe, Nanxin Chen, Tetsuji Ogawa, Tetsunori Kobayashi, “Mask CTC: NonAutoregressive End-to-End ASR with CTC and Mask Predict,” 21st Annual Conference of International
Speech Communication Association (Interspeech), October 2020.
12. Yosuke Higuchi, Masayuki Suzuki, Gakuto Kurata, “Speaker Embeddings Incorporating Acoustic Conditions for Diarization," 2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), May 2020.
13. Yosuke Higuchi, Naohiro Tawara, Tetsunori Kobayashi, Tetsuji Ogawa, “Speaker Adversarial Trainingof DPGMM-based Feature Extractor for Zero-Resource Languages,” 20th Annual Conference of International Speech Communication Association (Interspeech), September 2019.

International Conference (Co-Author)
1. Tomoki Ariga, Yosuke Higuchi, Kazutoshi Hayasaka, Naoki Okamoto, and Tetsuji Ogawa, “Parody Detection Using Source-Target Attention With Teacher-Forced Lyrics,” 2024 IEEE International Conferenceon Acoustics, Speech and Signal Processing (ICASSP), April 2024.
2. Masao Someki, Nicholas Eng, Yosuke Higuchi, Shinji Watanabe, “Segment-Level Vectorized Beam Search
Based on Partially Autoregressive Inference," 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), December 2023.
3. Tomoki Ariga, Yosuke Higuchi, Mitsunori Kanno, Rie Shigyo, Takato Mizuguchi, Naoki Okamoto, Tetsuji
Ogawa, “Spotting Parodies: Detecting Alignment Collapse Between Lyrics and Singing Voice," 31stEuropean Signal Processing Conference (EUSIPCO), September 2023.
4. Huaibo Zhao, Yosuke Higuchi, Yusuke Kida, Tetsuji Ogawa, Tetsunori Kobayashi, “Mask-CTC-based
Encoder Pre-training for Streaming End-to-End Speech Recognition," 31st European Signal Processing
Conference (EUSIPCO), September 2023.
5. Brian Yan, Siddharth Dalmia, Yosuke Higuchi, Graham Neubig, Florian Metze, Alan W Black, Shinji
Watanabe, “CTC Alignments Improve Autoregressive Translation," 17th Conference of the European
Chapter of the Association for Computational Linguistics (EACL), May 2023.
6. Yifan Peng*, Siddhant Arora*, Yosuke Higuchi, Yushi Ueda, Sujay Kumar, Karthik Ganesan, SiddharthDalmia, Xuankai Chang, Shinji Watanabe, “A Study on the Integration of Pre-Trained SSL, ASR, LMand SLU Models for Spoken Language Understanding,” 2022 IEEE Spoken Language Technology Workshop (SLT), January 2023.
7. Masao Someki, Yosuke Higuchi, Tomoki Hayashi, Shinji Watanabe, “ESPnet-ONNX: Bridging a Gap
Between Research and Production," Asia-Pacific Signal and Information Processing Association Annual
Summit and Conference 2022 (APSIPA), November 2022.
8. Keqi Deng*, Zehui Yang*, Shinji Watanabe, Yosuke Higuchi, Gaofeng Cheng, Pengyuan Zhang, “Improving Non-Autoregressive End-to-End Speech Recognition with Pre-trained Acoustic and Language
Models,” 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),May 2022.
9. Huaibo Zhao, Yosuke Higuchi, Tetsuji Ogawa, Tetsunori Kobayashi, “An Investigation of Enhancing CTC
Model for Triggered Attention-based Streaming ASR,” Asia-Pacific Signal and Information Processing
Association Annual Summit and Conference 2021 (APSIPA), December 2021.
10. Shinji Watanabe, Florian Boyer, Xuankai Chang, Pengcheng Guo, Tomoki Hayashi, Yosuke Higuchi,Takaaki Hori, Wen-Chin Huang, Hirofumi Inaguma, Naoyuki Kamo, Shigeki Karita, Chenda Li, JingShi, Aswin Shanmugam Subramanian, Wangyou Zhang, “The 2020 ESPnet Update: New Features,Broadened Applications, Performance Improvements, and Future Plans,” 2021 IEEE Data Science &Learning Workshop (DSLW), June 2021.
11. Pengcheng Guo, Florian Boyer, Xuankai Chang, Tomoki Hayashi, Yosuke Higuchi, Hirofumi Inaguma,Naoyuki Kamo, Chenda Li, Daniel Garcia-Romero, Jiatong Shi, Jing Shi, Shinji Watanabe, Kun Wei,Wangyou Zhang, Yuekai Zhang, “Recent Developments on ESPnet Toolkit Boosted by Conformer,” 2021
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), June 2021.
12. Hirofumi Inaguma, Yosuke Higuchi, Kevin Duh, Tatsuya Kawahara, Shinji Watanabe, “Orthros: Nonautoregressive End-to-end Speech Translation with Dual-decoder,” 2021 IEEE International Conferenceon Acoustics, Speech and Signal Processing (ICASSP), June 2021.
Preprint
1. Yosuke Higuchi, Tetsuji Ogawa, Tetsunori Kobayashi, “Harnessing the Zero-Shot Power of InstructionTuned Large Language Model in End-to-End Speech Recognition,” arXiv preprint arXiv:2309.10524,
September 2023.
2. Hirofumi Inaguma, Yosuke Higuchi, Kevin Duh, Tatsuya Kawahara, Shinji Watanabe, “Nonautoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring,” arXiv preprint
arXiv:2109.04411, September 2021.