Hiroki Furuta
Tokyo, Japan | furuta@weblab.t.u-tokyo.ac.jp | website: https://frt03.github.io | Phone +81 90 9228 5209

Education
The University of TokyoTokyo, Japan

Ph.D. Student
April 2022 - (March 2025)• Department of Technology Management for Innovation, Graduate School of Engineering• Supervisor: Yutaka Matsuo (Chairperson, AI Strategy Council, Cabinet Office, Japan)• Research Topic: Web Navigation with Large Language Models, Foundation Models for Decision Making
Master of Engineering
April 2020 - March 2022
• Department of Technology Management for Innovation, Graduate School of Engineering• Supervisor: Yutaka Matsuo, Advisor (Adjunct Professor): Shixiang Shane Gu• Research Topic: Deep Reinforcement Learning for Continuous Control, Data-Centric View of Deep Reinforcement
Learning• Coursework: Statistical Machine Learning, Deep Learning, Artificial Intelligence, Advanced Programming Techniques, Internet Engineering, Biosignal Processing, etc. (GPA: 3.9)
• Dean’s Award, from Graduate School of Engineering (2022)
Bachelor of Engineering
April 2016 - March 2020
• Department of Systems Innovation, School of Engineering• Research Topic: Application of Machine Learning for Space Exploration Research, Object Detection for PlanetaryScience Images• Coursework: Statistical Machine Learning, Artificial Intelligence, Image Processing, Applied Statistics, Data Mining,
Algorithms and Data Structures, Mathematical Programming and Optimization, Probability theory, Human Interface
Engineering, Multi-Agent System, System Control Theory, Complex Theory, etc.

Employment
Google DeepMind

July 2022 – (March 2025)

Student Researcher Program
Tokyo, Japan• Research Topic: Real-World Web Navigation with Large Language Models. Multimodal Web Navigation.
• Host: Heiga Zen, Izzeddin Gur , Collaborator: Kuang-Huei Lee, Aleksandra Faust, Shixiang Shane Gu, etc.

Matsuo Institute, Inc.

September 2019 – March 2022

Part-time Student Researcher
Tokyo, Japan• Research Topic: Reinforcement and Imitation Learning for Robotics. Object Detection for Autonomous Driving.

NABLAS Inc.

August 2018 – March 2019

Research InternTokyo, Japan• Research Topic: Probabilistic Generative Models for Anomaly Detection of Chemistry Industrial Process.

Selected Publication
Conference Publications• Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, Ian Fischer. A Human-Inspired Reading Agentwith Gist Memory of Very Long Contexts. ICML 2024.
• Izzeddin Gur∗ , Hiroki Furuta∗ , Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra
Faust. (∗ Equal Contribution). A Real-World WebAgent with Planning, Long Context Understanding, andProgram Synthesis . ICLR 2024 (Oral)• Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin
Gur. Multimodal Web Navigation with Instruction-Finetuned Foundation Models. ICLR 2024. (Also
accepted to ICLR 2023 Workshop on Multimodal Representation Learning as Spotlight, Project Page)• Open X-Embodiment Collaboration, et al (One of 170+ authors). Open X-Embodiment: Robotic Learning
Datasets and RT-X Models. ICRA 2024. (Project Page)

• Hiroki Furuta, Yusuke Iwasawa, Yutaka Matsuo, Shixiang Shane Gu. A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation. ICLR 2023 (notable-top-25%). (Also
accepted to NeurIPS 2022 Offline Reinforcement Learning Workshop as Contributed Talk, Project Page)• Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu. Generalized Decision Transformer for Offline Hindsight
Information Matching. ICLR 2022 (Spotlight). (Project Page)• Hiroki Furuta, Tadashi Kozuno, Tatsuya Matsushima, Yutaka Matsuo, Shixiang Shane Gu. Co-Adaptation ofAlgorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning. NeurIPS
2021.
• Hiroki Furuta, Tatsuya Matsushima, Tadashi Kozuno, Yutaka Matsuo, Sergey Levine, Ofir Nachum, Shixiang
Shane Gu. Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning. ICML 2021. (Also accepted to ICLR 2021 Workshop on Never-Ending RL as ContributedTalk)• Tatsuya Matsushima∗ , Hiroki Furuta∗ , Yutaka Matsuo, Ofir Nachum, Shixiang Gu (∗ Equal Contribution).
Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimization. ICLR 2021.
Preprints• Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo. Interpreting Grokked Transformers in
Complex Modular Arithmetic. (ICLR 2024 Workshop Bridging the Gap Between Practice and Theory in Deep
Learning, Oral)• Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, Izzeddin Gur. Exposing Limitations of Language Model
Agents in Sequential-Task Compositions on the Web. arXiv preprint arXiv:2311.18751, 2023.

Academic Activitites• Co-organizer for Ecological Theory of RL Workshop at NeurIPS 2021.
• Reviewer for International Conference on Learning Representations (ICLR), 2022† , 2023, 2024.
• Reviewer for Neural Information Processing Systems (NeurIPS), 2021, 2022† , 2023† .
• Reviewer for International Conference on Machine Learning (ICML), 2021, 2022, 2023, 2024.
• Reviewer for Transactions on Machine Learning Research (TMLR).
• Reviewer for Advanced Robotics (AR).
• Program Committee for Foundation Models for Decision Making Workshop at NeurIPS 2022, 2023.
†Highlighted as Top Reviewer

Awards and Honours• Forbes JAPAN 30 UNDER 30 2023, Forbes JAPAN, 2023
• The Japan Society for the Promotion of Science Research Fellow (DC1), April 2022 - March 2025 (2,400,000
JPY + research grants 1,100,000 JPY per year).
• Toyota/Dwango Scholarship for Advanced Artificial Intelligence Researcher, April 2021 - March 2022 (1,200,000JPY).
• Dean’s Award (Research), from Graduate School of Engineering, The University of Tokyo, 2022.
• Specially Selected Bachelor Thesis Award, from Department of Systems Innovation, School of Engineering, The
University of Tokyo, 2020. “Automated Identification of Gravel Particles in High-Resolution Images of Asteroids using
Convolutional Neural Networks”.
• Specially Selected Student Award, Global Consumer Intelligence Endowed Chair, The University of Tokyo, 2018.

Technical Skills
Languages: Python (Expert), MATLAB (Proficient), JavaScript (Proficient), ROS (Proficient), C (Proficient), R (Familiar), Java (Familiar)
Frameworks:JAX (Expert), TensorFlow (Expert), PyTorch (Expert), NumPy (Expert)

Teaching
World Models Endowed Course
Winter 2023
Lecturer
Tokyo, Japan• Held in Winter 2023 at The University of Tokyo (about 300 students).
• Giving lectures about Transformer, Foundation Models and Large Language Models.
Deep Reinforcement Learning Course
Summer 2020 – Spring 2023
Organizer, Lecturer and Teaching AssistantTokyo, Japan• Held in Summer 2020, Spring 2021, Fall 2021, and Spring 2023 at The University of Tokyo (about 150 students percourse).
• Giving lectures about Imitation Learning, Offline Reinforcement Learning, Unsupervised (Reward-Free) Reinforcement Learning, and Foundation Models.

Research InterestMy expertise lies in building real-world autonomous agents. Initially, I focused on data-driven deep reinforcement learning(RL). However, with the emergence of GPT-3, I pivoted to explore the revolutionary potential of Large Language Models(LLMs) for decision-making tasks, with a particular emphasis on the web navigation domain. LLMs offer a distinct advantage:
their inherent common sense and understanding of preferred behaviors eliminate the need for extensive training from scratch,
unlike RL agents.
I would like to realize a multimodal generalist agent to control all digital devices (web browsers, computers, and smartphones) across any platform via the graphical user interface. Throughout my successive works on web agents, I hypothesizethat the key to success would be (1) the development of automated actor and episode evaluator via prompting for scalable
data collection, and (2) finetuning the most capable model with dataset as large as possible (including both success andfailure).
In addition, such a generalist agent for controlling all digital devices can have huge impacts to the entire society aroundthe world from both research and product aspects. I have a strong passion for this direction.