Yuto Shibata

Yuto Shibata ๆŸด็”ฐ ๅ„ชๆ–—

PhD Student @ Keio University · Visiting @ CMU

I am a PhD student at Aoki Lab, Keio University, advised by Prof. Yoshimitsu Aoki and Dr. Mariko Isogawa.

I am staying at Carnegie Mellon University under Prof. Katerina Fragkiadaki, working on physics-based humanoid control — training simulated humanoid characters to perform human-like whole-body motion via reinforcement learning in physics simulation.

My research interests include computer vision, physics-based humanoid control, human motion modeling, robotics, and wireless human sensing.

CV Email Scholar GitHub Twitter LinkedIn

๐Ÿ“„ Publications

Sound-based Multi-Person 3D Pose Estimation
ECCV 2026
Sound-based Multi-Person 3D Pose Estimation
Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
arXiv coming soon
IEEE Open Journal of Signal Processing 2026
BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds
Yuto Shibata, Yusuke Oumi, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
arXiv
Sign-to-Speech Prosody Transfer
ICPR 2026 ๐Ÿ‡ซ๐Ÿ‡ท ๐Ÿ† Best Scientific Paper Award
Sign-to-Speech Prosody Transfer via Sign Reconstruction-based GAN
Toranosuke Manabe, Yuto Shibata, Shinnosuke Takamichi, Yoshimitsu Aoki
arXiv
CVPR 2026 ๐Ÿ‡บ๐Ÿ‡ธ
Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning
Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki
Project arXiv Code
UWB Localization
SenSys 2025 ๐Ÿ‡บ๐Ÿ‡ธ
Occlusion-Resilient UWB Localization Via Ceiling Reflection With Floor Anchor Installation
Yuto Shibata, Matthew Ishige, Hiroaki Murakami, Yoshihiro Kawahara
Paper
Formula-SED
ICASSP 2025 ๐Ÿ‡ฎ๐Ÿ‡ณ
Formula-Supervised Sound Event Detection: Pre-Training Without Real Data
Yuto Shibata, Keitaro Tanaka, Yoshiaki Bando, Hirokatsu Kataoka, Yoshimitsu Aoki
Project Paper
Federated Learning
NeurIPS 2024 Workshop ๐Ÿ‡จ๐Ÿ‡ฆ
Anatomical Three-Dimensional Style Transfer Enabling Efficient Federated Learning with Extremely Low Communication Costs
Yuto Shibata, Yasunori Kudo, Yohei Sugawara
arXiv
Acoustic Pose
BMVC 2024 ๐Ÿ‡ฌ๐Ÿ‡ง
Acoustic-based 3D Human Pose Estimation Robust to Human Position
Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
arXiv
Sound2Pose
CVPR 2023 ๐Ÿ‡จ๐Ÿ‡ฆ
Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals
Yuto Shibata, Yutaka Kawashima, Mariko Isogawa, Go Irie, Akisato Kimura, Yoshimitsu Aoki
Project Paper Code Dataset

๐Ÿ† Awards

Jul 2025
MIRU Excellent Paper Award at MIRU 2025 (The 28th Meeting on Image Recognition and Understanding)
Ranked 2nd out of 726 presentations. Paper: "Universal Human Pose Representation for Multi-Modal Active Sensing"
Aug 2024
MIRU Interactive Presentation Award at MIRU 2024 (The 27th Meeting on Image Recognition and Understanding)
Awarded to the author of a paper with an excellent interactive presentation.
Aug 2024
MIRU Student Encouragement Award at MIRU 2024 (The 27th Meeting on Image Recognition and Understanding)
Awarded to presenters of outstanding oral research presentations by student first authors.
Jul 2023
IPSJ Yamashita SIG Research Award
Awarded to presenters of particularly outstanding papers from workshops and symposia.
Jul 2022
Frontier Award at MIRU 2022 (The 25th Meeting on Image Recognition and Understanding)
Awarded to authors of pioneering papers with outstanding presentations. Paper: "้Ÿณ้Ÿฟไฟกๅทใ‚’็”จใ„ใŸไบบ็‰ฉใฎ3ๆฌกๅ…ƒๅงฟๅ‹ขๆŽจๅฎš"

๐ŸŽค Invited Talks

Jul 2023
Invited talk at MIRU 2023 — "Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals"