Yuto Shibata

PhD student at Keio University

I am a PhD student in the Aoki Lab at Keio University, advised by Prof. Yoshimitsu Aoki and Dr. Mariko Isogawa.

From June 2025 to June 2026, I was a visiting researcher at Carnegie Mellon University with Prof. Katerina Fragkiadaki, working on physics-based control of humans and humanoids, specifically human–human and human–humanoid interaction.

More broadly, I work on computer vision, human motion, robotics, and sensing people with sound and wireless signals.

CV  /  Email  /  Google Scholar  /  GitHub  /  X  /  LinkedIn

Yuto Shibata

Publications

Humanoid-human interaction (coming soon)
VLM-Guided Contact Retargeting and Generative Partner Modeling for Deployable Humanoid–Human Interaction
Yuto Shibata*, Kotaro Amaya*, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki
CoRL 2026
* equal contribution
Sound-based Multi-Person 3D Pose Estimation
Sound-based Multi-Person 3D Pose Estimation
Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
ECCV 2026
project / arXiv / code / dataset
BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds
Yuto Shibata, Yusuke Oumi, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
IEEE Open Journal of Signal Processing, 2026
project / paper / arXiv / code / dataset
Sign-to-Speech Prosody Transfer
Sign-to-Speech Prosody Transfer via Sign Reconstruction-based GAN
Toranosuke Manabe, Yuto Shibata, Shinnosuke Takamichi, Yoshimitsu Aoki
ICPR 2026  Best Scientific Paper Award
project / arXiv / code
Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning
Yuto Shibata, Kashu Yamazaki, Lalit Jayanti, Yoshimitsu Aoki, Mariko Isogawa, Katerina Fragkiadaki
CVPR 2026
project / arXiv / code
UWB Localization
Occlusion-Resilient UWB Localization Via Ceiling Reflection With Floor Anchor Installation
Yuto Shibata, Matthew Ishige, Hiroaki Murakami, Yoshihiro Kawahara
SenSys 2025
paper
Formula-SED
Formula-Supervised Sound Event Detection: Pre-Training Without Real Data
Yuto Shibata, Keitaro Tanaka, Yoshiaki Bando, Hirokatsu Kataoka, Yoshimitsu Aoki
ICASSP 2025
project / paper / code
Federated Learning
Anatomical Three-Dimensional Style Transfer Enabling Efficient Federated Learning with Extremely Low Communication Costs
Yuto Shibata, Yasunori Kudo, Yohei Sugawara
NeurIPS 2024 Workshop
arXiv
Acoustic Pose
Acoustic-based 3D Human Pose Estimation Robust to Human Position
Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
BMVC 2024
arXiv
Sound2Pose
Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals
Yuto Shibata, Yutaka Kawashima, Mariko Isogawa, Go Irie, Akisato Kimura, Yoshimitsu Aoki
CVPR 2023
project / paper / code / dataset

Awards

Jul 2025
MIRU Excellent Paper Award, MIRU 2025 2nd out of 726 presentations. "Universal Human Pose Representation for Multi-Modal Active Sensing"
Aug 2024
MIRU Interactive Presentation Award, MIRU 2024
Aug 2024
MIRU Student Encouragement Award, MIRU 2024
Jul 2023
IPSJ Yamashita SIG Research Award
Jul 2022
MIRU Frontier Award, MIRU 2022 "音響信号を用いた人物の3次元姿勢推定"

Talks

Jul 2023
Invited talk, MIRU 2023 "Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals"