Outline and Objectives

This interdisciplinary workshop focuses on the latest advancements in human-in-the-loop robot learning, which integrate human multi-modal input (e.g. natural language, gestures, and haptic interaction) and online feedback (e.g., rewards, corrections, and preferences) to improve robot manipulation performance, adapt to new situations and align better with human intentions. Recent breakthroughs in foundation models, such as Large Language Models (LLMs), Vision-Language Models (VLMs), and Vision-Language-Action Models (VLAs), provide unprecedented perception and reasoning capabilities. However, their effective integration into robotics remains an emerging and underexplored challenge, especially for manipulation planning and control. This workshop will explore how foundation models and human-in-the-loop learning can be synergistically integrated to effectively enhance the robot learning process through active and intuitive human participation. We will delve into three critical themes:

1. How can foundation models be leveraged for adaptive and generalizable learning for robot manipulation planning and control in com plex dynamic environments where robots continuously interact with the physical world?
2. What are the best practices for integrating real-time human feedback to refine learning processes and improve alignment with human intentions?
3. How to design adaptive learning frameworks to ensure safety and robustness in human-robot interactions and collaborations?

Topics of interest

Topics of interest include but are not limited to:

• Human-AI collaboration for robot learning
• Human-AI hybrid intelligence
• Foundation models for robot manipulation
• Transfer learning and fine-tuning of foundation models for robotic manipulation
• Knowledge representation and reasoning in robots
• Human feedback in robot learning
• Human-in-the-loop reinforcement learning
• Learning from demonstrations and corrections
• Interactive robot manipulation learning
• Multi-task robot learning
• Architectures and frameworks for human-in-the-loop learning
• Cognitive models for robot learning
• Adaptive human-robot interaction
• Safety and robustness in human-robot collaboration

Program

Invited Speakers

Dr. Huazhe Xu

Assistant Professor, Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, China; Co-founder of GalaxeaAI

Talk Title: Robot Generalization with the Power of Large Vision Models

Dr. Jianlan Luo

Chief Scientist, AGIBOT, Shanghai, China

Talk Title: Intelligent Physical Agents: High-Performance Human-in-the-loop Learning for Generalist Robots

Abstract

Robot learning has advanced significantly in recent years, positioning it as an effective tool for achieving scalable, flexible robotic autonomy. However, the large-scale real-world adoption of such learning-based robotic systems remains challenging, for which they must fulfill stringent real-world performance criteria to be viable. In this talk, I will describe algorithms and principles for building high-performance robotic learning systems. I’ll start by examining a range of high-performance "robot specialist" systems. These systems are tailored to address key deployment factors such as reliability, robustness, and cycle time, which has ultimately paved the way for their industrial adoption. I will then proceed to describe mechanisms to build “robot generalist” foundation models by bootstrapping the aforementioned robot specialists. To conclude, I'll further discuss the connections between these two types of systems and methods for enabling these systems to execute complex, long-horizon tasks suitable for open-world deployment.

Prof. David Navarro-Alarcon

Associate Professor, Department of Mechanical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China

Talk Title: Collective Shaping of Multi-Particle Systems

Abstract

This talk presents our group's results in the development of methods to autonomously manipulate multi-particle aggregates with robotic systems. We demonstrate how robots can effectively transport and shape multiple dispersed particles through coordinated pushing actions. Our solution combines two key innovations: intelligent task planning powered by Vision Language Models (VLMs), and precise trajectory execution using adaptive shape representation. The VLMs enable the system to understand tool affordances and execute non-prehensile manipulation primitives. For tracking and controlling particle aggregates, we introduce an efficient parametrization method using truncated Fourier series to represent the evolving contour of particle groups. This mathematical framework allows us to compute optimal trajectories while maintaining group cohesion during manipulation. Through live demonstrations and experimental results, we'll show how our system successfully performs complex manipulation tasks while adapting to changing particle distributions. This research advances the field of robotic manipulation and opens new possibilities for applications in materials handling, assembly operations, and even in food processing scenarios.

Prof. Alberto Sanfeliu

Full Professor of Computational Sciences and Artificial Intelligence, Universitat Politècnica de Catalunya, UPC, Barcelona, Spain.

Talk Title: Human-Robot-Avatar Interaction and Collaboration in Dynamic Environments using Foundational Models

Dr. Marta Lagomarsino

Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy

Talk Title: Understanding Humans, Empowering Robots: Online Assessment of Human Psycho-Physical State and Expert Feedback for Adaptive Robot Behaviour

Prof. Dongheui Lee

Full Professor, Faculty of Electrical Engineering and Information Technology, Technische Universität Wien, Austria

Talk Title: Robot Assembly Task Learning from Human Guide

Organizers

Jianzhuang Zhao, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: jianzhuang.zhao@iit.it
website: https://www.iit.it/it/people-details/-/people/jianzhuang-zhao

Xing Liu, Professor, Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, P.R.China
e-mail: xingliu@nwpu.edu.cn
website: https://www.researchgate.net/profile/Xing-Liu-33

Marta Lagomarsino, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: marta.lagomarsino@iit.it
website: https://www.iit.it/it/people-details/-/people/marta-lagomarsino

Francesco Tassi, Post-Doc, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: francesco.tassi@iit.it
website: https://www.iit.it/it/people-details/-/people/francesco-tassi

Shufei Li, Post-Doc, Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, P.R. China
e-mail: shufei.li@outlook.com
website: https://scholar.google.com/citations?user=CpCQmkwAAAAJ&hl=en&oi=ao

Gustavo Jose Giardini Lahr, Assistant Professor, Hospital Israelita Albert Einstein, San Paulo, Brazil
e-mail: gustavo.lahr@einstein.br
website: https://www.einstein.br/pesquisa/instituto-do-cerebro

Chenguang Yang, Professor, Fellow, IEEE, School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool, UK
e-mail: Chenguang.Yang@liverpool.ac.uk
website: https://www.liverpool.ac.uk/people/charlie-yang

Xuguang Lan, Professor, Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, P.R. China
e-mail: xglan@mail.xjtu.edu.cn
website: https://gr.xjtu.edu.cn/en/web/zeuslan

Panfeng Huang, Professor, Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, P.R.China
e-mail: pfhuang@nwpu.edu.cn
website: https://www.researchgate.net/profile/Panfeng-Huang

Arash Ajoudani, Senior Researcher Tenured, Human-Robot Interfaces and Interaction (HRII) Lab, Italian Institute of Technology (IIT), Genova, Italy
e-mail: arash.ajoudani@iit.it
website: https://www.iit.it/people-details/-/people/arash-ajoudani

This workshop engages three PhD students, Elena Merlo (Italian Institute of Technology, Italy), Zihao Liu (Northwestern Polytechnical University, China), and Zifan Wang (The Hong Kong University of Science and Technology, China) who will collaboratively design promotional campaigns and coordinate hands-on demonstrations during local arrangements.