Research [ 05 ] Embodied interaction system

Embodied Agentic Interaction System

An embodied interaction system brings social intelligence research into an experience people can use, guide and correct.

Published: 2026-08-21
Author: Chen Li

FIG. 01The system, as designed

The question: How can these capabilities come together in a system people can collaborate with and correct?
Where it stands: Decisions are made on the device. The robot keeps working with the network down, and raw video never leaves it. Which parts run on hardware, which run in simulation, and which are still being built is marked in the film itself.
What the figure is: A designed diagram of the whole system, with each part marked as running on hardware, running in simulation, or still being built. Running one behaviour model across more than one body is a design goal in that diagram, not something we do today.

We study how attention, understanding, expression and feedback can work together in an agentic interaction system. Robots are the concrete platform for this research; the goal is to support collaboration with people.

Attention, understanding and prediction, and expression have to run on the robot in milliseconds, with slower judgment left to the cloud. And the experience produced by a real interaction has to find its way back into the system.

That is why we start from the foundations that shape the whole system, and do the most basic work first.