Event Driven Perception for Robotics

Biological Intelligence arises from the tight interaction between the brain and the body, that co-evolved to optimise the resources needed for perception and action and their results. Part of the computation is performed by the body and sensors and the way information is acquired shapes how computation is performed. Efficient and robust artificial intelligence can emerge from similar processes, supporting the development of Embodied Neuromorphic Intelligence, that studies the synergies between body and computation.

Robots need to be aware of human presence and actions for safety and to engage in collaborative actions. We leverage on the low-latency and high temporal resolution of event-cameras to detect and track human beings and infer their actions online by high frequency human pose estimation, gaze tracking, blink detection and face pose estimation.

Perceiving the motion of a target is essential for a successful interaction with a dynamic environment, in which objects and the robot itself move simultaneously. Event-cameras allow to track fast-moving targets, without losing information “between frames,” as a moving object triggers events from all pixels along its spatio-temporal trajectory.