The activity of the Lab is focused on the study of the neural plasticity that underlies the organization of the human sensorimotor system and its capacity to learn motor skills in the context of a complex environment. This also includes cognitive aspects in the neural control of movements because we believe in “embodied intelligence”: we see adaptive behavior as the emergent property of the bi-directional interactions of the nervous system with the body and the environment. This implies a continuous exchange of signals/energy between the nervous system, the body and the environment and it means that the motor neuronal input is shaped by the biophysics of the sensory organs and the motor neuronal output is transformed by the biomechanics of the body creating, at the same time, a large set of constraints & affordances.
Within this background the Lab investigates paradigms of motor learning and control by using visuo-haptic, dynamic virtual environments and measuring the motor response (in terms of kinematics and kinetics) and the corresponding neural correlates (in terms of muscle activity –EMG- and brain activity – EEG and NIRS).
The haptic part of the virtual reality system is provided by a several haptic robots (figure1) that have been designed in the lab (BdF in mono- or bi-manual configuration, IIT-Wrist robot, haptic grasp unit).
Figure 1: haptic devices designed at human behavioiur lab at IIT.
A-Braccio di Ferro BDF in bimanual configuration, B-Haptic Grasp unit, C- IIT Wrist Robot, D- 12DOFs Haptic Workstation for Bimanual manipulation of distal and proximal arms.
The emphasis is on closed-loop, adaptive systems in which the “assistance” provided by the robot during the execution of a task is modulated as a function of the actual performance and the ultimate goal of the task.
Two basic application areas are considered:
A) advanced Human-Robot Interfaces, based on a deep knowledge of the human operator and his/her computational capabilities;
B) self-adaptive robot-therapy of neurological patients (multiple sclerosis, post-ictus hemiplegia, Parkinson disease).
A few examples of the ongoing research activity are:
- Inter-limb interference during bimanual adaptation to dynamic environments
- Visuo-proprioceptive integration for compensating dynamic perturbations of wrist pointing movements
- Minimally assistive reaching strategy in robot therapy
- Adaptive robot assistance in tracking movements
Inter-limb interference during bimanual adaptation to dynamic environments
The two robots generate two independent curl viscous force fields that tend to deviate the hands laterally in the clockwise or counterclockwise directions. The focus of the study is on the patterns of interference between the two learning processes (for the left and right hands, respectively).
Figure 3: Movement trajectories at the beginning (left) and end (right) of the force field adaptation phase, for two typical subjects in the CW-CW group and CCW-CW group. Lines in grey and black denote, respectively, the left and right hand. Scale bar: 2 cm.
Visuo-proprioceptive integration for compensating dynamic perturbations of wrist pointing movements
The subjects perform pointing movements with their wrist, with various combinations of visual/kinesthetic perturbations input by means of the visual feedback and the wrist device.
Figure 4: IIT Wrist Robot.
It has 3 DoFs: Flexion/Extension, Pronation/Supination, Adduction/Abduction. One motor is used for F/E and P/S; two motors for A/A.
Figure 5: Wrist robot and Virtual Reality.
Visual representation of the target position and the actual pointing direction, as two circles, in relation with the manipulandum degrees of freedom. The two circles are visualized against a structured background (a stripe pattern). The overall picture could be fixed, with respect to the computer screen, or rotated in an harmonic manner. A: familiarization setup, B:kinesthetic disturbance; C: visual disturbance; D: visuo-kinesthetic disturbance.
Figure 6 example of trajectories for different target sets.
Black trajectories start from the center to each peripheral targets while grey trajectory are towards the center from the peripheral four targets and are mirrored below in the figure for graphical clarity sake; in (A) Familiarization (F), (B) visual (V), (C) kinesthetic (K) , (D) visuokinaesthetic (VK+), (E) visuo-kinaesthetic 180° phase lag (VK-) and (F) visuo-kinaesthetic 90° phase (VKP); target set (scale bar 2.5 cm).
Minimally assistive reaching strategy in robot therapy
The robot helps severe hemiplegic patients to perform reaching movements by generating assistive force fields of minimal intensity, in order to foster the emergence of active, goal oriented movements in a quasi-paralyzed limb.
Figure 7 A view from above of a subject holding the manipulandum.
The subject’s shoulders are strapped to a chair; the forearm is attached to a sliding support; the wrist is stabilized by means of a skateboard wrist brace and the hand grasp by means of a Velcro holder.
The targets are arranged on three layers: A, B, C. The C layer is placed in front of a virtual wall.
The basic sequence of target activation is A - C - B - A and it is repeated 3 9 7 9 3 = 63 times in a random order.
Note that the target distance in the figure is twice the real distance for graphical reaso.
Figure 8 Evolution of the speed profile from the initial to the final session, performed by a subject for the same movement (from the central position of the A layer to the central position of the C layer).
In general, all the subjects were characterized by a trend to quicker and smoother reaching movements in spite of the fact that the level of the assistance force was progressively reduced
Adaptive robot assistance in tracking movements
The robot helps the patient in tracking a moving target (simple Lissajous figure-of-eight) with an assistive force field that is modulated automatically as a function of the performance.
Figure 9 The Force field generator uses an impedance control scheme, with the direct drive of the robot actuators, in such a way to transmit to the handle a force vector computed as a function of the kinematic state of the robot (sampling frequency: 1 kHz).
The Adaptive Controller modulates the gain of the force field as a function of the evaluated performance, according to a non-monotonic training protocol. Continuous vectors: continuous time control; Dotted vectors: intermittent control.
Figure 10 Left - The top panel replicates the picture on the computer screen that includes the figure-of-eight path (black), the moving target (red circle), and the hand position (whitish car-shaped). The middle and bottom panels show the two tracking directions used in the experiments: CWright_CCWleft (blue), CCWrightCWleft (red). A – H are the eight control points used by the algorithm of performance evaluation. Right - The top set of 4 figures is related to a subject with a sever impairment level (FMA= 4). The bottom set of figures is related to a subject who is affected in a lighter way (FMA= 25). Blue denotes the clockwise-right/counterclockwise-left sequence; Red denotes the counterclockwise-right/clockwise-left sequence.
