Mirror Neurons and Interaction Lab

The motor system has been classically considered as an output module for planning and executing motor commands (Young, 1970) organized into a hierarchical structure that contains abstract motor intentions down to motor implementation parameters in dissociable cortical, subcortical, and spinal components (Graziano, 2006). For instance, on the abstract end of this continuum, neurophysiological research has shown that in monkey area F5, a premotor area, complex hand and mouth movements are represented (Rizzolatti et al., 1988). Neurons in this region discharge during the execution of goal-directed actions (i.e., grasping, manipulating, tearing, or holding), whereas they do not discharge during similar movements made with other purposes. This evidence suggests that F5 premotor neurons are able to generalize action goals.

In addition to their motor properties, however, several F5 neurons also show complex visual responses (visuomotor neurons). Canonical neurons discharge when the monkey observes graspable objects or executes grasping actions upon those objects (Murata et al., 1997). Mirror neurons discharge both when the monkey executes and observes another individual making the same action in front of it (Gallese et al., 1996). In humans, a growing body of neuroimaging evidence indeed indicates that vPM and posterior Broca's area (BA44) appear to have properties similar to monkey area F5 (Grafton et al., 1997; Fazio et al., 2009).

1. Action-Perception Network
2. Syntax of action
3. Speech-Perception Network
4. Automatic Speech Recognition
5. Music as a model of sensorimotor interactions

References
Fazio, P., Cantagallo, A., Craighero, L., D’Ausilio, A., Roy, A. C., Pozzo, T., Calzolari, F., Graniere, E., & Fadiga, L. (2009). Encoding of human action in Broca’s area. Brain, 132(Pt 7), 1980-8.
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119 ( Pt 2), 593-609.
Grafton, S. T., Fadiga, L., Arbib, M. A., & Rizzolatti, G. (1997). Premotor cortex activation during observation and naming of familiar tools. NeuroImage, 6(4), 231-6.
Graziano, M. (2006). The organization of behavioral repertoire in motor cortex. Annual Review of Neuroscience, 29, 105-34.
Murata, A, Fadiga, L., Fogassi, L., Gallese, V., Raos, V., & Rizzolatti, G. (1997). Object representation in the ventral premotor cortex (area F5) of the monkey. Journal of neurophysiology, 78(4), 2226-30.
Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., & Matelli, M. (1988). Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements. Experimental Brain Research. 1988;71(3):491-507.
Young, R.M. Mind, brain and adaptation in the nineteenth century. Cerebral localization and its biological context from Gall to Ferrier, Clarendon Press, Oxford (1970).

Action-Perception Network

The motor system has been shown to activate during observation of grasping actions but also by meaningless hand actions, suggesting that the motor cortex code the observed movements (Fadiga et al., 1995). This modulation was also shown to follow the time course of a reaching-grasping action (Gangitano et al., 2001) and anticipate the actor's muscle activation in a somatotopic manner (Borroni et al., 2005). Moreover, there is some indications that corticospinal modulation scales with the force required to execute the action, thus suggesting that the observer may infer actor's muscle contraction information out of visual kinematic cues (Senot et al., 2011). Therefore, it is plausible that the motor system might be able to extract at least some muscle control parameters by observing fine details of the actor's movement kinematics.

The Mirror Neurons and Interaction Lab studies the level of description the motor system is able to extract from the observed action.

Specific projects investigate:

• The level of description of the motor representation elicited via action observation
• The role played by the motor system in action recognition
• The role played by the motor system in object recognition

References
Borroni, P., Montagna, M., Cerri, G., & Baldissera, F. (2005). Cyclic time course of motor excitability modulation during the observation of a cyclic hand movement. Brain Research, 1065(1-2), 115-24.Fadiga, L., Craighero, L., & Olivier, E. (2005). Human motor cortex excitability during the perception of others’ action. Current Opinion in Neurobiology, 15(2), 213-8.
Gangitano, M., Mottaghy, F. M., & Pascual-Leone, A. (2001). Phase specific modulation of cortical motor output during mevement observation. Neuroreport, 12, 1489-92. Senot, P., D’Ausilio, A., Franca, M., Caselli, L., Craighero, L., & Fadiga, L. (2011). Effect of weight-related labels on corticospinal excitability during observation of grasping: a TMS study. Experimental Brain Research, 211(1), 161-7.

Syntax of action

Broca’s area plays a pivotal role in language, action and music (Fadiga, Craighero, D’Ausilio, 2009). Broca’s area involvement in language production has a long history, recently though, it’s role has been extended also to receptive functions (Friederici, 2002). Broca’s area is also at the center of a brain network for the encoding of action goals, either observed or executed. Finally Broca’s area was found implicated in the encoding of musical syntax much the way it does encode language structures (Koelsch, 2006). Patients with a lesion centered in Broca’s area were impaired in an action-sequencing task suggesting the intriguing possibility that Broca’s area could represent action’s syntactic rules rather than the basic motor program to execute them (Fazio et al., 2009). Actions are denoted by a relevant behavioral goal that, in order to be achieved, requires the composition of simpler movements. Simple movements do not necessarily posses a goal that motivates their execution. On the other hand, these might be part of very different actions, associated to different goals. Actions and simple movements are also composed of primitives representing the spatio-temporal sequence of muscle activations. These action hierarchies resemble the complex structures shown in other domains, such as music and language and, more interestingly, the experimental manipulation of these complex structures is associated with the activation of premotor and Broca’s areas, regardless of the domain of study (e.g., music or language). Hence, Broca’s area might be a center of a brain network encoding hierarchical structures regardless of their use in action, language or music.

The Mirror Neurons and Interaction Lab studies action syntax and the role played by Broca’s area in supramodal syntax encoding.

Specific projects investigate:

• EEG markers of action syntax violation
• Action-Language-Music shared resources investigation
• Supramodal syntax processing in aphasic patients (action sequencing tasks, knot making tasks)

References
Fadiga, L., Craighero, L., & D’Ausilio, A. (2009). Broca’s area in language, action, and music. Annals of the New York Academy of Sciences, 1169, 448-58.
Fazio, P., Cantagallo, A., Craighero, L., D’Ausilio, A., Roy, A. C., Pozzo, T., Calzolari, F., et al. (2009). Encoding of human action in Broca’s area. Brain, 132(Pt 7), 1980-8.
Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2), 78-84.
Koelsch, S., Fritz, T., V Cramon, D. Y., Müller, K., & Friederici, A. D. (2006). Investigating emotion with music: an fMRI study. Human Brain Mapping, 27(3), 239-50.

Speech-Perception Network

Speech perception has been shown to activate the motor system of the listener (Fadiga et al., 2002; and many others). This motor activation is somatotopically organized such that listening to tongue-produced sounds (i.e. [r] or [l] sounds) activates the tongue motor representation. However, it has been disputed that such motor activations may be simply correlated discharges that are not related with the real process of speech sound decoding (Toni et al., 2008). Recently, the selective interference with speech production centers proved effective in altering subject's performance in speech discrimination tasks (D'Ausilio et al., 2009). All together these data may suggest that the motor system is causally related to the perception of speech.

The Mirror Neurons and Interaction Lab studies the role of the motor system in speech perception and the conditions that favor its recruitment.

Specific projects investigate:

• The role played by the motor system in speech recognition
• The contextual factors favoring the recruitment of the motor system
• Computational models of the functional contribution of the motor system to speech perception

References
D’Ausilio, A., Pulvermüller, F., Salmas, P., Bufalari, I., Begliomini, C., & Fadiga, L. (2009). The motor somatotopy of speech perception. Current Biology, 19(5), 381-5.
Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifically modulates the excitability of tongue muscles : a TMS study. European Journal of Neuroscience, 15, 399-402.
Toni, I., de Lange, F. P., Noordzij, M. L., & Hagoort, P. (2008). Language beyond action. Journal of Physiology (Paris), 102(1-3), 71-9.

Automatic Speech Recognition (ASR)

While human beings show an excellent ability to understand one another's speech, independently of the speaker, the accent, the noise, etc., the robustness to speech variability of state-of-the-art ASR systems is still a great challenge. More reliable ASR systems may be achieved by (explicitly) using speech production knowledge (King et al., 2007). The motivation behind such approach is the well-known regular and largely invariant behavior of the vocal tract during speech production

By using simultaneous recordings of articulatory and acoustic streams during speech production it is possible to learn an Acoustic-Articulatory Mapping (AAM) function that allows to recover the articulatory information from speech when only speech is available (i.e., when the listener is recognizing someone else’s speech). The reconstruction of articulatory information can be seen as an extraction of robust features from the acoustic domain (Castellini et al., 2011). We consider this as the main (but not the only) motivation behind the use of measured articulatory information for ASR.

The Mirror Neurons and Interaction Lab investigates the contribution of the motor information to Automatic Speech Recognition.

Specific projects study:

• Novel AAM functions that goes through hierarchical representations of the acoustic and motor domains.
• The combined use of acoustic and motor information for “unsupervised” ASR systems (basic units such as sub-words, typically phonemes are automatically learned from data)
• Audio-Visual Speech Recognition

References
Castellini, C., Badino, L., Metta, G., Sandini, G., Tavella, M., Grimaldi, M., & Fadiga, L. (2011). The use of phonetic motor invariants can improve automatic phoneme discrimination. PloS one, 6(9), e24055.
King, S., Frankel, J., Livescu, K., McDermott, E., Richmond, K., & Wester, M. (2007). Speech production knowledge in automatic speech recognition. Journal of the Acoustical Society of America, 121(2), 723-42.

Music as a model of sensorimotor interactions

Coordinated action is one of the basic abilities for social interactions. Coordinated action might be conceived as a successful degree of synchrony or complementarity between actions performed by at least two individuals. Among human-specific abilities requiring inter-individual coordinative efforts, some have gained more attention than others, such as speech communication or action perception/production (Newman-Norlund et al., 2008). On the other hand, other complex behaviors might be similarly productive and yet less explored fields of research - such as music. Language, action and music indeed share several intriguing similarities that might be usefully exploited to foster further research on inter-individual communication dynamics (Fadiga, Craighero, D’Ausilio, 2009). Since musicians are a perfect model to study sensory-motor brain plasticity and organization (D’Ausilio et al., 2006), we use them as a model of how effective communication is shaped by efficient gestures coordination.

The Mirror Neurons and Interaction Lab studies how kinematic measures of ensemble musician’s action may shed some light on the complex dynamical network of interactions among them.

Specific projects investigate:

• Analysis of movement coordination among musicians (orchestras and quartets) via the Granger Causality method
• Analysis of movement communication among musicians (orchestras and quartets) via information theoretic approaches
• Audience facial temperature measures to quantify musical entrainment and emotional contagion

References
Newman-Norlund, R.D., van Schie, H.T., van Zuijlen, A.M., & Bekkering, H. (2007). The mirror neuron system is more active during complementary compared with imitative action. Nature Neuroscience. 2007 Jul;10(7):817-8.
D'Ausilio, A., Altenmüller, E., Olivetti Belardinelli, M., & Lotze, M. (2006). Cross-modal plasticity of the motor cortex while listening to a rehearsed musical piece. European Journal of Neuroscience. 2006 Aug;24(3):955-8.
Fadiga, L., Craighero, L., & D’Ausilio, A. (2009). Broca’s area in language, action, and music. Annals of the New York Academy of Sciences, 1169, 448-58.