The Multiscale Brain Communication Line studies the neural mechanisms responsible for our capacity to communicate with others. This research is essential to develop new brain interfaces, specifically conceived for human use, to transduce and computationally decode neural signals.
To this purpose, we are studying the mechanisms by which the brain processes and understands the communicative behaviors of other individuals to efficiently decode the brain signals related to communicative intentions. We are applying innovative and biologically-compatible technologies to the problem of automatic speech and action recognition (Speech and Communication Team) and we are designing a new generation of brain electronic devices characterized by reduced invasiveness, improved resolution, ultrasensitivity and capability to record and stimulate brain regions (Neurotechnologies Team).
In summary, with a critical focus on translational methodologies (single unit recordings, Micro-ECoG, fMRI, EEG, TMS), our research goal is to advance knowledge on brain functioning to help building the next generation of brain-computer interfaces. The group research activities span from basic research to applied one along three main research lines:
- Design and fabrication of long-term stable neural interfaces with high signal-to-noise ratio and spatio-temporal resolution.
- Research on brain centers and circuits involved in action/speech understanding
- Research on new efficient methods for automatic speech recognition from audio and multimodal signals (e.g., audio-visual)
Collaborations with other National and international labs are fundamental. A particularly intense collaboration is running between CTNSC@UniFe and the Neurosurgery Unit of Udine Hospital (M. Skrap).
(F. Biscarini, M. Murgia, M. Bianchi, S. Carli, F. Ciarpella, E. Zucchini, E. Delfino, L. Fadiga)
Information transmission and processing within the brain takes place by electro-chemical signaling. For this reason, one of the most efficient ways to access this information is to connect electrodes to the brain. We are actively pursuing the manufacturing of long term implants for transduction and stimulation in animals and humans, along two directions: i) whole-organic multifunctional devices; ii) microelectrode arrays (MEAs), both integrated on flexible substrates. Developing new devices for translational operations implies also the investigation device biocompatibility, assessment of the immune reaction of the living system to the device implant, and the long-term efficacy, reliability, and robustness, in vitro and in vivo.
The Neurotechnologies Team is a multidisciplinary group devoted to developing novel neural/device interfaces. The team includes expertise in organic electronics, nanotechnology, material sciences, biotechnology and neurophysiology. Experimental activity is carried out in the following facilities: nanochemistry lab, organic electronics lab, microscopy lab, cells culture lab, histology lab, neurophysiology labs. The group access the in-house animal facility for in-vivo experiments.
Specifically, we are developing
1) new organic bioelectronics devices, based on a) conducting polymer electrodes, obtained either by direct electrodeposition on templates, or laser scan ablation on the thin film, and b) molecular and polymer semiconductors. This activity involves also the collaboration with scientists from the laboratory of Organic Electronics (LEO) at UNIMORE. The substrate is made of biocompatible and biodegradable polymer foils. These devices may surrogate MEA’s with the advantage of direct amplification of ultra-low potentials evoked. Other architectures we are developing instead release chemical cues, like drugs and growth factors, that are anchored to active materials, on command upon voltage/current, or light pulse triggers.
2) soft and flexible intracortical and epicortical microelectrode arrays with low impedance, higher charge transfer capability and biocompatibility thanks to the use of nanocomposite high surface area coatings and hydrogel encapsulation. In parallel, we develop techniques for the bio-hybrid electrode integration in the brain tissue by using autologous cells derived from the host organism. The acute and long-term biocompatibility, recording and stimulation performance of the developed devices is tested in-house by performing neurophysiological tests followed by immunofluorescence techniques.
- Di Lauro M., Benaglia S., Berto M., Bortolotti C. A., Zoli M., Biscarini F. (2018). Exploiting interfacial phenomena in organic bioelectronics: Conformable devices for bidirectional communication with living systems. Colloids Surf B Biointerfaces, 168:143-147.
- Carli S., Trapella C., Armirotti A., Fantinati A., Ottonello G., Scarpellini A., Prato M., Fadiga L., Ricci D. (2018). Biochemically Controlled Release of Dexamethasone Covalently Bound to PEDOT. Chem. Eur. J., DOI: 10.1002/chem.201801499
- Vomero M., Castagnola E., Ciarpella F., Maggiolini E., Goshi N., Zucchini E., Carli S., Fadiga L., Kassegne S., Ricci D. (2017). Highly Stable Glassy Carbon Interfaces for Long-Term Neural Stimulation and Low-Noise Recording of Brain Activity.Sci Rep, 7, 40332.
- Kosugi A., Takemi M., Tia B., Castagnola E., Ansaldo A., Sato K., Awiszus F., Seki K., Ricci D., Fadiga L., Iriki A., Ushiba J. (2018). Accurate motor mapping in awake common marmosets using micro-electrocorticographical stimulation and stochastic threshold estimation. J Neural Eng, 15(3), 036019.
- Di Lauro M., Berto M., Giordani S., Benaglia G., Schweicher D., Vuillaume C. A., Bortolotti Y. H., Geerts F., Biscarini F. (2017) Liquid‐Gated Organic Electronic Devices Based on High‐Performance Solution‐Processed Molecular Semiconductor. Adv Electron Mater, 3(9), 1700159.
- Giordani M., Berto M., Di Lauro M., Bortolotti C. A., Zoli M., Biscarini F. (2017). Specific Dopamine Sensing Based on Short-Term Plasticity Behavior of a Whole Organic Artificial Synapse. ACS Sensors, 2, 1756–1760.
(L. Badino, P. Cardellicchio, A. Casile, A. D’Ausilio, L. Fadiga, M. Galluccio, P.M. Hilt, M. Marini, S. Mukherjee, L. Pasa, T. Pozzo, R. Tavarone, A. Tomassini, R. Turrisi, R. Viaro)
Social interaction plays a central role in shaping our cognitive capacities during child development and throughout our life. Successful interaction requires the ability to send and receive information between individuals. In a sense, individuals might be conceptualized as processing units embedded within a multi-agent complex system and specialized for the interpretation of specific social messages. This fundamental capacity, ultimately enabling the emergence of cognition, is based on the function of a specific neural circuit allowing the fast and accurate decoding of others’ verbal and non-verbal messages during interaction.
The Speech and Communication Team investigates these aspects by employing a highly multidisciplinary approach including expertise in the fields of neuroscience, psychology, computer science and engineering and using a mixture of cutting-edge neurophysiological (micro-electrocorticography, single unit recordings, micro-stimulation, electroencephalography and transcranial magnetic stimulation), behavioral (eye-tracking, body motion capture) and computational techniques (machine learning, multivariate analyses, nonlinear data analyses).
Specifically, we investigate the brain mechanisms allowing us to understand and use verbal (Speech Perception Network) and non-verbal (Action Perception Network, Syntax of Action) communication in everyday life. In parallel, we design computational systems capable of human-like performance in understanding verbal (Automatic Speech Recognition, ASR) and non-verbal interactions (Non-verbal Sensorimotor Communication). We have been developing bio-inspired audio and audio-visual ASR systems that exploit knowledge of how speech is produced. Our ASR systems are designed for very challenging tasks, such as recognition of speech from distant speakers, in multi-talker environments (i.e., “cocktail party” task), and from speakers with dysarthria. Currently, some of our systems are integrated in robotic platforms (the ICub and R1 platforms) and mobile applications for specific patient categories (ALLSpeak and ECOMODE-Facilitator).
The goal of the Speech and Communication Team is to advance fundamental research on how the brain makes us capable of smooth social interactions and to design automatic brain-inspired systems that will allow natural human-machine interaction.
- Badino L., Canevari C., Fadiga L., Metta G. (2016). Integrating articulatory data in deep neural network-based acoustic modelling. Computer Speech and Language, vol. 36, pp. 173-195.
- Badino L. (2016). Phonetic context embeddings for DNN-HMM phone recognition. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2016, pp. 405-409.
- Badino L., Canevari C., Fadiga L., Metta G. (2014). An auto-encoder based approach to unsupervised learning of subword units. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 7634-7638
- Hilt P., Bartoli E., Ferrari E., Jacono M., Fadiga L., D’Ausilio A. (2017). Action observation effects reflect the modular organization of the human motor system. Cortex, 95, 104-118.
- Volpe G., D’Ausilio A., Badino L., Camurri A., Fadiga L., (2016). Measuring social interaction in music ensembles. Phil Trans Roy Soc B, 371(1693), 20150377.
- Bartoli E., D’Ausilio A., Berry J., Badino L., Bever T., Fadiga L. (2015). Listener-speaker perceived distance predicts the degree of motor contribution to speech perception. Cereb Cortex, 25(2), 281-288.
Our laboratories host state-of-the-art facilities for motion capture, neurophysiology, histology, cell culture, material science, electrochemical and electrical characterization.
- Neuronavigated Transcranial Magnetic Stimulation, High density Electroencephalography, Eye-tracking, Optical Motion Tracking and ElectroMagnetic Articulography
- Tethered and wireless multichannel neural recording and stimulation, Neuron Tracing Fluorescence Microscopy, Histology Sectioning Microtome, Primary Cells Culture facilities
- Galvanostats/Potentiostats, Electropolymerization, High Resolution Optical Microscopy, LCR meter, Electrometers, Dual Source Meters, Scanning Probe Microscope, Probe station, Contact Angle Measurement, Plasma, Profilometer;
- Processing/fabrication/characterization line for organic electronics materials and devices: high-vacuum chambers for metal evaporation and organic thin film sublimation; glove boxe with spin coating; dip coating and vertical deposition.
- Translational Neurophysiology on Humans – Miran Skrap - Neurochirurgia, Ospedale di Udine
- Ultraflexible electrode arrays - Guglielmo Fortunato – CNR-IMM – Roma
- Polyimide Based Ultraconformable arrays- Thomas Stieglitz - Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering - IMTEK, University of Freiburg, Germany
- Glassy carbon electrode arrays - Sam Kassegne - San Diego State University - USA
- Marmoset motor cortex mapping – Atsushi Iriki – RIKEN Brain Science Institute – Saitama – Japan
- Action perception and motor control - Thierry Pozzo – CTNSC@IIT and INSERM - U1093 Cognition, Action, and Sensorimotor Plasticity, Dijon, France
- Motor intention understanding - Cristina Becchio – RBCS@IIT and Università di Torino, Italy
- The shared syntax of action, music and language - Stefan Kölsch - University in Bergen, Norway
- Computational investigation of action primitives - Yiannis Aloimonos - University of Maryland, USA
- The syntax of action, objects affordances and language - Katerina Pastra - Cognitive Systems Research Institute and Institute for Language and Speech Processing, Athens, Greece
- Object affordances in humans and robots - Jose Santos-Victor - Instituto Superior Técnico, Institute of Systems and Robotics, Lisboa, Portugal
- The motor system in speech and language perception - Friedemann Pulvermüller - Institut für Deutsche und Niederländische Philologie, Berlin, Germany
- Automatic speech recognition for robotics- Giorgio Metta – iCub@IIT>
- Articulatory automatic speech recognition and acoustic inversion – Raman Arora – Center for Language and Speech Processing, Johns Hopkins University
- Machine learning techniques for automatic speech recognition – Massimiliano Pontil –Computational and Statistical Learning, IIT
- Automatic speech recognition for dysarthric speech – Frank Rudzicz – University of Toronto
- Goal-directed sensorimotor coordination in group interaction - Andrea Gaggioli and Giuseppe Riva – Università Cattolica di Milano and IRCCS Istituto Auxologico Italiano, Milano, Italy
- Sensorimotor signaling - Giovanni Pezzulo - Institute of Cognitive Sciences and Technologies (ISTC-CNR), Roma
- Sensorimotor entrainment to musical ensembles - Gualtiero Volpe and Antonio Camurri – University of Genova
- Complex social interaction in musical ensembles - Peter Keller - University of Western Sydney, Australia
- Impedance spectroscopy and device characterization-Henrique L. Gomes- Electronic Engineering-Universidade do Algarve, Faro, Portugal.
- Analysis of signals and molecular modelling- Francesco Zerbetto- Alma Mater Università di Bologna
- Organic electronics biosensors – Carlo Augusto Bortolotti – Università di Modena e Reggio Emilia, Modena.