Temporal dynamics in the multisensory brain

Abstract

In this work, I investigate the mechanisms with which the brain manages temporal coherence between sensory signals. An overview of relevant literature is given, and current theories about how sensory signals are combined in brain and behaviour are introduced. Key unknowns about the temporal dynamics of auditory-visual integration are identified and addressed within four investigations. In the first study, I assess whether cues to the onset of a auditory-visual pair affect sensitivity to their temporal asynchrony. It is shown that regularly timed cues shorten the temporal window of integration compared with irregular cues. This demonstrates that attention can affect how sensory signals are bound. In the second experiment, speech-like asynchronous stimuli are presented for an extended duration whilst perceptual simultaneity is monitored. In this manner, the time-course of temporal adaptation is tracked over time. Adaptation occurs when the presented asynchrony is visual-leading, but not when it is auditory-leading. This may suggest that temporal recalibration in the auditory-leading direction is not a consequence of adaptation. In the third investigation, the neural correlates of the time-course of temporal adaptation are measured. Increased activity in frontal and parietal areas occurred during perceptual asynchrony, this replicates previous work and further promotes that these regions provide top-down modulation of the mechanisms of temporal simultaneity. Increased activity is present in the posterior cingulate cortex whilst the brain is maintaining an adapted state, compared with during adaptation. This region may act as a con ict monitor and compensator for temporal asynchrony. Lastly, I investigate the extent to which a highly prevalent inhibitory neurotransmitter affects performance in a multisensory behavioural task. There is a possible correlation between the concentration of gamma-aminobutyric acid in the parietal lobe and the overall strength of integration effects. Finally, the impact and future directions of this work are discussed in the context of current literature.