Timing in a multisensory context
Xinyue Wang, M.A.
The overall perspective of the project is to explore human timing behaviours with multi-modal inputs, ranging from abstract perceptual stimuli to real-life scenarios. Understanding how people time is crucial to explain and predict the perception of intervals when listening to and producing music. Moreover, the interaction of environmental factors such as music tempo and dance movements demands in particular a probe into the timing in the multisensory context.
To begin with, the project explored human timing models and their applications in music listening and production. The overview, detailed in the first review study, highlighted a key question that the project is trying to answer with corresponding experiments: whether our timing mechanism is a central or distributed system. The two hypotheses differ most critically on the presence of “modality effect” in timing performance. The central mechanism suggests a universal clock, which underlines consistent timing accuracy and variability across sensory modalities, while the distributed mechanism assumes that timing is a local function that varies by modality.
To explore the centrality of the human timing model, the second study attempted to find: 1. Whether a modality effect was present; 2. If yes, which between audition and vision dominated tempo judgment. A large proportion of past evidence based on the modality-specific timing view supports higher timing accuracy with audition over vision, tactile, and other modalities in timing and interval judgment. The studies, however, often adopted abstract stimuli such as dots/flashes (visual) and pure tones (auditory). The second study therefore presented point-light figures (PLF) derived from human movements and drum beats in a bisection paradigm. Participants exhibited higher reliance on PLF tempo than the auditory tempo, suggesting the dominance of visual inputs on the condition of its high predictability that was more easily attended to.
Study two adopted constant tempo with both modalities, while in real-life scenarios, however, we often encounter tempo changes. The third study intends to examine time perception with tempo-accelerating/-decelerating audiovisual stimuli. The study will be composed of two parts: 1) psychophysics measurements: tempo change detection threshold, temporal sensitivity, and duration judgment via a 2-forced-alternative choice paradigm; 2) aesthetic and affect evaluation: rating of the performance quality, the emotional valence, and arousal level of the performances via the pairwise comparison task. The experiments are expected to provide insight into the presence of modality effects in time perception when tempo changes, as well as the aesthetic and emotional endowments in this context.
The last study aims at investigating multisensory timing in music-accompanied Tai Chi practice. The study assimilates real-life scenarios where somatosensory and auditory (verbal instructions with or without music) inputs interact with the perceived durations and the passage of time. Altogether, the four studies progressively identify the centrality of the human timing mechanism by pinning down the modality effect, examine this effect with lab-controlled variations from constant to changing audiovisual tempo, and explore its application in real-life scenarios.
Publications
Wang, X., & Wöllner, C. (in press). Time as the ink that music is written with: a review of the internal clock models and their explanatory power in audiovisual perception. Jahrbuch der Deutschen Gesellschaft für Musikpsychologie [Yearbook of Music Psychology].
Wang, X., Wöllner, C., & Shi, Z. (submitted). Perceiving tempo in incongruent audiovisual presentations of human motion: Evidence for a visual driving effect.
Wang, X., Wöllner, C., & Shi, Z. (2020). Tempo in audiovisually inconsistent contexts: Visual information dominates perceived tempo. In the 5th International Conference on Time perspectives (ICTP), rescheduled to 2021.
Wang, X., Wöllner, C., & Shi, Z. (2019). How does time fly in inconsistent audiovisual contexts? An exploratory study with a temporal bisection paradigm. In 35. Tagungsband der Deutschen Gesellschaft für Musikpsychologie (pp. 88–90), Katholische Universität Eichstätt-Ingolstadt, Germany.
Wang, X., Shi, Z., & Wöllner, C. (2019). Temporal Entrainment Effect: Can Music Enhance our Attention Resolution in Time? In Bötsch, I. & von Georgi, R. (Eds.), 12th International Conference of Students of Systematic Musicology (SysMus19). Book of Abstracts (pp. 54–55). 10th-12th September 2019, Berlin, Germany.
Kontakt: Xinyue Wang