Visual world eyetracking has become a primary source of data on how listeners integrate multiple sources of linguistic and contextual evidence during spoken language comprehension. In a visual world study, an eyetracker is used to monitor listeners gaze to various visually presented referents while they interpret speech. Researchers often consider the probability of gazing at various referents over time as a direct measure of information integration. However, there are questions about whether this practice is valid. First, is the relationship between comprehension and eye gaze unidirectional (with comprehension influencing gaze), or can gaze also influence comprehension (e.g., do listeners process a word differently depending upon where they happen to be looking when they hear the word)? Second, does the elevated probability of gazing at a referent reflect information integration, or just information access? Finally, what patterns of gaze should be taken as evidence for successful integration, and what patterns provide evidence for dissociation? I will argue that visual world data do not directly measure information integration, but can be made to do so if certain practices are followed. An experiment shows that under typical circumstances, comprehension influences gaze, but not vice versa. I will then discuss how Bayesian ideas about evidence accrual can be exploited for distinguishing access from integration, as well as for identifying patterns of integration versus dissociation during online processing. Bayesian concepts of prior evidence and diagnosticity can be directly mapped onto terms in a logistic regression analysis. The application of this probabilistic approach will be illustrated with data from an experiment that examined the influence of prior linguistic versus extralinguistic context on language processing. This analysis suggests that previous work using the visual world paradigm may have contributed to an overly optimistic view of the degree of interactivity in the spoken language comprehension system.

If you would like to meet with the speaker, please contact Maria Staudte.