Visual search during cued language production
Speaker: Moreno Coco
Abstract:
Reference is the cognitive mechanism that binds real-world entities to their conceptual counterparts. Recent psycholinguistic studies using eye-tracking have shed light on the mechanisms that allows reference to be shared across linguistic and visual modalities. It is unclear, however, whether vision plays an active role during linguistic processing. In this talk, I present an eye-tracking study that investigates the interaction between language production and visual search using naturalistic scenes within a referentially ambiguous setting. We study production because of the active and independent role played by visual factors during the retrieval of information for linguistic encoding. Naturalistic scenes lead to more realistic visual responses; referential ambiguity reveals the interplay of visual and linguistic disambiguation strategies.
In a 2 X 2 eye-tracking experiment, participants were asked to look and describe a set of naturalistic scenes after being prompted with a cue word. Two visual factors (Minimal Vs Cluttered) on the density of visual information contained in the scene, were crossed with two linguistic factors (Animacy Vs Inanimacy) on the animacy of the cue word. Each scene contained always two visual referents (e.g. MAN/SPONGE) ambiguous in respect with the cue word(e.g. man, sponge). Preliminary results show that both clutter and animacy influence eye-movements responses. For example: 1) objects corresponding to the Animacy of the cue are fixated significantly more before naming, or 2) a Minimal scene makes the visual search more efficient, longer fixations to Inanimate objects when retrieving information to describe it. Our findings show that during sentence production, visual factors such as clutter do not only influence the visual search that retrieves objects to be encoded, but also interact with linguistic factors, such as the animacy of the cue word. This suggests a model in which visual and linguistic processing interact closely.
