Computational Linguistics Colloquium

Thursday, 16 May, 16:15, Seminar Room, Building 17

Prosody and error-handling in spoken interactions

A central claim in many conversational theories is that dialogue partners 'negotiate' on the information which is being exchanged. In order to make sure that information is grounded, they tend to follow a particular procedure, which involves exchanging specific forms of feedback that can be positive ('go on') or negative ('go back'). In this talk, I will argue that prosodic features can play an important role for monitoring an ongoing dialogue in this way. I will present three types of results. First, using data from two different corpora, i.e., a set of Japanese task-oriented dialogues (collaboration with Atsushi Shimojima, Hanae Koiso and Yasuhiro Katagiri) and a set of Dutch human-machine interactions (collaboration with Emiel Krahmer, Mariet Theune and Mieke Weegels), I will show that utterances that function as negative signals tend to differ prosodically from positive ones, in that they are often produced with more 'marked' settings. Second, I will show how these findings are potentially useful for automatic speech recognition and understanding, in that they can be exploited to improve the error handling component of a dialogue manager in a spoken dialogue system. This study is based on analyses of interactions of users with an American English spoken dialogue system (collaboration with Julia Hirschberg and Diane Litman). Finally, I will present results of a joint study with Björn Granström and David House (CTT, KTH, Stockholm, Sweden) in which we did an experiment with synthetic stimuli on the cue value of prosodic and visual features for signaling positive and negative feedback. For this experiment, we created audiovisual stimuli with a Talking Head, using the WaveSurfer software developed at CTT.

If you would like to meet with the speaker, please contact Bettina Braun.