Phonetic Convergence in Human-Machine Communication (CHI-C)
Funded by the German Research Foundation (DFG, 4/2016-3/2017 and 11/2017-3/2022) and internal funding
This research project is concerned with phonetic convergence in human-computer interaction. Phonetic convergence denotes an increase in similarity of the speech patterns of two interlocutors in communicative interaction. This phenomenon has been found and studied thoroughly in human-human interaction. There is now an increased interest to explore phonetic convergence in human-computer interaction (HCI) as well, since spoken dialogue systems are becoming more and more integrated in our everyday life.
In the first year of funding, we have demonstrated that human experimental subjects show patterns of phonetic convergence when being exposed to synthetic voices in a shadowing experiment (see figure below) that are similar to the convergence patterns observed in human-human interaction.
In the second funding phase, we extended the experimental approach to a Wizard-of-Oz scenario
and found that human interlocutors of our simulated spoken dialog system Mirabella
adapted their segmental pronunciation and question intonation to her
(see Aufgabe 1 and 4 below, respectively).
The project also aims to develop a quantitative model of phonetic convergence in spoken HCI,
build synthetic voices that are capable of adapting their speech output to the user’s speech patterns,
and derive implications for the design of conversational interfaces in speech technology.
Principal investigators
Bernd Möbius (UdS) and Ingmar Steiner (until 2020, now Fraunhofer IAIS)
Researchers
Iona Gessinger (Saarland Univ., PhD 2021)
Eran Raveh (Saarland Univ., PhD 2021)
Collaboration partners
Bistra Andreeva (UdS)
Michelle Cohn, Kristin Predeck, and Georgia Zellou (UC Davis)
Sébastien Le Maguer (Trinity College Dublin)
Antje Schweitzer (IMS Stuttgart)
Ingo Siegert (U Magdeburg)
Maya Twig (HU Jerusalem) and Oded Zehavi (U Haifa)
Student assistants
Nauman Fakhar, Jens Neuerburg, Johannah O'Mahony