advanced course, lecture, 2 SWS, 3 LP/ECTS, LSF/HIS 57831
B.Sc. CL; M.Sc. LST; B.Sc. und M.Sc. Inf; B.Sc. und M.Sc. CuKT
Tue 16.15-17.45, E1.3/0.01 (HS I)
None, but some background in computational linguistics, speech science, or signal processing will be advantageous.
Written exam at the end of the winter semester.
Speech synthesis is an essential component of any system relying on intuitive human-machine communication. Speech synthesis systems are also used in phonetic research to gain further insight into speech production and acoustic properties of speech. This advanced course offers an introduction to text-to-speech (TTS) synthesis systems and strategies. Various approaches to speech synthesis are presented, including formant synthesis, concatenative synthesis, and state-of-the-art corpus-based unit selection synthesis. Linguistic text analysis and natural language processing modules typically included in TTS systems are covered as well.
This course will be complemented in the summer semester 2012 by a research seminar with hands-on components.
Date | Topic | Slides | Assignment |
25.10. | Introduction Synthesis strategies |
Taylor 2009, ch. 3.1-3.4 | |
15.11. | Formant synthesis | Taylor 2009, ch. 13 | |
22.11. | Concatenative synthesis I | Taylor 2009, ch. 14 | |
29.11. | Concatenative synthesis II | Taylor 2009, ch. 14 | |
06.12. | Unit Selection synthesis | Taylor 2009, ch. 16 | |
13.12. | Statistical parametric (HMM) synthesis | Taylor 2009, ch. 15 | |
20.12. | Prosody: Duration and intonation modeling | Taylor 2009, ch. 9 | |
10.01. | Linguistic text analysis I | Taylor 2009, sec. 7.4, 8.3-8.5 Möbius 2001, ch. 3-6 |
|
17.01. | Linguistic text analysis II | (see 10.01.) | (see 10.01.) |
24.01. | TTS systems: Festival, Mary | Clark/etal:2007b pdf | |
31.02. | Concept-to-speech TTS system evaluation |
prepare questions | |
07.02. | Questions and answers | ||
Wed 15.02. 14:00-16:00 |
Written exam E1.3/0.01 (HS I) |
|
Registration for exam by Jan 27, 2012! |
Mon 16.04. 14:00-16:00 |
Written exam (resit) C7.2/5.09 |
|
Registration for exam by Mar 31, 2012! |
BibTex entries of all references (books, papers, URL):
@Book{Allen/etal:1987, author = {Allen, Jonathan and Hunnicutt, M.~Sharon and Klatt, Dennis}, title = {From Text to Speech: {T}he {MIT}alk System}, publisher = {Cambridge University Press}, year = 1987, address = {Cambridge}, annote = {tts, formant synthesis, textbook} } @Proceedings{Alter/etal:1997, title = {Concept to Speech Generation Systems---Proceedings of a Workshop in conjunction with 35th Annual Meeting of the Association for Computational Linguistics (Madrid, Spain)}, year = 1997, editor = {Kai Alter and Hannes Pirker and Wolfgang Finkler} } @InProceedings{Black/Taylor:1994, author = {Black, Alan W. and Taylor, Paul}, title = {C\textsc{hatr}: a generic speech synthesis system}, booktitle = {Proceedings of the International Conference on Computational Linguistics (Kyoto, Japan)}, volume = 2, year = 1994, pages = {983--986} } @Book{Breiman/etal:1984, author = {Breiman, Leo and Friedman, Jerome~H. and Olshen, Richard~A. and Stone, Charles~J.}, title = {Classification and Regression Trees}, publisher = {Wadsworth \& Brooks}, year = 1984, address = {Pacific Grove, CA} } @InCollection{Campbell:1992, author = {Campbell, W. Nick}, title = {Syllable-based segmental duration}, editor = {Bailly, G{\'e}rard and Beno{\^{\i}}t, Christian and Sawallis, Thomas R.}, booktitle = {Talking Machines: Theories, Models, and Designs}, publisher = {Elsevier}, year = 1992, address = {Amsterdam}, pages = {211--224} } @Article{Campbell:1999, author = {Campbell, W. Nick}, title = {A call for generic-use large-scale single-speaker speech corpora and an example of their application in concatenative speech synthesis}, journal = {Technical Publications, ATR Interpreting Telecommunications Research Laboratories}, year = 1999, pages = {42--47}, annote = {unit selection} } @Article{Carlson/Granstrom:1991, author = {Carlson, Rolf and Granstr{\"o}m, Bj{\"o}rn}, title = {Speech synthesis development and phonetic research---a personal introduction}, journal = {Journal of Phonetics}, year = 1991, volume = 19, pages = {3--8}, annote = {synthesis} } @Book{Clark/Yallop:1995, author = {Clark, John and Yallop, Colin}, title = {An Introduction to Phonetics and Phonology}, publisher = {Blackwell}, year = 1995, address = {Oxford}, edition = {2nd}, note = {1st edition 1990} } @Book{Clark/etal:2007a, author = {Clark, John and Yallop, Colin and Fletcher, Janet}, title = {An Introduction to Phonetics and Phonology}, publisher = {Blackwell}, year = 2007, address = {Oxford}, edition = {3rd}, annote = {textbook, phonetics} } @Article{Clark/etal:2007b, author = {Clark, Robert A.~J. and Richmond, Korin and King, Simon}, title = {Multisyn: Open-domain unit selection for the {Festival} speech synthesis system}, journal = {Speech Communication}, year = 2007, volume = 49, number = 4, pages = {317--330}, annote = {unit selection, synthesis, Festival, voice building, overview} } @Article{Dudley:1939a, author = {Homer Dudley}, title = {The vocoder}, journal = {Bell Labs Record}, year = 1939, volume = 17, pages = {122--126} } @Book{Dutoit:1997, author = {Dutoit, Thierry}, title = {An Introduction to Text-to-Speech Synthesis}, publisher = {Kluwer}, year = 1997, address = {Dordrecht}, annote = {Review by Eileen Fitzpatrick in CL 24 (2), 1998, 322--323}, annote = {textbook, synthesis, tts} } @Article{Fant:1953, author = {Fant, Gunnar}, title = {Speech communication research}, journal = {Ing. Vetenskaps Akad. Stockholm}, year = 1953, volume = 24, pages = {331--337}, annote = {formant synthesis, OVE I} } @Book{Fant:1960, author = {Fant, Gunnar}, title = {Acoustic Theory of Speech Production}, publisher = {Mouton}, year = 1960, address = {The Hague} } @InCollection{Fujisaki:1983, author = {Fujisaki, Hiroya}, title = {Dynamic characteristics of voice fundamental frequency in speech and singing}, booktitle = {The Production of Speech}, publisher = {Springer}, year = 1983, editor = {MacNeilage, Peter F.}, address = {New York}, pages = {39--55} } @Article{Fujisaki:1987, author = {Fujisaki, Hiroya}, title = {A note on the physiological and physical basis for the phrase and accent components in the voice fundamental frequency contours}, journal = {Annual Bulletin of the Research Institute for Logopedics and Phoniatrics (Tokyo)}, year = 1987, volume = 21, pages = {165--175} } @Article{Fujisaki/etal:1979b, author = {Fujisaki, Hiroya and Hirose, Keikichi and Ohta, K.}, title = {Acoustic features of the fundamental frequency contours of declarative sentences in {J}apanese}, journal = {Annual Bulletin of the Research Institute for Logopedics and Phoniatrics (Tokyo)}, year = 1979, volume = 13, pages = {163--172} } @Book{Gibbon/etal:1997, editor = {Gibbon, Dafydd and Moore, Roger and Winski, Richard}, title = {Handbook of Standards and Resources for Spoken Language Systems}, publisher = {Mouton de Gruyter}, year = 1997, address = {Berlin}, annote = {ISBN 3-11-015366-1; Review by Jan P. H. van Santen in Computational Linguistics 24 (3), 512--515}, annote = {EAGLES} } @Article{Holmes:1973, author = {Holmes, John N.}, title = {The influence of glottal waveform on the naturalness of speech from a parallel formant synthesizer}, journal = {IEEE Transactions AU}, year = 1973, volume = 21, pages = {298--305} } @Article{House/etal:1965, author = {House, Arthur~S. and Williams, Carl and Hecker, Michael H.~L. and Kryter, Karl~D.}, title = {Articulatory testing methods: Consonantal differentiation with a closed-response set}, journal = {Journal of the Acoustical Society of America}, year = 1965, volume = 37, pages = {158--166}, annote = {evaluation, assessment}, annote = {modified rhyme test (MRT)} } @InProceedings{Hunt/Black:1996, author = {Andrew J. Hunt and Alan W. Black}, title = {Unit selection in a concatenative speech synthesis system using a large speech database}, booktitle = {Proceedings of the {IEEE} International Conference on Acoustics and Speech Signal Processing (M{\"u}nchen, Germany)}, year = 1996, volume = 1, pages = {373--376}, annote = {unit selection} } @InProceedings{Iwahashi/Sagisaka:1993, author = {Iwahashi, Naoto and Sagisaka, Yoshinori}, title = {Duration modelling with multiple split regression}, booktitle = {Proceedings of the European Conference on Speech Communication and Technology (Berlin, Germany)}, year = 1993, volume = {??}, pages = {329--332}, annote = {tts, prosody, duration} } @InProceedings{Jekosch:1993, author = {Jekosch, Ute}, title = {Speech quality assessment and evaluation}, booktitle = {Proceedings of the European Conference on Speech Communication and Technology (Berlin, Germany)}, year = 1993, volume = 2, pages = {1387--1394}, annote = {tts, evaluation, assessment}, annote = {cluster identification (CLID) test} } @Article{Kaplan/Kay:1994, author = {Kaplan, Ronald and Kay, Martin}, title = {Regular models of phonological rule systems}, journal = {Computational Linguistics}, year = 1994, volume = 20, pages = {331--378} } @Book{Kempelen:1791, author = {Kempelen, Wolfgang von}, title = {{Mechanismus der menschlichen Sprache nebst Beschreibung einer sprechenden Maschine}}, publisher = {J. V. Degen}, year = 1791, address = {Wien}, note = {Facsimile Neudruck, 1970, der Ausgabe Wien 1791 mit einer Einleitung von Herbert E. Brekle und Wolfgang Wildgen. Friedrich Frommann, Stuttgart} } @Article{Klatt:1980a, author = {Klatt, Dennis H.}, title = {Software for a cascade/parallel formant synthesizer}, journal = {Journal of the Acoustical Society of America}, year = 1980, volume = 67, number = 3, pages = {971--980}, annote = {synthesis, Klatt80 synthesizer} } @Article{Kratzenstein:1782, author = {Kratzenstein, Christian Gottlieb}, title = {Sur la naissance de la formation des voyelles}, journal = {Journal de Physique}, year = 1782, volume = 21, pages = {358--380}, note = {French translation of: Tentamen coronatum de voce, Acta Acad. Petrog., 1780} } @Book{Leeuwen:1990, editor = {van Leeuwen, J.}, title = {Handbook of Theoretical Computer Science}, publisher = {Elsevier, Amsterdam; MIT Press, Cambridge, MA}, year = 1990, volume = {B}, annote = {FST, fsm; ISBN 0 444 88074 7} } @Book{Mobius:1993a, author = {M{\"o}bius, Bernd}, title = {{Ein quantitatives Modell der deut\-schen Intona\-tion---Analyse und Synthese von Grund\-frequenz\-ver\-l{\"a}ufen}}, publisher = {Niemeyer}, year = 1993, OPTnumber = 305, OPTseries = {Linguistische Arbeiten}, address = {T{\"u}bingen} } @Article{Mobius:1999, author = {M{\"o}bius, Bernd}, title = {{The Bell Labs German text-to-speech system}}, journal = {Computer Speech and Language}, year = 1999, volume = 13, pages = {319--358}, annote = {tts, german} } @Book{Mobius:2001a, author = {M{\"o}bius, Bernd}, title = {German and Multilingual Speech Synthesis}, publisher = {University of Stuttgart}, year = 2001, series = {Arbeitspapiere des Instituts f{\"u}r Maschinelle Sprachverarbeitung (Univ. Stuttgart), AIMS 7 (4)}, pages = {1--300}, annote = {synthesis, textbook, tts} } @InProceedings{Mobius/Santen:1996, author = {M{\"o}bius, Bernd and van Santen, Jan}, title = {Modeling segmental duration in {G}erman text-to-speech synthesis}, booktitle = {Proceedings of the International Conference on Spoken Language Processing (Philadelphia, PA)}, year = 1996, volume = 4, pages = {2395--2398} } @Article{Mohri:1997, author = {Mohri, Mehryar}, title = {Finite-state transducers in language and speech processing}, journal = {Computational Linguistics}, year = 1997, volume = 23, number = 2, pages = {269--311} } @InCollection{Mohri/etal:1998, author = {Mohri, Mehryar and Pereira, Fernando and Riley, Michael}, title = {A rational design for a weighted finite-state transducer library}, booktitle = {Lecture Notes in Computer Science 1436}, publisher = {Springer}, year = 1998, editor = {Wood, D. and Yu, S.}, address = {New York}, pages = {144--158}, annote = {fsm} } @InProceedings{Nikleczy/Olaszy:2003, author = {Nikl{\'e}czy, P. and Olaszy, Gabor}, title = {A reconstruction of {Farkas Kempelen's} speaking machine}, booktitle = {Proceedings of the European Conference on Speech Communication and Technology (Geneva, Switzerland)}, year = 2003, pages = {2453--2456}, annote = {Kempelen, speaking machine, synthesis} } @Article{Ohman:1967, author = {{\"O}hman, Sven E. G.}, title = {Word and sentence intonation: a quantitative model}, journal = {Speech Transmission Laboratory---Quarterly Progress and Status Report}, year = 1967, volume = {2--3}, pages = {20--54} } @Article{Ohman/Lindqvist:1966, author = {{\"O}hman, Sven E. G. and Lindqvist, Jan}, title = {Analysis-by-synthesis of prosodic pitch contours}, journal = {Speech Transmission Laboratory---Quarterly Progress and Status Report}, year = 1966, volume = 4, pages = {1--6} } @InCollection{Olive/etal:1998, author = {Olive, Joseph and van~Santen, Jan and M{\"o}bius, Bernd and Shih, Chilin}, title = {Synthesis}, booktitle = {Multilingual Text-to-Speech Synthesis: The {B}ell {L}abs Approach}, editor = {Richard Sproat}, publisher = {Kluwer}, year = 1998, address = {Dordrecht}, chapter = 7, pages = {191--228} } @PhdThesis{Pierrehumbert:1980, author = {Pierrehumbert, Janet}, title = {The phonology and phonetics of {E}nglish intonation}, school = {MIT}, address = {Cambridge, MA}, year = 1980 } @InProceedings{Pols/etal:1992, author = {Pols, Louis C. W. and SAM-Partners}, title = {Multi-lingual synthesis evaluation methods}, booktitle = {Proceedings of the International Conference on Spoken Language Processing (Banff, Alberta)}, year = 1992, volume = 1, pages = {181--184}, annote = {tts, evaluation} } @Book{PompinoMarschall:1995, author = {Pompino-Marschall, Bernd}, title = {{Einf{\"u}hrung in die Phonetik}}, publisher = {de Gruyter}, year = 1995, address = {Berlin} } @InCollection{Riley:1992, author = {Riley, Michael D.}, title = {Tree-based modeling for speech synthesis}, editor = {Bailly, G{\'e}rard and Beno{\^{\i}}t, Christian and Sawallis, Thomas R.}, booktitle = {Talking Machines: Theories, Models, and Designs}, publisher = {Elsevier}, year = 1992, address = {Amsterdam}, pages = {265--273} } @InProceedings{Santen:1992d, author = {van Santen, Jan P.~H.}, title = {Diagnostic perceptual experiments for text-to-speech system evaluation}, booktitle = {Proceedings of the International Conference on Spoken Language Processing (Banff, Alberta)}, year = 1992, volume = 1, pages = {555--558}, annote = {tts, evaluation} } @Article{Santen:1993b, author = {van Santen, Jan P.~H.}, title = {Exploring \textit{N}-way tables with sums-of-products models}, journal = {Journal of Mathematical Psychology}, year = 1993, volume = 37, number = 3, pages = {327--371} } @Article{Santen:1993c, author = {van Santen, Jan P.~H.}, title = {Perceptual experiments for diagnostic testing of text-to-speech systems}, journal = {Computer Speech and Language}, year = 1993, volume = 7, pages = {49--100} } @InCollection{Santen:1998a, author = {van Santen, Jan P. H.}, title = {Timing}, booktitle = {Multilingual Text-to-Speech Synthesis: The {B}ell {L}abs Approach}, editor = {Sproat, Richard}, publisher = {Kluwer}, year = 1998, address = {Dordrecht}, pages = {115--139}, annote = {tts, duration} } @InCollection{Santen:1998b, author = {van Santen, Jan P. H.}, title = {Evaluation}, booktitle = {Multilingual Text-to-Speech Synthesis: The {B}ell {L}abs Approach}, editor = {Sproat, Richard}, publisher = {Kluwer}, year = 1998, address = {Dordrecht}, pages = {229--244}, annote = {tts, evaluation} } @InCollection{Schweitzer/etal:2006a, author = {Schweitzer, Antje and Braunschweiler, Norbert and Dogil, Grzegorz and Klankert, Tanja and M{\"o}bius, Bernd and M\"{o}hler, Gregor and Morais, Edmilson and S{\"a}uberlich, Bettina and Thomae, Matthias}, title = {Multimodal speech synthesis}, booktitle = {{SmartKom}: Foundations of Multimodal Dialogue Systems}, editor = {Wahlster, Wolfgang}, publisher = {Springer}, year = 2004, pages = {411--435}, annote = {sk, synthesis, unitsel} } @InProceedings{Silverman/etal:1992, author = {Silverman, Kim and Beckman, Mary and Pitrelli, John and Ostendorf, Mari and Wightman, Colin and Price, Patti and Pierrehumbert, Janet and Hirschberg, Julia}, title = {{ToBI: A standard for labelling English prosody}}, booktitle = {Proceedings of the International Conference on Spoken Language Processing (Banff, Alberta)}, year = 1992, volume = 2, pages = {867--870} } @InProceedings{Spiegel/etal:1988, author = {Spiegel, Murray and Altom, Mary Jo and Macchi, Marian and Wallace, Karen}, title = {Using a monosyllabic test corpus to evaluate the intelligibility of synthesized and natural speech}, booktitle = {Proceedings of the American Voice I/O Systems Conference}, year = 1988, OPTpages = {??}, annote = {tts, evaluation, assessment}, annote = {Bellcore test} } @InProceedings{Spiegel/etal:1989, author = {Murray Spiegel and Mary Jo Altom and Marian Macchi and Karen Wallace}, title = {A monosyllabic test corpus to evaluate the intelligibility of synthesized and natural speech}, booktitle = {Proceedings of the ESCA Workshop on Speech Input / Output Assessment and Speech Databases}, editor = {Institute of Phonetic Sciences, Amsterdam}, year = 1989, address = {Noordwijkerhout, The Netherlands}, pages = {??} } @TechReport{Sproat:1995a, author = {Sproat, Richard}, title = {{LEXTOOLS}: {T}ools for finite-state linguistic analysis}, institution = {AT\&T Bell Laboratories}, year = 1995, note = {11522-951108-10TM} } @InProceedings{Sproat:1995b, author = {Sproat, Richard}, title = {A finite-state architecture for tokenization and grapheme-to-phoneme conversion in multilingual text analysis}, booktitle = {{From text to tags---Issues in multilingual language analysis. Proceedings of the ACL SIGDAT Workshop}}, year = 1995, address = {University College, Belfield, Dublin, Ireland} pages = {65--72} } @Book{Sproat:1998, title = {Multilingual Text-to-Speech Synthesis: The {B}ell {L}abs Approach}, editor = {Sproat, Richard}, publisher = {Kluwer}, year = 1998, address = {Dordrecht}, annote = {ISBN 0-7923-8027-4; Review by Douglas O'Shaughnessy in Computational Linguistics 24(4), 1998, 656--658}, annote = {textbook, tts, synthesis} } @Article{Syrdal/etal:1997, author = {Syrdal, Ann K. and Conkie, Alistair and Stylianou, Yannis and Schroeter, J{\"u}rgen and Garrison, L.F. and Dutton, D.}, title = {Voice selection for speech synthesis}, journal = {Journal of the Acoustical Society of America}, year = 1997, volume = 102, number = 5, pages = 3191, note = {(abstract)}, annote = {tts} } @InProceedings{Syrdal/etal:1998b, author = {Syrdal, Ann K. and Conkie, Alistair and Stylianou, Yannis}, title = {Exploration of acoustic correlates in speaker selection for concatenative synthesis}, booktitle = {Proceedings of the International Conference on Spoken Language Processing (Sydney, Australia)}, year = 1998, volume = 6, pages = {2743--2746}, annote = {tts, inventory, speaker} } @Book{Taylor:2009, author = {Taylor, Paul}, title = {Text-to-Speech Synthesis}, publisher = {Cambridge University Press}, year = 2009, annote = {textbook, synthesis, tts} } @Book{Ungeheuer:1962a, author = {Ungeheuer, Gerold}, title = {{Elemente einer akustischen Theorie der Vokalartikulation}}, publisher = {Springer}, year = 1962, address = {Berlin} } @TechReport{Voiers/etal:1972, author = {Voiers, William and Sharpley, A. and Hehmsoth, C.}, title = {Research on diagnostic evaluation of speech intelligibility}, institution = {Cambridge Research Laboratories}, year = 1972, note = {Research Report AFCRL-72-0694}, annote = {evaluation, assessment}, annote = {evaluation, diagnostic rhyme test (DRT)} } @Article{Voiers:1983, author = {Voiers, William~D.}, title = {Evaluating processed speech using the {Diagnostic Rhyme Test}}, journal = {Speech Technology}, year = 1983, number = {Jan/Feb}, pages = {30--39}, annote = {evaluation, diagnostic rhyme test (DRT)} } @Article{Wheatstone:1838, author = {Wheatstone, Charles}, title = {Art. II. -- 1. On the vowel sounds, and on Reed Organ Pipes. By Robert Willis [...] 2. Le Méchanisme de la Parole, suivi de la Description d'une Machine Parlante. Par M. de Kempelen [...] 3. C.G. Kratzenstein. Tentamen Coronatum de Voce. [...]} journal = {The London and Westminster Review}, year = 1838, pages = {27ff.}, annote = {Review of von Kempelen, Kratzenstein, Willis} }