Vera Demberg - Teaching
Summer Semester 2011
Seminar on Incremental Processing
Incremental processing in its most strict interpretation means that
information is fully and eagerly integrated with previously processed
material as soon as it is available, i.e. hypotheses about syntactic
structures and semantic roles are made as soon as each word is
encountered. On the syntax level, strict incrementality would for
example mean that each word is integrated into a syntactic structure
immediately, i.e. the processor would not use any stack to keep seen
words for later decision taking. The opposite of incremental
processing would be a machine that first reads in the whole sentence,
then decide on part-of-speech tags, then builds up a syntactic
structure bottom-up, and only starts assigning semantic roles once
syntactic structures are completed.
This course will look at incremental processing from two different
viewpoints: 1) is there evidence (or counter-evidence) for strict
incremental processing in human sentence processing? 2) Incrementality
from an NLP perspective: How can incremental processing be realized,
what are the advantages and disadvantages of processing incrementally,
and what applications can profit from incremental processing?
The new time
slot of the seminar is from 12:30s.t. to 14:00.
New: We're going to have the additional session on MAY 28th, 2-6pm.
Also, I confirmed that you can choose to do an oral exam for this
course. Oral exams will take place some time during the first two
weeks of August or the last week of June / first week of July.
Introduction: The strict competence hypothesis
and Johnson (1991) Memory Requirements and Local
Ambiguities of Parsing Strategies
Steedman, M. (1992).Grammars and Processors
(Tech. Rep. MS-CIS-92-52). University of Pennsylvania, Department of
Computer and Information Science. To appear in H. Kamp & C. Rohrer,
(Eds.),Current Theories in Natural Language Understanding. The
- Shieber and Johnson (1993) Variations in incremental interpretation.
- Edward Stabler (1991)
Avoid the pedestrian's paradox. In Principel-based parsing:
computation and psycholinguistics, pp. 199-237
Psycholinguistic Experiments on Incrementality
Topic PSY1: Evidence for/against strict incrementality (meaning that each
word in a sentence is fully integrated as soon as it is encountered;
Topic PSY2: Incrementality and Prediction (choose 2:)
Topic PSY3: Local Coherence Effects
Topic PSY4: Incremental Production?
Topic PSY5: Arguments for incrementality from a linguistic view point
(strong linguistics background required)
Tabor et al. (2004) Effects of merely local syntactic coherence on
(2006) The interaction of top-down and bottom-up statistics in the
resolution of syntactic category ambiguity.
Incrementality in NLP
Topic NLP1: Incremental Parsing with a PCFG
Topic NLP2: Incremental Parsing with Dependency Grammars
Roark (2001) Probabilistic top-down parsing and language
modeling. In Computational Linguistics, 27(2), pages 249-276.
Topic NLP3: Incremental Parsing with Tree-Adjoining Grammars (choose 3a
Topic NLP4: Incremental Semantic Parsing
Topic NLP5: Incremental Parsing with Cascaded / Hierarchical HMMs
(2004) "Incrementality in Deterministic Dependency Parsing"
Menzel (2009) "Towards radically incremental parsing of natural
Incremental processing in NLP Applications
Topic APP1: Dialogue systems / interactions with agents
Topic APP2: Machine translation (Familiarity with CCG advantageous for
Topic APP3: Trade-off between incrementality (speed) and accuracy:
Topic APP4: Speech Recognition
Skantze, G., &
Hjalmarsson, A. (2010). Towards Incremental Speech Generation in Dialogue Systems. In Proceedings of SIGdial. Tokyo,
Skantze, G., & Schlangen, D. (2009)
Incremental dialogue processing
in a micro-domain. In Proceedings of the 12th Conference of the
European Chapter of the Association for Computational Linguistics
(EACL-09). Athens, Greece.
Schlangen, D., & Skantze, G. (2009) A general,
abstract model of incremental dialogue processing. In Proceedings of
the 12th Conference of the European Chapter of the Association for
Computational Linguistics (EACL-09). Athens, Greece.
Schuler, Wu, Schwarz (2009) A Framework for Fast Incremental
Interpretation during Speech Decoding, Computational Linguistics, Vol
35, No. 3, pages 313-343