Computational Psycholinguistics

Lecturer: Matthew Crocker
Format: Lectures with Tutorials (4 SWS, 6LP)
Programme: MSc in LS&T, Diplom CL, BA in CL

Times: Mon 14-16 (Lecture), Wed 14-16 (Tutorial)
Location: Room U.15, Building C7.1
Location: Tutorials will usually be held sometimes in the CIP-Raum
Language of Instruction: English
Course begin: Monday, 19 October 2009

Course Contents

This course will discuss current computational models of human language processing. We will consider both how computational linguistics can inform the development of psycholinguistic theories, and also how computational models can account for and explain (experimentally) observed human language processing behaviour. The course will begin with an introduction to psycholinguistic research, summarising both the key observations about human language understanding, and also presenting central theoretical debates including issues such as modularity, incrementality, and the psychological status of linguistic principles and representations. We will then consider a number of computational models of lexical and sentence level processing and language acquisition. The models covered exploit symbolic, probabilistic, connectionist, and also 'hybrid' computational mechanisms.

Week	Monday	Wednesday
1	Introduction: modularity, competence-performance, incrementality, ambiguity resolution and garden-path phenomena. [Lecture1.pdf]	Tutorial 1: Intro to COGENT, experiment modeling
2	Parsing and psychological reality: incrementality, memory load, and disambiguation. Implementing top-down, shift-reduce, and left-corner models. [Lecture2.pdf]	Tutorial 2: Using the COGENT Framework for Sentence Parsing. Shift-reduce parsing.
3	Lexical processing: Top-down and bottom-up models of spoken lexical access. [Lecture3.pdf]	Tutorial 3: Parsing in Cogent.
4	Syntactic processing 1: Syntactic theories of parsing and long-distance dependencies. [Lecture4.pdf]	Tutorial 4: Continuation of Tutorial 3.
5	Syntactic processing 2: Reanalysis and monotonic parsing. [Lecture5.pdf]	Tutorial 5: Back-tracking & top-down parsing.
6	Tutorial 6: COGENT: Left-corner parsing.	Tutorial 7: COGENT: Memory Load.
7	Tutorial 8: COGENT: Memory Load.	Tutorial 9: COGENT projects.
8	Probabilistic Models 1: Probabilistic models of lexical processing 1. [Lecture6.pdf]	Probabilistic Models 2: Probabilistic models of syntactic processing 2. [Lecture7.pdf]
9	Interactive Models: The interactive-activation model, and the competition-integration model. [Lecture8.pdf]	Tutorial: TBA
	Christmas Break	Christmas Break
10	Introduction to Connectionist Models. [Lecture9.pdf]	Tlearn Tutorial 1: Introduction to tLearn.
11	Learning Neural Networks: Phonology and Morphology [Lecture10.pdf]	Tlearn Tutorial 2: Reading aloud.
12	Simple recurrent networks: processing sequences [Lecture11.pdf]	Tlearn Tutorial 3: The English past-tense.
13	SRNs: Learning syntax, and starting small [Lecture12.pdf]	Course Overview, Q & A [ CourseReview.pdf]
14	Tlearn Tutorial 4: More SRNs	Exam: Wednesday, February 3, 2010 @ 14:00 *sharp* Location: TBA Example Exam: [CPMockExam.pdf]

Tutorials
For details about the tutorials, please keep an eye on Garance's tutorial page.

Software
The course will use two systems for experimenting with computational models of human language processing.

• Cogent: for symbolic cognitive models and simulations
  
• Tlearn: for simple connectionist models

Both systems are freely available, for Mac OS, Linux, and Windows operating systems.

Essential Readings

• Matthew Crocker. Computational Psycholinguistics. In: Clark, Fox & Lappin (eds), Handbook of Computational Linguistics and Natural Language Processing, Blackwell, London, UK, 2010.
• Matthew Crocker. Mechanisms for Sentence Processing. In: Garrod & Pickering (eds), Language Processing, Psychology Press, London, UK, 1999.
• Dan Jurafsky. Probabilistic Modeling in Psycholinguistics. In Bod et al (eds.). Probabilistic Linguistics. The MIT Press, 2003.
• P. McLeod, K. Plunkett and E. T. Rolls (1998). Introduction to Connectionist Modelling of Cognitive Processes. Oxford University Press. Chapters: 1-5, 7, 9.
• Ken McRae, Michael Spivey-Knowlton, Michael Tanenhaus. Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension. Journal of Memory and Language, 38, 283ñ312 (1998)
• K. Plunkett and J. Elman (1997). Exercises in rethinking innateness: A Handbook for Connectionist Simulations. MIT Press. Chapters: 1-8, 11, 12.

Additional Literature

• N. Chater and M. Christiansen (1999). Connectionism and natural language processing. Chapter 8 of Garrod and Pickering (eds.): Language Processing. Psychology Press.
• M. Christiansen and N. Chater (1999). Connectionist Natural Language Processing: The State of the Art. Cognitive Science, 23(4): 417-437.
• Richard Cooper. Modelling High-Level Cognitive Processes. Lawrence Earlbaum, Mahwah, NL, 2002.
• Matthew Crocker and Steffan Corley. Modular Architectures and Statistical Mechanisms: The Case from Lexical Category Disambiguation. In: Merlo & Stevenson (eds), The Lexical Basis of Sentence Processing, John Benjamins, Amsterdam (in press).
• Matthew Crocker and Thorsten Brants. Wide Coverage Probabilistic Sentence Processing. Journal of Psycholinguistic Research; 29(6):647-669, 2000.
• J. Elman et al. (1996). Chapter 2: Why Connectionism? In: Rethinking Innateness. MIT Press.
• Daniel Jurafsky. A Probabilistic Model of Lexical and Syntactic Access and Disambigiuation. Cognitive Science, 20, 137-194 (1996).
• J. Elman (1990). Finding Structure in Time. Cognitive Science, 14: 179-211.
• J. Elman (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7: 195-225.
• J. Elman (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48: 71-99.
• M. Seidenberg and M. MacDonald (1999). A Probabilistic Constraints Approach to Language Acquisition and Processing. Cognitive Science, 23(4): 569-588.
• M. Steedman (1999). Connectionist Sentence Processing in Perspective. Cognitive Science, 23(4): 615-634.