Computational Linguistics Colloquium

Thursday, 15 December 2011, 16:15
Conference Room, Building C7 4

Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus

The terms `deep' and `shallow' are frequently used to characterize or contrast different approaches to parsing. Inevitably, such informal notions lack a clear definition, and there is little evidence of community consensus on the relevant dimension(s) of depth, let alone agreement on applicable metrics. At its core, the implied dichotomy of approaches alludes to differences in the interpretation of the parsing task.

In order to obtain a fine-grained evaluation of parser accuracy over naturally occurring text, we study 100 examples each of ten reasonably frequent linguistic phenomena, randomly selected from a parsed version of the English Wikipedia. We construct a corresponding set of gold-standard target dependencies for these 1000 sentences, operationalize mappings to these targets from seven state-of-the-art parsers, and evaluate the parsers against this data to measure their level of success in identifying these dependencies.

If you would like to meet with the speaker, please contact Yi Zhang.