Using substitutability to evaluate words classification
Speaker: Stella Frank
Abstract:
Previous models of word category acquisition have been evaluated using gold standard categories. However, children's syntactic categories change during language development, so evaluating against a fixed and final set of adult categories is not appropriate for these models. I propose a new measure - substitutable precision and recall - based on the idea that words which occur in similar syntactic environments share the same category. This measure was used to evaluate three standard category acquisition models (hierarchical clustering, frequent frames, Bayesian HMM) and the results correlate well with those obtained using two gold-standard-based measures.
