Unbounded
Structures from Finite Statistical Methods
James
Henderson
University of Edinburgh
Many problems in natural language processing require structured
representations of unbounded size. Syntactic parsing has been the
canonical example of such a problem (parse trees can be arbitrarily
large), but there are many others. On the other hand, most
methods in machine learning and statistics are designed for
unstructured finite representations. One popular example is
log-linear models (a.k.a. maximum entropy models), which estimate
probabilities for fixed length feature vectors. Much of the work
on statistical parsing can be characterized in terms of its choice of
mapping from unbounded syntactic structures to fixed length feature
vectors. I will present several approaches to defining this mapping,
namely hand-crafted feature sets, kernel methods, feed-forward neural
networks, and recurrent neural networks. These methods differ in
the amount of task-specific knowledge which must be built in, the
generality of the models which can be learned, and the performance
which has been achieved.
back to IGK4 schedule