Incremental, Predictive Parsing with Psycholinguistically Motivated
Tree-Adjoining Grammar

Frank Keller
Institute for Language, Cognition and Computation
School of Informatics, University of Edinburgh
http://homepages.inf.ed.ac.uk/keller/

Psycholinguistic research shows that key properties of the human
sentence processor are incrementality, connectedness (partial
structures contain no unattached nodes), and prediction (upcoming
syntactic structure is anticipated). However, there is currently no
broad-coverage parsing model with these properties. In this talk, we
present a probabilistic parser for a new version of Tree-Adjoining
Grammar (TAG). Our framework instantiates incrementality and
connectedness, from which prediction then follows naturally. We train
the parser on a TAG-transformed version of the Penn Treebank and show
that it achieves performance comparable to existing incremental TAG
parsers. From a psycholinguistic perspective, the key innovation of
our parser is an explicit mechanism for generating and verifying
syntactic predictions. We show that this mechanism makes it possible
to capture both locality effects and surprisal effects, and thus unify
a body of experimental results that have so far been accounted
separately.

Joint work with Vera Demberg and Alexander Koller.