Processing urban youth language

(Ines Rehbein)

Language data from registers and domains other than written newspaper
text can pose severe challenges for Natural Language Processing. A case
in point is informal spoken communication which is characterised by its
expressivity (shown, e.g., by the use of interjections and exclamatives)
and interactivity (use of backchannel signals and question tags).  In
addition, the data includes disfluencies such as repetitions, repairs,
breaks and fillers (uh, uhm) and syntactic structures which are common
in informal spoken language but will not occur in canonical written text
from the newspaper domain.

We present the KiezDeutsch-Korpus (KiDKo), a corpus of spontaneous
peergroup dialogues of adolescents from multiethnic and monoethnic areas
of Berlin, annotated with parts of speech, chunks and topological
fields. In the talk, we will shortly describe the processing pipeline
used for building the corpus, and then focus on (semi-supervised)
methods to address the data sparseness problem and to improve POS
tagging/parsing accuracies.