Ruprecht-Karls-Universität Heidelberg
Institut für Computerlinguistik

Bilder vom Neuenheimer Feld, Heidelberg und der Universität Heidelberg

Grounded NLP for Spatial Navigation

Module Description

Course Module Abbreviation Credit Points
BA-2010[100%|75%] CS-CL 6 LP
BA-2010[50%] BS-CL 6 LP
BA-2010[25%] BS-AC 4 LP
BA-2010 AS-CL 8 LP
Master SS-CL, SS-TAC 8LP
Lecturer Raphael Schumann
Module Type Proseminar / Hauptseminar
Language English
First Session 30.04.2020
Time and Place Thursday, 14:15-15:45, INF 306 / SR 13
Commitment Period tbd.

Prerequisite for Participation

•Basic knowledge in probability, statistics, and linear algebra, e.g. Mathematical Foundations and Statistical Methods for Computational Linguistics.

•Basic knowledge in neural networks, e.g., Neural Networks: Architectures and Applications for NLP


•Active participation (read papers and submit/ask questions)

•Paper presentation and discussion

•Implementation project


Spatial Navigation is the ability of an agent to orientate itself in a given environment. This ability can be examined by measuring the success of tasks that include reaching a goal location or finding a certain object. The environment is usually 2-dimensional and artificially created.
The role of NLP in this setting is twofold. In one case, the instructions are given by a human in natural language and an artificial agent learns to interpret these in order to navigate the environment. The natural language understanding of the artificial agent is grounded in the reward of fulfilling the given task. In the other case, the navigation instructions are generated by NLP and a human agent is on the receiving end. Here, the natural language generation is grounded in the reward. Recent research focuses on bridging this setting to the real world by using geospatial maps or StreetView as part of the simulated environment.

Module Overview


Date Session Materials


In this seminar we will read and discuss literature on both cases mentioned above:

Human natural language navigation instructions -> NLP -> agent (e.g. MacMahon et al., 2006, Chen et al., 2019)

NLP navigation instructions -> human agent (e.g. Daniele et al., 2016 , Vries et al., 2018)

» More Materials

zum Seitenanfang