Grounding SMT in Perception and Action
Summary: Grounded statistical machine translation (SMT) introduces the concept of a task-specific evaluation of translation quality and offers the opportunity to deploy task-specific feedback on translations as data for learning SMT systems. The main challenge of our project is the investigation of new mechanisms to train and evaluate SMT systems by grounding them in interactions with the world. We will focus on response-based learning in which the only supervision signal available to the learner is the response from acting in the world. An example are translations of executable database queries where a supervision signal can be extracted from executing the translated query against the database. Another example is feedback from human translators in grounded scenarios.