Ruprecht-Karls-Universität Heidelberg
Institut für Computerlinguistik

Bilder vom Neuenheimer Feld, Heidelberg und der Universität Heidelberg

Recent Advances In Sequence-To-Sequence Learning

Module Description

Course Module Abbreviation Credit Points
BA-2010 AS-CL 8 LP
Master SS-CL, SS-TAC 8 LP
Lecturer Tsz Kin Lam
Module Type Hauptseminar
Language English
First Session 24.10.2019
Time and Place Thursday, 14:15-15:45
INF 326 / SR 27
Commitment Period tbd.

Prerequisite for Participation

•Basic knowledge in probability, statistics, and linear algebra, e.g. Mathematical Foundations and Statistical Methods for Computational Linguistics.

•Basic knowledge in neural networks, e.g., Neural Networks: Architectures and Applications for NLP



•Regular and active attendance of seminar

•Reading of rest of papers from seminar reading list

•Presentation of paper(s) from seminar reading list

•Implementation project or written term paper


Neural Sequence-To-Sequence Learning (Seq2Seq) is about finding mappings between input and the output sequences using neural networks. The “sequence” can be of many different forms, e.g., financial time series, DNA, audio signals, texts, and images. The two sequences can be of the same or different modality, leading to various applications such as textual or spoken machine translation, text summarisation, caption generation, and semantic parsing.

Seq2Seq is a fully data-driven approach. It has been empirically shown to perform better than traditional statistical learning approaches in many scenarios, especially on large-scale problems. Additionally, it is end-to-end which naturally prevents error propagation as happening in cascaded systems.

In this seminar, we will discuss recent advances in Seq2Seq learning with three main themes centred around Machine Translation. We first start from (1) Neural Network Architectures for Seq2Seq, e.g., ConvSeq2Seq, Transformer and RNMT+. We then move to (2) Low-resources setting with strong focus on leveraging unpaired data, e.g., Fusion Techniques, Back-Translation and Dual Learning. Finally, (3) we will break the left-to-right generation order and move to a novel regime, called Non-autoregressive Neural Sequence Generation, e.g., Levenshtein Transformer and Insertion Transformer.

Module Overview


Date Session Materials


» More Materials

zum Seitenanfang