Ruprecht-Karls-Universität Heidelberg
Institut für Computerlinguistik

Bilder vom Neuenheimer Feld, Heidelberg und der Universität Heidelberg

Generalization in Deep Learning

Module Description

Course Module Abbreviation Credit Points
BA-2010 AS-CL 8 LP
Master SS-CL, SS-TAC 8 LP
Lecturer Mayumi Ohta
Module Type Hauptseminar
Language English
First Session 09.11.2020
Time and Place Monday, 10:30-12:00 11:15-12:45, Online (HeiConf)
Commitment Period 8th Feb. 2021 - 17th. Feb. 2021

[Update on 15. February 2021]

The presentation on Double Descent has been postponed until next week. Instead, we will have a short review-session this week.

[Update on 8. February 2021]

Commitment period: 8th Feb. - 17th Feb.

[Update on 25. January 2021]

We have no session on 1. Feb. 2021 (Mo).

[Update on 14. December 2020]

We have no session on 14. Dec. 2020 (Mo).

[Update on 18. November 2020]

We have no session on 23. Nov. 2020 (Mo).

I recommend to watch the lecture series Statistical Machine Learning Part 38 - 44 (Prof. Ulrike von Luxburg, University of Tübingen)
It's a very nice introduction to get an overview on Statistical Learning Theory.

[Update on 6. November 2020]

I've sent the registered students an info email with HeiConf link. If you haven't received the email, please contact me.

[Update on 4. November 2020]

If you haven't registered, but still want to participate, please write an email to the secretary sekretariat [at] cl.uni-heidelberg with the following information:

  • your name
  • your student ID (``Matrikelnummer'')
  • your email address
  • your degree program (``Studiengang'')
  • which institute you belong to
  • which semester you are in (``Fachsemester'', i.e. first semester in MA)

See here for late registration.

[Update on 2. November 2020]

We won't use Moodle in this seminar. I'm now preparing an info email to the registered students including HeiConf link. I'll announce here when I send you the info email.

[Update on 26. October 2020]

The time slot has been changed to 11:15-12:45.


Students have to register for this course until 26.10.2020. This applies for CL and Non-CL students and for freshmen as well as higher semesters. To enrol, please follow the instructions here: course registration .

If you have any question, please contact the lecturer by email: ohta[at]

Prerequisite for Participation

  • Statistical Methods for CL (or comparable)
  • Basic knowledge on Neural Networks and Optimization Theory
  • Assessment

  • Regular and active participation
  • Paper presentation
  • Implementation project
  • Content

    Deep neural networks have seen great success in a wide range of applications, but why they generalize well remains still a mystery. A number of researches have tackled the problem to uncover the generalization mystery in deep learning models from both theoretical and empirical perspectives. In this seminar, we will study theoretical proofs and experimental generalization bounds. We will discuss evaluations of complexity measures, such as VC-dimension, PAC-Bayes, Sharpness measure, Optimization-based or Compression-based measure, and the correlation of different measures to generalization in image classification experiments.

    Students are expected to reimplement one of those complexity measures and extend to some sequence-to-sequence tasks in their term project.

    » More Materials

    zum Seitenanfang