Ruprecht-Karls-Universität Heidelberg
Institut für Computerlinguistik

Bilder vom Neuenheimer Feld, Heidelberg und der Universität Heidelberg

Generalization in Deep Learning

Module Description

Course Module Abbreviation Credit Points
BA-2010 AS-CL 8 LP
Master SS-CL, SS-TAC 8 LP
Lecturer Mayumi Ohta
Module Type Hauptseminar
Language English
First Session 09.11.2020
Time and Place Monday, 10:30-12:00, Online
Commitment Period tbd.

Registration

Students have to register for this course until 26.10.2020. This applies for CL and Non-CL students and for freshmen as well as higher semesters. To enrol, please follow the instructions here: course registration .

If you have any question, please contact the lecturer by email: ohta[at]cl.uni-heidelberg.de.

Prerequisite for Participation

  • Statistical Methods for CL (or comparable)
  • Basic knowledge on Neural Networks and Optimization Theory
  • Assessment

  • Regular and active participation
  • Paper presentation
  • Implementation project
  • Content

    Deep neural networks have seen great success in a wide range of applications, but why they generalize well remains still a mystery. A number of researches have tackled the problem to uncover the generalization mystery in deep learning models from both theoretical and empirical perspectives. In this seminar, we will study theoretical proofs and experimental generalization bounds. We will discuss evaluations of complexity measures, such as VC-dimension, PAC-Bayes, Sharpness measure, Optimization-based or Compression-based measure, and the correlation of different measures to generalization in image classification experiments.

    Students are expected to reimplement one of those complexity measures and extend to some sequence-to-sequence tasks in their term project.

    Literature

    (tbd.)

    » More Materials

    zum Seitenanfang