Bilder vom Neuenheimer Feld, Heidelberg und der Universität Heidelberg

Lehrveranstaltungen
LSF
Ressourcen	Fachschaft
Studien-FAQ	Technik-FAQ

Robust and Parameter-Efficient Finetuning

Kursbeschreibung

Studiengang	Modulkürzel	Leistungs- bewertung
BA-2010[100%\|75%]	CS-CL	6 LP
BA-2010[50%]	BS-CL	6 LP
BA-2010[25%]	BS-AC, BS-FL	4 LP
BA-2010	AS-CL	8 LP
Master	SS-CL, SS-TAC	8 LP

Dozenten/-innen	Raphael Schumann
Veranstaltungsart	Proseminar / Hauptseminar
Sprache	English
Erster Termin	19.04.2023
Zeit und Ort	Mittwoch, 17:15-18:45, INF 325 / SR 24
Commitment-Frist	tbd.

Teilnahmevoraussetzungen

Completion of Programming I and Introduction to Computational Linguistics or similar introductory courses
Introduction to Neural Networks and Sequence-To-Sequence Learning (or equal)

Leistungsnachweis

Paper presentation
Implementation project
Prepare questions for each session's paper

Inhalt

Finetuning masked/large language models allows to adapt them for specific tasks. This course covers methods to finetune such models while avoiding catastrophic forgetting and reducing computational cost. The methods include adapters , weight regularization and input interpolation.

Seminarplan

Datum

Sitzung