Neural Networks: Architectures and Applications for NLP
Module Description
| Course | Module Abbreviation | Credit Points |
|---|---|---|
| BA-2010[100%|75%] | CS-CL | 6 LP |
| BA-2010[50%] | BS-CL | 6 LP |
| BA-2010[25%] | BS-AC | 4 LP |
| BA-2010 | AS-CL | 8 LP |
| Master | SS-CL, SS-TAC | 8 LP |
| Lecturer | Julia Kreutzer |
| Module Type | |
| Language | English |
| First Session | 24.10.2018 |
| Time and Place | Wednesday, 16:00-17:30, INF 327 / SR 3 |
| Commitment Period | January 8 - January 20 |
Prerequisite for Participation
Assessment
Class attendance, assignments
Inhalt
This course covers basic techniques and architectures for machine learning with artificial neural networks with a focus on NLP applications. A deep learning programming framework will be introduced and used for implementing the exercises of the class assignments. We will pursue the following questions: What are neural networks? What is deep learning? How can we implement and train neural networks? Why are these models so successful? Which architectures are suitable for which types of problems? What are the core challenges when processing natural language with neural networks? The goal of this course is to gain an understanding of the principles and basic architectures of neural networks. By the end of the course, students will be able to implement neural models for their own NLP applications. Grading is based on the credits obtained in the assignments.
Module Overview
Agenda
| Date | Session | Materials |
| 24.10.18 | Orga | Orga |
| 31.10.18 | Intro: What is DL? | Session01, Assignment 1, Instructions, Notebook demo |
| 7.11.18 | SGD, Backpropagation, Computation Graph | Session02 |
| 14.11.18 | Hyperparameters, Initialization, Regularization, Learning Rates | Session03, TF Intro |
| 21.11.18 | NLP from Scratch, Word Embeddings | Session04, Assignment 2 |
| 28.11.18 | Convolutional Neural Networks | Session05, More TF, Tensorboard, Code A1 Exercise 15 |
| 05.12.18 | Recurrent Neural Networks | Session06 |
| 12.12.18 | Gated Recurrent Neural Networks: LSTM, GRU | Session07 |
| 19.12.18 | Char-RNNs for text generation | Session08, Code+Data |
| 09.01.19 | Encoder-Decoder Models, Neural Machine Translation (1) | Session09, Assignment 3 |
| 16.01.19 | Attention, Neural Machine Translation (2) | Session10, Feedback Survey (Moodle) |
| 23.01.19 | Guest Lecture: "Deep Learning for NLP with Graphs and Trees" by Joost Bastings | Session11 |
| 30.01.19 | Multi-Task Learning, Transfer Learning | Session12 |
| 06.02.19 | Hackathon: Sarcastic Headline Detection |
Literature
