Large Language Models: Concepts, Methods, and Applications

Module Description

Participants

All advanced CL Bachelor students and all CL master students. Students from MSc Data and Computer Science or MSc Scientific Computing with Field of Application Computational Linguistics are welcome after getting permission from the lecturer. MSc Scientific Computing students can only take the course as HS for 8 LP.  If the seminar should be oversubscribed, CL students will have priority.  

Prerequisites for Participation

Module Prerequisites

Transformer, Deep Learning

Assessment

• Participation: 20%
• Project: 50%
• Oral Exam after the project: 30%

Contents

This seminar introduces the basic concepts, methods, and applications of large language models. Topics include prompt engineering, instruction tuning, fine-tuning, retrieval-augmented generation, reasoning, LLM agents, long-context models, and efficient deployment. The course emphasizes conceptual understanding, practical use cases, and current limitations.

Course Content:

1. Foundations and Research Landscape of Large Language Models
2. From Base Models to Assistants: Instruction Tuning and Alignment
3. Prompt Engineering and In-context Learning
4. Adapting Large Language Models: Fine-tuning and Parameter-Efficient Methods
5. Retrieval-Augmented Generation and External Knowledge
6. LLM Reasoning and Test-time Computation
7. Tool Use and LLM Agents
8. Long-context Models, Memory, and Personalization
9. Multimodal Large Language Models
10. Efficient Use and Deployment of Large Language Models

By the end of this seminar, students will be able to:

• Understand the main concepts, capabilities, and limitations of large language models.
• Explain key approaches for adapting and enhancing LLMs, including instruction tuning, prompt engineering, fine-tuning, and retrieval-augmented generation.
• Describe major research topics such as reasoning, tool use, agents, long-context processing, multimodal models, and efficient deployment.
• Compare different LLM methods and identify suitable approaches for specific tasks and applications.
• Critically evaluate representative research papers, experimental results, and practical limitations.
• Discuss current challenges and possible future research directions in large language models.

» More Materials