Sommer Semester 2025
Masters Seminar: Generative Models on Text
Large Language Models (such as GPT2, GPT3, GPT4, Llama, T5) and Intelligent Chatbots (such as ChatGPT, Claude, Gemini and Copilot) are a very timely topic.
Contents: N-gram language models, neural language modeling, word2vec, RNNs, Transformers, BERT, RLHF, ChatGPT, multilingual alignment, prompting, transfer learning, domain adaptation, linguistic knowledge in large language models
Instructors: Prof. Alexander Fraser, Marion Di Marco
Location: Room D.2.11
Time: Tuesday 16:15 – 17:45
Lectures
| Lecture 1 | 29. 04. 2025 | Organization and Introduction to Linguistic Concepts | ||
| Lecture 2 | 06. 05. 2025 | N-gram Models (without section 3.7) | ||
| (Dan Jurafsky and James H. Martin (2025). Speech and Language Processing) | ||||
| Slides | ||||
| Lecture 3 | 13. 05. 2025 | Bengio et al. (2003): A Neural Probabilistic Language Model. | ||
| (Journal of Machine Learning Research 3, 1137-1155) | ||||
| Lecture 4 | 20. 05.2025 | Talk by Prof. Victoria Nash: AI and the Evolution of Digital Childhood | ||
| May 20, 2025 16:15-18:00 in Room D.0.0.1 | ||||
| Lecture 5 | 27. 05. 2025 | Smith (2019): Contextual Word Representations: A Contextual Introduction. | ||
| (arXiv) | ||||
| Lecture 6 | 03. 06. 2025 | Lena Voita. NLP Course: Neural Language Models and | ||
| Sequence to Sequence and Attention (Web Tutorial) | ||||
| 10. 06. 2025 | Whitsun Vacation – no lecture | |||
| Lecture 7 | 17. 06. 2025 | Vaswani et al. (2017): Attention Is All You Need (NIPS) | ||
| Lecture by Dr. Lukas Edman | ||||
| Lecture 8 | 24. 06. 2025 | Devlin et al. (2019): BERT: Pre-training of Deep Bidirectional Transformers | ||
Literature
Speech and Language Processing
Dan Jurafsky and James H. Martin (2024; 3rd ed. draft)
Paper Presentations