University of Potsdam, MSc CogSys, Summer 2019
AM11/AM12: Recent NLP Highlights
| Instructor | Prof. Dr. David Schlangen |
| david.schlangen@uni-potsdam.de | |
| Office Hours | Tuesday, 5-6pm |
| Class Hours | Tuesday, 2-4pm |
| Class Room | |
| Class Website | https://compling-potsdam.github.io/sose19-am11-nlp-highlights/ |
Course Description
In this class we will read and discuss recent influential and promising papers from Computational Linguistics / NLP conferences and journals. Participants will take turns presenting papers – introducing background knowledge, if necessary – and leading the discussion. All participants will be expected to participate actively in the discussions.
This class assumes that you are able to read and understand current NLP papers. Hence it is advisable to have completed BM1, and ideally also an introduction to machine learning / deep learning.
Course Objectives
After completing this course, you will
- have a good idea of what the current state of the art in some subfields of CL is
- will have encountered some current NLP models in depth
- have learned about how scientific articles are written
- have improved your presentation skills
Background Readings
The papers that will be discussed are listed on the course website. As reference when working through the papers, here is some potentially useful material:
- Yoav Goldberg, Neural Network Methods for Natural Language Processing, Morgan & Claypoole, 2017 (In the library; will try to get them to buy e-book version.)
- The Stanford “cs224n: Natural Language Processing with Deep Learning” class materials: http://web.stanford.edu/class/cs224n/
- Dan Jurafsky & James Martin, Speech and Language Processing 3rd ed., draft available online https://web.stanford.edu/~jurafsky/slp3/
- Some cheatsheets: Stanford CS 229 Machine Learning, Stanford CS 230 Deep Learning
Course Policy
This seminar is part of module AM11 (or module AM12), which is worth 6 credit points. There will be 15 meetings (= 30 hours), which leaves 150 hours for preparation, study, and writing your essay. Please plan your semester accordingly. (For example, plan at least 4 hours each week (60 hours) besides the 2 hours class time (30 hours), 30 hours for preparing your presentation, 60 hours for your essay.)
Grading Policy
The exam is a “Portfolioprüfung”, combining the grade for the class and the grade for an essay (of around 20, generously spaced, pages) with equal weights.
Grading Policy, Class
70% your presentation, 30% your contributions to the discussion.
Send questions by end of the week (Sunday) before the date; this will help the presenter prepare.
Criteria for evaluating the presentations:
- Did the presenter understand the presented material, and understand what they didn’t understand?
- Did the presenter introduce the background sufficiently well for others to understand where the paper is coming from?
- Use of presentation materials
- Degree of Abstraction / Summarisation beyond paper. More than showing parts of the presented paper?
The goal of the presentation is to ensure that we all understand and appreciate the paper(s) that is / are presented. Focus on ensuring that everyone, or at least the majority of students, understand what has been done, why, and how. Backward-looking: What are they building on, what are they improving on (or at least are claiming to be)?
- Don’t try to copy the original presentation! Focus on explaining the background and the model. We have more time than the authors (who most likely only had 20 mins or less to present their work, but who presented it to a more advanced audience)
- We want a discussion.
-
Be critical, but also enthusiastic… Try to defend the choices made in the paper.
- What exactly are the authors trying to do?
- Why are they doing this?
- Are they doing what they are saying they are?
- What is the history of the problem? How has it been tackled before?
- What is the unique contribution of the paper?
- What are the results that have been achieved?
Criteria for evaluating contributions to discussion:
- Activity during the presentation: Asking questions, providing answers / helping presenter.
- Activity on piazza.
Grading Policy, Essay
The goal of the essay is to show that you understand the problem tackled in the paper, the motivation behind tackling it, and the approach proposed by the authors, and that you can speculate on possible improvements. If the presentation has been mostly backward-looking, be a bit more forward-looking. Has approach been taken up? What else could be done with it. Run the model, and test it with different data? Try to reproduce the results? More code-oriented (if applicable). (A secondary goal of this exercise is to force you to familiarize yourself with LaTeX..)
Widening the focus a bit. Discuss related papers (not just background papers).
There is an unfortunate tendency in the recent NLP literature to be ignorant of non-DL work. Bonus points for identifying earlier work on this topic and for relating it.
In this field, there are many people that write helpful blog posts explaining stuff, some of which we will see during the course (and some of which have become legendary, like Karpathy’s RNN post, or Olah’s LSTM explainer). Think of this as a possible audience for your essay.
Deadline for handing in the essay is September 15, 2019 (unless you apply for an extension well in advance, and for good reasons).
Attendance Policy
Pay attention.
Piazza… Discussion pre and post class.
E-mail Policy
I will try to be responsive with answering emails.. But first try piazza. (Possibly anonymously, if you are worried that your question might sound silly.)
If it is something administrative, email.