University of Potsdam, MSc CogSys, Summer 2019

AM11/AM12: Recent NLP Highlights

Instructor Prof. Dr. David Schlangen
Office Hours Tuesday, 5-6pm
Class Hours Tuesday, 2-4pm
Class Room  
Class Website

Course Description

In this class we will read and discuss recent influential and promising papers from Computational Linguistics / NLP conferences and journals. Participants will take turns presenting papers – introducing background knowledge, if necessary – and leading the discussion. All participants will be expected to participate actively in the discussions.

This class assumes that you are able to read and understand current NLP papers. Hence it is advisable to have completed BM1, and ideally also an introduction to machine learning / deep learning.

Course Objectives

After completing this course, you will

  • have a good idea of what the current state of the art in some subfields of CL is
  • will have encountered some current NLP models in depth
  • have learned about how scientific articles are written
  • have improved your presentation skills

Background Readings

The papers that will be discussed are listed on the course website. As reference when working through the papers, here is some potentially useful material:

Course Policy

This seminar is part of module AM11 (or module AM12), which is worth 6 credit points. There will be 15 meetings (= 30 hours), which leaves 150 hours for preparation, study, and writing your essay. Please plan your semester accordingly. (For example, plan at least 4 hours each week (60 hours) besides the 2 hours class time (30 hours), 30 hours for preparing your presentation, 60 hours for your essay.)

Grading Policy

The exam is a “Portfolioprüfung”, combining the grade for the class and the grade for an essay (of around 20, generously spaced, pages) with equal weights.

Grading Policy, Class

70% your presentation, 30% your contributions to the discussion.

Send questions by end of the week (Sunday) before the date; this will help the presenter prepare.

Criteria for evaluating the presentations:

  • Did the presenter understand the presented material, and understand what they didn’t understand?
  • Did the presenter introduce the background sufficiently well for others to understand where the paper is coming from?
  • Use of presentation materials
  • Degree of Abstraction / Summarisation beyond paper. More than showing parts of the presented paper?

The goal of the presentation is to ensure that we all understand and appreciate the paper(s) that is / are presented. Focus on ensuring that everyone, or at least the majority of students, understand what has been done, why, and how. Backward-looking: What are they building on, what are they improving on (or at least are claiming to be)?

  • Don’t try to copy the original presentation! Focus on explaining the background and the model. We have more time than the authors (who most likely only had 20 mins or less to present their work, but who presented it to a more advanced audience)
  • We want a discussion.
  • Be critical, but also enthusiastic… Try to defend the choices made in the paper.

  • What exactly are the authors trying to do?
  • Why are they doing this?
  • Are they doing what they are saying they are?
  • What is the history of the problem? How has it been tackled before?
  • What is the unique contribution of the paper?
  • What are the results that have been achieved?

Criteria for evaluating contributions to discussion:

  • Activity during the presentation: Asking questions, providing answers / helping presenter.
  • Activity on piazza.

Grading Policy, Essay

The goal of the essay is to show that you understand the problem tackled in the paper, the motivation behind tackling it, and the approach proposed by the authors, and that you can speculate on possible improvements. If the presentation has been mostly backward-looking, be a bit more forward-looking. Has approach been taken up? What else could be done with it. Run the model, and test it with different data? Try to reproduce the results? More code-oriented (if applicable). (A secondary goal of this exercise is to force you to familiarize yourself with LaTeX..)

Widening the focus a bit. Discuss related papers (not just background papers).

There is an unfortunate tendency in the recent NLP literature to be ignorant of non-DL work. Bonus points for identifying earlier work on this topic and for relating it.

In this field, there are many people that write helpful blog posts explaining stuff, some of which we will see during the course (and some of which have become legendary, like Karpathy’s RNN post, or Olah’s LSTM explainer). Think of this as a possible audience for your essay.

Deadline for handing in the essay is September 15, 2019 (unless you apply for an extension well in advance, and for good reasons).

Attendance Policy

Pay attention.

Piazza… Discussion pre and post class.

E-mail Policy

I will try to be responsive with answering emails.. But first try piazza. (Possibly anonymously, if you are worried that your question might sound silly.)

If it is something administrative, email.