View on GitHub


Deep Learning for NLP

Introduction to deep learning techniques for NLP. The goals of this course are:


Practical Info

Sharid Loáiciga

Tuesdays 4:15pm - 6:00pm
Runs from 21.04.2020 to 21.07.2020
(Room ONLINE until further notice

Session Date Content Preparation Material Release Due
1 21.04.2020 Introduction YG ch.2   reaction paragraph not required
2 28.04.2020 Revision of linear algebra & statistics ; Pytorch basics YG ch.2; M&S chs. 2,3,12 A1 rp + set up
3 05.05.2020 Feed forward networks (FFNs) YG chs. 3&4; Rao ch3; M&S ch.5; derivatives; backpropagation   rp
4 12.05.2020 QA + Word embeddings 1 video; YG ch.8   rp + A1
5 19.05.2020 Word embeddings 2 + intro to projects stanford a1 A2
6 26.05.2020 NNs training YG ch.5   rp
7 02.06.2020 no zoom meeting A2
8 09.06.2020 Recurrent neural networks (RNNs) Intro to LMs, ML chapter, blog post1, blog post2. One book + one blog necessary for rp. A3 rp
9 16.06.2020 QA + Special RNNs (gradient issues, stacked, GRUs, LSTMs) YG chs. 14 & 15   rp
10 23.06.2020 More RNNs (seq2seq), attention dependency parsing, machine translation   rp
11 30.06.2020 Paper discussion Transformer, What does BERT look at?   A3 + rp + group contracts (03.07.20) + pick project topic
12 07.07.2020 Convolutional NNs (CNNs), Start 4:30pm blog post, YG ch.13   rp
13 14.07.2020 Project proposal presentations     any late assignments (first time submission)
14 21.07.2020 Project proposal presentations any late assignments (first time submission)

Reading material

[YG] Goldberg, Yoav (2017). Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers.

[M&S] Moore, Will H. & David A. Siegel (2013). A Mathematics Course for Political and Social Research. Princeton University Press.
Videos here.

[DR] Rao, Delip & Brian McMahan (2019). Natural language processing with PyTorch: build intelligent language applications using deep learning. Beijing: O’Reilly. IMPORTANT: please choose ‘read online’ in order not to block the book.

All jupyter notebooks used in this course come from the companion repository by Rao & McMahan.

⭐️All the books are available through the UP network.


The following aspects are needed to pass the course:

All hand in deadlines refer to the day at 11:00pm

Late policy for assignments

There will be a second and final deadline for late submissions on July 14 21st, 11:00pm. If you fail to meet the first deadline for reaction paragraphs or assignments, you may use this second deadline.