Introduction to deep learning techniques for NLP. The goals of this course are:
- To understand the structure of neural networks
- To be familiar with central concepts in deep learning for NLP
- To know the most common deep learning models applied in NLP
- To implement your own deep learning models using the pytorch library
- Worth checking out: Troubleshooting Deep Neural Networks
- Moodle page up and running
- Slides and other materials available here after each session
Tuesdays 4:15pm - 6:00pm
Runs from 21.04.2020 to 21.07.2020
(Room 126.96.36.199) ONLINE until further notice
|1||21.04.2020||Introduction||YG ch.2||reaction paragraph not required|
|2||28.04.2020||Revision of linear algebra & statistics ; Pytorch basics||YG ch.2; M&S chs. 2,3,12||A1||rp + set up|
|3||05.05.2020||Feed forward networks (FFNs)||YG chs. 3&4; Rao ch3; M&S ch.5; derivatives; backpropagation||rp|
|4||12.05.2020||QA + Word embeddings 1||video; YG ch.8||rp + A1|
|5||19.05.2020||Word embeddings 2 + intro to projects||stanford a1||A2||—|
|6||26.05.2020||NNs training||YG ch.5||rp|
|7||02.06.2020||no zoom meeting||–||–||A2|
|8||09.06.2020||Recurrent neural networks (RNNs)||Intro to LMs, ML chapter, blog post1, blog post2. One book + one blog necessary for rp.||A3||rp|
|9||16.06.2020||QA + Special RNNs (gradient issues, stacked, GRUs, LSTMs)||YG chs. 14 & 15||rp|
|10||23.06.2020||More RNNs (seq2seq), attention||dependency parsing, machine translation||rp|
|11||30.06.2020||Paper discussion||Transformer, What does BERT look at?||A3 + rp + group contracts (03.07.20) + pick project topic|
|12||07.07.2020||Convolutional NNs (CNNs), Start 4:30pm||blog post, YG ch.13||rp|
|13||14.07.2020||Project proposal presentations|
|14||21.07.2020||Project proposal presentations||–||–||any late assignments (first time submission)|
[YG] Goldberg, Yoav (2017). Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers.
[M&S] Moore, Will H. & David A. Siegel (2013). A Mathematics Course for Political and Social Research. Princeton University Press.
[DR] Rao, Delip & Brian McMahan (2019). Natural language processing with PyTorch: build intelligent language applications using deep learning. Beijing: O’Reilly. IMPORTANT: please choose ‘read online’ in order not to block the book.
All jupyter notebooks used in this course come from the companion repository by Rao & McMahan.
⭐️All the books are available through the UP network.
The following aspects are needed to pass the course:
- reaction paragraph for each pack of preparation material assigned per week
- 3 assignments completed (A1, A2, A3)
- project work: presentation of a project proposal and written report of completed project.
All hand in deadlines refer to the day at 11:00pm
Late policy for assignments
There will be a second and final deadline for late submissions on July
14 21st, 11:00pm. If you fail to meet the first deadline for reaction paragraphs or assignments, you may use this second deadline.