CS 190I Deep Learning (Winter 2023)

Course Description

Deep Learning has been driving the progress of AI in the past decade and has found versatile applications in many products and everyday life. Examples include recommendation systems for online videos, automatic language translation, smart home assistants, creative art design, and autonomous driving vehicles. This course will introduce general principles, methods, network architectures, and applications of Deep Learning. We cover neural network architectures including convolutional neural networks, recurrent neural networks, Transformer, and graph neural networks. We will cover techniques for designing loss, training, and inference methods. We focus on both the principles, analytical skills, and implementation practice. This course is suitable for undergraduate students and graduate students who wants to pursue career in AI or do research in deep learning.

Instructor

Lei Li (Office Hour: Monday 7-8pm, 2121 HFH, book a slot here)

Teaching Assistant

Krushna Shah (Office Hour: Tuesday 2-3pm, Trailer 936)
Zoey Song (Office Hour: Thursday 5-6pm, HH 2014)
Danqing Wang (Office Hour: Wednesday 11-12am, HH 2014)

Time and Location

Monday and Wednesday, 2-3:15pm, CHEM 1171

Recitation sessions:

Friday 9-9:50am, PHELP 1444
Friday 10-10:50am, PHELP 1440
Friday 11-11:50am, PHELP 1440

Textbook

[D2L] Dive into Deep Learning, Aston Zhang, Zachary Lipton, Mu Li, Alexander Smola. available online.

The textbook below is a great resource for those hoping to brush up on the prerequisite mathematics background for this course.

Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Free online.

Prerequisites

Prerequisites: Students need to grasp knowledge in Linear algebra, Calculus, Probability and Statistics, basic data structure and algorithms, and significant experience in computer programming (python or C++).

CS 130A, 130B, MATH 3B, MATH 6A, PSTAT 120A, 120B.

Homework Submission & Grading

Please submit your homework at gradescope.
There are a total of three late days allowed for all assignments (combined).

Discussion Forum

We will use Ed platform. sign up here

Policy

Please read the following Link carefully!

Syllabus

#	Date	Topic	Reading	Homework
1	1/9	Introduction	Chap 1, 2 of D2L	HW1
2	1/11	Linear Models, Vector Calculus	Chap 3 of D2LC
	1/13	Recitation: recitation_week1
	1/16	holiday. no class
3	1/18	Logistic Regression, Cross Entropy	Chap 4 of D2L	MP1 out
	1/20	Recitation: recitation_week2
4	1/23	Feedforward Network, Empirical Risk Minimization, Gradient Descent	Chap 5 of D2L
5	1/25	Learning FFN	Chap 5 of D2L	HW1 due, HW2 out
	1/27	Recitation: recitation_week3
6	1/30	Model Evaluation	Chap 6 of D2L
7	2/1	Regularization and other training techniques	Chap 6 of D2L
	2/3	Recitation: recitation_week4
8	2/6	Convolutional Neural Networks	Chap 7 of D2L
9	2/8	Convolutional Neural Networks	Chap 7 of D2L
	2/10	Recitation:recitation_week5
10	2/13	ResNet and other CNN variants	Chap 8 of D2L	HW2 Due, HW3 out
11	2/15	Optimization for ML	Chap 12 of D2L
	2/17	Recitation:recitation_week6		MP1 Due, MP2 out
	2/20	holiday. no class
12	2/22	Object Detection	Chap 14 of D2L
	2/24	Recitation: recitation_week7
13	2/27	Recurrent Neural Networks	Chap 9 of D2L
14	3/1	Sequence-to-Sequence Learning and Transformer	Chap 10, 11 of D2L	HW3 Due
	3/3	Recitation:recitation_week8
15	3/6	Pretrained Language Models	Chap 15.8-15.10, Chap 16.6, 16.9 of D2L, BERT, GPT3, InstructGPT
16	3/8	Graph Neural Networks
	3/10	Recitation: Annotated Transformer
17	3/13	Variational Auto-Encoder	VAE, Sentence VAE
18	3/15	Guest Lecture on Industrial Application of Deep Learning		MP2 Due
	3/17	Recitation: Final Prep
	3/xx	Final Exam