CS 190I Deep Learning (Winter 2023)

Course Description

Deep Learning has been driving the progress of AI in the past decade and has found versatile applications in many products and everyday life. Examples include recommendation systems for online videos, automatic language translation, smart home assistants, creative art design, and autonomous driving vehicles. This course will introduce general principles, methods, network architectures, and applications of Deep Learning. We cover neural network architectures including convolutional neural networks, recurrent neural networks, Transformer, and graph neural networks. We will cover techniques for designing loss, training, and inference methods. We focus on both the principles, analytical skills, and implementation practice. This course is suitable for undergraduate students and graduate students who wants to pursue career in AI or do research in deep learning.

Instructor

Lei Li (Office Hour: Monday 7-8pm, 2121 HFH, book a slot here)

Teaching Assistant

Time and Location

Monday and Wednesday, 2-3:15pm, CHEM 1171

Recitation sessions:

Textbook

The textbook below is a great resource for those hoping to brush up on the prerequisite mathematics background for this course.

Prerequisites

Prerequisites: Students need to grasp knowledge in Linear algebra, Calculus, Probability and Statistics, basic data structure and algorithms, and significant experience in computer programming (python or C++).

CS 130A, 130B, MATH 3B, MATH 6A, PSTAT 120A, 120B.

Homework Submission & Grading

Discussion Forum

We will use Ed platform. sign up here

Policy

Please read the following Link carefully!

Syllabus

#
Date
Topic
Reading
Homework
1
1/9
Introduction
Chap 1, 2 of D2L
HW1
2
1/11
Linear Models, Vector Calculus
Chap 3 of D2LC


1/13 Recitation: recitation_week1


1/16 holiday. no class

3
1/18
Logistic Regression, Cross Entropy
Chap 4 of D2L
MP1 out

1/20 Recitation: recitation_week2

4
1/23
Feedforward Network, Empirical Risk Minimization, Gradient Descent
Chap 5 of D2L
5
1/25
Learning FFN
Chap 5 of D2L
HW1 due, HW2 out

1/27 Recitation: recitation_week3

6
1/30
Model Evaluation Chap 6 of D2L

7
2/1
Regularization and other training techniques Chap 6 of D2L


2/3 Recitation: recitation_week4

8
2/6
Convolutional Neural Networks Chap 7 of D2L
9
2/8
Convolutional Neural Networks
Chap 7 of D2L


2/10 Recitation:recitation_week5

10
2/13
ResNet and other CNN variants
Chap 8 of D2L
HW2 Due, HW3 out
11
2/15
Optimization for ML
Chap 12 of D2L


2/17 Recitation:recitation_week6
MP1 Due, MP2 out

2/20
holiday. no class


12
2/22
Object Detection
Chap 14 of D2L


2/24 Recitation: recitation_week7

13
2/27
Recurrent Neural Networks Chap 9 of D2L
14
3/1
Sequence-to-Sequence Learning and Transformer Chap 10, 11 of D2L
HW3 Due

3/3 Recitation:recitation_week8

15
3/6
Pretrained Language Models Chap 15.8-15.10, Chap 16.6, 16.9 of D2L, BERT, GPT3, InstructGPT

16
3/8
Graph Neural Networks

3/10 Recitation: Annotated Transformer

17
3/13
Variational Auto-Encoder VAE, Sentence VAE

18
3/15
Guest Lecture on Industrial Application of Deep Learning
MP2 Due

3/17 Recitation: Final Prep


3/xx
Final Exam