165B Machine Learning (Winter 2022)

Focus on Deep Learning

Course Description

Machine Learning is about developing systems that automatically improve their performance through experience. It has found applications in many AI systems and products. Examples include systems that recommend online videos, automatic translating languages, and autonomous driving vehicles. This course focus on a sub-field of machine learning -- Deep Learning, with moderate introduction to general learning concepts and methods. We cover topics such as supervised learning, unsupervised learning, self-supervised learning, and various neural network architectures including convolutional neural networks, recurrent neural networks, Transformer, and graph neural networks. We will cover techniques for designing loss, training and inference methods. We focus on both the principles, analytical skills and implementation practice. This course is suitable for undergraduate students and graduate students who wants to pursue career in AI or do research in machine learning.

Instructor

Lei Li
Office Hour: Monday 4-5pm (on zoom)

Teaching Assistant

Time and Location

Monday/Wednesday 11:00am-12:15pm  On Zoom for the first two weeks (if in-person teaching, at NH 1006)

TA sessions on Wednesday (use the same zoom)

Textbook

The textbook below is a great resource for those hoping to brush up on the prerequisite mathematics background for this course.

Prerequisites

Prerequisites: Students need to grasp knowledge in Linear algebra, Calculus, Probability and Statistics, basic algorithms, and significant experience in computer programming (python, C++, or Java).

CS 130A&130B, MATH 3B, 6A, PSTAT 120A, 120B.

Homework Submission & Grading

Discussion Forum

We will use Ed platform. Please signup here.

Policy

Please read the following Link carefully!

Syllabus

#
Date
Topic
Reading
Homework
1
1/3
Introduction
Chap 1 of GBC
HW1
2
1/5
Linear Models, Vector Calculus
Chap 5 of GBC


1/5 Recitation: slide1, slides2

3
1/10
Logistic Regression, Cross Entropy
Chap 5 of GBC

4
1/12
Feedforward Network, Empirical Risk Minimization, Gradient Descent
Chap 6 of GBC
HW1 due, HW2 out

1/12 Recitation: slide, note


1/17 holiday. no class

5
1/19
Backpropagation and Autograd



1/19 Recitation:

6
1/24
Evaluation and Regularization Chap 7 of GBC

7
1/26
Convolutional Neural Network Chap 9 of GBC

8
1/31
ResNet and other CNN variants
HW2 Due, HW3 out
9
2/2
Learning for CNN


10
2/7
Optimization for ML


11
2/9
Object Detection


12
2/14
Recurrent Neural Networks


13
2/16
Sequence-to-sequence Models
HW3 Due, HW4 out

2/21
holiday. no class


14
2/23
Transformer

15
2/28
NLP Pretraining

16
3/2
Graph Neural Network

17
3/7
Autoencoder and VAE
HW4 due on 3/8
18
3/9
GAN


3/17 12pm
Final Exam