Course Objective: Deep learning is a rapidly evolving field within machine learning that has made significant advancements in recent years. It has a wide range of applications, including natural language processing (NLP), computer vision, and speech recognition, among others. This course will offer a comprehensive overview and the mathematical foundations of deep learning, introducing key techniques and architectures that have become standard in various applications.

This course is currently offered under two different course IDs.
AIL721: Deep Learning - For old students only.
AIL7021: Deep Learning - For new students only.

Prerequisites:

• A foundational course in AI or ML.
• Proficiency in Python: All class assignments will be in Python. We will mainly use Pytorch, Numpy for implementations.
• Basic knowledge of Calculus & Linear Algebra.
• Basic knowledge of Probability and Statistics.

Registration Forms: For prerequisite waiver request please signup on this form . Registration may be allowed subject to class size.

Grading Scheme: Minor - 30%, Major - 35%, Assignments - 25%, Quizes - 10%.

Attendance Policy: Institute default (<75% attendance leads to grade being lowered by one).

Audit Pass Criteria: Marks equivalent to B- or higher, plus >=75% attendance.

Lecture Hall & Time: LH316, Tuesday and Friday 2:00-3:30pm.

Office Hours: By appointment only.

Class Communication: Moodle.

Tentative List of Topics:

Week No.	Lecture Dates	Topics
1	Jan 02, 06	Course logistics, Brief history, Motivation
2	Jan 09, 13	Multilayer Perceptron, Shallow Neural Network, Deep Neural Network
3	Jan 16, 20	Regression, Classification with Neural Network, Loss functions
4	Jan 23, 27	Optimization, Backpropagation, Regularization
5	Jan 30, Feb 3	Convolutional Neural Networks, Residual Networks, Recurrent Neural Networks
6	Feb 6, 10	LSTM, Backpropagation-Through-Time, Sequence-to-Sequence Model
7	Feb 13, 17	Attention Mechanism, Transformers, Intro to LLMs
-	Feb 20 - 27	Minor Exam
-	Mar 2 - 8	Mid Semester Break
8	Mar 10, 13	Graph Neural Network
9	Mar 17, 20	Deep Reinforcement Learning
10	Mar 24, 27	Normalizing flows, Latent Variable Model
11	Apr 7, 10	Variational Autoencoders, Diffusion Models
12	Apr 11, 14	Generative Adversarial Networks, Self-Supervised Learning
13	Apr 17, 18	Contrastive Learning, Bayesian Deep Learning
14	Apr 21, 24	Advanced Topics: Deep Multi-Task and Meta Learning (if time allows)

Reference Textbooks & Papers:

• [1] Understanding Deep Learning. Simon J.D Prince. MIT Press 2023 [pdf]
• [2] Deep Learning: Foundations and Concepts. Springer (2024). Christopher M. Bishop and Hugh Bishop [pdf]
• [3] Speech and Language Processing (3rd ed. draft) Dan Jurafsky and James H. Martin [pdf]