Course Objective: Deep learning is a rapidly evolving field within machine learning that has made significant advancements in recent years. It has a wide range of applications, including natural language processing (NLP), computer vision, and speech recognition, among others. This course will offer a comprehensive overview and the mathematical foundations of deep learning, introducing key techniques and architectures that have become standard in various applications.

Prerequisites:

• A foundational course in AI or ML.
• Proficiency in Python: All class assignments will be in Python. We will mainly use Pytorch, Numpy for implementations.
• College level Calculus & Linear Algebra.
• Basic knowledge of Probability and Statistics.

Class Location & Schedule: LH421; Tuesday 5pm - 6pm, Wednesday 12 Noon - 1pm, Friday 5pm - 6pm.

Grading Scheme: Mid-term - 25%, End-term - 30%, Assignments - 20%, Project - 25%.

Attendance Policy: Institute default (<75% attendance leads to grade being lowered by one).

Audit pass Criteria: Marks equivalent to B- or higher, plus >=75% attendance

Office Hours: Thursday 2-3 pm or by appointment.

Planned Syllabus:

Week No.	Lecture Dates	Topic	Deadlines	Reference Materials
1	03 Jan	Introduction — Brief history and motivation	-	-
2	07 Jan - 10 Jan	Supervised learning: multilayer perceptron (shallow neural nets); why deep?	-	[1] - Ch3, Ch4
3	15 Jan - 21 Jan	Loss functions, optimisation, backpropagation	-	[1] - Ch5,6; [2] - Ch8
4	22 Jan - 24 Jan	Overfitting, regularisation	-	[1]-Ch 8,9
5	28 Jan - 31 Jan	Deep learning for Computer Vision - CNNs, Other Architectures, Residual Nw	-	[1] - Ch10, Ch11
6	04 Feb - 07 Feb	Deep learning for NLP - Basics, RNN	7th Feb: Assgn-1
7	11 Feb - 14 Feb	Deep learning for NLP - BPTT, LSTM, Seq-to-Seq	15th Feb: Project Proposal	-
8	18 Feb - 19 Feb	Deep learning for NLP - Attention mechanism, Transformers, LLMs	-
-	21 Feb - 27 Feb	Mid Term Exam	-
9	28 Feb - 07 Mar	Guest Lec. on traning LLMs by Dr. Maksim Tkachenko, Graph Neural Netowrk (GNN)	-
-	10 Mar - 16 Mar	Semester Break	-
10	18 Mar - 21 Mar	Graph Neural Networks (GNN) Contd.	-
11	25 Mar - 28 Mar	Deep Reinforcement Learning	-
12	01 Apr - 04 Apr	Deep Reinforcement Learning Contd., RL With Human Feedback (RLHF)	-
13	08 Apr - 11 Apr	Generative Models - Generative Adversarial Network (GAN)	-
14	15 Apr - 18 Apr	Variational Auto-encoders, Diffusion Models, Normalizing Flows	-
15	22 Apr - 25 Apr	Course Project Presentation	-

Reference Textbooks & Papers:

• [1] Understanding Deep Learning. Simon J.D Prince. MIT Press 2023 [pdf]
• [2] Deep Learning: Foundations and Concepts. Springer (2024). Christopher M. Bishop and Hugh Bishop [pdf]
• [3] Speech and Language Processing (3rd ed. draft) Dan Jurafsky and James H. Martin [pdf]