Course Objective: Deep learning is a rapidly evolving field within machine learning that has made significant advancements in recent years. It has a wide range of applications, including natural language processing (NLP), computer vision, and speech recognition, among others. This course will offer a comprehensive overview and the mathematical foundations of deep learning, introducing key techniques and architectures that have become standard in various applications.

Prerequisites:

• A foundational course in AI or ML.
• Proficiency in Python: All class assignments will be in Python. We will mainly use Pytorch, Numpy for implementations.
• College level Calculus & Linear Algebra.
• Basic knowledge of Probability and Statistics.

Class Location & Schedule: LH421; Tuesday 5pm - 6pm, Wednesday 12 Noon - 1pm, Friday 5pm - 6pm.

Grading Scheme: Mid-term - 25%, End-term - 30%, Assignments - 20%, Project - 25%.

Attendance Policy: Institute default (<75% attendance leads to grade being lowered by one).

Audit pass Criteria: Marks equivalent to B- or higher, plus >=75% attendance

Office Hours: Thursday 2-3 pm or by appointment.

Planned Syllabus:

Week No.	Lecture Dates	Topic	Reference Materials
1	03 Jan	Introduction — Brief history and motivation	-
2	07 Jan - 10 Jan	Supervised learning: multilayer perceptron (shallow neural nets); why deep?	Prince”s Book - Ch3, Ch4
3	15 Jan - 21 Jan	Loss functions, optimisation, backpropagation	-
4	22 Jan - 24 Jan	Overfitting, regularisation	-
5	28 Jan - 31 Jan	Deep learning for Computer Vision - Basic models (CNNs)	-
6	04 Feb - 07 Feb	Deep learning for Computer Vision - Other architectures, pre-trained models	-
7	11 Feb - 14 Feb	Deep learning for NLP - Basic models (RNNs, LSTM, GRU)	-
8	18 Feb - 19 Feb	Deep learning for NLP - Attention mechanism, Transformers	-
-	21 Feb - 27 Feb	Mid Term Exam	-
9	28 Feb - 07 Mar	Deep learning for NLP - Transformers continued, Guest Lecture on traning LLMs by Dr. Maksim Tkachenko	-
-	10 Mar - 16 Mar	Semester Break	-
10	18 Mar - 21 Mar	Graph Neural Networks (GNN)	-
11	25 Mar - 28 Mar	Deep Reinforcement Learning	-
12	01 Apr - 04 Apr	Deep Reinforcement Learning Contd., Reinforcement Learning With Human Feedback (RLHF)	-
13	08 Apr - 11 Apr	Generative Adversarial Network (GAN)	-
14	15 Apr - 18 Apr	Variational Auto-encoders, Diffusion Models	-
15	22 Apr - 25 Apr	Course Project Presentation	-

Reference Textbooks & Papers:

• Understanding Deep Learning. Simon J.D Prince. MIT Press 2023 [pdf]
• Deep Learning: Foundations and Concepts. Springer (2024). Christopher M. Bishop and Hugh Bishop [pdf]