ICS 504 Deep Learning

(Term 222)

Course Information
Prerequisites
Textbook & Resources
Schedule
Grading
Late Policy

Current Catalog Description

This course will introduce students to the deep Learning models and their applications in real world. Foundations of deep learning networks training and optimization. Deep learning models for spatial and temporal data processing. Analysis of prominent deep learning models such as Convolutional Neural Networks (CNNs), Recurrent and Recursive Networks, Long-Short Term Memory (LSTM), Encoder-decoder, Transformers, Transfer learning, One-shot learning, and Generative Adversarial Networks (GANs).

Instructor

Hamzah Luqman

Class meetings: Sunday, Tuesday 17:00 - 18:15 pm (Artificial Intelligence MasterMaster Program); Building 59 room 1011; Sunday, Tuesday 18:45 - 20:00 pm (Visual Computing Master Program); DTV248-019
Office hours: Tuesday (10:30am-11:30am) and by appointment!
Blackboard: https://blackboard.kfupm.edu.sa/
MS Teams: ICS 504-01; ICS 504-03
Instructor Email: hluqman@kfupm.edu.sa

Course Objectives

Learn the foundations of Deep Learning and its applications in real world
Learn how to implement, train and evaluate deep neural networks
Learn various deep neural networks architectures such as CNNs, RNNs, LSTM, GAN, and Capsule Networks and their applications

Course Learning Outcomes

After completion of this course, the student shall be able to:

Explain advantages of deep learning with respect to the alternative machine learning approaches
Describe different neural network architectures and their usage in different applications
Implement, train, and evaluate deep neural networks using existing software libraries
Explore multiple deep learning architectures and understand how to fine-tune and continuously improve models
Apply deep learning to various AI tasks

Prerequisites

ICS 502 Machine Learning

Textbook

No official textbook but I recommend the following book:

Dive into Deep Learning

Tentative Schedule

Item	Topics	Additional readings, if any	Quizzes/Assignments
1	Course logistics
2	Introduction to deep learning Foundation of neural network and deep learning
3	Images Clasification and KNN	CS231: Image classification CS231: Linear classification
4	Foundation of neural network and deep learning
5	Neural networks training and optimization Perceptron and LMS	DL book: Regularization for DL DL book: Optimization for Training Deep Models
6	Feedforward networks and Loss functions	DL book: Deep Feedforward Nets	Assignment 1
7	Back propagation Calculus of back propagation	CS231: Optimization and Backpropagation
8	Convergence in neural networks Rates of convergence Loss surfaces Learning rates, and optimization methods RMSProp, Adagrad, Momentum	CS231: Optimization and Backpropagation DL book: Optimization for Training Deep Models
9	Stochastic gradient descent Optimization Acceleration Overfitting and regularization Tricks of the trade: Choosing a divergence (loss) function Batch normalization Dropout	CS230: Neural Networks training
10	Convolutional Neural Networks (CNNs)	DL book: Convolutional Networks CS231: CNN	Quiz 01
11	Convolutional Neural Networks Models of vision		Assignment 2
12	Backpropagation through CNNs Increasing output map size Transform invariance Alexnet, Inception, VGG
13	Transfer Learning	TCS231: Transfer Learning and Fine-tuning CNNs
	Midterm
14	Recurrent Neural Networks (RNNs) Modeling series Back propagation through time (BPTT) Bidirectional RNNs	DL book RNN chapter	Assignment 3
15	Stability Exploding/vanishing gradients Long Short-Term Memory Units (LSTMs) and GRU	Understanding LSTM Networks
16	Sequence To Sequence Modeling
17	Auto-encoder models		Quiz 02
18	Attention Models	Attention? Attention!
19	Transformers	Attention Is All You Need The Illustrated Transformer	Assignment 4
20	Generative Adversarial Networks (GANs)
20	Presentation by Dr.Issam Laradji
	Final Exam

Grading

15% Quizzes
20% Homework (4 homeworks)
15% Project
20% Midterm
30% Final exam

Late policy for deliverables

Every student has 4 free late days (one day per homework) for this course.
After all free late days are used up, penalty is 25% for each additional late day.

Important notes

Students are expected to be courteous toward the instructor and their classmates throughout the duration of this course.
All cell phones and e-devices must be “on silent” mode during the classes and “turned off” during the exam times.
Attendance is taken at the beginning of the class.
Unexcused Lecture Absences Policies: Two late attendances are considered as one absence. Six unexcused absences result in DN grade.
An unexcused absence can become an excused absence ONLY by an official letter and must be presented as soon as possible.
Assignments must be submitted on the due date.
No make up for exams or any other class work will be made.
Any issues related to grading should be raised within one week of the release of the grades
ZERO-TOLERANCE for CHEATING, whether in exams, quizzes, written/programming assignments, and course project. Plagiarism, copying and other anti-intellectual behavior are prohibited by the university regulations.