2023-Spring-CS5814-Intro to Deep Learning
Graduate Class, CS, VT, 2023
Information on this page has not been updated since January 13, 2023. Students should go to Canvas for the most up-to-date course syllabus and materials.
Time and Location: Tuesday and Thursday, 12:30 PM - 1:45 PM. NCB 270.
Instructor: Xuan Wang
- Office: Torgersen 3160K
- Phone: (540) 231-4061
- Email: xuanw [at] vt [dot] edu
- Office Hours: TBD
TAs: TBD
Attendance: Students are required to attend this class in person. The lectures will not be recorded or offered online.
Mask Requirement: Masks are highly recommended for all students, regardless of vaccination status, attending the class in person.
Course Websites: Canvas (slides and assignments), Piazza (discussions)
Course Description
Deep Learning has gained a lot of popularity due to its recent breakthrough results in many real-world applications such as speech recognition, machine translation, image understanding, and robotics. The primary idea of deep learning is to build high-level abstractions of the data through multi-layered architectures. This course introduces the fundamental principles, algorithms, and applications of deep learning. It will provide an in-depth understanding of various concepts and popular techniques in deep learning. This course is mainly designed for graduate students who are interested in studying deep learning techniques and their practical applications. Basic knowledge and understanding of machine learning and data mining algorithms are required.
The course begins with a thorough treatment of deep feedforward networks along with various regularization and optimization techniques used for efficiently learning these models. Different forms of the network architectures such as convolutional networks, recurrent neural networks, and autoencoders will be discussed in detail. Other advanced concepts such as deep generative models and deep reinforcement learning will also be covered. Finally, the course will conclude with a discussion on a few real-world application domains where deep learning techniques have produced astonishing results.
Prerequisites
CS5525 Data Analytics I (or) CS5824 Advanced Machine Learning.
Books
There is no single textbook that will be used in this course. The students might find the following books to be useful.
- Deep Learning, I. Goodfellow, Y. Bengio and A. Courville, MIT Press, 2016.
- Neural Networks and Deep Learning: A Textbook, C. Aggarwal, Springer, September 2018.
- Dive into Deep Learning – an interactive online book, A. Zhang, Z. Lipton, M. Li, and A. Smola from Amazon
Grading Policy
This is a tentative grading policy. The final grades will be relative to others in the class.
- 40% Homework Assignments
- 20% Midterm Exam
- 20% Final Exam
- 20% Final Project
Late and Missed Work
All assignments are due on the date assigned at the listed time. No late assignments will be accepted. Make-up exams will not be offered except for extreme circumstances. Contact the instructor as soon as possible to make arrangements. Documentation of the circumstance may be required.
Lecture Schedule
This is a tentative lecture schedule. All the slides and homework assignments will be released on Canvas.
| Date | Topic and Slides | Events |
|---|---|---|
| 01/17 (Tue) | Introduction | |
| 01/19 (Thu) | Applied Math and Machine Learning Basics | |
| 01/24 (Tue) | Applied Math and Machine Learning Basics | |
| 01/26 (Thu) | Applied Math and Machine Learning Basics | HW1 out |
| 01/31 (Tue) | Deep Feedforward Networks | |
| 02/02 (Thu) | Deep Feedforward Networks | |
| 02/07 (Tue) | Regularization for Deep Learning | |
| 02/09 (Thu) | Regularization for Deep Learning | HW1 due |
| 02/14 (Tue) | Optimization for Deep Learning | |
| 02/16 (Thu) | Optimization for Deep Learning | HW2 out |
| 02/21 (Tue) | Convolutional Networks | |
| 02/23 (Thu) | Convolutional Networks | |
| 02/28 (Tue) | Recurrent and Recursive Networks | |
| 03/02 (Thu) | Recurrent and Recursive Networks | HW2 due |
| 03/7 (Tue) | Spring Break (No Class) | |
| 03/9 (Thu) | Spring Break (No Class) | |
| 03/14 (Tue) | Practical Methodology for Deep Learning | Midterm out |
| 03/16 (Thu) | Practical Methodology for Deep Learning | |
| 03/21 (Tue) | Autoencoders | Midterm due |
| 03/23 (Thu) | Autoencoders | HW3 out |
| 03/28 (Tue) | Representation Learning | |
| 03/30 (Thu) | Representation Learning | |
| 04/04 (Tue) | Structured Probabilistic Models | |
| 04/06 (Thu) | Structured Probabilistic Models | HW3 due |
| 04/11 (Tue) | Deep Generative Models | |
| 04/13 (Thu) | Deep Generative Models | HW4 out |
| 04/18 (Tue) | Deep Reinforcement Learning | |
| 04/20 (Thu) | Deep Reinforcement Learning | |
| 04/25 (Tue) | Applications (CV, NLP, etc.) | |
| 04/27 (Thu) | Applications (CV, NLP, etc.) | HW4 due |
| 05/02 (Tue) | Societal Impacts and Ethics |
Accommodation Statement
Virginia Tech welcomes students with disabilities into the University’s educational programs. The University promotes efforts to provide equal access and a culture of inclusion without altering the essential elements of coursework. If you anticipate or experience academic barriers that may be due to disability, including but not limited to ADHD, chronic or temporary medical conditions, deaf or hard of hearing, learning disability, mental health, or vision impairment, please contact the Services for Students with Disabilities (SSD) office (540-231-3788, ssd@vt.edu, or visit ssd.vt.edu). If you have an SSD accommodation letter, please meet with the instructors privately during office hours as early in the semester as possible to deliver your letter and discuss your accommodations. You must give the instructors a reasonable notice to implement your accommodations, which is generally 5 business days and 10 business days for final exams.
Academic Integrity Statement
The tenets of the Virginia Tech Graduate Honor Code will be strictly enforced in this course, and all assignments shall be subject to the stipulations of the Graduate Honor Code. Any suspected violations of the Honor Code will be promptly reported to the honor system. Honesty in your academic work will develop into professional integrity. The faculty and students of Virginia Tech will not tolerate any form of academic dishonesty. For more information on the Graduate Honor Code, please refer to the GHS Constitution.
Absence Policy
Regular class attendance is expected of all students. However, attendance will not be taken and will not be used in determining your final course grade in this class.