Deep Learning
These notes follow Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola's Dive into Deep Learning and emphasize the book's central style: concepts, mathematics, and runnable code together. The path starts with tensors, data preparation, linear algebra, calculus, probability, and automatic differentiation, then builds complete training loops for regression and classification before moving to modern architectures.
The later pages cover the main deep learning families: multilayer perceptrons, convolutional networks, recurrent networks, attention, transformers, NLP applications, computer vision systems, recommender systems, GANs, reinforcement learning, Gaussian processes, and hyperparameter optimization. Code examples use PyTorch for portability. For classical context, compare these notes with machine learning; for prerequisites, see linear algebra and probability.
- Tensors and Data Preprocessing
- Math for Deep Learning
- Linear Regression and Training Loops
- Softmax Classification and Generalization
- Multilayer Perceptrons and Regularization
- PyTorch Builders Guide
- Convolutional Neural Networks
- Modern CNNs
- Sequence Modeling and RNNs
- Gated RNNs and Sequence-to-Sequence
- Attention and Transformers
- Pretrained Transformers and BERT
- Optimization Algorithms
- Computational Performance
- Computer Vision Applications
- NLP Pretraining and Applications
- Generative Adversarial Networks
- Recommender Systems
- Reinforcement Learning and Bayesian Tuning