Deep Learning

Why This Course Matters

Deep neural networks are often treated as opaque black boxes. True engineering intuition comes from understanding the progressive evolution of architectures: how we moved from simple rule-based expert systems to McCulloch-Pitts neurons, Rosenblatt’s Perceptrons, smooth Sigmoid units, continuous Gradient Descent, deep Feedforward Networks, Backpropagation, modern Optimizers (Momentum, RMSProp, Adam), CNNs, and recurrent LSTMs.

What You Will Build & Master

1. The ML framework Data, Tasks, Models, Loss Functions, Learning Algorithms, and Evaluation.
2. Biological & Binary Neurons: McCulloch-Pitts formulation, threshold search, boolean functions, and linear separability geometry.
3. Rosenblatt’s Perceptron: Real-valued hyperplanes, perceptron loss, convergence theorem proofs, and limitations.
4. The Calculus of Learning: First-order Taylor series approximation, analytical derivation of gradient descent, partial derivatives, and loss surface contours.
5. Deep Feedforward Networks & Backprop: Rigorous chain rule derivation, computation graphs, and pure NumPy implementations.
6. Advanced Optimizers: Overcoming ravines and saddle points with Momentum, NAG, AdaGrad, RMSProp, and Adam.
7. Computer Vision & CNNs: Spatial convolution arithmetic, kernels, max pooling, LeNet, and ResNet architectures in PyTorch.
8. Sequence Modeling & LSTMs: Temporal unrolling, BPTT, and gated memory architectures for sequence prediction.