0. Basics / prerequisites 0.1 Data types and numerical precision 0.2 Linear algebra basics 0.3 GPU memory hierarchy and parallelism 0.4 CUDA basics for ML workloads
1. Math and GPU fundamentals 1.1 GEMM, reductions, and matrix math 1.2 Memory bandwidth vs compute-bound workloads 1.3 Tensor cores and mixed precision 1.4 CUDA kernels for attention and softmax
2. Models 2.1 Open-weight vs proprietary models 2.2 Dense vs MoE models 2.3 Base models vs instruction-tuned models
3. Architecture 3.1 Transformer basics 3.2 Encoder-only, decoder-only, encoder-decoder 3.3 Attention and transformer blocks
4. Core components 4.1 Tokenization 4.2 Embeddings and positional encoding 4.3 MLP, normalization, and attention
5. Training 5.1 Pretraining 5.2 Fine-tuning and alignment 5.3 Optimizers and training infrastructure 5.4 Distributed training and parallelism
6. Inference 6.1 Prefill and decode 6.2 KV cache and batching 6.3 Inference optimization and serving engines 6.4 How to size GPUs for training and inference
7. RAG and agents 7.1 Retrieval fundamentals 7.2 Vector databases and reranking 7.3 Tools, memory, and agent loops
8. Multimodality and evaluation 8.1 Vision, audio, and video models 8.2 Reasoning and test-time compute 8.3 Evaluation, safety, and guardrails
9. Production 9.1 GPUs and hardware 9.2 CUDA kernels and optimized serving 9.3 Deployment and autoscaling 9.4 Observability, cost, and SLOs