This course develops several methods that are central to modern optimization and learning problems under uncertainty. These include dynamic programming, linear quadratic regulator, Kalman filter, multi-armed bandits and reinforcement learning. Representative applications and numerical methods are emphasized.