The problem of minimizing an expected value is ubiquitous in machine learning, from approximate Bayesian inference to acting optimally in a Markov decision process. Progress on this problem may drive advances in methods for generating novel images, unsupervised discovery of object relations, or continuous control. This course will introduce students to various methodological issues at stake in this problem and lead them in a discussion of its modern developments. Introductory topics may include stochastic gradient descent, gradient estimation, policy and value iteration, and variational inference. The class will have a major project component.