The related areas of risk-aware control theory, stochastic control theory, and reinforcement learning are becoming ever more important in the modern age of data. In lectures, we focus on studying mathematical foundations and the pros and cons of existing methods to highlight research gaps (e.g., quality-of-approximation guarantees vs. scalability to high-dimensional systems). Lecture topics include introductions to measure theory, Borel spaces, continuous-state Markov decision processes (MDPs), finite-horizon MDP problems, stochastic safety analysis, solution methods via value iteration, risk functionals, risk-aware control theory, and parametric approximation methods. The exploration of additional topics and applications related to stochastic control and reinforcement learning is encouraged through literature critiques and research projects. This course is designed to practice and enhance creative thinking skills and to launch and inspire your research.