MACHINE LEARNING APPROXIMATION OF RISK-AVERSE POLICIES IN MARKOV DECISION PROCESSES WITH AVERAGE-VALUE-AT-RISK

Abstract

This thesis investigates the integration of Average-Value-at-Risk (AVaR) into Approximate Dynamic Programming (ADP) frameworks for solving Markov Decision Processes (MDPs) under uncertainty. We develop analytical and numerical methods to derive risk-averse policies for a quadratic cost minimization problem with stochastic dynamics. The problem is formulated as a nested optimization where the inner problem determines optimal actions for given states and risk thresholds, while the outer problem identifies optimal thresholds. We compare analytical solutions with Monte Carlo simulations across various risk aversion levels. To address computational challenges in real-time applications, we develop polynomial regression models that accurately approximate the optimal policies. The resulting models preserve critical structural features of the original solutions while enabling efficient implementation. Our findings contribute to the growing field of risk-aware decision-making by providing both theoretical insights and practical methods for implementing AVaR-based optimization in complex stochastic environments. The proposed machine learning approach offers a computationally efficient alternative to traditional numerical methods without sacrificing accuracy, making risk-sensitive optimization more accessible for real-world applications in finance, robotics, healthcare, and energy management.