Maximum Principle and Dynamic Programming Viscosity Solution Approach

This book is concerned with optimal control problems of dynamical systems described by partial differential equations (PDEs). The content covers the theory and numerical algorithms, starting with open-loop control and ending with closed-loop control. It includes Pontryagin s maximum principle and the Bellman dynamic programming principle based on the notion of viscosity solution. The Bellman dynamic programming method can produce the optimal control in feedback form, making it more appealing for online implementations and robustness. The determination of the optimal feedback control law is of fundamental importance in optimal control and can be argued as the Holy Grail of control theory.

In fact, it has been realized since Pontryagin s time that the dynamic programming method, which leads to the Hamilton-Jacobi-Bellman (HJB) equation, would provide the feedback law, provided that the HJB equation is solvable. In other words, if the value function and its gradient are known, then the optimal control in closed-loop form can be obtained analytically. But unfortunately, no matter how smooth the coefficients of the HJB equation are, the classical solution may still not exist. Moreover, even if a classical solution exists, it may not be unique. The evolution happened when the viscosity solution was introduced by Michael G. Crandall and Pierre-Louis Lions in the early 1980s. According to the viscosity solution theory, the value function is usually the unique viscosity solution to the associated HJB equation. The introduction of the viscosity solution provided a rigorous mathematical foundation for the classical dynamic programming approach. With the exception of a few specific cases, obtaining an analytical solution for the HJB equation is not feasible due to its complexity. Therefore, the numerical solution is almost the best choice for finding the optimal control.

The book is organized into five chapters. Chapter 1 presents necessary mathematical knowledge. The following Part 1, consisting of Chapters 2 and 3, focuses on the open-loop control. The second part, the closed-loop control component is crucial in this monograph. We incorporate the notion of viscosity solution of partial differential equation with dynamic programming approach. The dynamic programming viscosity solution (DPVS) approach is then used to investigate optimal control problems. In each problem, the optimal feedback law is synthesized and numerically demonstrated. The last chapter, Chapter 5 presents multiple algorithms for the DPVS approach, including an upwind finite-difference scheme with the convergence proof. It is worth noting that the dynamic systems considered are primarily of technical or biologic origin, which is a highlight of the book.

This book is systematic and self-contained. It can serve the expert as a ready reference for control theory of infinite-dimensional systems. These chapters taken together would also make a one-seme