A Connection between the Adjoint Variables and Value Function for Differential Games
Keywords:
Nonzero-sum differential games, Maximum principle, Dynamic programming principle, super- and subdifferentials.
Abstract
In this paper, we present a deterministic two-player nonzero-sumd ifferential games (NZSDGs) in a finite horizon. The connection between the adjoint varaibles in the maximum principle (MP) and the value function in the dynamic programming principle (DPP) for differentail games is obtained in either case, whether the value function is smooth and nonsmooth. For the smooth case, the connection between the adjoint variables and the derivatives of the value function are equal to each other along optimal trajectories. Furthermore, for the nonsmooth case, this relation is represented in terms of the adjoint variables and the first-order super- and subdifferentials of the value function. We give an example to illustrate the theoretical results.References
\bibitem{AV} \textsc{S.M. Aseev, V.M Veliov}, \textit{Another view of the
maximum principle for infinite-horizon optimal control problems in economics}
, Russ. Math. Surv. 74(6), 963, 2019.
\bibitem{BCD} \textsc{M. Bardi and I. Capuzzo-Dolcetta}, \textit{Optimal
control and viscosity solutions of Hamilton-Jacobi-Bellman equations},
Birkhauser, Boston 1997.
\bibitem{BJ} \textsc{E. N. Barron and R. Jensen}, \textit{The Pontryagin
maximum principle from dynamic programming and viscosity solutions to
first-order partial differential equations}, Trans. Amer. Math. Soc, 298,
pp. 635-641,1986.
\bibitem{BO} \textsc{T. Ba\c{s}ar and G. J. Olsder}, \textit{Dynamic
Non-cooperative Game Theory}, second ed, SIAM, Philadelphia, 1999.
\bibitem{B} \textsc{R. Bellman}, \textit{Dynamic programming}, Princeton
University Press, Princeton, 1957.
\bibitem{Br} \textsc{A. Bressan}, \textit{Noncooperative differential games}
, Milan Journal of Mathematics, 79(2):357-427, 2011.
\bibitem{ChL} \textsc{L.Y. Chen, Q. Lu}, \textit {Relationships between the
maximum principle and dynamic programming for infinite dimensional
stochastic control systems}, J. Differential Equations, 358, 103-146, 2023.
\bibitem{CV} \textsc{F. H. Clarke, R. B. Vinter}, \textit{The relationship
between the maximum prinple and dynamic programming}, SIAM J. Control Optim,
25, 1291-1311, 1987.
\bibitem{CL} \textsc{M. G. Crandall, P. L. Lions }, \textit{Viscosity
solutions of Hamilton-Jacobi equations}, Trans. Amer. Math. Soc, 277, pp.
1-42, 1983.
\bibitem{CEL} \textsc{M. G. Crandall, L. C. Evans, P. L. Lions }, \textit{\
Some properties of viscosity solutions of Hamilton-Jacobi equations}, Trans.
Amer. Math. Soc, 282, pp. 487-502, 1984.
\bibitem{D} \textsc{R. Dorfman}, \textit{An economic interpretation of
optimal control theory}, Am. Econ. Rev. 59 . 817-831, 1969.
\bibitem{FR} \textsc{W. H. Fleming and R. W. Rishel}, \textit{Deterministic
and Stochastic Optimal Control}, Springer Verlag, New York, 1975.
\bibitem{HJX} \textsc{M.S. Hu, S.L. Ji, and X.L. Xue}, \textit{Stochastic
maximum principle, dynamic programming principle, and their relationship for
fully coupled forward-backward stochastic controlled systems}, ESAIM Control
Optim. Calc. Var, 26, Article no. 81, 2020.
\bibitem{I} \textsc{R. Isaacs}, \textit{Differential games}, Wiley, New
York, 1965.
\bibitem{JFN} \textsc{J.F. Nash}, \textit{Non-cooperative games}, Annals of
Mathematics, 54, 286-295, 1951.
\bibitem{NSW} \textsc{T. Nie, J.T. Shi and Z. Wu}, \textit{Connection
between MP and DPP for stochastic recursive optimal control problems:
viscosity solution framework in the general case}, SIAM J. Control Optim.
55, 3258-3294, 2017.
\bibitem{PC} \textsc{A. Pakniyat, P.E. Caines}, \textit{On the Relation
between the Minimum Principle and Dynamic Programming for Classical and
Hybrid Control Systems}, IEEE Transactions on Automatic Control, 2017.
\bibitem{P} \textsc{L. Pontryagin, V. Boltyanskii, R. Gamkrelidze, and E.
Mishchenko}, \textit{Mathematical theory of optimal processes}, Wiley, New
York, 1962.
\bibitem{Sh} \textsc{J.T. Shi},\textit{Relationship between maximum
principle and dynamic programming for stochastic differential games of jump
diffusions}, International Journal of Control. 87(4), pp. 693-703, 2014.
\bibitem{YZ} \textsc{J. Yong and X.Y. Zhou}, \textit{Stochastic Controls:
Hamiltonian Systems and HJB Equations}, Springer-Verlag, New York, 1999.
\bibitem{Z} \textsc{X.Y. Zhou}, \textit{Maximum principle, dynamic
programming, and their connection in deterministic control},J. Optim. Theory
Appl. {65}, 363--373, 1990.
maximum principle for infinite-horizon optimal control problems in economics}
, Russ. Math. Surv. 74(6), 963, 2019.
\bibitem{BCD} \textsc{M. Bardi and I. Capuzzo-Dolcetta}, \textit{Optimal
control and viscosity solutions of Hamilton-Jacobi-Bellman equations},
Birkhauser, Boston 1997.
\bibitem{BJ} \textsc{E. N. Barron and R. Jensen}, \textit{The Pontryagin
maximum principle from dynamic programming and viscosity solutions to
first-order partial differential equations}, Trans. Amer. Math. Soc, 298,
pp. 635-641,1986.
\bibitem{BO} \textsc{T. Ba\c{s}ar and G. J. Olsder}, \textit{Dynamic
Non-cooperative Game Theory}, second ed, SIAM, Philadelphia, 1999.
\bibitem{B} \textsc{R. Bellman}, \textit{Dynamic programming}, Princeton
University Press, Princeton, 1957.
\bibitem{Br} \textsc{A. Bressan}, \textit{Noncooperative differential games}
, Milan Journal of Mathematics, 79(2):357-427, 2011.
\bibitem{ChL} \textsc{L.Y. Chen, Q. Lu}, \textit {Relationships between the
maximum principle and dynamic programming for infinite dimensional
stochastic control systems}, J. Differential Equations, 358, 103-146, 2023.
\bibitem{CV} \textsc{F. H. Clarke, R. B. Vinter}, \textit{The relationship
between the maximum prinple and dynamic programming}, SIAM J. Control Optim,
25, 1291-1311, 1987.
\bibitem{CL} \textsc{M. G. Crandall, P. L. Lions }, \textit{Viscosity
solutions of Hamilton-Jacobi equations}, Trans. Amer. Math. Soc, 277, pp.
1-42, 1983.
\bibitem{CEL} \textsc{M. G. Crandall, L. C. Evans, P. L. Lions }, \textit{\
Some properties of viscosity solutions of Hamilton-Jacobi equations}, Trans.
Amer. Math. Soc, 282, pp. 487-502, 1984.
\bibitem{D} \textsc{R. Dorfman}, \textit{An economic interpretation of
optimal control theory}, Am. Econ. Rev. 59 . 817-831, 1969.
\bibitem{FR} \textsc{W. H. Fleming and R. W. Rishel}, \textit{Deterministic
and Stochastic Optimal Control}, Springer Verlag, New York, 1975.
\bibitem{HJX} \textsc{M.S. Hu, S.L. Ji, and X.L. Xue}, \textit{Stochastic
maximum principle, dynamic programming principle, and their relationship for
fully coupled forward-backward stochastic controlled systems}, ESAIM Control
Optim. Calc. Var, 26, Article no. 81, 2020.
\bibitem{I} \textsc{R. Isaacs}, \textit{Differential games}, Wiley, New
York, 1965.
\bibitem{JFN} \textsc{J.F. Nash}, \textit{Non-cooperative games}, Annals of
Mathematics, 54, 286-295, 1951.
\bibitem{NSW} \textsc{T. Nie, J.T. Shi and Z. Wu}, \textit{Connection
between MP and DPP for stochastic recursive optimal control problems:
viscosity solution framework in the general case}, SIAM J. Control Optim.
55, 3258-3294, 2017.
\bibitem{PC} \textsc{A. Pakniyat, P.E. Caines}, \textit{On the Relation
between the Minimum Principle and Dynamic Programming for Classical and
Hybrid Control Systems}, IEEE Transactions on Automatic Control, 2017.
\bibitem{P} \textsc{L. Pontryagin, V. Boltyanskii, R. Gamkrelidze, and E.
Mishchenko}, \textit{Mathematical theory of optimal processes}, Wiley, New
York, 1962.
\bibitem{Sh} \textsc{J.T. Shi},\textit{Relationship between maximum
principle and dynamic programming for stochastic differential games of jump
diffusions}, International Journal of Control. 87(4), pp. 693-703, 2014.
\bibitem{YZ} \textsc{J. Yong and X.Y. Zhou}, \textit{Stochastic Controls:
Hamiltonian Systems and HJB Equations}, Springer-Verlag, New York, 1999.
\bibitem{Z} \textsc{X.Y. Zhou}, \textit{Maximum principle, dynamic
programming, and their connection in deterministic control},J. Optim. Theory
Appl. {65}, 363--373, 1990.
Published
2024-08-19
How to Cite
Benmenni, R., & Nourreddine, D. (2024). A Connection between the Adjoint Variables and Value Function for Differential Games. Statistics, Optimization & Information Computing. https://doi.org/10.19139/soic-2310-5070-2115
Issue
Section
Research Articles
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
