Talk:Hamilton–Jacobi–Bellman equation

message: Um, I don't think the Hamilton-Jacobi-Bellman equation is the Hamilton-Jacobi equation anymore than let's say Shannon information is the thermodynamic entropy. Phys 02:57, 15 Aug 2004 (UTC)
thread_title: Physics and Optimal Control

message: Is there a reason for using the notation \langle a,b \rangle  to denote inner product here? I'd prefer ordinary matrix notation:  a^Tb . The latter is less confusing, since it can't be confused with other variations of scalar product, such as  \int a^\prime b \, \mathrm{d} x--PeR 12:08, 14 June 2006 (UTC)
replies: text: I think really the notation used should be \nabla and the \cdot. The notation for \frac {\delta} {\delta x} where x is a vector is not particularly intuitive.- (User) Wolfkeeper (Talk) 18:28, 20 May 2009 (UTC)

text: Yes, \nabla or \nabla_x seems good to me, FWIW. And a dot for the inner product, I like also. Encyclops (talk) 19:08, 20 May 2009 (UTC)
thread_title:  Bracket notation 

message: The current article claims that the HJB is a sufficient condition. That sounds wrong to me, because first of all the equation itself is not a sufficient condition: I assume what is meant is that "if V solves HJB, this suffices to conclude that it optimizes the objective". But is this true in general? I know that in discrete-time, infinite-horizon cases, a solution of the Bellman equation only serves to identify a candidate solution for the original sequence problem, that is, solving the Bellman equation is necessary but not sufficient for optimality. (See Stokey-Lucas-Prescott, Recursive Methods in Economic Dynamics. Theorem 4.3 shows that for an infinite horizon problem, satisfying the Bellman equation and an appropriate 'transversality condition' suffices for optimality; but the Bellman equation alone is not sufficient.)
Is the sufficiency claim in this article based on the fact that the example given has a finite horizon T? If so, this should be clarified, and it would be helpful to add more general cases too.
--Rinconsoleao (talk) 08:07, 30 May 2008 (UTC)
replies: text: The continuous time/continuous state case we are looking at here is more complex than the discrete time case you mention; there are some delicate technical issues that do not arise in d.t. control.  A number of "verification theorems" have been proven, using various assumptions.  The simplest theorems, one of which goes back to Bellman, says that if a control satisfies HJB and the terminal condition, then that control is optimal.  HJB => optimality.  In this sense HJB is a sufficient condition.  However, there could exist optimal solutions that are not smooth (not continuous or not differentiable), do not satisfy HJB, but are nevertheless optimal. There are also other verification theorems that establish HJB as necessary and sufficient, but that requires additional assumptions, so they are more restrictive.  We also have to ask what kind of "solutions" of HJB we are talking about, the "classical" PDE solutions that Bellman used or the modern viscosity solutions. Frankly, my knowledge of this area is not sufficient ;-) to give an overview of all these theorems. Encyclops (talk) 23:49, 30 May 2008 (UTC)
thread_title:  Sufficient condition? 

message: I do agree with this remark. For me, HJB is a necessary condition i.e. an optimal control should necessarily satisfied HJB. I refer to Oksendal (Stochastic differential equations, theorem 11.2.1).
The important question is to know if the finding of a solution of a HJB equation is sufficient. No. Once found, the solution of the HJB PDE has to satisfies some criteria (a verification theorem, Oksendal 11.2.2). 
HJB is necessary but not sufficient.
130.104.59.97 (talk) 09:31, 3 December 2009 (UTC)
thread_title:  Sufficient condition? A confirmation 

message: I wonder if the two last terms (before the big O) of the last equation should not be multiplied by dt.  —Preceding unsigned comment added by 61.26.5.133 (talk) 03:18, 13 March 2009 (UTC)
replies: text: I think you may be right. Any other opinions ? Encyclops (talk) 01:42, 16 March 2009 (UTC)

text: Agreed. Done. --Rinconsoleao (talk) 09:44, 8 June 2009 (UTC)
thread_title:  Multiply by dt? 

message: In section The partial differential equation I see a terminal condition which does not quite look like a meaningful terminal condition. Is it actually meant to be
:
V(x(T),T) = D(x(T))
and if so, may I correct that?
Thank you
--Andylong (talk) 11:32, 3 July 2012 (UTC)
thread_title:  Terminal condition 

message: Here we have a control problem with
x(0) = x0.
Okay. But often we solve control problems where in addition there is a terminal constraint
x(T) = xE.
I ran through the literature and I must say that I fail at finding the HJB in that case.
In particular, right now we use V(t,X) := min_(x,u){ int_t^T C(x(s),u(s)) ds + D(x(T)) s.t. x(t)=X }.
If we were to attempt V(t,X) := min_(x,u){ int_t^T C(x(s),u(s)) ds + D(x(T)) s.t. x(t)=X, x(T)=xE }
then V(T,X) for X!=xE would no longer be well-defined.
It would be crazy if that problem was unsolvable via the article's technique. Certainly, because xE can be chosen freely or as parameter of the optimization problem,
once the case xE has been presented in this article, the one-sided case can be deleted afterwards in favour of brevity.  — Preceding unsigned comment added by 2A02:908:1657:A860:71C7:F597:421E:F6B8 (talk) 21:47, 26 October 2021 (UTC)
thread_title:  Terminal Constraint