Talk:Bellman equation

message: Shuroo (talk) 09:19, 29 May 2009 (UTC)
thread_title:  General explanation 

message: I can't find the "principle of optimality" anywhere in Bellman's 1952 paper.  Perhaps this should be his 1957 book.  —Preceding unsigned comment added by 129.100.144.166 (talk) 17:01, 11 December 2007 (UTC)
replies: text: Principle of Optimality. An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.
--Rinconsoleao (talk) 19:57, 25 May 2009 (UTC)
thread_title: BPO

message: The equation given in the article is wrong, or at least not true in general. The right equation is
:V(x) = \max_{y \in \Gamma (x) } [F(x,y) + \beta V(T(x,y))], \forall x \in X.
where T(x,y) is the transformation of the state given a state x and an action y.
Shuroo  —Preceding unsigned comment added by Shuroo (talk • contribs) 20:27, 27 May 2009 (UTC)
thread_title: Notation for dynamic problems

message: Yet another argument for my notation is that it can be easily extended to stochastic problems (e.g., inventory management). To obtain the stochastic form of the equation we simply add E (expectation) before the second term since the new state is a random variable.
:V(x) = \max_{y \in \Gamma (x) } [F(x,y) + \beta E(V(T(x,y)))], \forall x \in X.
where T(x,y) is the transformation of the state given a state x and an action y.
Shuroo (talk) 09:21, 29 May 2009 (UTC)
thread_title: Notation for stochastic problems

message: "It breaks a dynamic optimization problem into simpler subproblems, writing the value of a decision problem at a certain point in time in terms of the payoff from some initial choices and the value of the remaining decision problem that results from those initial choices, as Bellman's Principle of Optimality prescribes." this sentance is VERY long and obfuscated ; it may deserve a reformulation dont you think  —Preceding unsigned comment added by 124.180.239.182 (talk) 09:35, 20 July 2010 (UTC)
thread_title: Wording

message: As I understand it, the mathematical expression given under the Bellman's Principle of Optimality section is incorrect.  The claim is made that it is the same as (the RHS of, presumably) the previous equation, but I don't think that's true -- as it stands, it is actually a description of a greedy solution, not a solution involving the principle of optimal substructure.  The reason being that the first term greedily selects the action at time 0 giving maximum payoff, when in fact a suboptimal action at this time may maximise the value overall by bringing the system into a different state at time 1 whose value (maximum possible discounted sum of payoffs), when added to this (suboptimal) time-0 payoff, is greater.  Not sure what the correct expression would be I'm afraid.  130.123.96.22 (talk) 03:18, 15 February 2011 (UTC)
replies: text: No, the expression does not refer to a 'greedy' algorithm-- in other words, it is not maximizing the first-period payoff in isolation. The control a0 is chosen so that it maximizes the first-period payoff PLUS the continuation value of the rest of the problem (which need not imply that it maximizes the first-period payoff by itself, as a greedy algorithm would). I will add another set of parentheses to make this clearer. Rinconsoleao (talk) 09:06, 15 February 2011 (UTC)
thread_title: Greedy?

message: I think the term "value function" deserves an article alone!Luizabpr (talk) 19:07, 21 May 2011 (UTC)
replies: text: belated, I gave it one. Cheers. --bender235 (talk) 01:57, 23 June 2020 (UTC)
thread_title: Value function 

message: Hello fellow Wikipedians,
I have just modified  on Bellman equation. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20050110161049/http://www.wu-wien.ac.at/usr/h99c/h9951826/bellman_dynprog.pdf to http://www.wu-wien.ac.at/usr/h99c/h9951826/bellman_dynprog.pdf
When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at ).
Cheers.—InternetArchiveBot (Report bug) 11:00, 30 October 2016 (UTC)
thread_title:  External links modified 

message: Hello fellow Wikipedians,
I have just modified one external link on Bellman equation. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20071012194420/http://decision.csl.uiuc.edu/~meyn/pages/book.html to http://decision.csl.uiuc.edu/~meyn/pages/book.html
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 15:19, 17 July 2017 (UTC)
thread_title:  External links modified 

message: Isn't that basically the main application of Bellman equation? 98.156.185.48 (talk) 02:06, 29 September 2023 (UTC)
thread_title:  Reinforcement learning isn't mentioned?