Talk:Forward–backward algorithm

message: The O_1 matrix listed in the Forward Algorithm section is inconsistent with the one presented in the Example section, even though both the transition matrix (T) and the emission matrix (B) are the same between the two sections. I believe the O_1 matrix listed in the example section is correct. This would be much more clear if there were more hidden states than emissions or vice versa, since the matrix dimensions would be indicative of whether states are being conditioned on emissions or if emissions are being conditioned on states. No edits made.
---  — Preceding unsigned comment added by 129.67.44.219 (talk) 16:44, 26 March 2013 (UTC)
thread_title: Untitled

message: I am against the proposed merge of forward algorithm into this article, because the forward algorithm is an important algorithm in its own right; it is probably more frequently employed than the forward-backward algorithm. Since the forward algorithm is used as a subprocedure of the forward-backward algorithm (it is precisely step 1 in the description), I would propose the following: We make the description of the algorithm a template, which we then simply include in both articles. 131.159.0.7 (talk) 17:18, 21 July 2010 (UTC)
replies: text: Removed the merge proposal since it's been around for a year, nobody defended it and I'm the third reader to disagree here. The forward algorithm is useful in its own right and deserves its own article. Qwertyus (talk) 09:46, 2 August 2011 (UTC)
replies: text: Someone took a bold step and replaced "Forward algorithm" with a redirect (edit was 30. september 2011 kl. 19:34‎ User:Max Libbrecht). I assume good faith, but having seen the discussion here, and no discussion anywhere else that supports the effective deletion, I'm going to revert the Forward algorithm page to being a separate item. I might also put a "not to be confused with..." --mcld (talk) 10:58, 2 May 2012 (UTC)
thread_title:  Merge with Forward Algorithm 

message: I extended the page with a brief overview a semi formal description of the algorithm, referenced some performance improvements and included an example. I personally feel that the matrix based description is best to follow. Additionally I felt that it is most helpful to understand the workings of this algorithm through a numerical example. Therefore I included a (rather extensive) example based on an example presented in the widely used AI text book of Russel and Norvig. I also referenced a repository of source code where java code implementing the algorithm may be found. 
I am generally new to editing at Wikipedia but tried to follow the guidelines I found to be relevant. If the content is not found suitable, contains errors or is otherwise unsuitable, I certainly welcome feedback, critique and any edits, including deletions, deemed necessary :)
BJJV (talk) 11:08, 26 May 2008 (UTC)
thread_title:  Page Cleanup 

message: It was a terrible idea to explain this algorithm in terms of matrices. Yes, it makes it so that you can elegantly describe the algorithm with a single formula, but it has many disadvantages, including:
#It makes the algorithm inaccessible to people that aren't comfortable with matrix math.
#It performs superfluous computations that would not be done in an efficient implementation.
#It hides context, and makes it easy to forget the meaning of columns and rows.
#The matrix math shown here is broken anyway. It shows row vectors where column vectors should be, and implies impossible calculations with matrices that aren't compatible for multiplication. When the smoothed values are computed, the symbol "x" is used, which implies cross-product, but straight scalar multiplication is used.
If we *really must* use matrix math to explain this algorithm, it needs to be cleaned up a lot. Frankly, I think the traditional explanation that uses probabilities and intuition is much better suited for an encyclopedic article. I propose that we rip out the matrix math completely. Also, I agree very much with the comment made earlier that this algorithm is so similar to the Viterbi algorithm that they should be explained together, and in a consistent manner.  —Preceding unsigned comment added by 128.187.80.2 (talk) 20:33, 3 February 2009 (UTC)
replies: text: Strongly agree. I work with matrix maths a lot, but it's obvious to me that the explanation given in this page is quite hard to understand. It seems to be oriented towards computational efficiency rather than clear exposition. The bit with the emission probabilities placed into a diagonal matrix in particular, really rams the point home that the layout is not so we can understand it! --mcld (talk) 11:05, 2 May 2012 (UTC)
thread_title: Explaining this algorithm with Matrices

message: It has been proposed since 2009 to merge BCJR algorithm into this article. Any thoughts? --KarlB (talk) 22:18, 7 July 2012 (UTC)
replies: text: The merge tag was only added and no discussion was initiated. I am therefore closing the merge request as improper and removing the tags. WTF? (talk) 18:18, 11 October 2012 (UTC)
thread_title:  Merge with BCJR algorithm 

message: Unfortunate that there's no discussion, since the explanation of BCJR & viterbi, when applied in the digital domain, is considerably easier to understand than the HMM version given here.  Unfortunately, the BCJR article is currently a stub with no content in it :-(  Anyway, just to make this article easier to understand, I think its important to expand on these other, easier-to-explain, easier-to-comprehend algorithms. User:Linas (talk) 00:46, 9 November 2013 (UTC)
thread_title:  [[ECC]] & BCJR 

message: The notation used on this page is inconsistent with the notation used on the HMM, forward algorithm, and Viterbi algorithm pages. Namely, in the latter cases, x represent the hidden states and y the observations, whereas here o represent the hidden states with x being the observations. I know that generally there is no standard notational convention across the many HMM papers, but for this set of articles, it would be much less confusing to stick with a consistent notation. Thoughts?  — Preceding unsigned comment added by OhGodItsSoAmazing (talk • contribs) 19:32, 7 December 2013 (UTC)
thread_title:  Consistent notation across HMM articles 

message: I've been looking for a reference that describes the efficient extension when using an order greater than one. I know the basic trick is to take (Q x Q) --> Q', then use this space which maps from [q_k at t-1 and q_l at t] to [q_j at t and q_i at t+1]. Clearly q_l=q_j. This causes many zeros in the transition probability matrix. 
Without modifying the algorithm, time is wasted on these zeros. A second order HMM implemented naively has a run time of O(N^4 T). But if we squeeze the zeros out in the algorithm, then this reduces to O(N^3 T). A third order HMM implemented naively runs in O(N^6 T), but squeezing the zeros out to avoid the algorithm running over them would run in O(N^4 T). I've done some work myself, but without references it would count as original research.
I'll provide one possible extension in the forward pass below (using the forward algorithm listed here in non-matrix form http://www.cs.sjsu.edu/~stamp/RUA/HMM.pdf):
f0:1(i) = pi(i)*b_i(O_1)
f0:2(j,i) = pi(j,i)*b_{j,i}(O_2), using pi(j,i) as the joint prior probability of state q_j followed by state q_i. Alternatively
f0:2(j,i) = f0:1(j) * T{j}{i} * b_{j,i}(O_2), but here T{j,i} is an element of the first-order transition matrix
f0:t(j,i) = sum_{l,k}^{N} f0:{t-1}(k,l) * T_{k,l}{j,i} * B{j,i}(O_t}, but q_l=q_j. Squeezing the zeros out give:
f0:t(j,i) = sum_{k}^{N} f0:{t-1}(k,j) * T_{k,j}{j,i} * B{j,i}(O_t}, for i and j in 1, 2, ..., N
Mouse7mouse9 20:20, 28 January 2014 (UTC)
thread_title:  Request for efficient higher order extension