Talk:Baum–Welch algorithm

message: Could Template:HMM example be put to use on this page?  (Granted we might have to modify it a bit to make it more general) -- Kowey 14:16, 14 Nov 2004 (UTC)
thread_title:  Suggestion 

message: There is a document written by Welch on IEEE Information Theory Society Newslectter, December 2003. See
http://www.itsoc.org/publications/nltr/it_dec_03final.pdf
I scrutinized the description, and I also think it is wrong.
We have:
gamma_t(i): the probability that an HMM is in state s_i at time t given the observations O and the parameters Lambda.               (1)
gamma_t(i)=P(s_i=q_t | O, Lambda)        (2)
gamma_t(i)=alpha_t(i)*beta_t(i)
           --------------------          (3)
             P(O|Lambda)
(3) is the Forward-Backward probability of state s_t(i), the probability one is in state s_i at t given time t.  So this is the role the Foward-Backward algorithm plays in B W.
Now, let's look at the probability of going through a TRANSITION from s_i at time t to s_j at time t+1. Let's call it "chi'
Chi(i,j) is the probability of being in state s_i at time t, and being in state s_j at time t+1 given the observations and the parameters.                       (4)
Chi(i,j)=P(s_i=q_t, s_j=q_t+1 | O, Lambda) (5)
Chi(i,j)=alpha_t(i)a_i_jb_j(ot+1)*beta_t+1(j)
         --------------------
           P(O|Lambda)      (6)
    T-1
SIGMA   Chi(i,j) is the expected number of
    t=1
        transitions from s_i to s_j
    T-1
SIGMA   gamma_t(i) is the expected number of
    t=1
        transitions from s_i
                   T-1
               SIGMA   Chi(i,j)
                   t=1
Updated a_i_j =------------------------
                   T-1
               SIGMA   gamma_t_(i)
                   t=1
It is the last step I am not getting.  Certainly the enumerator is not constant, as the current article claims, nor is it equal to the probability of the entire string. I cannot see why updated a_i_j is greater than the old value.
References
Cutting, D., and Kupiec, J., Pedersen, and P Sibun "A Practical Part-of-Speech Tagger", Xerox Palo Alto Research Center.
ChengXiang Zhai, A Brief Note On The Hidden Markov Models, Lecture Notes, Dept. of CS, University of Illinois at Urbana-Champaign.
tp://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-345Automatic-Speech-RecognitionSpring2003/4C063D4A-3B8B-4F05-B475-85AAD0D66586/0/lecture10.pdf
Koos van der Wilt Koos van der Wilt 00:52, 13 May 2007 (UTC)
replies: text: Were you discussing the article or the Welch presentation? Which one do you feel wrong? I think the latter, please confirm. I've not read your contribution, but I remember several headaches when trying to make sense of this algorithm in Pattern Classification (2ed) by Richard Duda et al. (); I was convinced the description was wrong, and I rememeber I thought it was difficult to fix it.
--Blaisorblade (talk) 03:09, 24 June 2008 (UTC)
thread_title:  Content Discussion 

message: I think the link is confusing. The current text of Forward-backward algorithm seems to describe a different algorithm, which uses a forward and a backward pass but combines them for other purposes (at least, it appears so to me). Also the Baum-Welch algorithm does such a forward and backward pass, so it is called forward-backward algorithm somewhere (for instance in Pattern Classification (2ed) by Richard Duda et al. ()).
--Blaisorblade (talk) 03:09, 24 June 2008 (UTC)
thread_title:  Confusion over Forward-backward algorithm 

message: In the formula for the epsilon variables, i and j are both free indices, and used for summation. 2001:6B0:1:1DF0:C185:CDB5:7AF7:29BA (talk) 17:58, 19 October 2013 (UTC)
thread_title:  Ambiguous indices 

message: There is one chicken and we observe whether it lays eggs or not at a day. Why are the observations then like NN, NE, EE, EN, that is, why are there two observations each day? I don't get it. HenningThielemann (talk) 17:33, 25 January 2015 (UTC)
thread_title:  Confusing chicken and egg example 

message: The description ends with "These steps are now repeated iteratively until a desired level of convergence." What steps? Although it would be nice to specify how convergence is measured/detected, I'll leave that aside for now. Isn't the algorithm meant to be evaluated for T=1, then T=2, T=3, and so on until a desired level of convergence is attained? If true, the text should reflect that (it currently does not, since the "steps" are not linked with T). Urhixidur (talk) 15:54, 24 October 2017 (UTC)
thread_title:  Convergence? 

message: I have written the following code to reproduce the numbers shown in the example.
!pip install hmmlearn
import numpy as np
from hmmlearn import hmm
np.random.seed(42)
model = hmm.MultinomialHMM(n_components=2, n_iter=100)
model.startprob_ = np.array([0.2, 0.8])
model.transmat_ = np.array([[0.5, 0.5],
                            [0.3, 0.7]])
model.emissionprob_ = np.array([[0.3, 0.7],
                               [0.8, 0.2]])
model.sample(100)
model.fit(0,0,0,0,0,1,1,0,0,0)
Chrism2671 (talk) 16:57, 11 April 2022 (UTC)
thread_title:  Reproduction