How does AI work? Part 3
Tags: ai, ml, math, machinelearning
Published: 2023-01-25 21:35:33

---
series: How does AI work?
---
## Part 3 — Training Models

In Parts 1 and 2 of this series we built our models by inspection. This was a good way to introduce the concepts involved but in reality what we’ll want to do is “train” our models using the data itself. This means setting the states’ output probabilities and transition probabilities. The more observations we have, the better.

The technique we’ll use is called **Viterbi alignment**. First we set the initial values for the transition and output probabilities and then we iterate in a series of refinements until we converge. That is, until the output probabilities no longer change with a new iteration. Let’s dig a little further into this process.

First we set the initial values:
* Roughly divide each observation into the number of states in the model e.g. for 3 states divide each observation into 3 parts of similar size.
* Get average number of data points (frames) per observation.
* Divide average number of frames per observation by number of states to get average frames per state (time expected to remain in a state).
* Set {% katex inline %}P({% endkatex %}*transition out of each state*{% katex inline %})=1/{% endkatex %}*average frames per state*.
* {% katex inline %}P({% endkatex %}*self transition*{% katex inline %})=1-P({% endkatex %}*transition out of each state*{% katex inline %}){% endkatex %}. We’ll now refer to transitions out and to the same state simply as {% katex inline %}P(transitions){% endkatex %}. Initially all states will have the same probabilities.
* Calculate mean and standard deviation for each state.

Let’s go over those steps with a real example. Let’s say we have three observation sequences. Like in our previous examples, we’re tracking only one feature: the change in the position of one hand on the y axis.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations1.webp)
###### Figure 1. Initial values

Our three observations in *Figure 1* have 16, 5, and 14 frames respectively. That gives us approximately 12 average frames per observation. Assuming we’re still working with 3 states in our model, 12 avg frames/3 states = 4 avg frames (data points) per state. If we’re spending 4 frames in each state on average, then we should expect to escape each state {% katex inline %}1/4{% endkatex %} of the time. As a result, the self transition probability is {% katex inline %}3/4{% endkatex %}. So {% katex inline %}P({% endkatex %}*transition out*{% katex inline %})=0.25{% endkatex %} and {% katex inline %}P({% endkatex %}*self transition*{% katex inline %})=0.75{% endkatex %}.

Let’s divide each observations into similarly sized thirds (for three states).

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations2.webp)
###### Figure 2. Dividing each observation into 3 states

Now calculate the mean and standard deviation for each state using the data points for each state as we just defined them.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations3.webp)
###### Figure 3. Gaussians for each state

Now comes the iteration. We’ll see if moving the boundaries between the states will lead to a better explanation of the data. We’re looking to see if we can get Gaussians with a lower variance. We’ll converge once we no longer need to move any boundaries and none of the output probabilities (mean and standard deviation) change.

This is the process at a high level. While not converged:

* Move the state boundaries in the observation sequences whenever a frame is closer to the neighboring state than to its current state as measured by the distance to the mean (in standard deviations).
* Recalculate {% katex inline %}P(transitions){% endkatex %} for each state ({% katex inline %}1/{% endkatex %}*average frames*{% katex inline %}){% endkatex %}.
* Recalculate output probabilities for each state (mean and standard deviation).

Let’s apply these steps to our example. On the first sequence, look at the boundary between states 1 and 2. The {% katex inline %}5{% endkatex %} is much closer to the mean of state 1 {% katex inline %}(5.4){% endkatex %} than to the mean of state 2 {% katex inline %}(-0.6){% endkatex %} as measured in standard deviations so that boundary should move to the right moving the {% katex inline %}5{% endkatex %} to state 1.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations4.webp)
###### Figure 4. The boundary between S1 and S2 moves to the right

Let’s keep going. What about the {% katex inline %}2{% endkatex %} that is now at the new boundary between states 1 and 2? As shown in Figure 5, the {% katex inline %}2{% endkatex %} is a little more than {% katex inline %}1{% endkatex %} standard deviation away from the mean of S1 and less than {% katex inline %}1{% endkatex %} standard deviation from the mean of S2, so it’s still better in the second state (for now). By the time we finish this iteration the mean and standard deviation for the states will have changed as data points move to another state.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations5.webp)
###### Figure 5. Determining if the boundary needs to move

Let’s see if any other boundaries move in this iteration. The {% katex inline %}-1{% endkatex %} is not going to move, it’s way closer to {% katex inline %}-0.6{% endkatex %} than to {% katex inline %}-5{% endkatex %}. In the second observation, the {% katex inline %}-7{% endkatex %} is closer to the mean of S3, so that boundary moves.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations6.webp)

In the last observation, the {% katex inline %}3{% endkatex %} is closer to the mean of S1 and the {% katex inline %}-5{% endkatex %} is right at the mean of S3, so those move.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations7.webp)

How about the {% katex inline %}-3{% endkatex %} in S2 of the last observation? It’s {% katex inline %}2.4/3.1=0.774{% endkatex %} standard deviations away from state 2 and {% katex inline %}2/2.6=0.769{% endkatex %} standard deviations away from state 3 so we’ll move it to state 3.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations8.webp)

We’re done moving boundaries so let’s recalculate everything. First the transition probabilities:

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations9.webp)
###### New transition probabilities

And then the new output probabilities (Gaussians):

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations10.webp)
###### New output probabilities

We’ve reached the end of the first iteration! Let’s start the second iteration and see if we need to move any boundaries given our new mean and std dev for the outputs.

In the first observation, the {% katex inline %}2{% endkatex %} is now closer to S1 than to S2 so it moves to S1.

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations11.webp)

The {% katex inline %}-1{% endkatex %} stays in S2 in both observations 1 and 3. No more moves needed so let’s recalculate everything. First the transition probabilities:

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations12.webp)
###### New transition probabilities

And then the output probabilities (Gaussians):

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations13.webp)
###### New output probabilities

We finished the second iteration! In the next iteration none of the boundaries are going to move nor the output probabilities going to change so we’ve converged! We have now initialized the values for each state of our HMM. We:
* Assigned a state to each time frame in each of our observations.
* Calculated the resulting averages for all values assigned to each state.
* Calculated the average amount of time we expect to stay in each state.

Here’s our final result:

![Observations](https://raw.githubusercontent.com/santisbon/reference/main/assets/img/observations14.webp)
###### Final transition and output probabilities