Finding the probability of a state at a given time in a Markov chain | Set 2 - GeeksforGeeks

07 Jul, 2021
Given a Markov chain G, we have the find the probability of reaching the state F at time t = T if we start from state S at time t = 0.A Markov chain is a random process consisting of various states and the probabilities of moving from one state to another. We can represent it using a directed graph where the nodes represent the states and the edges represent the probability of going from one node to another. It takes unit time to move from one node to another. The sum of the associated probabilities of the outgoing edges is one for every node.Consider the given Markov Chain( G ) as shown in below image:  
Examples:  
Input : S = 1, F = 2, T = 1
Output: 0.23
We start at state 1 at t = 0, 
so there is a probability of 0.23 
that we reach state 2 at t = 1.

Input: S = 4, F = 2, T = 100
Output: 0.284992
 
In the previous article, a dynamic programming approach is discussed with a time complexity of O(N2T), where N is the number of states. Matrix exponentiation approach: We can make an adjacency matrix for the Markov chain to represent the probabilities of transitions between the states. For example, the adjacency matrix for the graph given above is:
    
We can observe that the probability distribution at time t is given by P(t) = M * P(t – 1), and the initial probability distribution P(0) is a zero vector with the Sth element being one. Using these results, we can get solve the recursive expression for P(t). For example, if we take S to be 3, then P(t) is given by,
    
If we use effective matrix exponentiation technique, then the time complexity of this approach comes out to be O(N3 * log T). This approach performs better than the dynamic programming approach if the value of T is considerably higher than the number of states, i.e. N.Below is the implementation of the above approach:  
C++
Java
Python3
C#
Javascript
// C++ implementation of the above approach#include <bits/stdc++.h>using namespace std; // Macro to define a vector of float#define vf vector<float> // Function to multiply two matrices A and Bvector<vf > multiply(vector<vf > A,    vector<vf > B, int N){    vector<vf > C(N, vf(N, 0));    for (int i = 0; i < N; ++i)        for (int j = 0; j < N; ++j)            for (int k = 0; k < N; ++k)                C[i][j] += A[i][k] * B[k][j];    return C;} // Function to calculate the power of a matrixvector<vf > matrix_power(vector<vf > M, int p, int n){    vector<vf > A(n, vf(n, 0));    for (int i = 0; i < n; ++i)        A[i][i] = 1;     while (p) {        if (p % 2)            A = multiply(A, M, n);        M = multiply(M, M, n);        p /= 2;    }     return A;} // Function to calculate the probability of// reaching F at time T after starting from Sfloat findProbability(vector<vf > M, int N, int F,                                        int S, int T){    // Storing M^T in MT    vector<vf > MT = matrix_power(M, T, N);     // Returning the answer    return MT[F - 1][S - 1];} // Driver codeint main(){    // Adjacency matrix    // The edges have been stored in the row    // corresponding to their end-point    vector<vf > G{ { 0, 0.09, 0, 0, 0, 0 },                   { 0.23, 0, 0, 0, 0, 0.62 },                   { 0, 0.06, 0, 0, 0, 0 },                   { 0.77, 0, 0.63, 0, 0, 0 },                   { 0, 0, 0, 0.65, 0, 0.38 },                   { 0, 0.85, 0.37, 0.35, 1.0, 0 }};     // N is the number of states    int N = 6;         int S = 4, F = 2, T = 100;     cout << "The probability of reaching " << F << " at time "                << T << "\nafter starting from " << S << " is "                << findProbability(G, N, F, S, T);     return 0;}
// Java implementation of the above approachclass GFG{     // Function to multiply two matrices A and B    static double[][] multiply(double[][] A,            double[][] B, int N)    {        double[][] C = new double[N][N];        for (int i = 0; i < N; ++i)            for (int j = 0; j < N; ++j)                for (int k = 0; k < N; ++k)                    C[i][j] += A[i][k] * B[k][j];        return C;    }     // Function to calculate the power of a matrix    static double[][] matrix_power(double[][] M, int p, int n)    {        double[][] A = new double[n][n];        for (int i = 0; i < n; ++i)            A[i][i] = 1;         while (p > 0)        {            if (p % 2 == 1)                A = multiply(A, M, n);            M = multiply(M, M, n);            p /= 2;        }        return A;    }     // Function to calculate the probability of    // reaching F at time T after starting from S    static double findProbability(double[][] M,                    int N, int F, int S, int T)    {        // Storing M^T in MT        double[][] MT = matrix_power(M, T, N);         // Returning the answer        return MT[F - 1][S - 1];    }     // Driver code    public static void main(String[] args)    {        // Adjacency matrix        // The edges have been stored in the row        // corresponding to their end-point        double[][] G = { { 0, 0.09, 0, 0, 0, 0 },                        { 0.23, 0, 0, 0, 0, 0.62 },                        { 0, 0.06, 0, 0, 0, 0 },                        { 0.77, 0, 0.63, 0, 0, 0 },                        { 0, 0, 0, 0.65, 0, 0.38 },                        { 0, 0.85, 0.37, 0.35, 1.0, 0 } };         // N is the number of states        int N = 6;         int S = 4, F = 2, T = 100;         System.out.printf(                "The probability of reaching " + F +                " at time " + T + "\nafter starting from " +                        S + " is %f",                findProbability(G, N, F, S, T));    }} // This code is contributed by Rajput-Ji
# Python implementation of the above approachfrom typing import List # Function to multiply two matrices A and Bdef multiply(A: List[List[float]], B: List[List[float]],             N: int) -> List[List[float]]:    C = [[0 for _ in range(N)] for _ in range(N)]    for i in range(N):        for j in range(N):            for k in range(N):                C[i][j] += A[i][k] * B[k][j]    return C # Function to calculate the power of a matrixdef matrix_power(M: List[List[float]], p: int, n: int) -> List[List[float]]:    A = [[0 for _ in range(n)] for _ in range(n)]    for i in range(n):        A[i][i] = 1    while (p):        if (p % 2):            A = multiply(A, M, n)        M = multiply(M, M, n)        p //= 2    return A # Function to calculate the probability of# reaching F at time T after starting from Sdef findProbability(M: List[List[float]], N: int, F: int, S: int,                    T: int) -> float:     # Storing M^T in MT    MT = matrix_power(M, T, N)     # Returning the answer    return MT[F - 1][S - 1]  # Driver codeif __name__ == "__main__":     # Adjacency matrix    # The edges have been stored in the row    # corresponding to their end-point    G = [[0, 0.09, 0, 0, 0, 0], [0.23, 0, 0, 0, 0, 0.62],         [0, 0.06, 0, 0, 0, 0], [0.77, 0, 0.63, 0, 0, 0],         [0, 0, 0, 0.65, 0, 0.38], [0, 0.85, 0.37, 0.35, 1.0, 0]]     # N is the number of states    N = 6    S = 4    F = 2    T = 100     print(        "The probability of reaching {} at time {}\nafter starting from {} is {}\n"        .format(F, T, S, findProbability(G, N, F, S, T))) # This code is contributed by sanjeev2552
// C# implementation of the above approachusing System; class GFG{     // Function to multiply two matrices A and B    static double[,] multiply(double[,] A,                            double[,] B, int N)    {        double[,] C = new double[N, N];        for (int i = 0; i < N; ++i)            for (int j = 0; j < N; ++j)                for (int k = 0; k < N; ++k)                    C[i, j] += A[i, k] * B[k, j];        return C;    }     // Function to calculate the power of a matrix    static double[,] matrix_power(double[,] M, int p, int n)    {        double[,] A = new double[n,n];        for (int i = 0; i < n; ++i)            A[i, i] = 1;         while (p > 0)        {            if (p % 2 == 1)                A = multiply(A, M, n);            M = multiply(M, M, n);            p /= 2;        }        return A;    }     // Function to calculate the probability of    // reaching F at time T after starting from S    static double findProbability(double[,] M,                    int N, int F, int S, int T)    {        // Storing M^T in MT        double[,] MT = matrix_power(M, T, N);         // Returning the answer        return MT[F - 1, S - 1];    }     // Driver code    public static void Main(String[] args)    {        // Adjacency matrix        // The edges have been stored in the row        // corresponding to their end-point        double[,] G = { { 0, 0.09, 0, 0, 0, 0 },                        { 0.23, 0, 0, 0, 0, 0.62 },                        { 0, 0.06, 0, 0, 0, 0 },                        { 0.77, 0, 0.63, 0, 0, 0 },                        { 0, 0, 0, 0.65, 0, 0.38 },                        { 0, 0.85, 0.37, 0.35, 1.0, 0 } };         // N is the number of states        int N = 6;         int S = 4, F = 2, T = 100;         Console.Write("The probability of reaching " + F +                " at time " + T + "\nafter starting from " +                        S + " is {0:F6}",                findProbability(G, N, F, S, T));    }} // This code is contributed by 29AjayKumar
<script> // Javascript implementation of the above approach// Function to multiply two matrices A and Bfunction multiply(A, B, N){    var C = Array.from(Array(N), ()=>Array(N).fill(0));    for (var i = 0; i < N; ++i)        for (var j = 0; j < N; ++j)            for (var k = 0; k < N; ++k)                C[i][j] += A[i][k] * B[k][j];    return C;} // Function to calculate the power of a matrixfunction matrix_power(M, p, n){    var A = Array.from(Array(n), ()=>Array(n).fill(0));    for (var i = 0; i < n; ++i)        A[i][i] = 1;    while (p > 0)    {        if (p % 2 == 1)            A = multiply(A, M, n);        M = multiply(M, M, n);        p = parseInt(p/2);    }    return A;} // Function to calculate the probability of// reaching F at time T after starting from Sfunction findProbability(M, N, F, S, T){     // Storing M^T in MT    var MT = matrix_power(M, T, N);         // Returning the answer    return MT[F - 1][ S - 1];} // Driver code// Adjacency matrix// The edges have been stored in the row// corresponding to their end-pointvar G = [ [ 0, 0.09, 0, 0, 0, 0 ],                [ 0.23, 0, 0, 0, 0, 0.62 ],                [ 0, 0.06, 0, 0, 0, 0 ],                [ 0.77, 0, 0.63, 0, 0, 0 ],                [ 0, 0, 0, 0.65, 0, 0.38 ],                [ 0, 0.85, 0.37, 0.35, 1.0, 0 ] ];                 // N is the number of statesvar N = 6;var S = 4, F = 2, T = 100;document.write("The probability of reaching " + F +        " at time " + T + "<br>after starting from " +                S + " is " +        findProbability(G, N, F, S, T).toFixed(6)); // This code is contributed by rrrtnx.</script>
The probability of reaching 2 at time 100
after starting from 4 is 0.284991
 
Time Complexity: O(N3 * logT) Space Complexity: O(N2) 
vaibhav29498
Rajput-Ji
29AjayKumar
sanjeev2552
rrrtnx
matrix-exponentiation
Probability
Technical Scripter 2018
Graph
Mathematical
Matrix
Technical Scripter
Mathematical
Matrix
Graph
Probability
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Kruskal’s Minimum Spanning Tree Algorithm | Greedy Algo-2
Topological Sorting
Bellman–Ford Algorithm | DP-23
Floyd Warshall Algorithm | DP-16
Detect Cycle in a Directed Graph
Program for Fibonacci numbers
Write a program to print all permutations of a given string
C++ Data Types
Set in C++ Standard Template Library (STL)
Coin Change | DP-7