Introduction to Solving Basic Bayesian Networks with OpenBUGS in R | by Tri Nguyen | Towards Data Science

A Bayesian network is a probabilistic graphical model that represents a set of variables and their conditional dependencies via an oriented acyclic graph. “Oriented” means that the arrows connecting the nodes of the graph are oriented (i.e. has directions) so the conditional dependencies have a one-way direction. “Acylic” means that it is impossible to loop through the network. It can only go up or down.
Let’s start with an example:
Manchester United is competing in the English Premier League. They have been struggling to keep up with their image of one of the world’s biggest club in recent years after the retirement of Sir Alex Ferguson, the most successful manager in the club’s history. Last year, they only finished 6th in the league. As such, at the beginning of the 2019/20 season, the fans believe that the club will finish in Top 3 of the league with only a probability of 0.2, in 4th place with the probability of 0.3, and outside of Top 4 with the probability of 0.5.
These probabilities hold true only if the club’s key player is Marcus Rashford is not severely injured throughout the season. The probability of a severe injury happening to him is 0.2. If that happens, the probabilities for the above end of season positions are 0.1, 0.2, and 0.7 respectively.
If they finish in Top 3, they will be qualified for next year’s Champions League Group Stage. If they finish 4th, they will have to go through Champions League Qualification Round so the probability of them proceeding to the Champions League Group Stage is reduced to 0.7.
Manchester United is also competing in the Europa League, second-tier to the Champions League in Europe. As a former winner, they have a 0.6 probability of winning the Europa League again if Rashford is fit. If Rashford is injured, the probability is reduced to 0.4. If they win the Europa League, they are guaranteed to have a place in next year’s Champions League Group Stage with 0.99 probability even if they don’t get into Top 3 unless UEFA changes the rules for next year.
If the club is qualified for next year’s Champions League Group Stage, they will have a 0.7 chance to sign one of their top transfer targets, Bruno Fernandes from Sporting Lisbon. If not, the chance to sign Fernandes is reduced to 0.4.
At the beginning of the 2020/2021 season, Bruno Fernandes is a Manchester United player.
- What is the probability that Rashford is injured?
- What is the probability that the club was qualified for Champions League?
- What is the probability that the club finished in Top 3?
- What is the probability that the club finished 4th?
- What is the probability that the club won the Europa League?
Some questions of this style are straight forward to solve analytically using Bayes’ theorem formula when the number of variables and dependencies involved is small. However, for problems like the above example, it will be much easier to use software applications for the Bayesian analysis to find the probabilities in question.
Here, we will use the library “R2OpenBUGS” in R to solve for those probabilities. The library is based on the OpenBUGS software, which is for the Bayesian analysis of complex statistical models using Markov chain Monte Carlo (MCMC) methods. The “BUGS” stands for (Bayesian inference Using Gibbs Sampling). As such, the installation of OpenBUGS is required for the library to run. The installation instruction can be found here.
First, we import the library “R2OpenBUGS” and define the path where the OpenBUGS program is installed.
Then, we will define our model and
Here, MR denotes the state of Marcus Rashford and when MR is around 2, Rashford is injured and we assign that event to MRinjured. Note that in this case, when MR is returned around 1, Rashford is fit. The order of these events could be switched up depending on how we assign the probabilities later. We can almost map those events like (MRfit, MRinjured) -> (1, 2) in this case.
Similarly, other variables are defined. Note that, since how well Manchester United do in the English Premier League (EPL) depends on the state of Marcus Rashford, “MR” variable is included when we define the EPL variable. Other dependencies follow similar fashion.
Next, we assign the data (i.e. the probabilities, hard evidence) that we have.
We see that BF is set to 1. That is because we know from the problem that Bruno Fernandes signs with United at the end of the season. Hence, that is hard evidence and we assign the BF variable to 1.
The rest of the data are given probabilities. Probabilities with dependencies have the form of matrices. Please note that the way matrices are defined here is different from the native matrix configuration on OpenBUGS itself. More info about how to define matrix on OpenBUGS software can be found here.
Next, we need to follow a default, compulsory step called Initialization. However, that is more applicable to problems involving continuous random distributions and we do not need to initialize any value here. However, the package will return an error if we do not assign the initialization step something, so we assign it to NULL.
Now, we can run the model to calculate the probabilities in question given the hard evidence.
“debug” is default to False. However, it is recommended to switch it to True as it is easier to see the errors in case the model does not run. We can now run the codes to get the model running. Here, we are going to run and iterate the model 100,000 times and then discard the first 1,000 results to ensure any large initial variations are not included.
When the model is run, OpenBUGS will open and display a window similar to the above.
It contains all the answers we are looking for but if we want everything to be in R, we will need to close the application so the R codes could finish running. Then we use the following code to display the results in R.
Given the news that Bruno Fernandes signed with United, the probability that Manchester United got qualified for next year’s Champions League is 81%. The probability that they won the Europa League also gets higher to 62.5%. The probabilities that they got Top 3 or 4th place in the Premier League are also marginally higher at 20.8% and 30.3%.
Noticeably, the probability that Marcus Rashford is injured is lowered to 17.6% because he plays an important role in the team’s performance and directly affects the event that the team is qualified for UCL, which directly affects whether Fernandes decides to sign with United or not.
library(R2OpenBUGS)OpenBUGS.pgm = "c:/Program Files (x86)/OpenBUGS/OpenBUGS323/OpenBUGS.exe"#Setting up the modelmodel <- function() {   MR ~ dcat(p.MR[]); MRinjured <- equals(MR, 2);  EPL ~ dcat(p.EPL[MR,]); EPLTop3 <- equals(EPL,1); EPL4th <- equals(EPL,2); EPLElse <- equals(EPL,3);  Europa ~ dcat(p.Europa[MR,]); WonEL <- equals(Europa,1);  UCL ~ dcat(p.UCL[EPL,Europa,]); InUCL <- equals(UCL,1);  BF ~ dcat(p.BF[UCL,])}#Hard evidence: 1 is TRUE and 2 is FALSEdata <- list( BF = 1,p.MR = c(0.8,0.2),p.Europa = structure(.Data =  c(0.6,0.3,                         0.4,0.7), .Dim = c(2,2)),p.EPL = structure(.Data = c(0.2,   0.1,                       0.3,   0.2,                    0.5,   0.7), .Dim = c(2,3)),p.UCL = structure(.Data = c(1, 0.99, 0.99,     1, 0.7, 0,                    0, 0.01, 0.01,     0, 0.3, 1), .Dim = c(3,2,2)),p.BF = structure(.Data = c( 0.7,  0.4,                            0.3,  0.6), .Dim = c(2,2)) )#Initializationinits <- NULL#Run BUGS and save resultsout <- bugs(data = data,                 inits = inits,                 parameters.to.save = c("MRinjured", "EPLTop3", "EPL4th", "EPLElse", "WonEL", "InUCL"),                model.file = model,                 digits = 5,                n.chains = 1,                n.burnin = 1000,                n.iter = 100000,                OpenBUGS.pgm=OpenBUGS.pgm,                 WINE = WINE,                 WINEPATH = WINEPATH,                useWINE=F,                debug = T,                working.directory=getwd(),                DIC = F                )print(out$summary)