Bayesian hierarchical modeling
Bayesian hierarchical modelling is a statistical model written in multiple levels (hierarchical form) that estimates the parameters of the posterior distribution using the Bayesian method. The sub-models combine to form the hierarchical model, and Bayes' theorem is used to integrate them with the observed data and account for all the uncertainty that is present. The result of this integration is the posterior distribution, also known as the updated probability estimate, as additional evidence on the prior distribution is acquired. Frequentist statistics may yield conclusions seemingly incompatible with those offered by Bayesian statistics due to the Bayesian  treatment of the parameters as random variables and its use of subjective information in establishing assumptions on these parameters. As the approaches answer different questions the formal results aren't technically contradictory but the two approaches disagree over which answer is relevant to particular applications. Bayesians argue that relevant information regarding decision-making and updating beliefs cannot be ignored and that hierarchical modeling has the potential to overrule classical methods in applications where respondents give multiple observational data. Moreover, the model has proven to be robust, with the posterior distribution less sensitive to the more flexible hierarchical priors. Hierarchical  modeling is used when information is available on several different levels of observational units. For example, in epidemiological modeling to describe infection trajectories for multiple countries, observational units are countries, and each country has its own temporal profile of daily infected cases. In decline curve analysis to describe oil or gas production decline curve for multiple wells, observational units are oil or gas wells in a reservoir region, and each well has each own temporal profile of oil or gas production rates (usually, barrels per month). Data structure for the hierarchical modeling retains nested data structure. The hierarchical form of  analysis and organization helps in the understanding of multiparameter problems and also plays an important role in developing computational strategies. Philosophy. Statistical methods and models commonly involve multiple parameters that can be regarded as related or connected in such a way that the problem implies a dependence of the joint probability model for these parameters. Individual degrees of belief, expressed in the form of probabilities, come with uncertainty. Amidst this is the change of the degrees of belief over time. As was stated by Professor José M. Bernardo and Professor Adrian F. Smith, “The actuality of the learning process consists  in the evolution of individual and subjective beliefs about the reality.” These subjective probabilities are more directly involved in the mind rather than the physical probabilities. Hence, it is with this need of updating beliefs that Bayesians have formulated an alternative statistical model which takes into account the prior occurrence of a particular event. Bayes' theorem. The assumed occurrence of a real-world event will typically modify preferences between certain options. This is done by modifying the degrees of belief attached, by an individual, to the events defining the options. Suppose in a study of the effectiveness of cardiac treatments , with the patients in hospital "j" having survival probability formula_1, the survival probability will be updated with the occurrence of "y", the event in which a controversial serum is created which, as believed by some, increases survival in cardiac patients. In order to make updated probability statements about formula_1, given the occurrence of event "y", we must begin with a model providing a joint probability distribution for formula_1 and "y". This can be written as a product of the two distributions that are often referred to as the prior distribution formula_4 and the sampling distribution formula_5 respectively: Using the basic  property of conditional probability, the posterior distribution will yield: This equation, showing the relationship between the conditional probability and the individual events, is known as Bayes' theorem. This simple expression encapsulates the technical core of Bayesian inference which aims to incorporate the updated belief, formula_8, in appropriate and solvable ways. Exchangeability. The usual starting point of a statistical analysis is the assumption that the "n" values formula_9 are exchangeable. If no information – other than data "y" – is available to distinguish any of the formula_1’s from any others, and no ordering or grouping of the parameters can be made, one  must assume symmetry among the parameters in their prior distribution. This symmetry is represented probabilistically by exchangeability. Generally, it is useful and appropriate to model data from an exchangeable distribution as independently and identically distributed, given some unknown parameter vector formula_11, with distribution formula_4. Finite exchangeability. For a fixed number "n", the set formula_9 is exchangeable if the joint probability formula_14 is invariant under permutations of the indices. That is, for every permutation formula_15 or formula_16 of (1, 2, …, "n"), formula_17 Following is an exchangeable, but not independent and identical (iid), example: Consider an urn with a red ball and  a blue ball inside, with probability formula_18 of drawing either. Balls are drawn without replacement, i.e. after one ball is drawn from the "n" balls, there will be "n" − 1 remaining balls left for the next draw. Since the probability of selecting a red ball in the first draw and a blue ball in the second draw is equal to the probability of selecting a blue ball on the first draw and a red on the second draw, both of which are equal to 1/2 (i.e. formula_20), then formula_21 and formula_22 are exchangeable. But the probability of selecting a  red ball on the second draw given that the red ball has already been selected in the first draw is 0, and is not equal to the probability that the red ball is selected in the second draw which is equal to 1/2 (i.e. formula_23). Thus, formula_21 and formula_22 are not independent. If formula_26 are independent and identically distributed, then they are exchangeable, but the converse is not necessarily true. Infinite exchangeability. Infinite exchangeability is the property that every finite subset of an infinite sequence formula_21, formula_28 is exchangeable. That is, for any "n", the sequence formula_9 is exchangeable. Hierarchical  models. Components. Bayesian hierarchical modeling makes use of two important concepts in deriving the posterior distribution, namely: Suppose a random variable "Y" follows a normal distribution with parameter "θ" as the mean and 1 as the variance, that is formula_30. The tilde relation formula_31 can be read as "has the distribution of" or "is distributed as". Suppose also that the parameter formula_11 has a distribution given by a normal distribution with mean formula_33 and variance 1, i.e. formula_34. Furthermore, formula_33 follows another distribution given, for example, by the standard normal distribution, formula_36. The parameter formula_33 is called the hyperparameter, while  its distribution given by formula_36 is an example of a hyperprior distribution. The notation of the distribution of "Y" changes as another parameter is added, i.e. formula_39. If there is another stage, say, formula_33 follows another normal distribution with mean formula_41 and variance formula_42, meaning formula_43, formula_44formula_41 and formula_42 can also be called hyperparameters while their distributions are hyperprior distributions as well. Framework. Let formula_47 be an observation and formula_1 a parameter governing the data generating process for formula_47. Assume further that the parameters formula_50 are generated exchangeably from a common population, with distribution governed by a hyperparameter formula_51. The  Bayesian hierarchical model contains the following stages: The likelihood, as seen in stage I is formula_55, with formula_56 as its prior distribution. Note that the likelihood depends on formula_51 only through formula_1. The prior distribution from stage I can be broken down into: With formula_51 as its hyperparameter with hyperprior distribution, formula_61. Thus, the posterior distribution is proportional to: Example. To further illustrate this, consider the example: A teacher wants to estimate how well a student did on the SAT. The teacher uses information on the student’s high school grades and current grade point average (GPA) to come up  with an estimate. The student's current GPA, denoted by formula_64, has a likelihood given by some probability function with parameter formula_11, i.e. formula_66. This parameter formula_11 is the SAT score of the student. The SAT score is viewed as a sample coming from a common population distribution indexed by another parameter formula_51, which is the high school grade of the student (freshman, sophomore, junior or senior). That is, formula_69. Moreover, the hyperparameter formula_51 follows its own distribution given by formula_61, a hyperprior. To solve for the SAT score given information on the GPA, All information in the problem will  be used to solve for the posterior distribution. Instead of solving only using the prior distribution and the likelihood function, the use of hyperpriors gives more information to make more accurate beliefs in the behavior of a parameter. 2-stage hierarchical model. In general, the joint posterior distribution of interest in 2-stage hierarchical models is: 3-stage hierarchical model. For 3-stage hierarchical models, the posterior distribution is given by: Bayesian nonlinear mixed-effects model. The framework of Bayesian hierarchical modeling is frequently used in diverse applications. Particularly, Bayesian nonlinear mixed-effects models have recently received significant attention. A  basic version of the Bayesian nonlinear mixed-effects models is represented as the following three-stage: Stage 1: Individual-Level Model formula_78 Stage 2: Population Model formula_79 Stage 3: Prior formula_80 Here, formula_81 denotes the continuous response of the formula_82-th subject at the time point formula_83, and formula_84 is the formula_85-th covariate of the formula_82-th subject. Parameters involved in the model are written in Greek letters. formula_87 is a known function parameterized by the formula_88-dimensional vector formula_89. Typically, formula_90 is a `nonlinear' function and describes the temporal trajectory of individuals. In the model, formula_91 and  formula_92 describe within-individual variability and between-individual variability, respectively. If Stage 3: Prior is not considered, then the model reduces to a frequentist nonlinear mixed-effect model. A central task in the application of the Bayesian nonlinear mixed-effect models is to evaluate the posterior density: formula_93 formula_94 formula_95 The panel on the right displays Bayesian research cycle using Bayesian nonlinear mixed-effects model. A research cycle using the Bayesian nonlinear mixed-effects model comprises two steps: (a) standard research cycle and (b) Bayesian-specific workflow. Standard research cycle involves literature review, defining a problem and specifying the research  question and hypothesis. Bayesian-specific workflow comprises three sub-steps: (b)–(i) formalizing prior distributions based on background knowledge and prior elicitation; (b)–(ii) determining the likelihood function based on a nonlinear function formula_96; and (b)–(iii) making a posterior inference. The resulting posterior inference can be used to start a new research cycle.