Markov kernel
In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finite state space. Formal definition. Let formula_1 and formula_2 be measurable spaces. A "Markov kernel" with source formula_1 and target formula_2 is a map formula_5 with the following properties: In other words it associates to each point formula_12 a probability measure formula_13 on formula_2 such that, for every measurable set formula_15, the map formula_16 is measurable  with respect to the formula_17-algebra formula_8. Examples. Simple random walk on the integers. Take formula_19, and formula_20 (the power set of formula_21). Then a Markov kernel is fully determined by the probability it assigns to a singleton set formula_22 with formula_23 for each formula_24: Now the random walk formula_26 that goes to the right with probability formula_27 and to the left with probability formula_28 is defined by where formula_30 is the Kronecker delta. The transition probabilities formula_31 for the random walk are equivalent to the Markov kernel. General Markov processes with countable state space. More generally take formula_32 and  formula_33 both countable and formula_34. Again a Markov kernel is defined by the probability it assigns to singleton sets for each formula_35 We define a Markov process by defining a transition probability formula_37 where the numbers formula_38 define a (countable) stochastic matrix formula_39 i.e. We then define Again the transition probability, the stochastic matrix and the Markov kernel are equivalent reformulations. Markov kernel defined by a kernel function and a measure. Let formula_42 be a measure on formula_11, and formula_44 a measurable function with respect to the product formula_17-algebra formula_46 such that then formula_48 i.e. the mapping defines a  Markov kernel. This example generalises the countable Markov process example where formula_42 was the counting measure. Moreover it encompasses other important examples such as the convolution kernels, in particular the Markov kernels defined by the heat equation. The latter example includes the Gaussian kernel on formula_51 with formula_52 standard Lebesgue measure and Measurable functions. Take formula_54 and formula_55 arbitrary measurable spaces, and let formula_56 be a measurable function. Now define formula_57 i.e. Note that the indicator function formula_60 is formula_61-measurable for all formula_59 iff formula_63 is measurable. This example allows us to think of a Markov kernel as a  generalised function with a (in general) random rather than certain value. Galton–Watson process. As a less obvious example, take formula_64, and formula_11 the real numbers formula_66 with the standard sigma algebra of Borel sets. Then with i.i.d. random variables formula_68 (usually with mean 0) and where formula_69 is the indicator function. For the simple case of coin flips this models the different levels of a Galton board. Composition of Markov Kernels and the Markov Category. Given measurable spaces formula_70, formula_71 we consider a Markov kernel formula_72 as a morphism formula_73. Intuitively, rather than assigning to each formula_12 a sharply  defined point formula_75 the kernel assigns a "fuzzy" point in formula_33 which is only known with some level of uncertainty, much like actual physical measurements. If we have a third measurable space formula_77, and probability kernels formula_73 and formula_79, we can define a composition formula_80 by The composition is associative by Tonelli's theorem and the identity function considered as a Markov kernel (i.e. the delta measure formula_82) is the unit for this composition. This composition defines the structure of a category on the measurable spaces with Markov kernels as morphisms first defined by Lawvere. The category has the empty  set as initial object and the one point set formula_83 as the terminal object. From this point of view a probability space is formula_84 is the same thing as a pointed space formula_85 in the Markov category. Probability Space defined by Probability Distribution and a Markov Kernel. A probability measure on a measurable space formula_70 is the same thing as a morphism formula_87 in the Markov category also denoted by formula_88. By composition, a probability space formula_89 and a probability kernel formula_90 defines a probability space formula_91. It is concretely defined by Properties. Semidirect product. Let formula_93 be a probability  space and formula_26 a Markov kernel from formula_70 to some formula_11. Then there exists a unique measure formula_97 on formula_98, such that: Regular conditional distribution. Let formula_100 be a Borel space, formula_32 a formula_100-valued random variable on the measure space formula_103 and formula_104 a sub-formula_17-algebra. Then there exists a Markov kernel formula_26 from formula_107 to formula_100, such that formula_109 is a version of the conditional expectation formula_110 for every formula_111, i.e. It is called regular conditional distribution of formula_32 given formula_114 and is not uniquely defined. Generalizations. Transition kernels generalize Markov kernels in the sense that for  all formula_12, the map can be any type of (non negative) measure, not necessarily a probability measure.