# Permutation Invariant Strong Law of Large Numbers for Exchangeable Sequences

Authors: Stefan Tappe
Journal: Journal of Probability and Statistics
 (2021)

---

## Abstract

We provide a permutation invariant version of the strong law of large numbers for exchangeable sequences of random variables. The proof consists of a combination of the Komlós–Berkes theorem, the usual strong law of large numbers for exchangeable sequences, and de Finetti’s theorem.


---

## Body

## 1. Introduction

Kolmogorov’s strong law of large numbers (SLLN) for independent and identically distributed (i.i.d.) sequences of random variables has been generalized into several directions. It has, for example, been generalized for pairwise independent, identically distributed random variables in [1], for nonnegative random variables in [2], for dependent, mixing random variables in [3, 4], and for pairwise uncorrelated random variables in [5].There is also a version of the SLLN for exchangeable sequences. More precisely, letξnn∈ℕ be an exchangeable sequence of random variables on a probability space Ω,F,ℙ, let E be its exchangeable σ-algebra, and let T be its tail σ-algebra. If the sequence ξnn∈ℕ is integrable, then the SLLN for exchangeable sequences tells us that ξnn∈ℕ is almost surely Cesàro convergent; more precisely, we have the following result.Proposition 1.
Letξnn∈ℕ⊂L1 be an exchangeable sequence of integrable random variables. Then, ξnn∈ℕ is ℙ-almost surely Cesàro convergent to the limit(1)ξ=Eξ1|E=Eξ1|T.This result is well known; see, for example, [6], Example 12.15, or [7], page 185. The goal of this note is to establish the following permutation invariant version of the SLLN for exchangeable sequences.Theorem 1.
Letξnn∈ℕ⊂L1 be an exchangeable sequence of integrable random variables. We set ξ≔Eξ1|E. Then, the following statements are true:(1)
For every subsequencenkk∈ℕ and every permutation π:ℕ⟶ℕ, the sequence ξnπkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ(2)
For every permutationσ:ℕ⟶ℕ and every subsequence mkk∈ℕ, the sequence ξσmkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ(3)
We haveℙ-almost surely ξ=Eξn|E=Eξn|T for each n∈ℕIntuitively, the statement of Theorem1 is plausible. Indeed, de Finetti’s theorem, which is stated as Theorem 3 in the following, provides a connection between exchangeable sequences and conditional i.i.d. sequences, and in the present situation, it implies that the sequence ξnn∈ℕ is i.i.d. given E or given T.Let us briefly indicate the main ideas for the proof of Theorem1. Since exchangeability of the sequence is preserved under permutations, by Proposition 1, it follows that the sequences ξnπkk∈ℕ and ξσmkk∈ℕ are almost surely Cesàro convergent. However, it is not clear whether the limits of these two sequences coincide with ξ because their exchangeable σ-algebras can be different from E, and accordingly, their tail σ-algebras can be different from T. Nevertheless, note that, by exchangeability of the sequence, all these limits have the same distribution.In order to overcome the problem regarding the identification of the limits, we use the Komlós–Berkes theorem (see [8]), which is stated as Theorem 2 in the following. This result is an extension of Komlós’s theorem (see [9]); see also [10], Thm. 5.2.1, for another extension of Komlós’s theorem. The Komlós–Berkes theorem was also used in order to prove the von Weizsäcker theorem (see [11]); see also [10], Thm. 5.2.3, for a similar result and [12] for a note on the von Weizsäcker theorem.Coming back to the identification of the limits, the Komlós–Berkes theorem provides us with a subsequencenkk∈ℕ such that, for every permutation π:ℕ⟶ℕ, the sequence ξnπkk∈ℕ is almost surely Cesàro convergent to the same limit. Using this result, in three steps, we will show that, for every subsequence and every permutation, the corresponding sequence is almost surely Cesàro convergent to the same limit and that this limit is given by ξ. For the identification of the limits, we use results about conditional expectations which are provided in the Appendix section.


## 2. Proof of the Result

LetΩ,F,ℙ be a probability space. We denote by L1=L1Ω,F,ℙ the space of all equivalence classes of integrable random variables. Let ξnn∈ℕ be a sequence of random variables. Furthermore, let E be the exchangeable σ-algebra of the sequence ξnn∈ℕ, and let T be the tail σ-algebra of the sequence ξnn∈ℕ. We assume that the sequence ξnn∈ℕ is exchangeable; that is, for every finite permutation π:ℕ⟶ℕ, we have(2)ℙ∘ξnn∈ℕ=ℙ∘ξπnn∈ℕ,or equivalently, for all k∈ℕ, all pairwise different n1,…,nk∈ℕ, and all pairwise different m1,…,mk∈ℕ, we have(3)ℙ∘ξn1,…,ξnk=ℙ∘ξm1,…,ξmk.Remark 1.
Note that, for every subsequencenkk∈ℕ and every permutation π:ℕ⟶ℕ, the sequence ξnπkk∈ℕ is also exchangeable. Accordingly, for every permutation σ:ℕ⟶ℕ and every subsequence mkk∈ℕ, the sequence ξσmkk∈ℕ is also exchangeable.Lemma 1.
The following statements are true:(1)
For every subsequencenkk∈ℕ and every permutation σ:ℕ⟶ℕ, there exist a permutation π:ℕ⟶ℕ and a subsequence mkk∈ℕ such that σmk=nπk for all k∈ℕ(2)
For every subsequencenkk∈ℕ and every permutation π:ℕ⟶ℕ, there exists a permutation σ:ℕ⟶ℕ such that σnk=nπk for all k∈ℕProof.
(1)
We define the one-to-one mapτ:ℕ⟶ℕ as(4)τk≔σ−1nk,for eachk∈ℕ.Then, there exists a permutationπ:ℕ⟶ℕ such that τπk<τπk+1 for all k∈ℕ. Indeed, we define π inductively as follows. Let π1∈ℕ be the unique index such that(5)τπ1=minτk:k∈ℕ.Ifπ1,…,πp are already defined for some p∈ℕ, then let πp+1∈ℕ be the unique index such that(6)τπp+1=minτk:k∈ℕ∖π1,…,πp.Then,π is a permutation. We define the subsequence mkk∈ℕ as mk≔τπk for each k∈ℕ. Then, we have σmk=nπk for each k∈ℕ.(2)
We define the permutationσ:ℕ⟶ℕ as(7)σm≔nπkifm=nkfor somek∈ℕ,m,otherwise.
Then, we haveσnk=nπk for all k∈ℕ.For convenience of the reader, we state the Komlós–Berkes theorem and de Finetti’s theorem before we provide the proof of Theorem1.Theorem 2.
(Komlós–Berkes theorem). Letξnn∈ℕ⊂L1 be a sequence of integrable random variables such that supn∈ℕEξn<∞. Then, there exist a subsequence nkk∈ℕ and an integrable random variable ξ∈L1 such that, for every permutation π:ℕ⟶ℕ, the sequence ξnπkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ.Proof.
See [8].LetG⊂F be a sub-σ-algebra. A sequence ξnn∈ℕ of random variables is called independent and identically distributed (i.i.d.) given G if for every finite subset I⊂ℕ and all Borel sets Bi∈Bℝ, i∈I, we have ℙ-almost surely(8)ℙ∩i∈Iξi∈Bi|G=∏i∈Iℙξi∈Bi|Gindependence givenG,and for all n,m∈ℕ and every Borel set B∈Bℝ, we have ℙ-almost surely(9)ℙξn∈B|G=ℙξm∈B|Gidentical distributions givenG.Theorem 3 (de Finetti’s theorem).
Letξnn∈ℕ be a sequence of random variables. Then, the following statements are equivalent:(i)
The sequenceξnn∈ℕ is exchangeable(ii)
There exists a sub-σ-algebra G⊂F such that ξnn∈ℕ is i.i.d. given G
If the previous conditions are fulfilled, then we can chooseG=E or G=T.Proof.
See, for example, Thm. 12.24 of [6].Now, we are ready to provide the proof of Theorem1.Proof.
of Theorem1. By the Komlós–Berkes theorem (see Theorem 2), there exist a subsequence nkk∈ℕ and an integrable random variable ξ∈L1 such that, for every permutation π:ℕ⟶ℕ, the sequence ξnπkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ. Now, we proceed with the following three steps:Step 1: first, we show that, for every permutationσ:ℕ⟶ℕ, the sequence ξσnn∈ℕ is ℙ-almost surely Cesàro convergent to ξ. Indeed, by Lemma 1, there exist a permutation π:ℕ⟶ℕ and a subsequence mkk∈ℕ such that σmk=nπk for each k∈ℕ. By Remark 1 and Proposition 1, we have(10)ξ=Eξnπ1|Enπkk∈ℕ=Eξnπ1|Tnπkk∈ℕ,whereEnπkk∈ℕ denotes the exchangeable σ-algebra of the sequence ξnπkk∈ℕ and Tnπkk∈ℕ denotes the tail σ-algebra of the sequence ξnπkk∈ℕ. Furthermore, by Remark 1 and Proposition 1, the sequence ξσnn∈ℕ is ℙ-almost surely Cesàro convergent to the random variable(11)η≔Eξσ1|Eσnn∈ℕ=Eξσ1|Tσnn∈ℕ,whereEσnn∈ℕ denotes the exchangeable σ-algebra of the sequence ξσnn∈ℕ and Tσnn∈ℕ denotes the tail σ-algebra of the sequence ξσnn∈ℕ. By de Finetti’s theorem (see Theorem 3), we have(12)η=Eξnπ1|Eσnn∈ℕ=Eξnπ1|Tσnn∈ℕ.SinceEnπkk∈ℕ⊂Eσnn∈ℕ and Tnπkk∈ℕ⊂Tσnn∈ℕ, by (10), we obtain(13)ξ=Eη|Enπkk∈ℕ=Eη|Tnπkk∈ℕ.By exchangeability of the sequenceξnn∈ℕ, we have(14)ℙ∘1n∑i=1nξnπi=ℙ∘1n∑i=1nξσi,for eachn∈ℕ,and hence, by PropositionA.1, we obtain ℙ-almost surely ξ=η. In particular, if σ=Id, then by (11) and de Finetti’s theorem (see Theorem 3), we obtain ℙ-almost surely(15)ξ=η=Eξn|E=Eξn|T,for eachn∈ℕ.Step 2: now, letσ:ℕ⟶ℕ be an arbitrary permutation, and let mkk∈ℕ be an arbitrary subsequence. Then, the sequence ξσmkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ. Indeed, by Step 1 and de Finetti’s theorem (see Theorem 3), the sequence ξσnn∈ℕ is ℙ-almost surely Cesàro convergent to(16)ξ=Eξσ1|Eσnn∈ℕ=Eξσm1|Eσnn∈ℕ.Furthermore, by Remark1 and Proposition 1, the sequence ξσmkk∈ℕ is ℙ-almost surely Cesàro convergent to the random variable(17)ζ≔Eξσm1|Eσmkk∈ℕ.SinceEσmkk∈ℕ⊂Eσnn∈ℕ, by (16), we obtain(18)ζ=Eξ|Eσmkk∈ℕ.By exchangeability of the sequenceξnn∈ℕ, we have(19)ℙ∘1n∑i=1nξσi=ℙ∘1n∑i=1nξσmi,for eachn∈ℕ,and hence, by PropositionA.1, we obtain ℙ-almost surely ξ=ζ. Consequently, the sequence ξσmkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ.Step 3: now, letnkk∈ℕ be an arbitrary subsequence, and let π:ℕ⟶ℕ be an arbitrary permutation. By Lemma 1, there exists a permutation σ:ℕ⟶ℕ such that σnk=nπk for all k∈ℕ. Therefore, by Step 2, the sequence ξnπkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ, which concludes the proof.We can extend the statement of Theorem1 as follows.Proposition 2.
Letξnn∈ℕ⊂L1 be an exchangeable sequence of integrable random variables. We set ξ≔Eξ1|E. Then, for every subsequence nkk∈ℕ and all permutations π,σ:ℕ⟶ℕ, the sequence ξσnπkk∈ℕ is ℙ-almost surely Cesàro convergent to ξ. Furthermore, we have ℙ-almost surely ξ=Eξn|E=Eξn|T for each n∈ℕ.Proof.
By Lemma1, there exists a permutation τ:ℕ⟶ℕ such that τnk=nπk for all k∈ℕ. The mapping ρ:ℕ⟶ℕ given by ρ≔σ∘τ is also a permutation, and we have σnπk=ρnk for all k∈ℕ. Therefore, applying Theorem 1 concludes the proof.We conclude this section with the following consequence regarding Komlós’s theorem for exchangeable sequences, namely, letξnn∈ℕ⊂L1 be an exchangeable sequence of random variables. Then, Theorem 1 shows that both extensions of Komlós’s theorem (the Komlós–Berkes theorem from [8], which we have stated as Theorem 2, and Thm. 5.2.1 of [10]) are true with the original sequence ξnn∈ℕ; that is, we do not have to pass to a subsequence ξnkk∈ℕ.



---

*Source: 3637837-2021-11-27.xml*