de Finetti’s theorem and exchangeability

This week, we will start to work on multivariate models, and non-independence. The first idea to discuss non-independence will be to use the concept ofexchangeability. A sequence of random variable is said to be exchangeable if for all any permutation of A standard example is the case where, with, a necessary condition is that
Since this inequality should hold for all it comes that necessarily
de Finetti (1931): Let be a sequence of random variables with values in is exchangeable if and only if there exists a distribution function on such that Note that is the distribution function of random variable nice proof of that result can be found in Heath & Sudderth (1995) – see alsoSchervish (1995)Chow & Teicher (1997) or Durrett (2010) and also probably in several bayesian books because that result has a strong interpretation in bayesian inference (as far as I understood, see e.g. Jaynes (1982)).
From the exchangeability condition, for any permutation of, can be inverted in idea is then to extend the size of the vector, i.e. for all, define that, if we condition on, since given the sum of components of, all possible rearrangements of the ones among the elements are equally likely, we can write first idea is to work on the blue term, and to invocate a theorem of approximation of the hypergeometric distribution to a binomial distribution, when becomes large. Then and let denote the cumulative distribution function of idea is then to write the sum as an integral, with respect to that distribution, theorem is then obtained since, i.e. the case of non-binary sequences, there is an extension of the previous result,
Hewitt & Savage (1955): Let be a sequence of random variables with values in is exchangeable if and only if there exists a measure on such that is the measure associated to the empirical measure instance, in the Gaussian case mentioned earlier, if conditionally on, the are conditionally independent, with distribution The proof can be found in Kingman (1978) and is based on martingale arguments.
Note that in the Gaussian case, where are i.i.d. random variables. To go further on exchangeability and related topics, see Aldous (1985)  (see also here).
This construction can be used in credit risk, to model defaults in an homogeneous portfolio, see e.g. Frey (2001),


Assuming a Beta distribution for the latent factor, we can derive the probability distribution of the sum we assume that – given the latent factor – (either the company defaults, or not),, we can derive the (unconditional) distribution of the sum 

> proba=function(s,a,m,n){
+ b=a/m-a
+ choose(n,s)*integrate(function(t){t^s*(1-t)^(n-s)*
+ dbeta(t,a,b)},lower=0,upper=1)$value
+ }

Based on that function, it is possible to plot the probability distribution over In the upper corner is plotted the density of the Beta distribution.

> a=2
> m=.2
+ n=10
+ V=rep(NA,n+1)
+ for(i in 0:n){
+ V[i+1]=proba(i,a,m,n)}
> barplot(V,names.arg=0:10)

Those two theorems are extremely close,

De Finetti’s theorem: a random sequence of random variables is exchangeable if and only if‘s are conditionnally independent, conditionnally on some random variable

Hewitt-Savage’s theorem: a random sequence is exchangeable if and only if‘s are conditionnally independent, conditionnally on some sigma-algebra

Olshen (1974), proposed an interesting discussion about those theorems, see also in the Encyclopedia of Statistical Science,

The subtle difference between those two theorem is also discussed in Freedman (1965)

2 thoughts on “de Finetti’s theorem and exchangeability”

  1. thanks Julyan for the comment on the Bayesian paradigm… about the last comment, indeed, exchangeability (infinite exchangeability) is a strong concept of positive dependence. Moshe Shaked introduced the PDM concept (positive dependent by mixture) in 1977, and it is one of the strongest concept of positive dependence… But I don’t think I’ll have time to discuss too much concepts of (multivariate) stochastic orderings, even if I plan to discuss definitions of positive dependence (and perhaps, also, negative dependence).

  2. For the Bayesian paradigm justification: de Finetti’s representation theorem proves that under the exchangeability assumption for the observations, there exist a model and a prior distribution on its parameter, of which the observations are a random sample.

    It’s interesting to see that an infinite exchangeable sequence necessarily has a positive covariance (not only a Gaussian sequence, as anyway the covariance is constant), and for a finite sequence, it is bounded below by -sigma^2/(n-1).

Leave a Reply

Your email address will not be published. Required fields are marked *