This week, we will start to work on multivariate models, and non-independence. The first idea to discuss non-independence will be to use the concept ofexchangeability. A sequence of random variable  is said to be exchangeable if for all 

for any permutation  of . A standard example is the case where, with

and

Since , a necessary condition is that

i.e.

Since this inequality should hold for all  it comes that necessarily .
de Finetti (1931): Let  be a sequence of random variables with values in .  is exchangeable if and only if there exists a distribution function  on  such that

where . Note that  is the distribution function of random variable

A nice proof of that result can be found in Heath & Sudderth (1995) – see alsoSchervish (1995), Chow & Teicher (1997) or Durrett (2010) and also probably in several bayesian books because that result has a strong interpretation in bayesian inference (as far as I understood, see e.g. Jaynes (1982)).
From the exchangeability condition, for any permutation  of ,

that can be inverted in

The idea is then to extend the size of the vector , i.e. for all , define

so that, if we condition on ,

but since given the sum of components of , all possible rearrangements of the ones among the  elements are equally likely, we can write

The first idea is to work on the blue term, and to invocate a theorem of approximation of the hypergeometric distribution  to a binomial distribution , when  becomes large. Then

Let  and let  denote the cumulative distribution function of .

The idea is then to write the sum as an integral, with respect to that distribution,

The theorem is then obtained since , i.e.

In the case of non-binary sequences, there is an extension of the previous result,
Hewitt & Savage (1955): Let  be a sequence of random variables with values in .   is exchangeable if and only if there exists a measure  on  such that

where  is the measure associated to the empirical measure

and

For instance, in the Gaussian case mentioned earlier, if

then

where

i.e. conditionally on , the  are conditionally independent, with distribution . The proof can be found in Kingman (1978) and is based on martingale arguments.
Note that in the Gaussian case,  where  are i.i.d. random variables. To go further on exchangeability and related topics, see Aldous (1985)  (see also here).
This construction can be used in credit risk, to model defaults in an homogeneous portfolio, see e.g. Frey (2001),

Assuming a Beta distribution for the latent factor, we can derive the probability distribution of the sum

Since

if we assume that – given the latent factor –  (either the company defaults, or not),

i.e.

Thus, we can derive the (unconditional) distribution of the sum

i.e.

> proba=function(s,a,m,n){
+ b=a/m-a
+ choose(n,s)*integrate(function(t){t^s*(1-t)^(n-s)*
+ dbeta(t,a,b)},lower=0,upper=1)$value
+ }

Based on that function, it is possible to plot the probability distribution over . In the upper corner is plotted the density of the Beta distribution.

> a=2
> m=.2
+ n=10
+ V=rep(NA,n+1)
+ for(i in 0:n){
+ V[i+1]=proba(i,a,m,n)}
> barplot(V,names.arg=0:10)

Those two theorems are extremely close,

De Finetti’s theorem: a random sequence  of  random variables is exchangeable if and only if ‘s are conditionnally independent, conditionnally on some random variable .

Hewitt-Savage’s theorem: a random sequence  is exchangeable if and only if ‘s are conditionnally independent, conditionnally on some sigma-algebra 

Olshen (1974), proposed an interesting discussion about those theorems, see also in the Encyclopedia of Statistical Science,

The subtle difference between those two theorem is also discussed in Freedman (1965)