Tag Archives: proportion

Want to say one thing and the exact oppositive with strong confidence ?

No need to do politics. Just take a statistical course. And I do not talk about misinterpretation of statistics, but I talk about the mathematical foundations of statistical tests.
Consider the following parametric test, with a one-dimensional parameter: http://freakonometrics.blog.free.fr/public/perso2/test-lies-01.gif versus http://freakonometrics.blog.free.fr/public/perso2/test-lies-02.gif, for some (fixed) http://freakonometrics.blog.free.fr/public/perso2/test-lies-03.gif. A standard way of doing such a test is to consider an rejection region http://freakonometrics.blog.free.fr/public/perso2/test-lies-05.gif. The test works as follows: consider a sample http://freakonometrics.blog.free.fr/public/perso2/test-lies-06.gif,

  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-07.gif, then we accept http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif
  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-09.gif, the we reject http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif

For instance, consider the case of a Bernoulli sample, with probability http://freakonometrics.blog.free.fr/public/perso2/test-lies-62.gif. The standard idea is to define

http://freakonometrics.blog.free.fr/public/perso2/test-lies-13.gif

The rejection region is then based on statistic http://freakonometrics.blog.free.fr/public/perso2/test-lies-210.gif,

  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-25.gif, then we accept http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif
  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-22.gif, the we reject http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif

where threshold http://freakonometrics.blog.free.fr/public/perso2/test-lies-26.gif is taken so that the probability to make a first type error is http://freakonometrics.blog.free.fr/public/perso2/test-lies-28.gif(say 5%) using the Gaussian approximation for z. Here

http://freakonometrics.blog.free.fr/public/perso2/test-lies-30.gif

Thus, the acceptation region is then the green area below, while the rejection region is the red one, for http://freakonometrics.blog.free.fr/public/perso2/test-lies-210.gif.

Consider now the exact opposite test (with the same http://freakonometrics.blog.free.fr/public/perso2/test-lies-03.gif), http://freakonometrics.blog.free.fr/public/perso2/test-lies-51.gifversus http://freakonometrics.blog.free.fr/public/perso2/test-lies-52.gif. Here, we use the same statistics, and the test is

  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-22.gif, then we accept http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif
  • if http://freakonometrics.blog.free.fr/public/perso2/test-lies-25.gif, the we reject http://freakonometrics.blog.free.fr/public/perso2/test-H0.gif

where now

http://freakonometrics.blog.free.fr/public/perso2/test-lies-50.gif

Thus, now, the acceptation region is then the green area below, while the rejection region is the red one.

So if we summarize what we just said,

  • in the region on the left below, both test agree that http://freakonometrics.blog.free.fr/public/perso2/test-lies-55.gif
  • in the region on the right below, both test agree that http://freakonometrics.blog.free.fr/public/perso2/test-lies-57.gif
  • and in the region in blue, in the middle, the two tests disagree (one claims that http://freakonometrics.blog.free.fr/public/perso2/test-lies-55.gif, and the other one that http://freakonometrics.blog.free.fr/public/perso2/test-lies-57.gif)

Here is the evolution of the region as a function of http://freakonometrics.blog.free.fr/public/perso2/test-lies-56.gif (the size of the sample) when the sample frequency is 20%. With a small sample size, we can hardly say anything.

n=seq(1,100)
p=0.2
x1=p+qnorm(.95)*sqrt(p*(1-p)/n)
x2=p+qnorm(.05)*sqrt(p*(1-p)/n)
plot(n,x1,type="l",ylim=c(0,1))
polygon(c(n,rev(n)),c(x1,rev(x2)),col="light blue",border=NA)
lines(n,x1,lwd=2,col="red")
lines(n,x2,lwd=2,col="red")

One might say that those bounds are based on a Gaussian approximation which is not correct when http://freakonometrics.blog.free.fr/public/perso2/test-lies-56.gif is too small. So we can compute exact bounds,
y1=qbinom(.95,size=n,prob=p)/n
y2=qbinom(.05,size=n,prob=p)/n
polygon(c(n,rev(n)),c(y1,rev(y2)),col="blue",border=NA)
lines(n,y1,lwd=2,col="red")
lines(n,y2,lwd=2,col="red")

and we get

This is what we can observe if we use R statistical procedures, either the asymptotic one,

> prop.test(2,10,.5,alternative="less")
 
1-sample proportions test with continuity correction
 
data:  2 out of 10, null probability 0.5
X-squared = 2.5, df = 1, p-value = 0.05692
alternative hypothesis: true p is less than 0.5
95 percent confidence interval:
0.0000000 0.5100219
sample estimates:
p
0.2
 
> prop.test(2,10,.5,alternative="greater")
 
1-sample proportions test with continuity correction
 
data:  2 out of 10, null probability 0.5
X-squared = 2.5, df = 1, p-value = 0.943
alternative hypothesis: true p is greater than 0.5
95 percent confidence interval:
0.04368507 1.00000000
sample estimates:
p
0.2

or a more accurate one

> binom.test(2,10,.5,alternative="less")
 
Exact binomial test
 
data:  2 and 10
number of successes = 2, number of trials = 10, p-value = 0.05469
alternative hypothesis: true probability of success is less than 0.5
95 percent confidence interval:
0.0000000 0.5069013
sample estimates:
probability of success
0.2
 
> binom.test(2,10,.5,alternative="greater")
 
Exact binomial test
 
data:  2 and 10
number of successes = 2, number of trials = 10, p-value = 0.9893
alternative hypothesis: true probability of success is greater than 0.5
95 percent confidence interval:
0.03677144 1.00000000
sample estimates:
probability of success
0.2

Here, when the sample frequency is 20% and http://freakonometrics.blog.free.fr/public/perso2/test-lies-56.gif is equal to 10, we accept at the same time that theta is higher than 50% and lower than 50%.
And obviously it is not only a theoretical problem: it has obviously some strong implications. This morning, a good friend mentioned a post published some months ago, online here, about discrimination, and the lack of women with academic positions in mathematics, in France. As claimed by the author of the post“A Paris VI, meilleure université française selon son président, sur 11 postes de maitres de conférences, 5 filles classées premières. Il y a donc des filles excellentes ? A Toulouse, sur 4 postes, 2 filles premières. Parité parfaite. Mais à côté de cela, Bordeaux, 4 postes, 0 fille première. Littoral, 3 postes, 0 fille, Nice, 5 postes, 0 fille, Rennes, 7 postes, 0 fille…”.
Consider the latter one: in Rennes, out of 7 people hired last year, no woman. So in some sense, it looks obvious that there is some kind of discrimination ! Zero out of seven ! Well, if we consider the fact that around 30% of PhD thesis in mathematics were defended by women those years, we can also try to see is there if no “positive discrimination“, i.e. test http://freakonometrics.blog.free.fr/public/perso2/test-lies-60.gif where theta is the probability to hire a woman (just to be a little bit provocative).

> prop.test(0,7,.3,alternative="less")
 
1-sample proportions test with continuity correction
 
data:  0 out of 7, null probability 0.3
X-squared = 1.7415, df = 1, p-value = 0.09347
alternative hypothesis: true p is less than 0.3
95 percent confidence interval:
0.0000000 0.3719021
sample estimates:
p
0
 
Warning message:
In prop.test(0, 7, 0.3, alternative = "less") :
Chi-squared approximation may be incorrect
> binom.test(0,7,.3,alternative="less")
 
Exact binomial test
 
data:  0 and 7
number of successes = 0, number of trials = 7, p-value = 0.08235
alternative hypothesis: true probability of success is less than 0.3
95 percent confidence interval:
0.0000000 0.3481637
sample estimates:
probability of success
0

With no woman hired that year, we can still pretend that there was some kind of “positive discrimination“. An note that we do accept – with more confidence – the assumption of “positive discrimination” if we look at all universities together,

> prop.test(5+2,11+4+4+3+5+7,.3,alternative="less")
 
1-sample proportions test with continuity correction
 
data:  5 + 2 out of 11 + 4 + 4 + 3 + 5 + 7, null probability 0.3
X-squared = 1.021, df = 1, p-value = 0.1561
alternative hypothesis: true p is less than 0.3
95 percent confidence interval:
0.0000000 0.3556254
sample estimates:
p
0.2058824
 
> binom.test(5+2,11+4+4+3+5+7,.3,alternative="less")
 
Exact binomial test
 
data:  5 + 2 and 11 + 4 + 4 + 3 + 5 + 7
number of successes = 7, number of trials = 34, p-value = 0.1558
alternative hypothesis: true probability of success is less than 0.3
95 percent confidence interval:
0.0000000 0.3521612
sample estimates:
probability of success
0.2058824

So obviously, with small sample, almost anything can be claimed !

Tests statistiques et intervalles de confiance

Comme je l’avais énoncé rapidement à la fin du chapitre sur les intervalles de confiance, il existe une forme de dualité entre tests et intervalles de confiance. Par exemple, si on considère un test sur la moyenne, de la forme http://freakonometrics.blog.free.fr/public/perso2/testic-01.gifcontre http://freakonometrics.blog.free.fr/public/perso2/testic-02.gif dans un modèle Gaussien, http://freakonometrics.blog.free.fr/public/perso2/test-ic03.gif, la région d’acceptation (de niveau http://freakonometrics.blog.free.fr/public/perso2/testic-04.gif) sera de la forme

http://freakonometrics.blog.free.fr/public/perso2/testic-05.gif
c’est à dire si

http://freakonometrics.blog.free.fr/public/perso2/testic-06.gif
Aussi, pour qu’une valeur http://freakonometrics.blog.free.fr/public/perso2/testic-07.gif soit acceptée avec un niveau http://freakonometrics.blog.free.fr/public/perso2/testic-04.gif (erreur de première espèce associée à un test bilatéral), une condition nécessaire et suffisante est que http://freakonometrics.blog.free.fr/public/perso2/testic-07.gif appartienne à l’intervalle de confiance (symétrique) centré sur la moyenne empirique.
Il y a ainsi dualité entre deux concepts

  • prendre une valeur acceptée pour un test bilatéral de niveau http://freakonometrics.blog.free.fr/public/perso2/testic-04.gif
  • prendre une valeur appartenant à l’intervalle de confiance symétrique de niveau 1-http://freakonometrics.blog.free.fr/public/perso2/testic-04.gif

Considérons par exemple le test asymptotique du rapport de vraisemblance (évoqué ici), dont la région d’acceptation sera de la forme

http://freakonometrics.blog.free.fr/public/perso2/tesp100.gif
Étant donné un échantillon http://freakonometrics.blog.free.fr/public/perso2/testh03.gif, l’intervalle de confiance pour  est

http://freakonometrics.blog.free.fr/public/perso2/tesp101.gif
Numériquement, reprenons l’exemple du jeu de pile ou face abordé ici.

> n=20; alpha=.05
> X
y2=sample00:lic/ 2C-Wmensionay/c
l-lies-01*e-et=TRUEestersitneg> n=20; alpha=.05
> X
y2=funsuch e-et(p){2,10,.5,alternative="less")
 
1-
> 
> clines(n,x1,lwd=2,samplemessage:
In message:
In lines(n,x1,lwd=2,> n=20; alpha=.05
> X
y2=funsuch e-et(p){pport de vraisem>
<2145Nice,id="post-831" class="post-831 post type-post status-publish format-standard has-post-thuproptem 0.3, aterva.gif" /f sroptis lessropt Pa sropmg gin-of- probasropesesempore sropreakonome sroprnomee sropf="http: > hyp-- .en-> he.gifeakir/esesempore fact211;f="1;f="inttiques et intervalles de confiance" srcset="https://f-origin.hypotheses.org/4p-coradim.jptc th-"trxennpersf w0iles/2013/02/slot1-131x200.png 131w, https://f-origin.hypotheses.org/4p-coradim.jptc th-1sl4x7g.f

/te24iles/2013/02/slot1-131x200.png 131w, https://f-origin.hypotheses.org/4p-coradim.jptc th-50x3.f

/50iles/2013/02/slot1-131x200.png 131w, https://f-origin.hypotheses.org/4p-coradim.jptc th-80x59f

/80iles/2013/02/slot1-131x200.png 131w, https://f-origin.hypotheses.org/4p-coradim.jptc th-cs.x73f

/te0iles/2013/02/slot1-131x200.png 131w, https://f-origin.hypotheses.org/4p-coradim.jptc th-s.dx147f

/s.dir/253/files/2013/02/s w01.png 655w" w01.s="(max-width: 196px) 100vw, 196px" />

he.gifeakir/esesempore fact211;f="1;f="inte-etp. et intervalles de confiance

Iian, i.@aditig-rrel osictonfly, Ied ou4_m.d that yopubat twdon manon0% and-06.lle bg/rublontility to hirete thatmpNum.d tscr thaIed ou4_mces hihten, ide cve5 f/Mathelan con factBrazimina ap wetespconThis;avadem thh new-> phod.d th -06.e">2822M"1;Rc dseff, 62opub cons46,8%t rement spcon="Tesve div2mJosespS pré 68opub cons42,7%“2822-demg gns -06.gifpublic/pue cndttp ndit re2,2%̶a-06.e>postive émg gin con problont2.2%588ased meais ( usid éj l&nnfiet)lp thaf wethoseaa-06.e">2822a hp://ese60.gif"espronslt="2withdi://we cc/p
  • licnfianc“lg gnsese60.gif"y-06.e>po> wics.c/persesesempore f-rfoundaUsonfly, ;avf="hto2/testory, ina discrimd émg gin con problconthelng:etesesemporeobvithost-lies- that y> lonperso2col">coli/fb cd énomndtrde probabiperso2col">colvithnomndtrde9c/eartiersipto7.gif" / conr risfrom.ay that those bounds are cond éses.oberobvi217bg/ore onperso2col">colvithlg gnditcononsontabta mhisgif" psont1/ 2-docd scr0e7ics.hyworeawitematis les7.gif" / (tscuphod.oo sm6.gised onperso2colhnoyl">p //tocie-m:ervalgeo>) sere.fr/publsormhodso.ion<-m till1  uccriminamg gin con prob#8217;ie.fr/publsormhodso.ion<-m till1  lmeais p th#8217;ie.fr/publsormhodso.ion<-m till1  Somathegiodiffwetersnew> podsif tc. Bg/r wet, r0ewis thoe.gifeaetesesemporesmrimi;avd én="1;f="intobviontot obef" tisero ou;avana-06.e">ol -06.nne>ubpod g, a good ;av1993 ;av2822d éAmnr altSs-post-tbem̶ look a04.onfie.fr/ue">

    confi/files/2de-mptmageo2/.regue">

    l-docconfie.fr/publsormhodso.ion<-m till1  l="" />gif" obvif" elnccperso2col">col col perso2col>) ser/files/2de-mptmageo2/.regpublsormhodso.ion<-m till1  ) ser/files/2de-mptmageo2/.regpublsormhodso.ion<-m till1 ) ser/files/2de-mptmageo2/.regpublsormhodso.ion<-m till1  confi/files/2de-mptmageo2/.regue">

    col icsrttp a a-06.e>pfr/public/percolhnoyl">p //tocie-m:ervalgeo>) sere.fr/publsormhodso.ion<-m till1  ) ser/files/2de-mptmageo2/.regpublsormhodso.ion<-m till1 p //tocie-m:ervalgeo>) sere.fr/publsormhodso.ion<-m till1  .<1-rfoundaersitic1=ies(1.96/p://(4*n),regth(p)-rfoundaersitic2=1.96*p://(p-r.or))/p://(ngrfoundaersitditta>.<1rfoundaersitic31=1.96*p://(2*pTT2tta^2)/p://(ngrfoundaersitditta>.2rfoundaersitic32=1.96*p://(2*pTT2tta^2)/p://(ngrfoundaersitpten(p,ic32,type=-06.e>p/centnne,col=-06.e>pde-r-06.e>p-rfoundaersiticnls(p,ic31,col=-06.e>pone-06.e>p-rfoundaersiticnls(p,ic2-rfoundaersiticnls(p,ic-doty=2-rfoundaucce hd éa hre1antha cd édo4" e: cnlsont2 é"822eomndtrde̶uphod.oo sm cd éetin: cnlsoav2al pro>asyckre-t-04g>1anmir/mates: p 0.2

    :vanae.ftest">proont2oe.gsero ouhypem 0.3, atenri,e.gifeaeakir/dgiomul.oromberobvi217bg/oresifriminese60.gif"p:-Ilconfie.fr/publsormhodso.ion<-m till1 l-doIlconfie.fr/publsormhodso.ion<-m till1  Onconfie.fr/publsormhodso.ion<-m till1  col O)ceHf>On/ éese60.gif" /2oeop.ten/ im.l oflsontell, if5% (gised s-56smalgifea e:rimin/ é2%l50ead man usjust to berso2/t/files/2de-mptmageo2/.regue">

    lg gnsese60.gif"y-06.e>poontan 5hi-squa-06.lle&#rsiost anything can be claimed !