This week, we started to discuss *overdispersion* when modeling claims frequency. In my previous post, I discussed computations of empirical variances with different exposure. But I did use only one factor to compute classes. Of course, it is possible to use much more factors. For instance, using *cartesian products* of factors,

> X=as.factor(paste(sinistres$carburant,sinistres$zone, + cut(sinistres$ageconducteur,breaks=c(17,24,40,65,101)))) > E=sinistres$exposition > Y=sinistres$nbre > vm=vv=ve=rep(NA,length(levels(X))) > for(i in 1:length(levels(X))){ + ve[i]=Ei=E[X==levels(X)[i]] + Yi=Y[X==levels(X)[i]] + vm[i]=meani=weighted.mean(Yi/Ei,Ei) # moyenne + vv[i]=variancei=sum((Yi-meani*Ei)^2)/sum(Ei) # variance + cat("Class ",levels(X)[i],"average =",meani," variance =",variancei,"\n") + } Class D A (17,24] average = 0.06274415 variance = 0.06174966 Class D A (24,40] average = 0.07271905 variance = 0.07675049 Class D A (40,65] average = 0.05432262 variance = 0.06556844 Class D A (65,101] average = 0.03026999 variance = 0.02960885 Class D B (17,24] average = 0.2383109 variance = 0.2442396 Class D B (24,40] average = 0.06662015 variance = 0.07121064 Class D B (40,65] average = 0.05551854 variance = 0.05543831 Class D B (65,101] average = 0.0556386 variance = 0.0540786 Class D C (17,24] average = 0.1524552 variance = 0.1592623 Class D C (24,40] average = 0.0795852 variance = 0.09091435 Class D C (40,65] average = 0.07554481 variance = 0.08263404 Class D C (65,101] average = 0.06936605 variance = 0.06684982 Class D D (17,24] average = 0.1584052 variance = 0.1552583 Class D D (24,40] average = 0.1079038 variance = 0.121747 Class D D (40,65] average = 0.06989518 variance = 0.07780811 Class D D (65,101] average = 0.0470501 variance = 0.04575461 Class D E (17,24] average = 0.2007164 variance = 0.2647663 Class D E (24,40] average = 0.1121569 variance = 0.1172205 Class D E (40,65] average = 0.106563 variance = 0.1068348 Class D E (65,101] average = 0.1572701 variance = 0.2126338 Class D F (17,24] average = 0.2314815 variance = 0.1616788 Class D F (24,40] average = 0.1690485 variance = 0.1443094 Class D F (40,65] average = 0.08496827 variance = 0.07914423 Class D F (65,101] average = 0.1547769 variance = 0.1442915 Class E A (17,24] average = 0.1275345 variance = 0.1171678 Class E A (24,40] average = 0.04523504 variance = 0.04741449 Class E A (40,65] average = 0.05402834 variance = 0.05427582 Class E A (65,101] average = 0.04176129 variance = 0.04539265 Class E B (17,24] average = 0.1114712 variance = 0.1059153 Class E B (24,40] average = 0.04211314 variance = 0.04068724 Class E B (40,65] average = 0.04987117 variance = 0.05096601 Class E B (65,101] average = 0.03123003 variance = 0.03041192 Class E C (17,24] average = 0.1256302 variance = 0.1310862 Class E C (24,40] average = 0.05118006 variance = 0.05122782 Class E C (40,65] average = 0.05394576 variance = 0.05594004 Class E C (65,101] average = 0.04570239 variance = 0.04422991 Class E D (17,24] average = 0.1777142 variance = 0.1917696 Class E D (24,40] average = 0.06293331 variance = 0.06738658 Class E D (40,65] average = 0.08532688 variance = 0.2378571 Class E D (65,101] average = 0.05442916 variance = 0.05724951 Class E E (17,24] average = 0.1826558 variance = 0.2085505 Class E E (24,40] average = 0.07804062 variance = 0.09637156 Class E E (40,65] average = 0.08191469 variance = 0.08791804 Class E E (65,101] average = 0.1017367 variance = 0.1141004 Class E F (17,24] average = 0 variance = 0 Class E F (24,40] average = 0.07731177 variance = 0.07415932 Class E F (40,65] average = 0.1081142 variance = 0.1074324 Class E F (65,101] average = 0.09071118 variance = 0.1170159

Again, one can plot the variance against the average,

> plot(vm,vv,cex=sqrt(ve),col="grey",pch=19, + xlab="Empirical average",ylab="Empirical variance") > points(vm,vv,cex=sqrt(ve)) > abline(a=0,b=1,lty=2)

An alternative is to use a tree. The tree can be obtained from another variable (the insured had, or had not, a claim, during the period considered) but it should be rather close to the one we would like to model (the *number *of claims over the period considered). Here, I did use the whole database (with more that 600,000 lines)

> library(tree) > T=tree((nombre>0)~as.factor(zone)+as.factor(puissance)+ + as.factor(marque)+as.factor(carburant)+as.factor(region)+ + agevehicule+ageconducteur,data=baseFREQ, + split = "gini",minsize =25000)

The tree is the following

> plot(T) > text(T)

Now, each knot defines a class, and it is possible to use it to define a class. Which is supposed to be homogeneous.

> X=as.factor(T$where) > E=sinistres$exposition > Y=sinistres$nbre > vm=vv=ve=rep(NA,length(levels(X))) > for(i in 1:length(levels(X))){ + ve[i]=Ei=E[X==levels(X)[i]] + Yi=Y[X==levels(X)[i]] + vm[i]=meani=weighted.mean(Yi/Ei,Ei) # moyenne + vv[i]=variancei=sum((Yi-meani*Ei)^2)/sum(Ei) # variance + cat("Class ",levels(X)[i],"average =",meani," variance =",variancei,"\n") + } Class 6 average = 0.04010406 variance = 0.04424163 Class 8 average = 0.05191127 variance = 0.05948133 Class 9 average = 0.07442635 variance = 0.08694552 Class 10 average = 0.4143646 variance = 0.4494002 Class 11 average = 0.1917445 variance = 0.1744355 Class 15 average = 0.04754595 variance = 0.05389675 Class 20 average = 0.08129577 variance = 0.0906322 Class 22 average = 0.05813419 variance = 0.07089811 Class 23 average = 0.06123807 variance = 0.07010473 Class 24 average = 0.06707301 variance = 0.07270995 Class 25 average = 0.3164557 variance = 0.2026906 Class 26 average = 0.08705041 variance = 0.108456 Class 27 average = 0.06705214 variance = 0.07174673 Class 30 average = 0.05292652 variance = 0.06127301 Class 31 average = 0.07195285 variance = 0.08620593 Class 32 average = 0.08133722 variance = 0.08960552 Class 34 average = 0.1831559 variance = 0.2010849 Class 39 average = 0.06173885 variance = 0.06573939 Class 41 average = 0.07089419 variance = 0.07102932 Class 44 average = 0.09426152 variance = 0.1032255 Class 47 average = 0.03641669 variance = 0.03869702 Class 49 average = 0.0506601 variance = 0.05089276 Class 50 average = 0.06373107 variance = 0.06536792 Class 51 average = 0.06762947 variance = 0.06926191 Class 56 average = 0.06771764 variance = 0.07122379 Class 57 average = 0.04949142 variance = 0.05086885 Class 58 average = 0.2459016 variance = 0.2451116 Class 59 average = 0.05996851 variance = 0.0615773 Class 61 average = 0.07458053 variance = 0.0818608 Class 63 average = 0.06203737 variance = 0.06249892 Class 64 average = 0.07321618 variance = 0.07603106 Class 66 average = 0.07332127 variance = 0.07262425 Class 68 average = 0.07478147 variance = 0.07884597 Class 70 average = 0.06566728 variance = 0.06749411 Class 71 average = 0.09159605 variance = 0.09434413 Class 75 average = 0.03228927 variance = 0.03403198 Class 76 average = 0.04630848 variance = 0.04861813 Class 78 average = 0.05342351 variance = 0.05626653 Class 79 average = 0.05778622 variance = 0.05987139 Class 80 average = 0.0374993 variance = 0.0385351 Class 83 average = 0.06721729 variance = 0.07295168 Class 86 average = 0.09888492 variance = 0.1131409 Class 87 average = 0.1019186 variance = 0.2051122 Class 88 average = 0.05281703 variance = 0.0635244 Class 91 average = 0.08332136 variance = 0.09067632 Class 96 average = 0.07682093 variance = 0.08144446 Class 97 average = 0.0792268 variance = 0.08092019 Class 99 average = 0.1019089 variance = 0.1072126 Class 100 average = 0.1018262 variance = 0.1081117 Class 101 average = 0.1106647 variance = 0.1151819 Class 103 average = 0.08147644 variance = 0.08411685 Class 104 average = 0.06456508 variance = 0.06801061 Class 107 average = 0.1197225 variance = 0.1250056 Class 108 average = 0.0924619 variance = 0.09845582 Class 109 average = 0.1198932 variance = 0.1209162

Here, when ploting the empirical variance (per knot) against the empirial average of claims, we get

Here, we can identify classes where remaining heterogeneity.