# Visualising a Classification in High Dimension

So far, when discussing classification, we’ve been playing on my toy-dataset (actually, I should no claim it’s mine, it is inspired by the one used in the introduction of Boosting, by Robert Schapire and Yoav Freund). But in ral life, there are more observations, and more explanatory variables.With more than two explanatory variables, it starts to be more complicated to visualise. For instance, consider

```MYOCARDE=read.table(
"http://freakonometrics.free.fr/saporta.csv",

where we have observations from people in E.R., for infarctus, and we want to understand who did survive, to get a predictive model. But before running some classifier, let us visualise our data. Since we have seven explanatory variables and our class (survival or death), we can go for a PCA.

```library(FactoMineR) # ACP (sur les var continues)
X=MYOCARDE[,1:7]
acp=PCA(X)```

To add the death/survival variable, treat it as numerical 0/1 variable (at least to get a direction)

```MYOCARDE2=MYOCARDE
MYOCARDE2\$PRONO=(MYOCARDE2\$PRONO=="SURVIE")*1
acp=PCA(MYOCARDE2,quanti.sup=8,graph=TRUE)``` The nice thing is that we see here where variables are colinear with that one. It is also possible to visualise individuals, and classes, too

```acp=PCA(MYOCARDE,quali.sup=8,graph=TRUE)
plot(acp, habillage = 8,col.hab=c("red","blue"))``` It looks like we should be able to derive a nice classifier here. At least, while projecting on the first two components, we can see our classes.

Now, can’t we get a classifier and visualise it on the first two components ?  Because PCAs are simply based on orthogonral projection, yes, we can (we have to keep in mind also that data were normalized here). Given two coordinates on the first two components plane (assume that the other 5 are null), given our transformation matrix, our normalizing components, and a classifier (here based on a logistic regression) we can get back to the original space, and classify that new data

```acp=PCA(X,ncp=ncol(X))
M=acp\$var\$coord
m=apply(X,2,mean)
s=apply(X,2,sd)
pred=function(d1,d2,Mat,reg){
z=Mat %*% c(d1,d2,rep(0,ncol(X)-2))
newd=data.frame(t(z*s+m))
names(newd)=names(X)
predict(reg,newdata=newd,type="response") }```

Consider now a logistic regression. Just to get something not too complicated (and get rid of non significant variables), we use a stepwise procedure to simplify the model,

```reg_tot=glm(PRONO~.,data=MYOCARDE,
family=binomial)
reg_tot=step(reg_tot)
Minv=solve(M)
p=function(d1,d2) pred(d1,d2,Minv,reg_tot)```

To visualize the isoprobability line (where an individual has a 50% chance to survive) use the following

```Outer <- function(x,y,fun) {
mat <- matrix(NA, length(x), length(y))
for (i in seq_along(x)) {
for (j in seq_along(y))
mat[i,j]=fun(x[i],y[j])}
return(mat)}

xgrid=seq(-5,5,length=251)
ygrid=seq(-5,5,length=251)
zgrid=Outer(xgrid,ygrid,p)```

and then, we add a contour line on our previous graph

```acp2=PCA(MYOCARDE,quali.sup=8,graph=TRUE)
plot(acp2, habillage = 8,col.hab=c("red","blue")) It is not too bad, but we should be able to do better. What if we keep  all variables here (even if they were not significant)

```reg_tot=glm(PRONO~.,data=MYOCARDE,
family=binomial)
Minv=solve(M)
p=function(d1,d2) pred(d1,d2,Minv,reg_tot)
zgrid=Outer(xgrid,ygrid,p)
acp2=PCA(MYOCARDE,quali.sup=8,graph=TRUE)
plot(acp2, habillage = 8,col.hab=c("red","blue")) 