Consider the simple case of a moving histogram (which is a very simple kernel). The idea is to recall that
is the slope close to point .
Then we use the empirical cumulative density to approximate the slope, i.e.
which can also be writen
Consider now the density seen as a random variable
where the‘s are i.i.d. where , with
Thus, observe that , but that’s not what we’re looking for… From Taylor’s expansion,
where the bias comes from the approximation of the density by some string. About the variance,
thus, since ,
We can observe that
is decreasing as , while the variance is increasing as . This is the standard bias-variance tradeoff in statistics.