The Bias-Variance Tradeoff
The Bias-Variance Tradeoff¶
This is a legacy chapter which will eventually be incorporated into Chapter 17. This chapter will remain here as an appendix until we finish Chapter 17.
Sometimes, we choose a model that is too simple to represent the underlying data generation process. Other times, we choose a model that is too complex—it fits the noise in the data rather than the data’s overall pattern.
To understand why this happens, we analyze our models using the tools of probability and statistics. These tools allow us to generalize beyond a few isolated examples to describe fundamental phenomena in modeling. In particular, we will use the tools of expectation and variance to uncover the bias-variance tradeoff.