15. Linear Models#

At this point in the book, we’ve covered, to different extents, the four stages of the data science lifecycle. We’ve talked about formulating questions and obtaining and cleaning data. We’ve used exploratory data analysis to better understand the data, and in this chapter, we extend the constant model, introduced in Chapter 4, to the linear model. Linear models are a popular tool in the last stage of the lifecycle—understanding the world.

Knowing how to fit linear models opens the door to all kinds of useful data analyses. We can use these models to make predictions—for example, environmental scientists developed a linear model to predict air quality based on air sensor measurements and weather conditions (see Chapter 12). In that case study, understanding how measurements from two instruments varied together enabled us to calibrate inexpensive sensors and improve their air quality readings. We can also use models to make inferences about the form of a relationship between features–for example, veterinarians used a linear model (see Chapter 18) to infer the coefficients for length and girth in a model for a donkey’s weight: \( Length ~+ ~ 2 \times Girth ~-~175 \). In that case study, the model enables vets working in the field to prescribe medication for sick donkeys. And, models can help describe relationships and provide insights–for example, in this chapter, we explore relationships between factors correlated with upward mobility, such as commute time, income inequality, and the quality of K-12 education. We carry out a descriptive analysis that follows an analysis social scientists have used to shape public conversation and inform policy recommendations.

In this chapter, we start by describing the simple linear model, which summarizes the relationship between two features with a line. We explain how to fit this line to data using the loss minimization approach first introduced in Chapter 4. Then, we introduce the multiple linear model, which models one feature using multiple, other features. To fit such a model, we use linear algebra and reveal the geometry behind fitting a linear model with squared error loss. Finally, we cover feature engineering techniques that let us include categorical features and transformed features when building models.