Data Science Framework

Throughout this chapter we focus on Sierra Investments, a small wealth management firm. Sierra is evaluating whether it should invest in and buy certain loans. To make this decision, Sierra compares the interest revenue from the loans to the costs of acquiring loans, including costs of late payments, defaults, and collections activities. Paige Baumann, the management accountant at Sierra Investments, is interested in exploring how she might use data science to help managers make more effective investment decisions by identifying the payoffs and risks associated with different investment choices.*

As she contemplates next steps, Paige goes through the data science framework, a seven-step decision-making process for applying machine learning techniques in business situations:

  1. Gain a business understanding of the problem
  2. Obtain and explore relevant data
  3. Prepare the data
  4. Build a model
  5. Evaluate the model
  6. Visualize and communicate insights
  7. Deploy the model

We discuss each of these steps in subsequent sections. As you go through these steps, keep the “big picture” in mind. Data Science relies on backtesting and feedback to choose among models. To do this, data scientists subdivide their data into a training sample (used in Step 4 to build a model), a cross-validation sample (used in step 5 to evaluate and choose among models), and a holdout sample (used in step 5 to test how the model would perform on completely new data). Backtesting helps the data scientist to decide how detailed a model to build. Too simple a model may miss important features for making good predictions; too detailed a model may introduce irrelevant features that weaken predictions.