Chapter 5 Interpretable Models

The most straightforward way to get to interpretable machine learning is to use only a subset of algorithms that create interpretable models.

Very common model types of this group of interpretable models are:

  • Linear regression model.
  • Logistic regression.
  • Decision trees.

In the following chapters we will talk about these models. Not in detail, only the basics, because there are already a ton of books, videos, tutorials, papers and more material. We will focus on how to interpret the models. This chapter covers linear models, logistic regression, and decision trees in more detail. It also lists some more.

All of the interpretable model types explained in this book are interpretable on a modular level, with the exception of the k-nearest neighbors method. The following table gives an overview over the interpretable model types and their properties. A model is linear if the association between features and target is modeled linearly. A monotonic model ensures that the relationship between a feature and the target outcome is always in the same direction over the whole range of the feature: an increase in the features value will consistently lead to either an increase or a decrease of the target outcome, but never both for this feature. Monotonicity is useful for the interpretation of a model, because it makes it easier to understand a relationship. Some models can automatically include interactions between the features for predicting the outcome. You can always include interactions into any kind of model by manually creating interaction features. This can be important for correctly predicting the outcome, but too many or too complex interactions can hurt interpretability. Some models only handle regression, some only classification, and some can manage to do both.

You can use this table to choose a suitable interpretable model for your task (either regression (regr.) or classification (class.)):

Algorithm Linear Monotone Interaction Task
Linear models Yes Yes No Regr.
Logistic regression No Yes No Class.
Decision trees No No Yes Class. + Regr.
RuleFit Yes No Yes Class. + Regr.
Naive Bayes Yes Yes No Class.n
k-nearest neighbours No No No Class. + Regr.

Terminology

  • Y is the target outcome.
  • X are the features (also called variables, covariables, covariates, or inputs).
  • \(\beta\) are regression weights (also called coefficients).