generalisation

Stability is a desirable property of clinical prediction models that can allow us to transfer models to new environments such as different hospitals or countries. In this post, we will look at how Directed Acyclic Graphs and causal reasoning can help us anticipte if a given prediction model is stable when moved between environments.

The area under the receiver operating characteristic (AUC) is arguably among the most frequently used measures of classification performance. In this post, I look at how performance estimated by AUC changes based on the covariate distribution in the evaluation set, even if we know the true model. This complicates the interpretation of external validation results if a change in this distribution is possible or even likely.