Why can feature importance be unstable when predictors are strongly correlated?

Why

Imagine three genes on the same pathway that rise and fall together. A random forest may split on gene A in one fit and gene B in another, and impurity-based importance spreads credit among them in ways that change with the random seed. Permutation importance, which measures how much a score drops when one feature is shuffled, has its own failure.

The scikit-learn documentation explains that when one of two correlated features is permuted, the model still reaches the information through the other, so both get a lower importance than they deserve. Its breast cancer example shows a forest that predicts well while every feature looks unimportant. The documentation also notes that impurity-based importance is biased toward high-cardinality numeric features and is computed on training data, while permutation importance can be computed on held-out data.

The remedies change the question rather than the number. Cluster correlated features, for example by Spearman correlation, and report importance per cluster, or permute whole groups together. Check stability across bootstrap fits.

Above all, treat importance as a property of this model on this data. It is not a ranking of biological causes, and a gene with low importance may simply have a correlated partner.