Feature Importance Metric

What is Feature Importance?

Feature Importance ranks variables by their contribution to model predictions. It helps identify which features are most influential.

Calculation Methods

Tree-Based (Random Forest, XGBoost)

• Based on how often features are used for splits
• How much they reduce impurity/loss

Permutation Importance

• Shuffle each feature and measure performance drop
• Works with any model

Interpretation

• High Importance: Strong predictive power
• Low Importance: Limited contribution to predictions
• Relative Values: Compare features within the same model

Uses

• Understand model behavior
• Feature selection (remove unimportant features)
• Scientific discovery (identify key biomarkers)

Related Terms

• Feature Selection: Choosing important features
• Random Forest: Algorithm providing importance
• XGBoost: Another importance-providing algorithm