Elena Rätsch: Exploring Differential Machine Learning Prediction Performance of Population Subgroups in the UK Biobank

BCCN Berlin / Technische Universität Berlin

Abstract

The incorporation of ML methods in biomedical research and healthcare applications raises significant fairness-related concerns as studies have revealed model performance disparities across subpopulations defined by sensitive attributes. We address this debate by conducting a disaggregated analysis of prediction performance of a regularized regression model across population subgroups defined by sensitive attributes (sex, townsend deprivation index, ethnic background) within the UK Biobank dataset. Specifically, we employ imaging-derived phenotypes extracted from T1-weighted structural brain MRI scans to predict the target phenotypes of age, fluid intelligence, and neuroticism. As this study aimed to evaluate the presence and causes of prediction performance disparities, we employed several regression metrics to capture patterns in subgroup-specific model prediction performance.

Key findings reveal significant performance disparities across all prediction targets and subgroups investigated. Further analysis employing test set matching methods exposes sampling bias and consequent dataset shifts as contributing factors to the observed model performance disparities. A qualitative assessment of simplified projections of the assumed linear feature-target relations provides evidence for distinct subgroup-specific variations in the underlying feature-target relation as additional drivers of model performance disparities in certain prediction scenarios. For some prediction scenarios, we were able to identify parallels between the observed subgroup-specific differences in the feature-target relation and bias phenomena in the training data. The findings emphasize the necessity of disaggregated model performance analysis, as differential causes for model performance disparities require apposite bias mitigation approaches.

 

 

Additional information:

Master thesis defense

 

Organized by:

Prof. Dr. Kerstin Ritter & Prof. Dr. Stefan Haufe