Daniel Vargas Herrera: Systematic comparison of machine and deep learning models for diagnosing schizophrenia based on structural brain MRI data

BCCN Berlin / Charité-Universitätsmedizin Berlin

 

Abstract

Schizophrenia is a serious mental disorder whose cause and biomarkers are yet to be unraveled. Applying machine learning approaches to high-dimensional neuroimaging data is a promising approach for the detection of novel patterns, possibly elucidating the underlying pathology in an unbiased way. We used a structural MRI data set comprising 65 patients and 79 healthy controls.

Here, we systematically compared the performance of classical machine learning algorithms such as support vector machines, logistic regression, and gradient boosting, paired with feature extraction methods as principal component analysis, F-score based feature selection, and brain parcellation. Additionally, we compared the performance of a three dimensional convolutional neural network trained on the complete brain scans. The generalization error was estimated using 5-fold cross-validation and an additional unseen test set. We assessed the influence of sex and age by analyzing the obtained accuracies separately for males and females, as well as for three age groups. Furthermore, we repeated the analysis using solely sex and age as predictors for schizophrenia. Lastly, we evaluated the classification ability of individual regions. Classical machine learning algorithms presented a large discrepancy between cross-validated (76-78%) and test set (55-68%) balanced accuracies. In contrast, the cross-validated balanced accuracies from the convolutional neural network (65-67.5%) did not have a large decrease in the test set (64%). Sex-specific results did not reveal a difference in performance between males and females. The performance of sex as a schizophrenia predictor stayed as expected at chance level. Age-specific results showed that classifiers improved their performance on participants in the age range of 27-35 (~70%). Training the models exclusively on age also yielded a high balanced accuracy 69 %. The use of individual regions to predict schizophrenia was not informative, as the test set accuracies stayed at chance level for all the estimators. This large discrepancy between cross-validated and test results suggest that the sample sizes are not large enough to provide a robust estimate of the generalization error for classical machine learning algorithms. The region-specific results suggest that the structure of individual brain regions cannot be used for disease classification at the scale of this dataset. The analysis based on age hints that the algorithms are learning correlations between age and clinical condition, rather than illness-specific patterns in brain structure. In spite of the limitations of this study, the proposed methodology can be validated by studies with larger sample sizes, and it can be further extended to other mental diseases.

 

Additional Information

Master Thesis Defense

If you would like to join the online talk, please contact graduateprograms(at)bccn-berlin.de

 

Organized by

Prof. Dr. Kerstin Ritter / Lisa Velenosi