Background: Diabetes mellitus is a major global health burden, and its early detection is essential for preventing serious complications. Machine learning, and eXtreme Gradient Boosting (XGBoost) in particular, performs strongly on routine clinical data; however, reported results are frequently optimistic because of hold-out evaluation and data leakage, and the resulting models are often difficult to interpret.
Objective: To develop and rigorously evaluate an interpretable XGBoost model for predicting the presence of diabetes from eight routine diagnostic measurements, using an unbiased, leakage-controlled evaluation design.
Methods: An openly available dataset of 1,168 patient records (771 non-diabetic and 397 diabetic; an approximately 66:34 class imbalance) with eight clinical features was analysed. Physiologically implausible zero values were treated as missing and imputed with the median inside a processing pipeline. Model selection and performance estimation were separated using a 5×5 nested cross-validation scheme, with all preprocessing confined to the training folds to prevent data leakage. Performance was assessed on out-of-fold predictions using accuracy, sensitivity, specificity, precision, F1 score, the area under the ROC curve (ROC-AUC), and the Brier score; probability calibration was examined; and the contribution of each feature was quantified with SHapley Additive exPlanations (SHAP).
Results: The model achieved an ROC-AUC of 0.823, an accuracy of 0.769, a specificity of 0.859, a sensitivity of 0.595, a precision of 0.684, an F1 score of 0.636, and a Brier score of 0.160. The near-diagonal calibration curve and low Brier score indicated that the predicted probabilities were well calibrated and could be interpreted as reliable risk estimates. SHAP analysis identified glucose, body mass index, and age as the most influential predictors, in agreement with established clinical risk factors for diabetes.
Conclusions: Under a leakage-controlled, unbiased evaluation, XGBoost provided moderate but trustworthy discrimination together with well-calibrated probabilities for diabetes prediction, while SHAP confirmed clinically plausible predictors. The comparatively conservative performance underscores the importance of nested cross-validation over simple hold-out splits for realistic model assessment. External validation and decision-threshold or class-imbalance strategies represent promising directions for future work.