Generalizing the application of machine learning predictive models across different populations: does a model to predict the use of renal replacement therapy in critically ill COVID-19 patients apply to general intensive care unit patients? - Critical Care Science (CCS)

TO THE EDITOR

The widespread use of machine learning has created the possibility of generating robust prediction models for individual patients; however, caution is needed in their use for heterogeneous critically ill populations.⁽⁾ Recent literature has demonstrated major advances in the field of acute kidney injury prediction and the need for renal replacement therapy (RRT).⁽⁾ In a large multicenter cohort, we evaluated how a previously published model⁽⁾ that predicts the need for RRT in coronavirus disease 2019 (COVID-19) intensive care unit (ICU) patients would perform in a general ICU patient.

Recently, using a data-driven methodology in a multicenter cohort of 14,374 critically ill COVID-19 patients, we developed and validated a machine learning prediction model to predict the use of RRT (the “COVID-19-RRT Model”).⁽⁾ In the present study, we performed an external validation of the “COVID-19-RRT Model” in a cohort of non-COVID-19 adult patients admitted to 126 ICUs in 2022 in a Brazilian private hospital network. The data were acquired using a solution used for quality assessment (Epimed Monitor).⁽⁾ The study was approved by the Institutional Review Board after providing informed consent (Instituto D’Or de Pesquisa e Ensino [IDOR], CAAE:17079119.7.0000.5249). The prediction performance was evaluated in terms of calibration (plots and Brier’s score) and discrimination (area under the ROC curve [AU-ROC]). A description of the materials and methods used are provided in the Supplementary Material (Table 1S, 2S and Figure 1S).

[…]