Modeling strategies for prognostic models with time-dependent treatment variables_Chinese Journal of Evidence-Based Medicine

Authors：

QIAN Di , JIN Zhichao ,  ZHAO Yanfang

1. Department of Military Health Statistics, Naval Medical University, Shanghai 200433, P. R. China;

Corresponding author：

ZHAO Yanfang, Email: zhyf715@126.com

Keywords：

Prognostic model; Marginal structure model; Simulation study

DOI：

10.7507/1672-2531.202310209

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To explore the method of constructing time-dependent variables of clinical prognostic model, and to combine marginal structure model with clinical prognostic model to provide a more accurate tool for individualized prognostic assessment of patients. Methods Through data simulation, a training dataset with sample size of 7 000 and a validation dataset with sample size of 3 000 were constructed, and the predictive efficacy of ignoring treatment model, baseline no-treatment model, baseline treatment prediction model and marginal structure prediction model were respectively compared under different follow-up times and different situations. Results At 2 follow-up time points, there was no significant difference between the marginal structure prediction model and the baseline treatment prediction model, but they were higher than the neglected treatment model and the baseline no treatment model. At 5 follow-up time points, the prediction ability of the marginal structure prediction model was significantly higher than that of the other three prediction models. Conclusion In the case of time-dependent treatment in the observational cohort, the change of treatment after baseline should be considered when constructing the clinical prognosis model, otherwise the prediction accuracy of the prognosis model will be reduced.

Citation： QIAN Di, JIN Zhichao, ZHAO Yanfang. Modeling strategies for prognostic models with time-dependent treatment variables. Chinese Journal of Evidence-Based Medicine, 2024, 24(4): 490-496. doi: 10.7507/1672-2531.202310209 Copy

1.	Flores M, Glusman G, Brogaard K, et al. P4 medicine: how systems medicine will transform the healthcare sector and society. Per Med, 2013, 10(6): 565-576.
2.	Moons KG, Royston P, Vergouwe Y, et al. Prognosis and prognostic research: what, why, and how. BMJ, 2009, 338: b375.
3.	Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
4.	Steyerberg EW, Moons KG, van der Windt DA, et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med, 2013, 10(2): e1001381.
5.	Grove WM, Zald DH, Lebow BS, et al. Clinical versus mechanical prediction: a meta-analysis. Psychol Assess, 2000, 12(1): 19-30.
6.	Royston P, Moons KG, Altman DG, et al. Prognosis and prognostic research: developing a prognostic model. BMJ, 2009, 338: b604.
7.	Altman DG, Vergouwe Y, Royston P, et al. Prognosis and prognostic research: validating a prognostic model. BMJ, 2009, 338: b605.
8.	Dekkers OM, Mulder JM. When will individuals meet their personalized probabilities. A philosophical note on risk prediction. Eur J Epidemiol, 2020, 35(12): 1115-1121.
9.	Piccininni M, Konigorski S, Rohmann JL, et al. Directed acyclic graphs and causal thinking in clinical risk prediction modeling. BMC Med Res Methodol, 2020, 20(1): 179.
10.	Sperrin M, Jenkins D, Martin GP, et al. Explicit causal reasoning is needed to prevent prognostic models being victims of their own success. J Am Med Inform Assoc, 2019, 26(12): 1675-1676.
11.	Pladet L, Luijken K, Fresiello L, et al. Clinical decision support for extracorporeal membrane oxygenation: will we fly by wire. Perfusion, 2023, 38(1_suppl): 68-81.
12.	Clift AK, Dodwell D, Lord S, et al. Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study. BMJ, 2023, 381: e073800.
13.	Pajouheshnia R, Damen JAAG, Groenwold RHH, et al. Treatment use in prognostic model research: a systematic review of cardiovascular prognostic studies. Diagn Progn Res, 2017, 1: 15.
14.	Liew SM, Doust J, Glasziou P. Cardiovascular risk scores do not account for the effect of treatment: a review. Heart, 2011, 97(9): 689-697.
15.	Liew SM, Doust J, Glasziou P. Systematic review did not consider problem of treatment effects. BMJ, 2012, 345: e4355.
16.	Pajouheshnia R, Schuster NA, Groenwold RHH, et al. Accounting for time‐dependent treatment use when developing a prognostic model from observational data: a review of methods. Stat Neerland, 2019, 74: 38-51.
17.	Cheong-See F, Allotey J, Marlin N, et al. Prediction models in obstetrics: understanding the treatment paradox and potential solutions to the threat it poses. BJOG, 2016, 123(7): 1060-1064.
18.	Groenwold RH, Moons KG, Pajouheshnia R, et al. Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings. J Clin Epidemiol, 2016, 78: 90-100.
19.	Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology, 2000, 11(5): 550-560.
20.	Sperrin M, Martin GP, Pate A, et al. Using marginal structural models to adjust for treatment drop-in when developing clinical prediction models. Stat Med, 2018, 37(28): 4142-4154.
21.	van Geloven N, Swanson SA, Ramspek CL, et al. Prediction meets causal inference: the role of treatment in clinical prediction models. Eur J Epidemiol, 2020, 35(7): 619-630.
22.	Alba AC, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA, 2017, 318(14): 1377-1384.
23.	Huang Y, Li W, Macheret F, et al. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc, 2020, 27(4): 621-633.
24.	Grant SW, Collins GS, Nashef SAM. Statistical primer: developing and validating a risk prediction model. Eur J Cardiothorac Surg, 2018, 54(2): 203-208.
25.	Lin L, Sperrin M, Jenkins DA, et al. A scoping review of causal methods enabling predictions under hypothetical interventions. Diagn Progn Res, 2021, 5(1): 3.
26.	Fehr J, Piccininni M, Kurth T, et al. Assessing the transportability of clinical prediction models for cognitive impairment using causal models. BMC Med Res Methodol, 2023, 23(1): 187.
27.	Dickerman BA, Dahabreh IJ, Cantos KV, et al. Predicting counterfactual risks under hypothetical treatment strategies: an application to HIV. Eur J Epidemiol, 2022, 37(4): 367-376.
28.	Mitra N, Roy J, Small D. The future of causal inference. Am J Epidemiol, 2022, 191(10): 1671-1676.
29.	Prosperi M, Guo Y, Sperrin M, et al. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nat Mach Intell, 2020, 2(7): 369-375.

1. Flores M, Glusman G, Brogaard K, et al. P4 medicine: how systems medicine will transform the healthcare sector and society. Per Med, 2013, 10(6): 565-576.
2. Moons KG, Royston P, Vergouwe Y, et al. Prognosis and prognostic research: what, why, and how. BMJ, 2009, 338: b375.
3. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
4. Steyerberg EW, Moons KG, van der Windt DA, et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med, 2013, 10(2): e1001381.
5. Grove WM, Zald DH, Lebow BS, et al. Clinical versus mechanical prediction: a meta-analysis. Psychol Assess, 2000, 12(1): 19-30.
6. Royston P, Moons KG, Altman DG, et al. Prognosis and prognostic research: developing a prognostic model. BMJ, 2009, 338: b604.
7. Altman DG, Vergouwe Y, Royston P, et al. Prognosis and prognostic research: validating a prognostic model. BMJ, 2009, 338: b605.
8. Dekkers OM, Mulder JM. When will individuals meet their personalized probabilities. A philosophical note on risk prediction. Eur J Epidemiol, 2020, 35(12): 1115-1121.
9. Piccininni M, Konigorski S, Rohmann JL, et al. Directed acyclic graphs and causal thinking in clinical risk prediction modeling. BMC Med Res Methodol, 2020, 20(1): 179.
10. Sperrin M, Jenkins D, Martin GP, et al. Explicit causal reasoning is needed to prevent prognostic models being victims of their own success. J Am Med Inform Assoc, 2019, 26(12): 1675-1676.
11. Pladet L, Luijken K, Fresiello L, et al. Clinical decision support for extracorporeal membrane oxygenation: will we fly by wire. Perfusion, 2023, 38(1_suppl): 68-81.
12. Clift AK, Dodwell D, Lord S, et al. Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study. BMJ, 2023, 381: e073800.
13. Pajouheshnia R, Damen JAAG, Groenwold RHH, et al. Treatment use in prognostic model research: a systematic review of cardiovascular prognostic studies. Diagn Progn Res, 2017, 1: 15.
14. Liew SM, Doust J, Glasziou P. Cardiovascular risk scores do not account for the effect of treatment: a review. Heart, 2011, 97(9): 689-697.
15. Liew SM, Doust J, Glasziou P. Systematic review did not consider problem of treatment effects. BMJ, 2012, 345: e4355.
16. Pajouheshnia R, Schuster NA, Groenwold RHH, et al. Accounting for time‐dependent treatment use when developing a prognostic model from observational data: a review of methods. Stat Neerland, 2019, 74: 38-51.
17. Cheong-See F, Allotey J, Marlin N, et al. Prediction models in obstetrics: understanding the treatment paradox and potential solutions to the threat it poses. BJOG, 2016, 123(7): 1060-1064.
18. Groenwold RH, Moons KG, Pajouheshnia R, et al. Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings. J Clin Epidemiol, 2016, 78: 90-100.
19. Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology, 2000, 11(5): 550-560.
20. Sperrin M, Martin GP, Pate A, et al. Using marginal structural models to adjust for treatment drop-in when developing clinical prediction models. Stat Med, 2018, 37(28): 4142-4154.
21. van Geloven N, Swanson SA, Ramspek CL, et al. Prediction meets causal inference: the role of treatment in clinical prediction models. Eur J Epidemiol, 2020, 35(7): 619-630.
22. Alba AC, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA, 2017, 318(14): 1377-1384.
23. Huang Y, Li W, Macheret F, et al. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc, 2020, 27(4): 621-633.
24. Grant SW, Collins GS, Nashef SAM. Statistical primer: developing and validating a risk prediction model. Eur J Cardiothorac Surg, 2018, 54(2): 203-208.
25. Lin L, Sperrin M, Jenkins DA, et al. A scoping review of causal methods enabling predictions under hypothetical interventions. Diagn Progn Res, 2021, 5(1): 3.
26. Fehr J, Piccininni M, Kurth T, et al. Assessing the transportability of clinical prediction models for cognitive impairment using causal models. BMC Med Res Methodol, 2023, 23(1): 187.
27. Dickerman BA, Dahabreh IJ, Cantos KV, et al. Predicting counterfactual risks under hypothetical treatment strategies: an application to HIV. Eur J Epidemiol, 2022, 37(4): 367-376.
28. Mitra N, Roy J, Small D. The future of causal inference. Am J Epidemiol, 2022, 191(10): 1671-1676.
29. Prosperi M, Guo Y, Sperrin M, et al. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nat Mach Intell, 2020, 2(7): 369-375.

Previous Article
Discussion on the construction of comprehensive evaluation model of post-marketing clinical safety of traditional Chinese medicine injection based on mixed methods research

Chinese Journal of Evidence-Based Medicine

Modeling strategies for prognostic models with time-dependent treatment variables

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Format

Content