Time Profile of Time-Dependent Area Under the ROC Curve for Survival Data
DOI:
https://doi.org/10.6000/1929-6029.2015.04.01.12Keywords:
Survival analysis, Prognostic models, Time-dependent AUC, Proportional hazards modelsAbstract
In the setting of survival analysis, the time-dependent area under the receiver operating characteristic curve (AUC) has been proposed as a discrimination measure of interest. In contrast with the diagnostic setting, the definitions of time-dependent sensitivity and specificity are required. This paper evaluates the time-dependent profile of the resulting AUC(t), which has not been previously assessed. We show that, even when the effect of a binary biomarker on the hazard rate is constant, the value of AUC(t) varies over time according to the prevalence of the marker. The Time-profile of the continuous biomarker is illustrated with numerical integration, and data on several prognostic factors in AML are examined.
References
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology 2010; 21(1): 128. http://dx.doi.org/10.1097/EDE.0b013e3181c30fb2 DOI: https://doi.org/10.1097/EDE.0b013e3181c30fb2
Schemper M, Henderson R. Predictive accuracy and explained variation in Cox regression. Biometrics 2000; 56(1): 249-55. http://dx.doi.org/10.1111/j.0006-341X.2000.00249.x DOI: https://doi.org/10.1111/j.0006-341X.2000.00249.x
O’Quigley J, Xu R. Explained variation in proportional hazards regression. In: Crowley J, Ed. Handbook of Statistics in Clinical Oncology, Marcel Dekker, New York 2001; pp. 397-409.
Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Statistics in Medicine 2004; 23(13): 2109-23. http://dx.doi.org/10.1002/sim.1802 DOI: https://doi.org/10.1002/sim.1802
Harrell FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models: issues in developingmodels, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 1996; 15: 361-387. http://dx.doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics 2005; 61(1): 92-105. http://dx.doi.org/10.1111/j.0006-341X.2005.030814.x DOI: https://doi.org/10.1111/j.0006-341X.2005.030814.x
Maisel A, Mueller C, Neath SX, Christenson RH, Morgenthaler NG, McCord J, Nowak RM, Vilke G, Daniels LB, Hollander JE, Apple FS, Cannon C, Nagurney JT, Schreiber D, Defilippi C, Hogan C, Diercks DB, Stein JC, Headden G, Limkakeng ATJ, Anand I, Wu AH, Papassotiriou, J, Hartmann O, Ebmeyer S, Clopton P, Jaffe AS, Peacock WF. Copeptin Helps in the Early Detection of Patients With Acute Myocardial Infarction Primary Results of the CHOPIN Trial (Copeptin Helps in the early detection Of Patients with acute myocardial INfarction). J Am CollCardiol 2013; 62(2): 150-60. http://dx.doi.org/10.1016/j.jacc.2013.04.011 DOI: https://doi.org/10.1016/j.jacc.2013.04.011
Aalen O. Nonparametric inference for a family of counting process. The Annals of Statistics 1978; 6(4): 701-726. http://dx.doi.org/10.1214/aos/1176344247 DOI: https://doi.org/10.1214/aos/1176344247
Therneau T, Grambsch P. Modeling Survival Data: Extending the Cox Model. Springer 2000. http://dx.doi.org/10.1007/978-1-4757-3294-8 DOI: https://doi.org/10.1007/978-1-4757-3294-8
Chambless LE, Diao G. Estimation of time-dependent area under the roc curve for long-term risk prediction. Statistics in Medicine 2006; 25(20): 3474-3486. http://dx.doi.org/10.1002/sim.2299 DOI: https://doi.org/10.1002/sim.2299
Heagerty PJ, Lumley T, Pepe MS. Time-dependent roc curves for censored survival data and a diagnosticmarker. Biometrics 2000; 56(2): 337-344. http://dx.doi.org/10.1111/j.0006-341X.2000.00337.x
Hung H, Chiang C. Estimation methods for time-dependent auc models with survival data. The Canadianmarker. Biometrics 2000; 56(2): 337-344. DOI: https://doi.org/10.1111/j.0006-341X.2000.00337.x
Uno H, Cai T, Tian L, Wei LJ. Evaluating prediction rules for t-year survivors with censored regression models. Journal of the American Statistical Association 2007; 102(478). http://dx.doi.org/10.1198/016214507000000149 DOI: https://doi.org/10.1198/016214507000000149
Song X, Zhou X, Ma S. Nonparametric receiver operating characteristic-based evaluation for survival outcomes. Statistics in Medicine 2012; 31(23): 2660-2675. http://dx.doi.org/10.1002/sim.5386 DOI: https://doi.org/10.1002/sim.5386
R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria 2012. R Foundation for Statistical Computing. ISBN 3-900051-07-0. http://www.R-project.org/
Blanche P. timeROC: Time-dependent ROC Curve and AUC for Censored Survival Data 2012. R package version 0.1. http://CRAN.R-project.org/package=timeROC
Heagerty PJ. Packaging by ParamitaSaha-Chaudhuri: risksetROC: Riskset ROC Curve Estimation from Censored Survival Data 2012. R package version 1.0.4. http://CRAN.R-project.org/package=risksetROC
Heagerty PJ. Packaging by ParamitaSaha-Chaudhuri: survivalROC: Time-dependent ROC Curve Estimation from Censored Survival Data 2013. R package version 1.0.3. http://CRAN.R-project.org/package=survivalROC
Potapov S, Adler W, Schmid M. survAUC: Estimators of Prediction Accuracy for Time-to-event Data 2011. R package version 1.0-4. http://CRAN.R-project.org/package=survAUC
Castaigne S, Pautas C, Terré C, Raffoux E, Bordessoule D, Bastie J, Legrand O, Thomas X, Turlure P, Reman O, de Revel T, Gastaud L, de Dunzburg N, Contentin N, Henry E, Marolleau J, Aljijakli A, Rousselot P, Fenaux P, Preudhomme P, Chevret S, Dombret H. Effect of gemtuzumabozogamicin on survival of adult patients with de-novo acute myeloidleukaemia (alfa-0701): arandomised, open-label, phase 3 study. The Lancet 2012; 379(9825): 1508-1516. http://dx.doi.org/10.1016/S0140-6736(12)60485-1 DOI: https://doi.org/10.1016/S0140-6736(12)60485-1
Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika 1994; 81(3): 515-526. http://dx.doi.org/10.1093/biomet/81.3.515 DOI: https://doi.org/10.1093/biomet/81.3.515
Blanche P, Latouche A, Viallon V. Time-dependent auc with right-censored data: a survey study. arXivpreprint arXiv:1210.6805 2012. DOI: https://doi.org/10.1007/978-1-4614-8981-8_11
Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation tothe variance and odds ratio of a continuous explanatory variable. BMC Medical Research Methodology 2012; 12(1): 82. http://dx.doi.org/10.1186/1471-2288-12-82 DOI: https://doi.org/10.1186/1471-2288-12-82
Agresti A. An introduction to categorical data analysis Wiley 1996; Vol. 135.
May M, Sterne J, Egger M. Parametric survival models may be more accurate than Kaplan-Meier estimates. BMJ 2003; 326(7393): 822. http://dx.doi.org/10.1136/bmj.326.7393.822 DOI: https://doi.org/10.1136/bmj.326.7393.822
Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Medical Research Methodology 2011; 11(1): 13. http://dx.doi.org/10.1186/1471-2288-11-13 DOI: https://doi.org/10.1186/1471-2288-11-13
Pepe MS, Kerr KF, Longton G, Wang Z. Testing for improvement in prediction model performance. Statistics in medicine 2013; 32(9): 1467-1482. http://dx.doi.org/10.1002/sim.5727 DOI: https://doi.org/10.1002/sim.5727
Austin PC, Steyerberg EW. Predictive accuracy of risk factors and markers: a simulation study of the effect ofnovel markers on different performance measures for logistic regression models. Statistics in Medicine 2013; 32(4): 661-672. http://dx.doi.org/10.1002/sim.5598 DOI: https://doi.org/10.1002/sim.5598
Harrell FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 1996; 15: 361-387. http://dx.doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4 DOI: https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
Uno H, Cai T, Pencina MJ, D’Agostino RB, Wei LJ. On the c-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 2011; 30(10): 1105-1117. DOI: https://doi.org/10.1002/sim.4154
Gönen M, Heller G. Concordance probability and discriminatory power in proportional hazards regression. Biometrika 2005; 92(4): 965-970. http://dx.doi.org/10.1093/biomet/92.4.965 DOI: https://doi.org/10.1093/biomet/92.4.965
Li J, Ma S. Time-dependent roc analysis under diverse censoring patterns. Statistics in Medicine 2011; 30(11): 1266-1277. DOI: https://doi.org/10.1002/sim.4178
Lambert J, Chevret S. Summary measure of discrimination in survival models based on cumulative/dynamic time-dependent ROC curves. Stat Methods Med Res 2014; [Epub ahead of print]. http://dx.doi.org/10.1177/0962280213515571 DOI: https://doi.org/10.1177/0962280213515571
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2015 J. Lambert, R. Porcher, S. Chevret
This work is licensed under a Creative Commons Attribution 4.0 International License.
Policy for Journals/Articles with Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
Policy for Journals / Manuscript with Paid Access
Authors who publish with this journal agree to the following terms:
- Publisher retain copyright .
- Authors are permitted and encouraged to post links to their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work .
